Clojurians Log v2

Forgot to follow up! Cljdoc is now using the new feed, works great! Thank you!

Excellent! My pleasure!

Thanks for digging in to this @lee! > Question1: The search API returns created , and the new feed returns released_at the values for these seem to be mostly the same, but maybe not always? created and released_at are intended to be the same value, but it looks like this is a bug! When we index a new release, we use the .lastModified of the pom file, not the created column from the database. (But when we do a full re-index, we use the created column, so this value isn't even consistent for a given release, as we do a full reindex every four hours). I suspect using the .lastModified of the pom file is a holdover from when we stored much less in the db, and relied on the pom files more (we moved away from that maybe eight years ago). I'll fix this to have search use the created column. > Observation2: The new data feed supports edn, which is great. I did notice that it might have a small inconsistency. ... Notice that :released_at is encoded as an #inst but :next_from is encoded as a string. This is not a problem at all for me and maybe intentional, but I noticed it and thought it best to share. This is intentional; I was worried about clients parsing the next_from, then converting it back to a string to make the next call, possibly losing the millisecond precision. I'll note in the wiki that this should be treated as an opaque value, and not parsed. I also considered encoding the value in some way to make it clearer that it is opaque, but didn't want to add that complexity. I'm open to suggestions here. > Question2: The search API and new data feed almost match in fields returned. The only field missing from the data feed is the description. I can get at the description through other means, but do you have any plans to include it in the data feed? I included only the bare minimum, but I'm happy to add description to save you the API call. > Observation3: The new data feed can find what seems to be the earliest of releases. This is because (as you noted above) the search index only has the latest release. There was a later release of nailgun on 2010-12-18 which replaced the first one in the index.

I'll create some issues for fixing the search/feed created inconsistencies and including the description, and try to get those implemented in the next couple of days!

And let me know if you need anything else!

https://github.com/clojars/clojars-web/issues/897 https://github.com/clojars/clojars-web/issues/898

Sweet! Thanks so much @tcrawley ❤️

My pleasure! I just released including the description in the feed.

On Observation2, I've really no suggestion. I suppose the :released_at could be encoded as a string for consistency? But really, there is nothing wrong with it as it is now - and a small update to the wiki would clarify.

Thanks! I'll switch cljdoc over to this, we are currently using https://github.com/cljdoc/cljdoc/blob/75fb923d3d60f94593a3a03361f1f8ee2705c5d0/src/cljdoc/server/release_monitor.clj#L37-L40, this should be a better fit.

Excellent! Yes, this should definitely be better! Let me know if you have any issues.

Note: I made a small change to this api: it now returns a next_from property that is the timestamp to get the next page. This should be a bit easier to use over next. next is still present in the response, but should be considered deprecated. I have updated the wiki page (https://github.com/clojars/clojars-web/wiki/Data#release-feed) to reflect this change.

Hiya @tcrawley, I'm trying out the new data feed. Looks great! I do have questions/observations. Question1: The search API returns created , and the new feed returns released_at the values for these seem to be mostly the same, but maybe not always? I probably don't really care, but I am curious enough to ask. For example: https://clojars.org/api/release-feed?from=2024-12-29T11:39:53.968Z Returns:

{
  "next": "/api/release-feed?from=2024-12-29T18:00:21.169Z",
  "next_from": "2024-12-29T18:00:21.169Z",
  "releases": [
    {
      "artifact_id": "stowaway",
      "group_id": "stowaway",
      "released_at": "2024-12-29T18:00:21.169Z",
      "version": "0.1.29"
    }
  ]
}

{
  "count": 2,
  "total-hits": 2,
  "results-per-page": 24,
  "offset": 0,
  "results": [
    {
      "jar_name": "stowaway",
      "group_name": "stowaway",
      "version": "0.1.29",
      "description": "Library for abstracting data storage from business logic",
      "created": "1735495214979"
    },
    {
      "jar_name": "automaton-simulation-de",
      "group_name": "org.clojars.hephaistox",
      "version": "4.5.4-la",
      "description": null,
      "created": "1735472393968"
    }
  ]
}

user=> (java.time.Instant/ofEpochMilli 1735495214979)
#object[java.time.Instant 0xc68a5f8 "2024-12-29T18:00:14.979Z"]

$ curl --header "Accept: application/edn" \?from\=2024-12-29T11:39:53.968Z
{:next "/api/release-feed?from=2024-12-29T18:00:21.169Z", :next_from "2024-12-29T18:00:21.169Z", :releases [{:artifact_id "stowaway", :group_id "stowaway", :released_at #inst "2024-12-29T18:00:21.169000000-00:00", :version "0.1.29"}]}

curl --header "Accept: application/edn" \?from\=2000-01-01T11:39:53.968Z | jet -t ':releases first' --pretty
{:artifact_id "nailgun",
 :group_id "org.clojars.ato",
 :released_at #inst "2009-11-15T04:55:16.666-00:00",
 :version "0.7.1"}

curl ''
{"count":0,"total-hits":0,"results-per-page":24,"offset":0,"results":[]}