datomic 2021-07-11 | Slack Archive

I'm getting these messages quite often, 3 at a time, when I'm starting my REPL:

Downloading: org/clojure/clojure/maven-metadata.xml from datomic-cloud

It leads to a quite slow REPL startup:

$ time clj -M:test:dev -e "(time (require 'datomic.ion))"
Downloading: org/clojure/clojure/maven-metadata.xml from datomic-cloud
Downloading: org/clojure/clojure/maven-metadata.xml from datomic-cloud
Downloading: org/clojure/clojure/maven-metadata.xml from datomic-cloud
"Elapsed time: 56.775926 msecs"
clj -M:test:dev -e "(time (require 'datomic.ion))"  52.34s user 2.88s system 335% cpu 16.463 total

I saw on stack overflow that I might want to adjust <updatePolicy>daily</updatePolicy> in my ~/.m2/settings.xml, but some commenters said it didn't work for them, so they just disabled snapshot version resolution. Since this phenomena does not occur on every startup, but I suspect only hourly, I'm not sure how to debug it. Is there a recommended way to make these checks happen less often?

onetom16:07:49

The repo is defined as:

:mvn/repos
 {"datomic-cloud" {:url ""}}

Alex Miller (Clojure team)21:07:13

what version of clj are you using?

Alex Miller (Clojure team)21:07:07

clj won't use those updatePolicy settings so I don't think that will have any effect (should be daily for things like snapshots)

Alex Miller (Clojure team)21:07:05

but what you're seeing here seems to be checking for clojure, a lib that should be found before that in other repos like maven central. it seems a little odd that you're even getting this at all (but maybe it's checking and rechecking because that metadata file is missing on the s3 maven repo)

Alex Miller (Clojure team)21:07:17

if you have an old clj, it's possible some changes that have been put in would help. 1.10.3.855 is current stable release

👍 2

onetom15:07:51

I've upgraded to 1.10.3.855, but that didn't help. then i've updated the clojure version in my deps.edn from 1.10.1 to 1.10.3 and now it's not looking for org/clojure/clojure/maven-metadata.xml on datomic-cloud. HOWEVER, i've just moved from martian/martian {:mvn/version "0.1.16"} to com.github.oliyh/martian {:mvn/version "0.1.17-SNAPSHOT"} and now im getting similar messages, roughly every 10-15 minutes, when I start a fresh Clojure CLI process, even just clojure -A:dev:test -Stree:

Downloading: com/github/oliyh/martian/0.1.17-SNAPSHOT/maven-metadata.xml from datomic-cloud
Downloading: com/github/oliyh/martian-clj-http/0.1.17-SNAPSHOT/maven-metadata.xml from datomic-cloud
Downloading: com/github/oliyh/martian-httpkit/0.1.17-SNAPSHOT/maven-metadata.xml from datomic-cloud
Downloading: martian/martian/0.1.17-SNAPSHOT/maven-metadata.xml from datomic-cloud
Downloading: martian/martian/0.1.17-SNAPSHOT/maven-metadata.xml from datomic-cloud
Downloading: com/github/oliyh/martian-clj-http/0.1.17-SNAPSHOT/maven-metadata.xml from datomic-cloud
Downloading: com/github/oliyh/martian-httpkit/0.1.17-SNAPSHOT/maven-metadata.xml from datomic-cloud
Downloading: martian/martian/0.1.17-SNAPSHOT/maven-metadata.xml from datomic-cloud
Downloading: com/github/oliyh/martian/0.1.17-SNAPSHOT/maven-metadata.xml from datomic-cloud

what's weird, is that the same artifact is reported multiple times. is that some retry logic?

Alex Miller (Clojure team)15:07:52

this may just be an artifact of the snapshot update logic. if it's checking every repo every time, it's never going to find that metadata file so it has to recheck again later

👍 2

Alex Miller (Clojure team)15:07:04

maven has the ability to mark repos as whether they even have snapshots and that would help here (we don't expose that option)

onetom16:07:28

I just read through https://maven.apache.org/settings.html#repositories That <repository> <snapshots> <enabled>false or <updatePolicy>daily</updatePolicy> would be a nice improvement to support. Now that a bare Clojure terminal REPL starts up in less than a second, getting rid of these other sources of "slow Clojure startup" would be a great quality of life improvement, especially for early stage projects, where REPL startups are more frequent. I'm even looking into Emacs and inf-clojure again, beacause NOT using nREPL would shave another 1.7s from the startup time...

onetom16:07:14

It occured to me that I should have looked into where this might have been reported, and indeed you have already said: > We don't currently expose the update policy settings of repositories but that's probably reasonable to do. here: https://clojure.atlassian.net/browse/TDEPS-97 , which is continued on this still open ticket, with patches, to support the maven repository options: https://clojure.atlassian.net/browse/TDEPS-101

Alex Miller (Clojure team)16:07:32

I don't know that I want what's in those patches exactly, which is why this hasn't been moved forward

Alex Miller (Clojure team)16:07:10

but it's helpful to me to know your experiences above

Drew Verlee21:07:03

I logged the the request sent to my websocket handler and it included a hashmap with the key "input" and "Context". "Input" in the log contains a json payload "{\"headers...". Which ideally would be serialized to edn, but regardless, i would expect calling (get request "Input") to return that string, instead, in the logs, im seeing it return null. I'm basically trying to do whats outlined here https://www.freecodecamp.org/news/real-time-applications-using-websockets-with-aws-api-gateway-and-lambda-a5bb493e9452/ And so i believe i should be able to translate :

event.requestContext.connectionId;

into (get-in event ["requestContext" connectionId"] Here is the handler:

(defn handler [request]
  (cast/event {:msg      "WebSocketRequest"
               ::request request
               ::input   (get-in request ["Input"] )
               ::keys    (keys request)
               ::type    (type request)})
  {:status 200 :body "foobar" })

And in the otuput we see the key "Input" but when i try to get the value, and log that, it comes back null.

{
    "TomattoBackendDatomicIonWebsocketKeys": [
        "Context",
        "Input"
    ],
    "TomattoBackendDatomicIonWebsocketRequest": {
        "Context": <removed>
        "Input": "{\"headers\": ...}"
    },
    "TomattoBackendDatomicIonWebsocketInput": null,

}

Joe Lane00:07:07

@U0DJ4T5U1 check https://docs.datomic.com/cloud/ions/ions-reference.html#entry-points Input and context are keywords. Of the request map. The value of :input is a json string so you’ll need to parse that.

Drew Verlee01:07:34

@U0CJ19XAM I agree, the logs are saying the value of Input is a json string. But that string should still be returned as the value of (get-in request ["Input"]) right?

Joe Lane01:07:12

Input is a keyword. See the docs I linked to. :input

Joe Lane01:07:10

(defn echo [{:keys [context input]}] input)

Drew Verlee01:07:01

ah, ok. I'll check the docs but i take your meaning.

Joe Lane01:07:37

The casted log messages can be a little misleading because they are required to be json. A consequence of this is that all keywords are converted to strings.

Drew Verlee02:07:59

It's reasonable, I'm not sure why I didn't think of that.

2021-07-11

Channels