It doesn’t have to read from RAM. It just have to reflect the value wrby the other thread. The cache coherence protocol might ensure that without forcing cpu caches to be written to the main memory

I think a slightly edited version of what you wrote above: > with volatiles all threads will see writes (non volatile don’t without some other synchronization that creates a happens-before constraint) > with volatiles, reads and writes can’t be atomic though - you can’t read and then write assuming that the value hasn’t changed Would be a pretty good summary.

I read both JCIP and the JMM a long time ago, and just remembering basically that has served me for pretty much all uses of volatile since then - I’ve forgotten all the details about the mechanics.

From the description, could be either GC pause or that the thread blocked on the query is holding a lock that others are waiting on. For the former, I’d turn on the jvm gc logging - its designed to be run under load and you can shunt them off to their own log. If the latter or something else, getting a thread dump (kill -3 on the pid or use jstack) will tell you what threads are doing when it locks up and what monitors are held

Ok, thanks much. I hadn’t thought much about the possibility of a shared lock, so this plus the GC logging gives us something to pursue. This issue has happened 3 times that I know of since July, so data gathering is slow, but now we know where to spread the nets.

http-kit is built on core.async, so if you're locking up the core.async thread pool (e.g. doing I/O inside of a go), that could cause problems too.

I'm not sure how http-kit feels about I/O inside of handler functions.

core.async now has a system property you can set to catch use of blocking async ops in go blocks. Might be worth turning that on in dev (not in prod!) to see if that’s happening

See doc at top of http://clojure.github.io/core.async/

We’re not using core.async in the app itself, not sure if that matters.

Thanks for the link, I’ll definitely check that out

Easy thing to check out at least

Oo, yes, this is going to come in very handy

@U09LZR36F What makes you think http-kit is based on core.async?

The haziness of memory 😁

https://github.com/http-kit/http-kit/blob/059deac93b1662077d52e1eb81bef9db8c89d746/project.clj#L12

That's got to be a lie, right 😁

No, http-kit is a light weight web server in Clojure + some Java without any deps

admittedly that sounded weird to me too :)

@U06CM8C3V are you using the async part of httpkit?

http://http-kit.github.io/http-kit/org.httpkit.server.html#var-as-channel Hmm, I'd always thought the channels in http-kit were core.async channels. I guess I'm just crazy :)

@U06CM8C3V Maybe this issue will have some relevant info. Not sure: https://github.com/http-kit/http-kit/issues/345

Is the request or response body very large as in, would it be problematic to hold it in memory at once? This can also be a problem with http-kit

We’re not doing anything fancy with http-kit, just

(http/start
     (-> env
         (assoc :handler #'handler/app)
         (assoc :max-body max-body)
         (assoc :threads server-threads)
         (update :io-threads #(or % (* 2 (.availableProcessors (Runtime/getRuntime)))))
         (update :port #(or port %))))

Hmm, that’s actually calling start from luminus.http-server

which is basically calling http-kit/run-server, wrapped inside a try/catch

Yeah, looks like the response could be pretty large sometimes. :thinking_face:

http-kit doesn't have an optimal scenario for this. I have made notes about that here: https://github.com/borkdude/babashka/wiki/HTTP-client-and-server-considerations#http-kit-holds-entire-requestresponses-in-memory I might switch to jetty in the future for babashka because of this. Although for babashka scripts it may not be a dealbreaker, I don't feel comfortable about it.

Also see: https://github.com/http-kit/http-kit/issues/90 Again, not sure if this is your issue, but it could be.

This is really helpful, thanks much

Wait, I take back what I said about a large response — the code I’m looking at pulls potentially a lot of data out of Mongo, but then it batches it out to offline processing (via future) for a later download. The http response itself is basically empty.

ok well, that's not it then.

what version of http-kit are you using?

2.5.0

ok. not sure what the issue is and if it's related to http-kit then, sorry

np, I appreciate the input and I learned some stuff, so all good

What mongo client do you use?

It's probably worth getting metrics in place for JVM memory stats (heap, memory use, etc)

we’re at monger 3.5.0

sounds kind of like https://github.com/michaelklishin/monger/issues/166

That doesn’t seem to be our issue, it’s happening for us when we’re querying for data.

@U06CM8C3V You won't get a lot of insight with New Relic on http-kit I'm afraid. We were using http-kit in production and we ended up switching (back) to Jetty because that has official support in New Relic and you get a lot more information.

We worked with New Relic quite a bit to try to get http-kit supported but in the end they just considered it too niche to expend any effort. We tried configuring transaction recognition and wrote some middleware to help with transaction boundary recognition, but in the end we just gave up.

All our apps are on Jetty now, except one built on Netty and we don't get a lot of the core web transaction metrics from New Relic on that either -- but we have a lot of custom metrics that we added ourselves, via New Relic's (now obsolete) metrics plugin library.

@U04V70XH6 out of curiosity, do you use ring-jetty or pedestal or ...?

ring-jetty -- we try to keep things as "stock" as possible so we get the best out of New Relic.

@U04V70XH6 Wow, that's good to know, thanks for sharing that. I'll look into switching our app back to ring-jetty

+1 newrelic and ring jetty also worked well for us, never saw a reason to pursue alternative http servers. even if you're using websockets there's good support for that in the sunng87 jetty adapter

We went with Netty for our websocket app because our frontend is JS and uses http://Socket.IO so we needed to support that on the backend and it's "easy" with netty-socketio (and we use that with Netty directly via Java interop).

I've been through that course but I don't recall the details of that example. When I get time later today I'll pull that lesson back up and see if I can provide guidance based on that.

@U04V70XH6 sounds great and looking forward to it. I'm really digging property-based testing so I'm hoping to learn as much as I can.

The three courses are really good. Some of the Advanced PBT stuff is pretty mind-blowing.

I have Advanced PBT queued up next.

OK, just re-watched that... The "recursive" part is there already: the generators for lines (and for words) are built on top of lower-level generators (not strictly recursive except in the sense that the generators call other generators). The main issue he leaves open to the reader is the question of whether the generated output is what you would expect from processing the input: and the empty lines (or words) case is quite a tricky one to think about. Consider this behavior of clojure.string/split-lines:

user=> (str/split-lines "\n\n")
[]
user=> (str/split-lines "\n \n")
["" " "]
user=> (str/split-lines "\n \n\n")
["" " "]
user=> (str/split-lines "\n \n\n ")
["" " " "" " "]

(a less charitable way to couch that is that he picked a really hard problem, ran into difficulty trying to specify the behavior, and punted on it 🙂 )

But the good part about this -- and he says this in the lesson -- is that trying to generatively test the function immediately uncovered several bizarre corner cases that are very hard to reason about! @US0NPHU9E

I think the part I'm getting caught up on is determining what I actually want the lines function to do. Presumably I don't want it to be lossy since I lose the inverse property and this would also make it equivalent to the existing str/split-lines function. If lines is not lossy, then I suppose I'm checking before \n to determine if I should add an empty string?

hmm, okay. I think I'll try that out and see where I get. Thanks for running through that, @U04V70XH6. Hopefully I'll have a satisfactory answer soon.

Yup, and that is exactly the hard part: should lines (or words) be lossy? Since words will throw away whitespace between items, it is definitely lossy: "foo bar" and "foo bar" must both yield ["foo" "bar"]

And then the question is how you convert from the generated output to an appropriate input, since one output should really generate a random set of inputs that should all map back to that one output.

And at that point, you'll be generating inputs from a seed output values (and now you're layering another set of generators on top of the two-or-more you already have).

i got lines solved. at least to a satisfactory answer 😄

i still need to do words but the process is making a little more sense and the flow of using the shrunken test input of a failing test to develop the code is coming together

i think i'm running into some memory issues with lein test-refresh or maybe with using quick-check in the repl but that's a different issue.

ah, i think it's because of recursion and i'm not spending any time on optimizing. because of the way clojure.test.check does its expansion and shrinkage, it probably isn't too friendly with recursive functions

spec 2 is still under active design. I would recommend using spec 1 for now.

thanks. It might be too soon to ask, but are there plans for some kind of compatibility shim or tool to ease the spec1 -> spec2 transition?

too early, but we will presumably have a guide, a tool, compatibility layer, something

that's what I figured. Looking forward to getting my hands on spec 2 whenever it's ready!

Ahah that's where the question originated

This does not look cool, but it does the job.

This is not perfect, but anyway:

(for [x (butlast report)
      y (rest report)
      :when (= 2020 (+ x y))]
  (* x y))

(defmacro info [x]
  (let [f  *file*
        md (meta &form)]
    (assoc md :file f)))

(info "hi")
=> {:line 10, :column 8, :file "/path/to/x.clj"}

Ooooh thanks! In a cross-lang discord someone mentioned how nice the dbg! macro\fn is in rust. I figured if we can get the file\line meta then the rest is a really easy macro 😄

Rust will always have a file source location. This is not necessarily true when you’ve got a reader

I think I understand. This may not supply expected info when eval’d from a REPL right?

That, if you eval it from the source buffer, or just call read-string or read ever. Basically only things that use the line numbering pushback reader when clojure requires and compiles things will have this information

I see, well sometimes is better than nothing.

yeah totally

Found it. Gotta reference cljs.analyzer/*cljs-file*