datomic 2019-07-06 | Slack Archive

yep

So this is a prod outage? And are you deploying ions?

I will ask some more questions on the case so the whole team can see.

not a prod outage fortunately, but one of my teams is wrapping a product increment on monday, and this is impacting that, and yes we’re using ions

stuarthalloway00:07:48

we'll get you sorted ASAP

steveb8n04:07:42

FWIW it’s great to see this kind of support response out in the open. builds confidence for me

☝️ 16

fdserr04:07:36

Has anyone got a workaround to enable the ping endpoint in a containerised Datomic without the thing blowing up? (0.9.5930, dev protocol)

docker run -v /config/:/config/ my-docker/transactor
Launching with Java options -server -Xms1g -Xmx1g -XX:+UseG1GC -XX:MaxGCPauseMillis=50
Critical failure, cannot continue: Error starting transactor
java.lang.RuntimeException: Unable to start ping endpoint localhost:9999
...
Caused by: java.lang.IllegalStateException: Insufficient configured threads: required=3 < max=3 for QueuedThreadPool[qtp1630087575]@61292997{STA
RTED,3<=3<=3,i=3,q=0}[ReservedThreadExecutor@c3b9daa{s=0/1,p=0}]

I know it is an upstream issue (Jetty). Poking Datomists for a possible hint to enable a transactor health check on K8s/Prom.... TIA!

favila05:07:45

https://forum.datomic.com/t/jetty-max-threads-error-when-enabling-ping-health/603/21

favila05:07:08

We worked around it by using one of the magic cpu limit numbers (k8s environment)

favila05:07:03

Unsure of root cause, if is jetty or java8

fdserr06:07:54

Golden, just works. Can't imagine the sweat you put in this. May I ask how you found out the existence of this hidden key? Thanks a bunch @U09R86PA4!

fdserr06:07:11

just saw the next-next post with it 😃

favila06:07:18

Actually what happened to us was we had it working fine (by accident it turns out) then we adjusted the limits later and it failed. We couldn’t believe it but we found the forum post as confirmation

favila06:07:32

I think we had 8 and lowered to 4 or something

fdserr07:07:41

> We couldn’t believe it but we found the forum post as confirmation 😃

fdserr04:07:33

BTW congrats Datomic team, the June release is packed with awesome features 🙏

🎉 8

pvillegas1221:07:46

In order for a transaction function to not be applied (in the atomicity sense), do you need to raise an exception?

favila00:07:16

Yes directly or indirectly

fdserr15:07:32

@U6Y72LQ4A Throwing is the way to stop a TX, AFAIK. Throwing clojure.lang.ExceptionInfo helps us deal with explicit business constraints (userland/maybe-recoverable) and we let the rest blow up ("system" error).

pvillegas1215:07:06

Perfect, thanks @U09R86PA4 @U05164QBS for confirming

2019-07-06

Channels