Fork me on GitHub
#datomic
<
2022-06-17
>
favila14:06:08

The latest datomic on-prem supports JDK17 (I think via an artemis upgrade). However we can’t get the analytics (presto server) product to run on 17. Is that known and expected or are we doing something wrong?

tcrawley14:06:53

it fails with:

2022-06-17T14:09:55.105Z	INFO	main	com.google.inject.Guice	An exception was caught and reported. Message: java.lang.NoClassDefFo
undError: Could not initialize class com.google.inject.internal.cglib.core.$MethodWrapper
java.lang.IllegalStateException: Unable to load cache item
	at com.google.inject.internal.cglib.core.internal.$LoadingCache.createEntry(LoadingCache.java:79)
	at com.google.inject.internal.cglib.core.internal.$LoadingCache.get(LoadingCache.java:34)
	at com.google.inject.internal.cglib.core.$AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:119)
	at com.google.inject.internal.cglib.core.$AbstractClassGenerator.create(AbstractClassGenerator.java:294)
	at com.google.inject.internal.cglib.reflect.$FastClass$Generator.create(FastClass.java:65)
	at com.google.inject.internal.BytecodeGen.newFastClassForMember(BytecodeGen.java:258)
	at com.google.inject.internal.BytecodeGen.newFastClassForMember(BytecodeGen.java:207)
	at com.google.inject.internal.ProviderMethod.create(ProviderMethod.java:69)
	at com.google.inject.internal.ProviderMethodsModule.createProviderMethod(ProviderMethodsModule.java:327)
	at com.google.inject.internal.ProviderMethodsModule.getProviderMethods(ProviderMethodsModule.java:135)
	at com.google.inject.internal.ProviderMethodsModule.configure(ProviderMethodsModule.java:105)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:347)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:356)
	at com.google.inject.spi.Elements.getElements(Elements.java:104)
	at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:137)
	at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:105)
	at com.google.inject.Guice.createInjector(Guice.java:87)
	at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:276)
	at io.prestosql.server.Server.doStart(Server.java:111)
	at io.prestosql.server.Server.lambda$start$0(Server.java:73)
	at io.prestosql.$gen.Presto_348____20220617_140950_1.run(Unknown Source)
	at io.prestosql.server.Server.start(Server.java:73)
	at io.prestosql.server.PrestoServer.main(PrestoServer.java:38)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.google.inject.internal.cglib.core.$MethodWrapper
	at com.google.inject.internal.cglib.core.$DuplicatesPredicate.evaluate(DuplicatesPredicate.java:104)
	at com.google.inject.internal.cglib.core.$CollectionUtils.filter(CollectionUtils.java:52)
	at com.google.inject.internal.cglib.reflect.$FastClassEmitter.<init>(FastClassEmitter.java:69)
	at com.google.inject.internal.cglib.reflect.$FastClass$Generator.generateClass(FastClass.java:77)
	at com.google.inject.internal.cglib.core.$DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
	at com.google.inject.internal.cglib.core.$AbstractClassGenerator.generate(AbstractClassGenerator.java:332)
	at com.google.inject.internal.cglib.core.$AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:96)
	at com.google.inject.internal.cglib.core.$AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:94)
	at com.google.inject.internal.cglib.core.internal.$LoadingCache$2.call(LoadingCache.java:54)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at com.google.inject.internal.cglib.core.internal.$LoadingCache.createEntry(LoadingCache.java:61)
	at com.google.inject.internal.cglib.core.internal.$LoadingCache.get(LoadingCache.java:34)
	at com.google.inject.internal.cglib.core.$AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:119)
	at com.google.inject.internal.cglib.core.$AbstractClassGenerator.create(AbstractClassGenerator.java:294)
	at com.google.inject.internal.cglib.reflect.$FastClass$Generator.create(FastClass.java:65)
	at com.google.inject.internal.BytecodeGen.newFastClassForMember(BytecodeGen.java:258)
	at com.google.inject.internal.BytecodeGen.newFastClassForMember(BytecodeGen.java:207)
	at com.google.inject.internal.ProviderMethod.create(ProviderMethod.java:69)
	at com.google.inject.internal.ProviderMethodsModule.createProviderMethod(ProviderMethodsModule.java:327)
	at com.google.inject.internal.ProviderMethodsModule.getProviderMethods(ProviderMethodsModule.java:135)
	at com.google.inject.internal.ProviderMethodsModule.configure(ProviderMethodsModule.java:105)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:347)
	at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:356)
	at com.google.inject.spi.Elements.getElements(Elements.java:104)
	at com.google.inject.spi.Elements.getElements(Elements.java:97)
	at io.airlift.configuration.ConfigurationFactory.registerConfigurationClasses(ConfigurationFactory.java:164)
	at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:223)
	... 5 more

favila14:06:20

follow up q: is datomic analytics using presto or trino now?

Joe Lane14:06:08

The Datomic Analytics connector is intended for use with Presto 348. It's the version packaged w/ the distribution.

favila14:06:07

why the mention of trino then?

Joe Lane14:06:49

Because you'll never be able to find the "Presto 348" docs. They modified the branding and SEO in place in the transition from "Presto" to "Trino".

tcrawley14:06:18

so Presto 348 is really Trino?

Joe Lane14:06:07

https://trino.io/blog/2020/12/27/announcing-trino.html • In the transition they introduced many breaking changes • https://trino.io/docs/current/release/release-348.html is now found on the Trino webpage • It appears they've taken down the docs as-of Presto 348. I thought they used to version their docs but it appears that isn't true anymore.

favila14:06:44

So this code is https://github.com/trinodb/trino/tree/348 and it’s released literally 12 days before the rebrand.

Joe Lane15:06:24

Off the top of my head I believe that is true.

favila15:06:33

Our confusion stems from the documentation/readmes which links to https://prestodb.io/ and calls it “presto”, but that’s just because distinction didn’t exist at analytics release and the docs weren’t updated?

favila15:06:58

which was pre-fork and pre-rebrand.

Joe Lane15:06:53

Whos documentation? Ours?

Joe Lane15:06:21

facepalm Please Hold.

favila15:06:37

$ cat README.txt 
Presto is a distributed SQL query engine.

Please see the website for installation instructions:

favila15:06:06

https://docs.datomic.com/on-prem/analytics/analytics-concepts.html does link to http://trino.io now though. But it calls it “presto SQL” (which granted is the old trino name)

favila15:06:53

Either way, I think our original Q is solved: doesn’t support Java 17 because underlying lib doesn’t

Joe Lane15:06:55

That link in the readme i believe used to link to the right place, however, it now redirects to https://prestodb.io/ which is incorrect.

favila15:06:48

situation is confusing because confusing

Joe Lane15:06:56

Thanks for understanding.

Joe Lane15:06:20

@U09R86PA4 RE: cat README.txt, can pwd for me? Trying to figure out who's readme that is.

favila15:06:13

presto-server

tcrawley15:06:26

Inside the datomic-pro distribution

Joe Lane15:06:44

I see what happened. We zipped up 348 when it was new and put it in S3. This is a violation of "cool url's don't change" caused by the rebrand.

plexus15:06:14

To be precise, Datomic bundles io.prestosql/presto-server 348. This product no longer exists, it was forked and superceded by com.facebook.presto/presto-server and io.trino/trino-server. I asked a while ago about upgrade plans but didn't get any response, so it's also not clear yet which side of the fork they will continue with.

plexus15:06:49

They did strip out non-datomic plugins from Presto, if you want those you can find the original distribution here https://repo.maven.apache.org/maven2/io/prestosql/presto-server/348/

Joe Lane15:06:16

We will continue with Trino

👍 2
jdkealy15:06:15

what’s the best way to do bulk updates in memory constricted environments ?

jdkealy15:06:21

i presume the worst way is to do a (->> query (map #(transact [{:db/id %, other:attr “xyz”}])

jdkealy15:06:38

i’ve got a pod with 2 gigs of ram, it seems to perform ok except in the case of bulk updates

jdkealy15:06:07

i guess async writes is the way to go… if you deref the results, that’s where you get into trouble ?

souenzzo00:06:37

(let [vs [...]
      tx-data (async/chan)
      void (async/chan (async/sliding-buffer 1))
      n 5]
  (async/pipeline-blocking n
    void 
    (map (fn [tx]
           @(d/transact conn tx-data)))
    tx-data)
    
  (doseq [i vs]
    (async/>!! tx-data i)))

souenzzo00:06:46

you can play with the value of n

jdkealy00:06:43

don’t you want to not deref the transact ?

jdkealy00:06:09

or does that help create back pressure ?

souenzzo02:06:16

Not sure. But in my head, is important to deref, to make sure that you will have only 5 transacts running concurrently

Drew Verlee10:06:51

Why use core async here? I'm probably missing something because i don't understand blocking pipeline.

souenzzo17:06:38

it gives you more control over your threads. the total number of, the coordination, etc.