Fork me on GitHub
#datomic
<
2018-11-21
>
mavbozo07:11:28

I tried using clojure 1.10.0-beta7 and datomic 0.9.5703 but during the uberjar runtime, I got this exception: java.lang.IllegalStateException: Attempting to call unbound fn: #'datomic.common/requiring-resolve

mavbozo07:11:50

I upgraded my uberjar from clojure 1.10.0-beta5 to clojure 1.10.0-beta7. looks like requiring-resolve was added to clojure.core in clojure 1.10.0-beta6

hawkey10:11:40

hi, is there any Datomic Console alternative?

Dustin Getz12:11:27

Hyperfiddle is coming but we aren't launched for self-host yet http://www.hyperfiddle.net/

👍 4
Dustin Getz12:11:27

Hyperfiddle is coming but we aren't launched for self-host yet http://www.hyperfiddle.net/

👍 4
dpsutton15:11:40

would a feature request like this have interest from the cognitect side? we've had some queries return way more than expected because typos caused clauses to not unify. Would there be interest for some type of "strict" mode that would throw an error if an identifier other than _ or a logical variable that starts with "_" (like ?_customer) did not unify?

dpsutton15:11:23

this current one was from ?schedule and ?schedlue not unifying

jchen18:11:28

Is there a way to shrink the size of blobs written to storage? Our transactors are trying to write 27MB at a time to MySQL (5.6.34), which throws the MySQL exception The size of BLOB/TEXT data inserted in one transaction is greater than 10% of redo log size. Increase the redo log size using innodb_log_file_size. We can't change that parameter without incurring downtime -- I'm hoping there's a knob we can turn in datomic to split that 27mb blob up. As it is, that exception means our transactors cycle every 2-3 minutes because they can't finish indexing

ambroise19:11:55

related to above, can I query in datalog for all transactions that have a size greater than N?

souenzzo19:11:52

via log

(filter (fn [{:keys [data]}]
          (> (count data) 3))
        (d/tx-range (d/log @config/conn) nil nil))

ambroise19:11:47

thanks will try!

marshall19:11:36

@jchen you should not have segments that large. are you making very large transactions?

marshall19:11:04

Datomic should not be used to store BLOBs or other large unstructured data

marshall19:11:06

Also, you should consider this a critical issue @jchen if your system can’t complete an indexing job it will eventually reach a point where it is no longer available for writes

marshall19:11:35

You need to get the system past this indexing job and resolve the underlying issue that is causing the large segments

marshall19:11:48

the most common causes for large segments are: storing large blob values, making very large transactions (many datoms), or storing large strings that are frequently updated in a way that the leading bits are unchanged (i.e. we’ve had users storing serialized HTML or CSS in a string and updating it frequently with updates that only alter content somewhere deep into the string)

Dustin Getz22:11:20

@marshall, how large strings are we talking (since this is Onprem and no 4k limit)

marshall22:11:51

i’d avoid anything that smells like a serialized value

marshall22:11:29

don’t have a specific size limit, but if it isnt a “fact” it doesnt belong in a datom

jchen19:11:39

Thanks for the reply @marshall. The index job eventually succeeded: we noticed that the :kv-cluster/create-val bufsize peaked at around 27.8mb while throwing exceptions, then slowly dropped to 26.8mb and succeeded. This is the second time in two days we've had this issue -- the first time it also seemed to resolve itself without any intervention.

marshall20:11:35

@jchen i would definitely look for any instances of the types of usage patterns i mentioned above Do you have a paid Datomic On-Prem license or is this with Starter?

jchen20:11:42

We've got a paid license. I think in general we don't have many large transactions (the largest tx we found in the last 2 days was 300kb), and we have some 50kb blob attributes

marshall20:11:13

@jchen i would suggest you file a support ticket Are the blob attributes frequently updated?

jchen20:11:12

how do I file a ticket? I don't see a way to do it on the datomic website