Fork me on GitHub
#xtdb
<
2020-08-24
>
jarohen08:08:03

I'm pretty sure it's not an official archive, but https://clojurians-log.clojureverse.org/ is a good record πŸ™‚

πŸ‘ 3
dominicm08:08:38

zulipchat also has an archive, which is searchable.

πŸ‘ 3
Las11:08:45

Hi, first of all, thanks for Crux! I read through https://opencrux.com/reference/transactions.html, but left wondering what happens if a submitted transaction fails. What are the failure modes? What is the idiomatic way to check from application code if a transaction succeeded?

πŸ™ 3
refset11:08:36

tx-committed? on the node (ICruxAPI) tells you whether the transaction succeeded. We don't provide a programmatic means of finding out exactly why a transaction failed, but there is logging and within a transaction function you can of course add your own custom logging

refset11:08:58

Failure modes for a match are strictly logical only, but a transaction function might also fail because: it can't compile, NPE etc.

Las12:08:44

Thanks, I was looking for something along the lines of https://docs.datomic.com/cloud/transactions/transaction-processing.html#anomalies in terms of development ergonomics, but I understand that this may not make sense in the context of Crux, i.e. optimistic async only tx processing

βœ”οΈ 3
refset12:08:58

Noted, we'll improve this area eventually. Thanks for the feedback πŸ™‚

πŸ‘ 3
Las13:08:33

To sum it up: I think a construct that allows submitting a transaction asynchronously and can be derefed to check if it succeeded would be a useful addition to improve developer ergoniomics.

πŸ‘Œ 6
zclj14:08:02

I have the following code:

(crux/q
 (crux/db crux-node)
 {:find   [(list 'eql/project '?e query)]
  :where '[[?e :user/e-mail ?e-mail]
	       [?e :user/id]]
  :args   [{:?e-mail e-mail}]})
When running this I intermittently get an exception:
{:type java.lang.IllegalArgumentException
 :message No implementation of method: :id->buffer of protocol: #'crux.codec/IdToBuffer found for class: clojure.lang.PersistentList
 :at [clojure.core$_cache_protocol_fn invokeStatic core_deftype.clj 583]} 
It happens more often from a lein test than from the REPL. Should I open an issue for this, or is it something on my end? Note that it only seem to happen where there are no user with the given e-mail.

jarohen15:08:15

Hi @U1G8B7ZD3 - I'll take a quick look to see if it's something obvious, but if not, it's worth an issue - thanks πŸ™‚

zclj15:08:35

Thanks. Let me know if an issue is needed, in that case I can provide a more complete stack trace

jarohen06:08:07

That'd be great, thanks πŸ™‚

jarohen08:08:55

created https://github.com/juxt/crux/issues/1057 so that I don't lose track of it πŸ™‚

tolitius18:08:17

is there a way to disable the indexer for an embedded crux/jdbc: i.e. do not replay the tx log on start / rely on kv index files? use case: one of the systems we have is write only it records events from various sources to a SQL database (postgres) [no crux involved] I’d like to add crux/put on each successful upserted batch to store history other systems are read and write that would work with both postgres data and crux history, so indexer there is needed but for the write only system: is it possible to avoid carrying a rocksdb index that this system does not need? the reason it is troublesome is twofold: 1. these are millions of transactions in short periods of time: disk space 2. the write system has several instances managed by nomad: carrying for dir structure, docker binds, etc. complicates ops and is not really needed for business

dominicm18:08:36

This was definitely under consideration at some point. You could probably make a nop kvstore protocol implementation.

refset20:08:39

Yep it was under consideration for a while...and then luckily we did something about it! It's called the "ingest client" API, for submitting transactions only: https://www.opencrux.com/reference/clojure-api.html#_new_ingest_client I think it should be perfect for your use case

refset21:08:24

ahh, of course, sorry πŸ™‚ So it's currently hard-coded for Kafka but we could certainly get it working with JDBC too, with a small amount of effort, as all the plumbing exists already. I can't make promises on timelines right this moment - are you blocked on your testing without having it? Feel free to message me direct if you want to share details on timelines or whatever

tolitius21:08:17

nah, I don’t want to create more urgent work for you. just knowing you are thinking about adding this for jdbc within reasonable timeframe is good enough. I can do some gymnastics with those indices for now (at least I think will be able to, will depend on their size)

βœ”οΈ 3
tolitius21:08:02

if I have time I’ll explore a noop https://github.com/juxt/crux/blob/master/crux-core/src/crux/kv.clj implementation as @U09LZR36F suggested. but I would of course prefer to have that working without a need of a plug )

πŸ‘ 3
jarohen06:08:16

agreed - indeed, have been removing the dependency on Kafka in the ingest client, currently queued up for the next major release

πŸ†’ 9
metal 3
refset11:08:25

^ awesome, I clearly failed to properly read the memo on that, thanks for chiming in @U050V1N74 πŸ™‚

πŸ™ 3