Fork me on GitHub
#xtdb
<
2023-04-17
>
Chris Lester00:04:23

Hi folks. I'm trying to start off with the quickstart and I'm getting a missing dependency from xtdb:

; Evaluating file: xtdb-client.clj
; Syntax error (FileNotFoundException) compiling at (xtdb/io.clj:1:1).
; Could not locate cognitect/anomalies__init.class, cognitect/anomalies.clj or cognitect/anomalies.cljc on classpath.
; Evaluation of file xtdb-client.clj failed: class clojure.lang.Compiler$CompilerException
I've just copied in the quickstart ns code for now into a namespace and trying to load it in the repl. Adding that dependency in deps.edn (to see what would happen) results in change to
; Evaluating file: xtdb-client.clj
; Syntax error reading source at (xtdb/io.clj:406:61).
; Invalid token: ::anomalies/category
; Evaluation of file xtdb-client.clj failed: class clojure.lang.Compiler$CompilerException
cl
Is there a configuration I'm missing? Polylith component deps:
{:paths ["src" "resources"]
 :deps {org.clojure/core.memoize {:mvn/version "1.0.257"}
        com.xtdb/xtdb-core {:mvn/version "1.23.1"}
        com.xtdb/xtdb-rocksdb {:mvn/version "1.23.1"}}
 :aliases {:test {:extra-paths ["test"]
                  :extra-deps {}}}}

seancorfield00:04:00

@US893PCLF Can you link me to the docs you're following and explain what xtdb-client.clj is?

seancorfield00:04:04

Since you're using Polylith, I'm wondering what your :dev alias looks like -- in particular, are you using :extra-paths for Cursive or :extra-deps for every other editor/IDE?

seancorfield00:04:40

For example, if I start a REPL with just com.xtdb/xtdb-core, I'm able to require just fine:

(~/clojure)-(!2003)-> clj -Sdeps '{:deps {com.xtdb/xtdb-core {:mvn/version "1.23.1"}}}'
Clojure 1.12.0-alpha2
user=> (require ')
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See  for further details.
nil
user=>
So I think this might be something in your env/project setup...?

seancorfield00:04:55

I'd also check your version of Clojure itself (`org.clojure/clojure` I mean): relies on :as-alias so it requires Clojure 1.11 (or 1.12):

[cognitect.anomalies :as-alias anomalies]

seancorfield00:04:31

If you're using Clojure 1.10 or earlier, that will silently fail when it gets to ::anomalies/categoy

seancorfield00:04:26

Which doesn't mention Polylith 🙂

Chris Lester00:04:26

... will check the clojure version as well. I'm guessing it is in my setup, just don't know where yet.

Chris Lester00:04:45

No, I'm adding it in as a component :)

seancorfield00:04:53

You may have Clojure 1.10 or earlier in your workspace-level deps.edn file.

Chris Lester00:04:26

Yep, 1.10.3 .. will update to later version.

seancorfield00:04:45

Also see my comment about your :dev alias and IDEs...

Chris Lester01:04:17

:dev {:extra-paths ["development/src"]
                  :extra-deps {org.clojure/clojure {:mvn/version "1.12.0"}
                               org.clojure/tools.deps {:mvn/version "0.17.1297"}

                  ;; pedestal deps
                               cli-matic/cli-matic {:mvn/version "0.5.4"}
                               com.taoensso/timbre {:mvn/version "6.0.2"}
                               expound/expound {:mvn/version "0.9.0"}
                               org.apache.commons/commons-lang3 {:mvn/version "3.12.0"}

                 ;; state management
                               aero/aero {:mvn/version "1.1.6"}
                               integrant/integrant {:mvn/version "0.8.0"}
                               integrant/repl {:mvn/version "0.3.2"}

                 ;; http responses
                               metosin/ring-http-response {:mvn/version "0.9.3"}

                  ;; json parsing
                               cheshire/cheshire {:mvn/version "5.11.0"}

                  ;; web services
                               io.pedestal/pedestal.immutant {:mvn/version "0.5.11-beta-1"}
                               io.pedestal/pedestal.route {:mvn/version "0.5.11-beta-1"}
                               io.pedestal/pedestal.service {:mvn/version "0.5.11-beta-1"}

                  ;; security
                               yogsototh/clj-jwt {:mvn/version "0.3.0"}

                  ;; spec tools
                               metosin/spec-tools {:mvn/version "0.10.5"}
                               commons-validator/commons-validator {:mvn/version "1.7"}
                               com.gfredericks/test.chuck {:mvn/version "0.2.14"}
                  ;; bases 
                               poly/ixi-cli {:local/root "bases/cli"}
                               poly/pedestal {:local/root "bases/pedestal"}

                  ;; components
                               poly/auth {:local/root "components/auth"}
                               poly/client-milvus {:local/root "components/client-milvus"}
                               poly/client-openai {:local/root "components/client-openai"}
                               poly/client-xtdb {:local/root "components/client-xtdb"}
                               poly/common-regex {:local/root "components/common-regex"}
                               poly/common-specs {:local/root "components/common-specs"}
                               poly/health {:local/root "components/health"}
                               poly/http-specs {:local/root "components/http-specs"}
                               poly/jwt-auth0 {:local/root "components/jwt-auth0"}
                               poly/notematic {:local/root "components/notematic"}
                               poly/responses-web {:local/root "components/responses-web"}
                               poly/storage {:local/root "components/storage"}}}

seancorfield01:04:39

OK, cool. Not Cursive 🙂

seancorfield01:04:28

{:mvn/version "1.12.0"} -- either 1.11.1 or 1.12.0-alpha2 -- there is no 1.12.0 GA yet

seancorfield01:04:23

(I'm using 1.12.0-alpha2 because I'm testing all the fun new stuff in it -- and we often run pre-release builds in production at work)

👍 2
Chris Lester01:04:41

Problem solved ... thx Sean!

2
seancorfield01:04:22

I'm not sure if XTDB docs make it clear anywhere that Clojure 1.11.1 is required...?

wotbrew09:04:35

Thanks for this, the :as-alias usages was introduced last patch. I've asked the team whether the next release should relax the requirement or if we should update the docs 🙏.

nivekuil17:04:32

how is UPDATE implemented in core2?

jarohen10:04:58

hey @U797MAJ8M 👋 loosely speaking, it's a query to find both the rows to update and the rest of each of the updated rows (i.e. the cols you haven't SET) and then follows a similar path to Datalog :puts

nivekuil01:04:53

is that query run on the thread which submits the tx? I'm wondering about the new consistency model since I'm not clear where the serialization happens now, e.g. two updates to the same document at the same time will result in one being silently overridden, since there isn't match support yet?

nivekuil01:04:19

or are the writes now deltas instead of whole documents?

refset10:04:33

> is that query run on the thread which submits the tx? nope, the query happens on the indexing thread (same principle as transaction functions or match operations) so the ~same caveats apply about not wanting to make those linearized UPDATEs too 'chunky' (else you'll stall the other writers) > are the writes now deltas instead of whole documents? also nope, underlying writes to object storage are operating in terms of whole documents ('rows') due to being schemaless, and there is (again) no explicit structural sharing across versions, beyond the potential for columnar compression of identical values

refset10:04:47

we haven't yet built anything approaching optimistic MVCC / row-level locking / Calvin at this point

nivekuil19:04:53

oh, so the node writes back to the object store after polling the tx log, as opposed to now where it happens in the same thread. are they still doing that with zero knowledge of other nodes or are they now part of one consumer group?

refset20:04:06

all still zero knowledge and relying on deterministic processing, and objects only ever get written once enough data has accumulated into a 'chunk' (defaulting to 100,000 rows, currently)