This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-03-18
Channels
- # announcements (4)
- # babashka (2)
- # beginners (72)
- # calva (2)
- # cider (18)
- # clj-kondo (30)
- # cljs-dev (2)
- # clojure (106)
- # clojure-austin (2)
- # clojure-europe (17)
- # clojure-italy (6)
- # clojure-nl (4)
- # clojure-uk (109)
- # clojurescript (31)
- # cursive (6)
- # data-science (2)
- # datomic (30)
- # events (1)
- # fulcro (20)
- # graphql (4)
- # jobs (2)
- # joker (8)
- # kaocha (2)
- # meander (31)
- # off-topic (1)
- # pathom (53)
- # re-frame (22)
- # reitit (1)
- # shadow-cljs (26)
- # specter (2)
- # sql (20)
- # testing (2)
- # tools-deps (2)
- # tree-sitter (1)
- # xtdb (20)
- # yada (6)
Hello I am trying to get started with crux using rocks-db. Does someone has a sample project that uses these two things
Hello @UGMEQUCTV 🙂 there are a few examples of projects using Crux+Rocks in the wild at the end of this post: https://juxt.pro/blog/posts/crux-dev-diary-january-2020.html
Are you looking to use Rocks for the event log as well as the KV store (a.k.a. "standalone"? Or would you be able to use JDBC / Kafka for event log storage?
Hi I have a starter project. I am trying to start a crux node, but I get an error
(ns crux-example.sample
(:require [crux.api :as crux]
[ :as io]
[crux-example.helpers :refer [easy-ingest]]))
;;
(def node
(crux/start-node {:crux.node/topology '[crux.standalone/topology
crux.kv.rocksdb/kv-store]
:crux.kv/db-dir "data/db"}))
This is the error message I get
Syntax error (ExceptionInfo) compiling at (form-init455416676044078120.clj:8:3).
Spec assertion failed
[crux.standalone/topology crux.kv.rocksdb/kv-store] - failed: (fn [id] (and (or (string? id) (keyword? id) (symbol? id)) (namespace (symbol id))))
@U899JBRPF thank you for your help 🙂
Never mind, I manage to solve the issue 🙂
Please disregard the above post
Does crux have a limit on string length like Datomic does (4096 characters)? Would storing, say, the contents of a bunch of blog articles in Crux be ok?
Hey, good question. I don't believe there are any such limits made explicit in the Crux code, so it's a case of looking through the Kafka / RocksDB / LMDB docs and config. For instance, Kafka by default is optimised for messages <1MB: > Out of the box, the Kafka brokers can handle messages up to 1MB (in practice, a little bit less than 1MB) with the default configuration settings, though Kafka is optimized for small messages of about 1K in size. (https://ibmstreams.github.io/streamsx.kafka/docs/user/ConsumingBigMessages/)
Rocks is apparently happy with 3GB though 🙂 > In general, RocksDB is not designed for large keys. The maximum recommended sizes for key and value are 8MB and 3GB respectively
The way Crux works means that large things get hashed automatically and so all the joins and lookups happen fast regardless of the size of the data stored
Strings longer than 128 bytes have a slightly different code path, but I don't think you would notice any performance impact: https://github.com/juxt/crux/blob/master/crux-core/src/crux/codec.clj#L79 https://github.com/juxt/crux/blob/master/crux-core/src/crux/codec.clj#L204
That was a significant pain point for me with Datomic in one of my projects
Regarding the hashing what is Crux using? SHA-3 or something?
@U054UD60U right now Crux uses SHA-1 (160 bits), so the same as git, but we are open to making this pluggable at some point. It can probably be done without too much change although it's not a priority for us currently https://github.com/juxt/crux/blob/master/crux-core/src/crux/hash.clj