Fork me on GitHub

Greetings everyone. I am experimenting with Crux and Kafka, trying to get a feel for what a production system would sound like. Could anyone kindly tell me where to find juxt/kafka-connect-crux:20.04-1.8.2-alpha (as instructed this specific version (20.04-1.8.2-alpha). The latest available version on Confluent Hub is 19.12-1.6.1-alpha .


Hi @U5URYMRV2 we've not released the 1.8.2 connector to Confluent Hub (it's a quite a manual process) but I can create a build or you can build it yourself from the repo


Cheers @U899JBRPF. I’ll try and build it from the repo and install it manually on my confluent local


Cool, happy to help, and if you're still lost in a couple of hours I can create a release for you 🙂


I've made a note to publish to Confluent Hub more regularly!


thanks Jeremy

🙏 1

Ok 😬— so lein package keep trying to pull 20.04-1.8.3-alpha-SNAPSHOT artifacts from clojars. I am on the 20.04-1.8.2-alpha tag (but it’s the same with master .


right, so you can either roll back the repo commits to the 1.8.2 release commit and try again, or you can lein sub install from the root and work with the SNAPSHOT versions in your app. I'd recommend the first approach


hi 🙂 I'm trying to setup pointing at my local crux node. It looks like I have to first start an http-server for my crux node, so I've added the dep:

juxt/crux-http-server               {:mvn/version "20.04-1.8.2-alpha"}
and have this config:
(defn rocks-config [data-dir]
  {:crux.node/topology                    '[crux.standalone/topology
   :crux.http-server 8099
   :crux.kv/db-dir                        (str (io/file data-dir "db"))
   :crux.standalone/event-log-dir         (str (io/file data-dir "eventlog"))
   :crux.standalone/event-log-kv-store    'crux.kv.rocksdb/kv
   :crux.standalone/event-log-sync?       true
   :crux.kv/sync?                         true})
but when I start the node I get:
Execution error at crux.topology/resolve-id (topology.clj:15).
No such var: api/open-q
anyone have an idea of what's wrong here?


I don't see what would obviously be wrong with open-q not being found. But still thinking about it. This though should be more obvious if you can share your curl command


this is happening when starting the node


(ns space.matterandvoid.server.crux-node
    [crux.api :as crux]
    [ :as io]
    [mount.core :refer [defstate]])
  (:import [crux.api ICruxAPI]))

(defn rocks-config [data-dir]
  {:crux.node/topology                    '[crux.standalone/topology
   :crux.http-server 8099
   :crux.kv/db-dir                        (str (io/file data-dir "db"))
   :crux.standalone/event-log-dir         (str (io/file data-dir "eventlog"))
   :crux.standalone/event-log-kv-store    'crux.kv.rocksdb/kv
   :crux.standalone/event-log-sync?       true
   :crux.kv/sync?                         true})

(defn start-crux-node ^ICruxAPI [storage-dir]
  (crux/start-node (rocks-config storage-dir)))

(comment (start-crux-node "crux-store"))

 ; repl output =>
(start-crux-node "crux-store")
Syntax error compiling at (crux/http_server.clj:189:16).
No such var: api/open-q


When I start the repl and inspect the crux.api ns - i see that there's no open-q var in there


these are my deps:

juxt/crux-core                      {:mvn/version "20.04-1.8.1-alpha"}
juxt/crux-rocksdb                   {:mvn/version "20.04-1.8.1-alpha"}
juxt/crux-metrics                   {:mvn/version "20.04-1.8.2-alpha"}
juxt/crux-http-server               {:mvn/version "20.04-1.8.2-alpha"}


ha, those 1.8.1 should be 1.8.2 woooops 🙂

🙂 1

so the server is working now:

{:crux.index/index-version 6, :crux.doc-log/consumer-state nil, :crux.tx-log/consumer-state nil, :crux.kv/kv-store "crux.kv.rocksdb.RocksKv", :crux.kv/estimate-num-keys 39, :crux.kv/size 623497}b
but the connection from the console is not working:


ah, it's a cors error, I think the docs mention that...


Yeah, you need to enable cors in your http server config


Although we may have now prevented that from being possible :thinking_face:


The reason being that the crux-console project is essentially mothballed atm, as we're working on an entirely new UI which is really a content-negotiated PWA that lives entirely on replacement http routes. Not long to go until we're ready to show that off 😉


just saw the handler is private, i tried a hack to use the var, but that didn't work. It's not essential at the moment, just wanted to try it out. Very cool! I'll just wait a bit then 🙂


sorry about that, I should have foreseen the issue! Building your own fork of crux-http-server is another option


what kind of use-cases for a console do you have in mind? e.g. something to help understand an existing data set? debugging?


i'm going through the churn phase of the data model for my app so i want to make ad-hoc queries against the data - deleting some data that got in a bad state, updating a doc to put it into the state i want - i'm doing it via some function helpers through the repl at the moment. I thought it may be faster to have a UI to dig through the data instead

👍 1

and no problem! - i'm going into all of this with the expectation that there will be some hurdles to jump over


cool, glad to hear you've got some kind of repl flow figured out already. The crux-console SPA is pretty nice for certain things, but it's not as handy as a repl 🙂