Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2023-01-18

Channels

#xtdb
<
2023-01-18
>
Jason03:01:31

I set up xtdb in memory (xtdb.api/start-node {}) in a dev setup, added a (xtdb.http-server/->xtdb-handler (clip/ref :xtdb-node) {}), registered it via 

(reitit.ring/ring-handler
   router
   (reitit.ring/routes
    (reitit.ring/create-resource-handler {:path "/js/" :root "/compiled/js/"})
    (when (-> env :debug) xtdb-handler)
    (ring/create-default-handler)))
and ran through the quickstart https://docs.xtdb.com/guides/quickstart/#_2_ingestand https://docs.xtdb.com/guides/quickstart/#_3_query. When I go to /_xtdb/status the page loads, when I go to attribute cardinalities I see the :xt/id and :user/name attributes, but when I try to query through the UI I get no results. Any idea what could be going on? Is xtdb-handler expected to work with an in memory node?

Jason03:01:19

whoops! https://docs.xtdb.com/guides/quickstart/#_1_setup and found this when I go to the page 

java.lang.IllegalArgumentException: No implementation of method: :write-body-to-stream of protocol: #'ring.core.protocols/StreamableResponseBody found for class: juxt.clojars_mirrors.muuntaja.v0v6v8.muuntaja.protocols.StreamableResponse
at clojure.core$_cache_protocol_fn.invokeStatic(core_deftype.clj:584)
at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:576)
at ring.core.protocols$eval51088$fn__51089$G__51079__51098.invoke(protocols.clj:8)
at ring.util.servlet$update_servlet_response.invokeStatic(servlet.clj:109)
at ring.util.servlet$update_servlet_response.invoke(servlet.clj:94)
at ring.util.servlet$update_servlet_response.invokeStatic(servlet.clj:98)
at ring.util.servlet$update_servlet_response.invoke(servlet.clj:94)
at ring.adapter.jetty$proxy_handler$fn__53078.invoke(jetty.clj:28)
at ring.adapter.jetty.proxy$org.eclipse.jetty.server.handler.AbstractHandler$ff19274a.handle(Unknown Source)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:833)

Jason04:01:44

https://clojurians.slack.com/archives/C0K65B20P/p1645728266176959?thread_ts=1645555586.499599&amp;cid=C0K65B20P @U01PAJR4N69, https://github.com/metosin/muuntaja/blob/83f7f707b69b3111668856804733c4fd17373159/modules/muuntaja/src/muuntaja/protocols.clj#L23, and https://github.com/ring-clojure/ring/blob/8af4ab93190dfe5b4827c14b416a4cb92e18cdaf/ring-core/src/ring/core/protocols.clj#L35 I was able to resolve it using 

(extend-protocol ring.core.protocols/StreamableResponseBody
  juxt.clojars_mirrors.muuntaja.v0v6v8.muuntaja.protocols.StreamableResponse
  (write-body-to-stream [body _ ^OutputStream output-stream]
    (with-open [out output-stream]
      ((.f body) ^OutputStream out))))

refset12:01:44

Ah, I'm glad you found the answer šŸ™‚ I'll make a note in the docs to help in future!

csmith20:01:35

Hi folks. Iā€™m trying to adopt xtdb in a project that is currently using datascript. Iā€™m struggling to make a suitable alternative for the set/subset? rule below because xtdb stores the card many data as separate tuples. The below results in calling clojure.set/subset? with #{1 2 4} for ?x and each of 1, 2, and 3 in turn. Of course this is incorrect and leads to an error. Iā€™m stumped on how to rewrite this rule without card many. 

(xt/submit-tx (get-conn) [[::xt/put {:xt/id 1 :set-attr #{1 2 3}}]])

  (xt/q (xt/db (get-conn))
        ā€™{:find [(pull ?e [*])]
          :where [[?e :set-attr ?val]
                  (subset? #{1 2 4} ?val)]
          :rules [[(subset? ?x ?y)
                   [(clojure.set/subset? ?x ?y)]]]})

refset20:01:58

Hey @U09RT5MKJ! So there are few things going on here, and I'm not so sure the cardinality-many default aspect is the main issue. I think the surest/quickest fix to your example would be: 

(xt/q (xt/db (get-conn))
        '{:find [(pull ?e [*])]
          :in [?s]
          :where [[?e :set-attr ?val]
                  (subset? ?s ?val)]
          :rules [[(subset? ?x ?y)
                   [(clojure.set/subset? ?x ?y)]]]}
       #{1 2 4})

refset20:01:55

Part of the confusion is that set literals in the query are interpreted as relations (I think this is unlike DataScript)

refset20:01:07

at least wherever a logic-var is valid

csmith20:01:20

I appreciate the help. I did not consider that possibility, but I did read that in the doc. did not click I guess

refset20:01:28

using :in will ensure it is passed as a scalar

csmith20:01:58

itā€™s funny because passing it as a parameter is of course more realistic. Me using a literal here was an early attempt at a POC that failed

refset20:01:30

it's definitely just a case of getting familiar with the rules (no pun intended) in play, and this is arguably surprising for anyone new to XT who has used DataScript etc. for a while

refset20:01:12

I think you could also use the set literal in the expression clause directly and it would work as you'd hoped 

[(clojure.set/subset? #{1 2 4} ?y)]
(I might be wrong though, it's getting late... šŸ˜…)

csmith20:01:14

Iā€™m getting the same result more or less

šŸ‘Œ 2
csmith20:01:29

(xt/q (xt/db (get-conn))
        '{:find [(pull ?e [*])]
          :in [?s]
          :where [[?e :set-attr ?val]
                  (subset? ?s ?val)]
          :rules [[(subset? ?x ?y)
                   [(prn ?x ?y)]]]}
        #{1 2 4})

csmith20:01:05

If I use prn to debug (of course the query is nil) but, I get: 

#{1 4 2} 1

csmith20:01:29

and 2 more entries with 2 and 3, so I think itā€™s the issue that card many data is ā€œflattenedā€ into separate tuples. Right?

refset20:01:05

ohhh, I see, I was mistaken about your intention, sorry(!)

csmith20:01:24

all good. Trying to compare set a to set b sort of thing (as opposed to contains? or similar)

refset20:01:43

in that case try: 

(xt/q (xt/db (get-conn))
        '{:find [(pull ?e [*])]
          :in [?s]
          :where [[(get-attr ?e :set-attr) ?val]
                  (subset? ?s ?val)]
          :rules [[(subset? ?x ?y)
                   [(clojure.set/subset? ?x ?y)]]]}
       #{1 2 4})

refset20:01:01

this get-attr function should return the values under ?val as a single scalar set, without decomposing it into a relation

csmith20:01:05

awesome, I think this gets me unstuck. Much appreciated

csmith20:01:31

It isnā€™t quite right. he ?val comes out as a vector rather than a set but I should be able to coerce that

csmith20:01:11

This looks a little gross but works correctly: 

(xt/q (xt/db (get-conn))
        '{:find [(pull ?e [*])]
          :in [?s]
          :where [[?e :set-attr _]
                  [(get-attr ?e :set-attr) ?val]
                  (subset? ?s ?val)]
          :rules [[(subset? ?x ?y)
                   [(set ?y) ?sy]
                   [(clojure.set/subset? ?x ?sy)]
                   ]]}
        #{1 2 3})

šŸ™Œ 2