This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-09-01
Channels
- # adventofcode (2)
- # announcements (3)
- # babashka-sci-dev (79)
- # beginners (76)
- # biff (2)
- # calva (32)
- # cider (2)
- # clj-kondo (42)
- # clj-on-windows (17)
- # clojure (28)
- # clojure-belgium (1)
- # clojure-berlin (1)
- # clojure-europe (95)
- # clojure-nl (4)
- # clojure-norway (4)
- # clojure-uk (5)
- # clojurescript (27)
- # conjure (5)
- # cursive (3)
- # data-science (16)
- # datomic (67)
- # graalvm (12)
- # hyperfiddle (36)
- # jobs (3)
- # jobs-discuss (1)
- # kaocha (2)
- # klipse (1)
- # leiningen (28)
- # lsp (16)
- # luminus (3)
- # malli (10)
- # nrepl (3)
- # off-topic (57)
- # other-languages (18)
- # re-frame (4)
- # reitit (8)
- # releases (1)
- # remote-jobs (1)
- # scittle (4)
- # shadow-cljs (7)
- # test-check (1)
- # tools-deps (4)
- # vim (11)
- # xtdb (25)
I’m on an application team using PostgreSQL. We’re considering Datomic Cloud. There is an analytics team that wants us to stream all our data to them via https://aws.amazon.com/kinesis/data-firehose/. I’m imagining there’s a way to hook up the https://blog.datomic.com/2013/10/the-transaction-report-queue.html to it somehow. Maybe I could write an Ion subscribing with the Datomic client that knows how to publish to Firehose?
Datomic Cloud's Client API doesn't support the Peer API's txReportQueue. You might want to install a transaction function that pushes each transaction to a queue as a side-effect. C.f. https://forum.datomic.com/t/tx-report-queue/316/4

Let's pretend I know nothing about Datomic, except for marketing and some YouTube videos (which is true 😆). What does it mean to install a transaction function
?
I guess that's what this is about? https://docs.datomic.com/cloud/transactions/transaction-functions.html#custom
Yes, so it's app-level code that you deploy for the transactor instead of for the user-level-ion-app. Ion can deploy code to: • Query groups • Primary group (incl. transactor)
Makes sense. Thanks!
You're welcome. Honestly, when discussing these things, we're in the darker corners of Datomic experience. It's much easier to get answers for Datomic On-Prem than for Cloud. OTOH, given its constraints, it seems it might be harder to shoot ourselves in the foot with Cloud than it is with On-Prem's more flexible design.
I'm struggling to make a query to list "recipes" in my datomic db which are either public or private.
I tried to use or
as in
'[:find (pull ?e pattern)
:in $ pattern account-id
;; Or clause:
:where (or [?e :recipe/public? true]
(and [?owner :account/account-id account-id]
[?e :recipe/owner ?owner]
[?e :recipe/public? false]))]
.. but that's failing with:
Caused by: java.lang.AssertionError: Assert failed: All clauses in 'or' must use same set of vars, had [#{?e} #{?e} #{?owner ?e}]
(apply = uvs)
Then I tried or-join
:
'[:find (pull ?e pattern)
:in $ pattern account-id
;; Or clause:
:where (or-join [?e]
[?e :recipe/public? true]
(and [?owner :account/account-id account-id]
[?e :recipe/owner ?owner]
[?e :recipe/public? false]))]
but that doesn't return what I want - it seems to only return recipes that are public.
Before, I had those as two separate queries and it worked well:
;; public recipes
[:find (pull ?e pattern)
:in $ pattern
:where [?e :recipe/public? true]]
;; private recipes
'[:find (pull ?e pattern)
:in $ ?account-id pattern
:where
[?owner :account/account-id ?account-id]
[?e :recipe/owner ?owner]
[?e :recipe/public? false]]
Here's the whole schema - it's from the Learn Pedestal course: https://github.com/jacekschae/learn-pedestal-course-files/blob/main/increments/19-load-dataset/src/resources/cheffy
Ok, silly mistake - it looks like my version is actually working and it's just a higher-level logic that transforms this into something that makes a test fail.
Well, it actually is a bit weird.
pull
requires me to use an argument name without question mark like pattern
.
But or-join
requires me to use one with question mark it seems like ?account-id
.
So the correct version of my query is:
`[:find (pull ?e [*])
:in $ pattern account-id
;; Or clauses:
;; `or-join` is needed because the second clause (`and`) uses a different set of variables
:where (or-join [?e ?account-id]
[?e :recipe/public? true]
(and [?e :recipe/public? false]
[?owner :account/account-id ?account-id]
[?e :recipe/owner ?owner]))]
I also had to specify both ?e
and ?account-id
in the list that comes as the first arg of or-join
.For plain account-id
I get this error:
Caused by: java.lang.RuntimeException: Unable to resolve symbol: account-id in this context
Ah, no, it simply doesn't work.
I forgot to replace one occurence.
With ?account-id
I get this error:
... 81 more
Caused by: java.lang.Exception: Unable to find data source: $__in__3 in: ($ pattern $__in__3)
using this query:
[:find (pull ?e pattern)
:in $ pattern ?account-id
:where (or-join [?e ?account-id] ; it's the same error whether I specify ?account-id here or not
[?e :recipe/public? true]
(and [?e :recipe/public? false]
[?owner :account/account-id ?account-id]
[?e :recipe/owner ?owner]))]
Ok, I found the real problem - sometimes, this ?account-id
input arg can be nil
.
It's this condition that makes it fail with such a weird error.
So I can include this clause only when ?account-id
is not nil:
[?owner :account/account-id ?account-id]
I need a single query for reasons. So I'm looking for the way to get both results from a single query, instead of having two queries (which actually are longer than a single one)
For now, it's seems it's the ?account-id
parameter that makes it fail when it's nil.
If there's a simpler way, I'm happy to learn it 🙂.
"or" condition doesn't sound to me like something unusual. But I'm Datomic noob and mostly familiar with SQL databases.
That's what I tried and it doesn't work - see https://clojurians.slack.com/archives/C03RZMDSH/p1662030547421749
or-join is very special case, i'm not sure you really need it here, but i've never run into a situation where i have used it
Again, I tried or
and it doesn't work:
Caused by: java.lang.AssertionError: Assert failed: All clauses in 'or' must use same set of vars, had [#{?e} #{?e} #{?owner ?e}]
(apply = uvs)
It's because I need ?owner
in the second set of rules.(and [?e :recipe/public? false]
[?owner :account/account-id ?account-id]
[?e :recipe/owner ?owner]))
i think you may make a rule for this, and i'm not sure the ?owner
is doing anythingIt should return only recipes whose owner is the account-id specified as the input argument
if the owner has a relation to their recipes, you can get that without a query, just a pull
I'm using pull, so I'm not sure I understand. I would appreciate a full query if you can provide that.
But I want a single query.
I'm trying to pass my query to an interceptor which executes it and that doesn't know anything about the query.
Interceptor uses datomic.api/query
function to get the results and pass it further.
i do almost everything in datomic via pulls, very rarely do i make queries, and they are usually doing stuff that is crazy (history stuff)
Again, I use it already and this is what I have now:
(da/query
{:query '[:find (pull ?e pattern)
:in $ pattern ?account-id
:where (or-join [?e]
[?e :recipe/public? true]
(and [?e :recipe/public? false]
[?owner :account/account-id ?account-id]
[?e :recipe/owner ?owner]))]
:args [db recipe-pattern account-id]})
you gotta use pulls when dealing with direct relations like this, it's just much simpler
I have complicated recipe-pattern already: https://github.com/jacekschae/learn-pedestal-course-files/blob/main/increments/43-list-conversations/src/main/cheffy/recipes.clj#L9-L28 Not sure if that can easily be combined with what I need.
you don't need all this complicated stuff in the where, just query all public recipies, and pull related ones on the owner
Sorry, I simply don't understand how to do this filtering with pull syntax.
I need to get the union of two sets:
• All the public recipes - returning all the information as specified by my recipe-pattern
• All the private recipes owned by given account-id
(if any)
That's a separate action that I do not want to perform because the actor executing the query doesn't know anything about it. In my case, the client would pass this:
{:query '[:find (pull ?e pattern)
:in $ pattern ?account-id
:where (or-join [?e]
[?e :recipe/public? true]
(and [?e :recipe/public? false]
[?owner :account/account-id ?account-id]
[?e :recipe/owner ?owner]))]
:args [db recipe-pattern account-id]})
and the executor would take it and pass to datomic.api/query
- it has no clue about some additional pull that needs to be executed.Maybe. But I feel like I'm making it more complicated just because of the desire to use pull 🙂
can you do this in 2 steps, where one is the query and another is just a pull (which can be done in a query as well)
Maybe I should think about allowing a sequence of queries, regardless whether this is the best approach here or not. Thanks for the ideas. I'll see what I can do.
pull is really good, though. you can get away with almost never using queries if you use pull
the client can ask for all public recipes, and all owner recipes, you get 2 very tiny queries, 1 line each.
i have a feeling it'll be faster too, cus who knows what datomic is trying to do with that or-join stuff
also, you probably want to do something special with the all recipes query (filter, sort, limit) and adding that stuff to an already complicated query is probably going to cause issues
Does Datomic have any support for java.time package?
It seems that I have to use an instance of java.util.Date
as a value of instant attribute.
I didn't find much about this topic, just an older discussion: https://forum.datomic.com/t/java-time/1406
Unfortunately Datomic doesn't let you extend the types it supports, but I have been using "Datomic Type Extensions" via https://github.com/magnars/datomic-type-extensions with this java.time-package https://github.com/magnars/java-time-dte for several years in different projects now. It is slightly leaky, but certainly useable.