Fork me on GitHub
#datomic
<
2020-03-27
>
johnj17:03:41

is there such thing as too many entities references in a cardinality many attribute?

motform17:03:32

I have another newbie question about the map form for queries, regarding quoting as it feels like I have misunderstood something.

(d/query '{:query {:find [?e]
                  :in [$ ?title]
                  :where [[?e :title ?title]]}
           :args [(d/db conn) "negroni"]})
gives me the error nth not supported on this type: Symbol. When I quote the nested query map, it works (not surprisingly, as we don’t wanna eval all the datalog symbols that it complains about if i leave the whole thing unquoted). This works in the repl, but I don’t see how I would write code that does this to the query map without reaching for a bunch of map manipulation, the thought of which gives me that feeling that I’m doing something wrong.

motform17:03:28

What confuses me is that in the docs, nothing is quoted. when querying with the map form. https://docs.datomic.com/cloud/query/query-executing.html It also shows the nested query map form for a /q and not /query invocation, which also confuses me, as i thought /q wanted a flat map and args as & args , supplied directly to the function

skuttleman18:03:18

I don't think you want to quote the args, right?

(d/query {:query '{:find [?e]
                   :in [$ ?title]
                   :where [[?e :title ?title]]}
          :args [(d/db conn) "negroni"]})

motform18:03:02

No, I guess not, but how do I only quote the query-map? This is the interface to the database, where parse-strainer takes a map of user input and builds a query-map through a cond-> pipeline

(defn strain [strainer]
  (let [query (parse-strainer strainer)]
    (d/query query)))

favila18:03:09

The purpose of quoting is so symbols like ?e don’t get expanded to current.ns/?e and lists like (some-rule) in the query don’t get evaluated.

favila18:03:29

you can accomplish the same clause by clause, or using forms like (list 'some-rule '?foo ?bar) or even ('some-rule '?foo ~'?bar)`

favila18:03:53

This is normal Clojure quoting, it’s not specific to datomic

favila18:03:15

the query just wants to see literal symbols ?e etc

motform18:03:22

yeah ok, that makes sense of course, I guess I don’t quote that much in Clojure otherwise.

motform18:03:56

But what I still don’t get is how they want to to use the api. If I have a fn that spits out the following map so that it might then be used to call d/query with, how do I quote only the :query submap?

{:query {:find [?e]
                   :in [$ ?title]
                   :where [[?e :title ?title]]}
          :args [(d/db conn) "negroni"]}
(update m :query quote) evals the symbols

favila18:03:28

if you already have the query map, why are you quoting it again?

motform18:03:08

im not! which is where I get confused, haha

motform18:03:12

(defn strain [strainer]
  (let [query (parse-strainer strainer)]
    (d/query query)))

motform18:03:15

is my end-point

favila18:03:19

if that is actually what your function returns, then you are done

favila18:03:36

just hand that map to d/q

motform18:03:43

my fn

(defn- parse-strainer [{:keys [:ingredients :search :type]}]
  (cond-> base-query
    ingredients (simple-query :ingredients '?ingredients ingredients)
    type (simple-query :type '?type type)
    search (fn-query :fulltext '?fulltext 'fulltext '$ search)))
(parse-strainer {:ingredients ["vodka" "cream"] :search ["russian"]})
{:query
 {:find [(pull ?e [:id :title])],
  :in [$ [?ingredients ...] [?fulltext ...]],
  :where
  [[?e :ingredients ?ingredients]
   [(fulltext $ :fulltext ?fulltext) [[?e ?n]]]]},
 :args [(d/db @*conn) ["vodka" "cream"] ["russian"]]}

motform18:03:07

but when i do (strain {:ingredients ["vodka" "cream"] :search ["russian"]}) i get Execution error (UnsupportedOperationException) at datomic.datalog/extrel-coll$fn (datalog.clj:300). nth not supported on this type: Symbol

favila18:03:43

where do your args come from?

favila18:03:47

your (d/db @*conn) is a literal list with d/db and conn elements

favila18:03:20

I think it’s trying to use it as a vector-type datasource, but d/db doesn’t support nth

motform18:03:41

oh shoot, so i should somewhere do like (def db (d/db conn) and have :arg [db [“foo”] [“bar”]?

favila18:03:59

“arg” is not syntax, it is real objects

favila18:03:52

(d/q query arg1 arg2) and (d/q {:query query :args [arg1 arg2]}) are equivalent

favila19:03:56

can you show the code that builds args?

motform19:03:03

ah, of course! I did not think of that at all

motform19:03:05

“its just data”

motform19:03:26

(defn strain [strainer]
  (let [{:keys [query args]} (parse-strainer strainer)]
    (apply d/q query (d/db @*conn) args)))
now works! before I hade a base map for my query that contained :args [(d/db @*conn)]that i then conjed onto the other args to

favila19:03:13

you can still do that, just don’t quote the args in your base map

favila19:03:56

the problem is that the db was literally a list, instead of the db object, so some quoting was going on that shouldn’t have.

motform19:03:32

i get that now, thank you so much for your help!

motform19:03:06

i guess i just assumed that i would be invoked somewhere along the line, don’t think i’ve encountered this before

motform19:03:26

datomic is the only place outside of macros where i’ve actually come across quoting

motform19:03:08

is there a best prefered way to pass the db argument around? in my cases, I’ve used (d/db @*conn) where conn is an atom holding a (d/connect uri), would it be “better” if it was just a var with a (d/db conn)?

favila19:03:31

like any code, avoid mutables

favila19:03:35

conn is a mutable

favila19:03:30

also, it allows some priviledge scoping: anyone can transact with a conn, but not with a db

motform19:03:30

that makes sense, thanks!

motform19:03:50

I guess i should just RT rest of the FM, that best practice page was really good!

motform19:03:30

do you have any good open source reference projects that use datomic that one could look at?

favila19:03:01

I know lots of libraries, but I can’t think of any apps offhand

motform09:03:13

no probs, I’m super thankful for all the help already! : )

motform09:03:41

Can I have one last quoting question? In my query, I want to pull and bind-coll with , which i can add to my q no problems,

{:query
 {:find
  [(pull ?e [:id :title :recipe :preparation :ingredients]) ...],
  :in [$ [?ingredients ...] [?fulltext ...]],
  :where
  [[?e :ingredients ?ingredients]
   [(fulltext $ :fulltext ?fulltext) [[?e ?n]]]]},
 :args [#{"gin" "rum"} #{"russian"}]}
However, when I run this, it tells me that Argument ... in :find is not a variable, despite that fact that it can handle the in the :in clause

motform10:03:12

Does it have something to do with the fact the the other invocations of are nested? should not, right?

favila11:03:47

This is a peer vs client api difference. Only the former supports destructuring in :find

motform13:03:24

Hm, then that’s strange, I thought that Datomic free only used the peer api? It works in my hand written queries, which is what made me confused

favila16:03:13

You are using free not cloud?

favila16:03:59

D/q with map arg is a client api thing, so I assumed you were using cloud

motform18:03:44

Haha, no, I’m a free leecher for the moment. d/q takes args in map or vector form, d/query take maps with :query and :args keys, as Ive understood it. I have no clue about how the client api, have not researched that yet

favila22:03:13

Nvm, this is the map form of query, so you need an extra vector

favila22:03:44

[:find a b c] => {:find [a b c]}. So [:find [a ...]] => {:find [[a ...]]}

johnj18:03:10

is transaction functions a common way to achieve referential integrity in datomic?

arohner18:03:56

Tried to use insecure HTTP repository without TLS:
 project: 
 com/datomic/datomic-lucene-core/3.3.0/datomic-lucene-core-3.3.0.jar 
 
This is almost certainly a mistake; for details see 

arohner18:03:28

I see that the datomic-pro-0.9-6024.pom contains

<repository>
   <id>project</id>
   <url></url>
  

Alex Miller (Clojure team)18:03:37

I think they were fixing up some stuff like this recently iirc, but I'm not on the team

Alex Miller (Clojure team)18:03:53

not sure if anyone is watching here atm

arohner18:03:06

I think I figured it out

arohner18:03:56

we’re hosting datomic-pro.jar in a private S3 repo. datomic-lucene-core also needs to be there. It wasn’t found in central, so it tried all other repos, and then complained about the http://

Alex Miller (Clojure team)18:03:34

ah, yes that is a common error reporting gotcha

Alex Miller (Clojure team)18:03:55

if it looks in N places and doesn't find it, it just reports the first or last place it looked as it doesn't know where it was expected to find it

Alex Miller (Clojure team)18:03:19

and often that is different than your expectation

em22:03:35

For Datomic Ions, are there hooks for system (component, integrant, etc.) start/stop? Wondering about best practices here

cjsauer15:03:30

In the past I’ve placed start calls at the beginning of each HTTP request, right before handing the request to the app handler. Calls to start are idempotent and are essentially no-ops if things are already started.

cjsauer15:03:16

@UNRDXKBNY so it’s basically side-effecting middleware.

cjsauer15:03:56

This gives you a well-defined place to assoc the started system onto the request map as well.

em19:03:03

@cjsauer Yeah, I've basically done the same for system start, though was a little worried about runtime performance and how much the no-op check would cost. But are there any good solutions for stop? Wondering about how to manage integrations like AWS API gateway and websockets, and having notifications on process cycling or auto-scaling

cjsauer19:03:47

Hm, I haven’t needed stop hooks myself. I wonder if you could tie into the Simple Workflow (SWF) hooks that Datomic sets up for coordinating deploys.

em20:03:07

Ooh, didn't know about that part of Datomic (thanks for the pointer!), though it makes sense that something like SWF was behind coordinating deploys. I wonder if there's some buried api for this that wasn't too much trouble/not supported to hook into

em19:03:03

@cjsauer Yeah, I've basically done the same for system start, though was a little worried about runtime performance and how much the no-op check would cost. But are there any good solutions for stop? Wondering about how to manage integrations like AWS API gateway and websockets, and having notifications on process cycling or auto-scaling