Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2022-03-08

Channels

#asami
<
2022-03-08
>
simongray14:03:11

Really happy with Asami, but still a bit of a n00b with the queries. How do I dynamically construct a query that ANDs a bunch of triples together? i.e. I have a variable set of mentions that I want to use to build a query such that the query looks/works like 

(d/q [:find '[?path ...]
      :where
      (and ['?e :document/mention m_1]
           ['?e :document/mention m_2]
           ['?e :document/mention ...]
           ['?e :document/mention m_n-1]
           ['?e :document/mention m_n])
      '[?e :file/extension "xml"]
      '[?e :file/path ?path]]
     conn)
I am unsure how to build a query that accomplishes this. Currently trying to build the and clause on in a let, inserting it into the rest of the query where every symbol is now individually quoted, but this is quite messy. Surely there is a better way? Help please! :S

simongray14:03:48

oh wait, I guess the AND is actually implicit, so one could do this right? 

(d/q (into '[:find [?path ...]
               :where
               [?e :file/extension "xml"]
               [?e :file/path ?path]]
             (for [m mention]
               ['?e :document/mention m]))
       conn)
Am I on the right track? Or is there a syntax shortcut to match against a set of values?

quoll14:03:26

Yes, the AND is implicit at the top level. Your previous query could work with ('and ...)

quoll14:03:28

also, I presume that the ['?e :document/mention ...] doesn't actually have an ellipses in it, right? Because that wouldn't work

simongray14:03:04

It is being concatenated using into 🙂

quoll14:03:10

Ooops, I missed the into

quoll14:03:23

that's why I deleted the message. I was just typing to say that 🙂

simongray14:03:31

Just wondering if into is the right way to go, I guess

quoll14:03:43

You don't need a for there. It's more idiomatic to map it instead: (map #(vector '?e :document/mention %) mention)

quoll14:03:26

Or... (map (fn [m] ['?e :document/mention m]) mention)

simongray14:03:43

Ok, but the principle is the same 🙂 I’m mostly wondering if there was a syntax shortcut to match against a set of values.

simongray14:03:17

or if the idiomatic way is to construct the query using regular Clojure functions

quoll14:03:17

Also, if you're constructing a query, you can use a map object instead: 

(d/q {:find '[?path ...]
      :where
        (into [[?e :file/extension "xml"]
               [?e :file/path ?path]]
              (map #(vector '?e :document/mention %)))}
     conn)

simongray14:03:40

Ah, that might be preferable

quoll14:03:54

Hmmm... I can only think of how you can match when ?e is to be matched against ANY of the mentions, not all of them 😕

simongray14:03:48

so like OR? How is that done? I will need to do that too 😛

simongray14:03:07

I guess I put them inside (or ...) ?

quoll14:03:41

(d/q '[:find [?path ...]
       :in $ [?m ...]      
       :where
         [?e :file/extension "xml"]
         [?e :file/path ?path]]
         [?e :document/mention ?m]))
       conn)

simongray14:03:07

thanks a lot, Paula!

simongray14:03:53

right - I keep forgetting that that exists too 😛

quoll14:03:17

Which I should ALSO describe in the Asami docs, because Asami does do things differently in many cases. But the :in forms are the same

simongray14:03:18

Yeah, I am also doing a bunch of RDF queries in Aristotle (i.e. Apache Jena) in my other project, so I get to learn all the subtle differences between RDF, Asami, and Datomic 😛

simongray14:03:32

the triplestore spectrum

quoll14:03:52

Ah... I've been motivated to wrap Asami in RDF/SPARQL. That will be happening soon

🚀 2
🎉 1
simongray14:03:20

Asami is really cool, just so you know 🙂

quoll16:03:50

Thanks for this comment! I'm not always motivated to spend time on Asami, but this helps a lot ❤️

❤️ 1
simongray14:03:37

the RDF heritage is very welcome here

simongray14:03:07

Personally, I wouldn’t be querying it using SPARQL, but I can definitely see the value in that

simongray14:03:39

but the RDF semantics coupled with Clojure data queries are… chef’s kiss

quoll14:03:29

Awww... thank you 🙂

quoll14:03:50

SPARQL is important, because it provides a clean way for people to access it outside of Clojure

rickmoynihan10:03:50

Just to say I’d be very interested in seeing this 👌

quoll15:03:07

Well, I'm hoping it's not TOO big a deal. Most of the operations have a 1:1 mapping, and Asami includes things like BIND, OPTIONAL and so on. Also, the not operator is actually just the SPARQL MINUS operator, rather than the precise semantics of Datomic's not. My approach will be to convert the grammar to Instaparse, then map the resulting structures to the current query language, since that language is: a) already a Clojure structure, which is what lexing/parsing is trying to do b) already in a form that can be executed by the query engine

rickmoynihan16:03:30

@U051N6TTC: We already have a very complete instaparse grammar that converts SPARQL into an EDN based AST representation, if you’re interested

rickmoynihan16:03:18

I can’t promise it’s 100% complete; but IIRC it’s pretty close

rickmoynihan16:03:41

e.g. 

(-> "
SELECT DISTINCT ?_ WHERE {
  ?s1 <<http://p>>/<> ?o1 .
  ?s2 a ?o2 .
  ?s3 uk-wfd:RiverBasinDistrict ?o3 .
  ?s4 uk-wfd: ?o4
}
ORDER BY ASC (?s)
LIMIT 10
"
    parse
    clojure.pprint/pprint)

;; =>
{:type :select,
 :modifier "DISTINCT",
 :projection
 [{:type :qvar, :x ?s} {:type :qvar, :x ?p} {:type :qvar, :x ?o}],
 :expr
 [{:type :triple,
   :s {:type :qvar, :x ?s},
   :p #object[grafter_2.rdf4j.sparql.path.Group 0x14367ef3 "[email protected]"],
   :o {:type :qvar, :x ?o}}
  {:type :triple,
   :s {:type :qvar, :x ?a},
   :p {:type :qvar, :x ?b},
   :o {:type :qvar, :x ?c}}],
 :modifiers
 [{:type :order-by, :modifier "ASC", :expr {:type :qvar, :x ?s}}
  {:type :limit, :n 10}]}

rickmoynihan16:03:18

Not that I want to take the fun out of writing such thing from you 😆

quoll16:03:21

There's an ego thing of building it from scratch. And for better or worse, people don't like external libs 😕 Asami is remarkably self-contained (many of the libs are also written by me)

rickmoynihan16:03:37

Sure… The code isn’t public yet; but you could always inline some bits if you found it useful… e.g. even just the file of instaparse compatible ebnf

rickmoynihan16:03:58

as iirc you can’t quite just copy/paste the grammar from the sparql spec

rickmoynihan16:03:04

then there’s about 300 lines of clojure to convert it into the edn ast representation — which you could choose to ignore or not

rickmoynihan16:03:46

I didn’t write it, a colleague did — so I’d want to run it by him first, but the plan was to open the code at some point anyway.

rickmoynihan16:03:28

it could pretty easily be cut down to have minimal dependencies

quoll14:03:42

People don't need to know what keywords are too. I figure I'll treat them as QNames

simongray15:03:07

(defn- entity->where-triples
  "Deconstruct partial `entity` description into triples for a search query."
  [entity]
  (->> (for [[k v] entity]
         (if (coll? v)
           (map #(vector '?e k %) v)
           [['?e k v]]))
       (apply concat)
       (set)))

(defn- entity->search-query
  "Build an entity search query from a partial `entity` description."
  [entity]
  (into '[:find [?id ...]
          :where [?e :db/ident ?id]]
        (entity->where-triples entity)))

(defn match-entity
  "Look up entity IDs in `conn` matching partial `entity` description."
  [conn entity]
  (d/q (entity->search-query entity) conn))

(defn search
  "Return matching entities in `conn` based on partial `entity` description."
  [conn entity]
  (map (partial d/entity conn) (match-entity conn entity)))
Here’s what I ended up doing. Works for my purposes as long as I am not having to do OR queries.

simongray15:03:28

Basically, I just want to be able to partially describe some entity and get the corresponding matches