Fork me on GitHub
#datascript
<
2020-09-08
>
oly10:09:40

couple of questions can you do sub queries ? using d/q inside an existing d/q errors for me and i can not find any examples 😕

pithyless14:09:53

@UU67HFS2X I'm not exactly following your use-case, but whenever you want to run sub-queries, just remember you can run multiple queries (possibly interleaved with data-munging using regular clojure functions); Sub queries like in SQL engines are necessary, because you want queries to be atomic (here you have a derefed immutable value); or to save on networking bandwidth (here your entire working set is already in memory).

oly14:09:07

should i run these queries separately ? ie (d/q [query]) (d/q [query]) then combine the results some how in my head I would do (dq [query (d/q query entity-id]) ie have a sub query that looks up based on values in the first query

oly14:09:43

I am finding what i want to do quite complex, i can get half way there but combining is the tricky part, I am pulling a product then combining all combinations of 2 attributes against that record but I want to use a different value when it exists

oly14:09:28

so think one product many prices, but not all prices exist in which case i want the record to be zero, I can select the price eid but this gives me many prices to one product and i need it to be a row ie product [price1 price2 price3] every product needs 3 prices but some may only have a single price entered in which case i should still get the 3 prices and the missing values should be 0

oly14:09:56

this way i can map over the result set and generate a csv

pithyless14:09:08

what's the relationship? :price/product or :product/price?

oly14:09:13

currently i am using distinct which is giving me a product entity and a set of price entities, my plan is to pull on each price entity and perhaps do a map merge to fill in the missing values

oly14:09:23

:price/product

oly14:09:06

so products do not have a price entity in them

oly14:09:26

(d/q '[:find ?product-entity (distinct ?e)
         :where [?e :price/product ?product-entity]] @conn)

oly14:09:53

so I can use that to get a product entity with a set of price entites against each product

pithyless14:09:54

what about something like?

(d/q '[:find (pull ?e [:product/id {:price/_product [:price/value]}])
       :where [?e :product/id]]
     db)

oly14:09:26

bear with me while i do some testing 🙂

pithyless14:09:02

The pull does a reverse-join (but will still return products even if no prices exist); Then you would need to do some data-wrangling on the resultset to get it in a form that you need for your CSV (adding default zeros, etc.)

oly14:09:46

yeah I might be able to work with this, got a couple of question can you use functions in a pull like you can in a query where

oly14:09:09

say i want to rename :price/value to be the composite of two other attributes ?

oly14:09:11

[(?make-keyword ?n ?c ?amount) ?kw]
So i figured out i can do this in a where, this fn just combines n and c into a keyword with amount as the value

pithyless14:09:20

no, that's what I meant about the data munging; You want to use where clauses to filter out the data you're interested; pull to fetch all the data you need to work with; and then clojure to do the data-munging 🙂

(let [results (d/q ...)
      final-results (some-data-munging ...)] ...) 

oly14:09:12

perhaps that's where i am going wrong trying to do to much in datalog

pithyless14:09:17

IMO, that kind of stuff only belongs in the where clause if you want to use ?kw for further filtering

oly14:09:44

okay good to know I was using it then doing (distinct ?kw) to get a set of values against a product with the set being the keyword i actually needed

pithyless14:09:41

A classic example is sorting:

(let [sorting-data (d/q '[:find ?only ?keys ?used ?for ?sorting ...])
      sorted (sort-by some-fn sorting-data)
      actual-data (d/pull-many [:data :to :view] (map :db/id sorted))])

pithyless14:09:18

^ notice that the keys you sorted by may not even show up in the pulled data you're interested in

oly14:09:09

yeah thanks for bearing with me, trying to expand the example to pull in the extra data I need

pithyless21:09:44

@UU67HFS2X - just as a followup, TIL that Datomic does also support a syntax for nested queries (it has certain limitations, but still...). No idea if that was ever ported to DataScript. 🤯 https://docs.datomic.com/cloud/query/query-data-reference.html#q

oly08:09:46

I think i actually saw that at some point, but could not find any examples when i needed it again so thanks for posting it 🙂

oly10:09:07

0. [ 54 #{ 1371 291 779 1101 734 ... } ]
Also give the above I want to get a couple of attributes from the entity ids in the set and combine them into a string, I think i can do this in a sub query but are there other ways, In the result above i am using distinct but i kind of want distinct grouped fields to work with which I will need to concat together to make strings

oly10:09:00

:find ?pid (distinct (str ?n1 "-" ?n2)) 
something like that possible perhaps ?

stathissideris13:09:26

@metasoarous this is what I’ve come up with so far, it almost looks like it would be worth looking into react for swing instead 😄 https://gist.github.com/stathissideris/7eb2d14990d0b23f8edbee53dbb7bc21

😂 3
stathissideris13:09:21

no attempts to avoid re-running queries, it’s not my main problem anyway — updating the UI is

stathissideris17:09:09

(refined after working on it a bit)

stathissideris19:09:37

@metasoarous apart from the fact that all of the queries run on every transaction, I also have the problem of hierarchy: if I have a listener for the windows and a listener for updating their contents, this code does not guarantee any order (you need to have created windows to update/add stuff within them)