Fork me on GitHub
#datomic
<
2019-01-23
>
Ben Hammond12:01:23

is there a nice way to pass sample size into a datalog query as a parameter?

(d/q '[:find (sample ?sample-size ?eid) . :in $ ?sample-size :where [?eid :organisation/id]]
     db
     3)
Execution error (ClassCastException) at datomic.aggregation/sample (aggregation.clj:63).
class clojure.lang.Symbol cannot be cast to class java.lang.Number (clojure.lang.Symbol is in unnamed module of loader 'app'; java.lang.Number is in module java.base of loader 'bootstrap')
has problems with variable binding

Ben Hammond12:01:00

I can do something like

((fn [sample-size]
   (d/q {:find [(list 'sample sample-size '?eid) '.]
         :in '[$]
         :where '[[?eid :organisation/id]]}
        db))
 3)
=> [17592186045475 17592186045480 17592186045485]
but seems like a pretty ugly solution

Ben Hammond12:01:03

I guess

((fn [sample-size] 
   (take sample-size 
         (shuffle 
          (d/q '[:find [?eid ...] :in $ :where [?eid :organisation/id]] db))))
 3)
=> (17592186045484 17592186045483 17592186045485)
is my best bet ..?

Ben Hammond12:01:13

abstracted as

(defn sampled-query
  "return no more than n items from datomic query, shuffled randomly"
  [n & qargs]
  (take n
        (shuffle (apply d/q qargs))))

favila14:01:22

Afaik each item in find can only use one bound var and each bound var can only be used once in find

favila14:01:28

So for eg you can’t put a pull expression in a binding then do (pull ?eid ?pull-expr) (violates first rule)

favila14:01:29

Nor can you do :find ?eid (pull ?eid [:my-attr]) (violates second rule)

favila14:01:03

And the error message you get will be utterly mysterious

favila14:01:48

So you can’t paramaterize sample size

souenzzo12:01:35

@ben.hammond try

(d/q '[:find (sample sample-size ?eid) .
       :in $ sample-size
       :where [?eid :organisation/id]]
     db 3)

Ben Hammond12:01:35

(d/q '[:find (sample sample-size ?eid) .
       :in $ sample-size
       :where [?eid :organisation/id]]
     db 3)
Execution error (ClassCastException) at datomic.aggregation/sample (aggregation.clj:63).
class clojure.lang.Symbol cannot be cast to class java.lang.Number (clojure.lang.Symbol is in unnamed module of loader 'app'; java.lang.Number is in module java.base of loader 'bootstrap')

Ben Hammond12:01:51

sample wants a number we've only given it a symbol

spieden19:01:34

hmm, i have a confounding situation where pull isn’t behaving as i expect. shouldn’t i see the same two :demux/id values in both situations below?

(q '{:find  [?did]
     :where [[?f :flowcell/id "HW27TBBXX"]
             [?d :demux/flowcells ?f]
             [?d :demux/id ?did]]})
=> #{["demux-id-two"] ["demux-id"]}

(pull [{:demux/_flowcells [:demux/id]}]
      [:flowcell/id "HW27TBBXX"])
=> #:demux{:_flowcells #:demux{:id "demux-id-two"}}
(`q` and pull are just partial applications of the fns from datomic.api)

favila20:01:40

:demux/flowcells is isComponent cardinality-one?

favila20:01:20

isComponent card1 reverse-refs are card-one

favila20:01:54

if this is a data shape you expect (multiple "demux"es sharing the same "flowcell" then :demux/flowcells should not be isComponent=true

favila20:01:19

if you retractEntity a demux entity the flowcell it points to will also be retracted

spieden23:01:11

@U09R86PA4 that was it! thanks. looks like i can’t alter the schema to make flowcell no longer a component, but i need to rethink this data model now anyway.

favila23:01:27

really? that should be possible

favila23:01:20

why do you think you can't alter this schema?

spieden17:01:16

i get:

{:errors ({:db/error :db.error/incompatible-schema-install,
                              :entity :demux/flowcells,
                              :attribute :db/isComponent,
                              :was true,
                              :requested false}),
               :db/error :db.error/invalid-install-attribute}

spieden17:01:54

i think i’m actually going to keep it as a component, though, and create new flowcell entities each time

spieden17:01:03

eh, this would be a major change actually with a big cascade

spieden17:01:43

oh i see, i’m just doing it wrong

spieden17:01:54

it wants a retraction instead of an assert false

spieden19:01:08

hmm, actually no the docs say it should work the way i’m trying

spieden19:01:41

seems like this is the first place where i can’t just :db.install/_attribute over what i have but need to detect update vs create and do :db.alter/_attribute instead

spieden19:01:07

ah nevermind. i just switched to using neither and seems good

kenny20:01:20

Why does an Ion parameter update request the creation of new physical resources?

henrik08:01:32

The logic is managed by CloudFormation. Certain parameters can't be updated in place, but mandates construction of a new resource. It has to do with AWS rather than Ions itself.

lilactown21:01:41

do I need to do a restart after updating the IAM policy attached to my compute node(s)?

lilactown21:01:05

welp, it looks like after I re-deployed it works now so… I guess so?

kenny21:01:08

I had invalid EDN in my Ion parameters and I got an exception that looks like this:

"Msg": "LoadIonsFailed",
    "Ex": {
        "Cause": "EOF while reading",
        "Via": [
            {
                "Type": "clojure.lang.Compiler$CompilerException",
                "Message": "java.lang.RuntimeException: EOF while reading, compiling:(config.clj:10:14)",
                "At": [
                    "clojure.lang.Compiler$InvokeExpr",
                    "eval",
                    "Compiler.java",
                    3700
                ]
            },
It would've been great if I got an error saying that ion/get-env failed due to an EOF error.

okocim23:01:57

is it possible to make use of a src-var in an aggregator, or is that just for datomic to know which source to use for passing in the coll to the aggregator? I’m trying to write a “best” aggregator that takes in a collection of entity ids, pulls in some further attributes from those entities, calculates a composite score, and returns the best composite. Can I access the database specified by the src-var passed into the aggregator, or is what I’m trying to do only possible in-memory on the client side?

okocim23:01:04

(d/q '[:find ?i (offer.calcs/best-composite $ ?o)
       :where
       [?s :store/id "demo-customer-shop"]
       [?q :quote/store ?s]
       [?q :quote/product-suite ?i]
       [?q :quote/offer ?o]]
     (db/latest-db))

;; here, :quote/offer is a composite ref with cardinality many

okocim23:01:07

something like that