[??? :strive_form_data/submitted_at ?some-value][(< ?some-value 1)] will work. But only you know what the ??? should be.

@U09R86PA4 amazing thank you so much! Can i ask, if i wanted some-value to be between 1 and 100 for example, how would i go about doing that? Using clojure syntax inside the query keeps leading to an error

[(<= 1 ?some-value)][(<= ?some-value 100)]

<= < > >= = != are special in queries. They’re not the normal clojure comparators

https://docs.datomic.com/on-prem/query/query.html#built-in-expressions

Cool that's how i did it. But i was afraid it would be inefficient since this basically does an implicit join?

all binding does an implicit join

how do you get the ?some-value we’ve been talking about in your query?

is it just [?nps :strive_form_data/submitted_at ?some-value]?

or is it on some other entity?

Ohh ok. Yes exactly how you just said

so the entity id is already known and bound; this is just retrieving the value and filtering

i.e. applying two predicates

this query is already scanning all answers/belongs_to

are either the second or third clause indexed?

https://docs.datomic.com/on-prem/best-practices.html#most-selective-clauses-first

@U09R86PA4 thank you very much! I wanted to ask, is there a way to do computation at the datomic query level? For example in sql we can use functions that divide numbers and uses the result as input of a child query. Is that possible in datalog as well?

Query rules are one way. You can also (on on-prem) call any function

Roughly.

Obviously, those zip files aren't in cause - they'less than 1 MB. And the dependencies in .m2 and gitlibs that must be copied from the S3 bucket don't weigh more than 150 MB.

Is the system quiet in that time? Eg no transactions?

On google cloud I remember some issue where their networking stack just drops idle tcp connections. I had to add keepalives into the kernel options somehow. IIRC it manifested like this, the peer looked like it went away and it only happened when the system was quiet

This was more than 5 years ago, my memory is hazy

no, i am developing a data processing framework with clojure, and currently i am stress testing it, and I am writting the results to datomic

so every second it has 2 tx of 100 datoms to save

so basically 200 a second, splitted between 2 transactions

I should also mention that these are dockerized, and I have also modified the keep alive values, for both containers, to make sure this isn’t a tcp connection issue

is it possible that the peer really did just go away for 10 secs, e.g. a long gc pause?

well, on the transactor i have set the max gc pause to be 50ms

on the peer, I haven’t modified that

is that something one should do?

those targets don’t apply when there’s a full gc and memory pressure. I’m really just suggesting that if you know the peer is busy or could have memory pressure on it, rule out that the timeout isn’t due to a GC pause

there are jvm startup flags that will log GC pause activity

to console or to a file

oh, ok, thanks for the pointers, much appreciated 🙂

I will check that out

I’m guessing no, as there’s no way to disambiguate which entity is being referred to via attribute identities.. ?

ie:

[[:db.fn/retractEntity [:a/id1 "a"]]
 [:db.fn/retractEntity [:a/id2 "b"]]
 {:thing/id "thing"
  :thing/stuff {:a/id1 "a"
                :a/id2 "b"}}]
 

You should run this to know for sure. I would expect all lookups to happen before retraction, so this will cause {:a/id1 "a" :a/id2 "b"} to expand to [:db/add id1a-entity-id :a/id1 "a"] etc. Since those same assertions are being retracted in the same transaction via the retractEntity, the transaction will fail with a conflict.

In general all lookups happen on the “before” db and all operations in a transaction are applied atomically. (Exceptions are composite tuples, which do read an intermediate state to know what to update the new values to; and entity predicates, which read an “after” db right before commit.) So it’s not ambiguous at all what a lookup in a transaction will do.

there’s no way, even in a transaction function, to see a “in-progress” or “partially-applied” database value

it does indeed fail with a conflict - will try with concrete IDs too (doesn’t work, sadly)

concrete ids will work IFF the entity ids for the retracts and the new assertions are different