The database function docs are kinda sketch on how one tx function can call another tx function. Is that possible, even?

is there a way to efficiently read through all datoms within a given partition? I'm thinking of using indexRange and the EAVT index. But don't know how to specify the lower and upper bound. Can i somehow find out what the lowest and highest entId is for a given partition?

@cezar: I'd question why you want to do something like that. This should do what you want though.

(d/datoms (d/filter db (fn [_ ^Datom datom]
                         (->> datom .e d/part (= part)))) :eavt)

what you showed is a sequential scan over the whole database. If I have a 1000 customers and each in its own partition and I want to trawl through a reasonably large collection of datoms (say 10,000,000 for each customer) it will be way more efficient to hone in only on the section of the index where their specific data is located. Isn't that what partitions are for?

Just query the database?

queries are very eager and I find that running d/q where the result is more than about 5,000,000 entities can easily blow up a Peer with OOM even on a Peer with Xmx4g

I prefer the lazy nature of d/datoms or indexRange or seekDatoms

for my use case here

The locality of reference is in :eavt, you could take-while with a pred on part - http://docs.datomic.com/clojure/#datomic.api/part — starting from the beginning of the partition with seek-datoms — http://docs.datomic.com/clojure/#datomic.api/seek-datoms

ah, figured it out. looks like seekDatoms on EAVT is the way to do this

thanks @bkamphaus for pointing me in the right direction

I can simply look use entIdAt with Date 0 and Date now to get all entities over a given partition

How do you lookup a database function so that you can call it from within another database function?

I ask because I find myself wanting to (mapcat (fn [x] [:my/fn db a b c]) values) which seems, odd? Hm. Maybe it’ll work.

Ha! It works! My tx function is huge with multiple functions, but I don’t think there’s much else I can do when updating a complicated “document-like” structure.

hi guys how do I avoid transacted value being printed to *out* in repl ? I import quite big amount of data and it's annoying.

@nxqd (do @(d/transact ….) :done) Something like that?

yeah totally forgot T_T thanks

@bkamphaus: I'm having a doubt about how Datomic works: are the indexes persisted in storage are immutable trees of segments, or is this just the in-memory representation?

in storage too

i usually do (-> (d/transact ...) :db-after) or ( .. :tx-data count)

@val_waeselynck: it’s turtles all the way down.

@zentrope: Yes, you can call txn functions from other txn functions. As long as they all eventually resolve to valid txdata (i.e. list of datoms)

@cezar: Yep, as you found (and @bkamphaus indicated), the most significant bits of the entity ID encode the partition. Thus, the partition is contiguous in EAVT index.

> it’s turtles all the way down. @marshall: sorry, I don't know what that means 😕 (not a native English Speaker)

@val_waeselynck: Sorry. Yes, it’s immutable segments everywhere.

@marshall: thanks! 🙂

https://en.m.wikipedia.org/wiki/Turtles_all_the_way_down

@val_waeselynck: As an english speaker, I do not understand either.

@marshall: Yes, it all makes sense if you think of a TX function as producing a data structure. Kinda like a macro.

I have a few questions regarding the console.

1) Does it work with mem databases?

2) From where do I get the dev-transactor.properties file?

3) The example command to start the console here http://docs.datomic.com/console.html:

bin/console -p 8080 dev datomic:

in that example, the "datomic:<dev://localhost:4334/>“ is the URI of the database

Where do I set that?

There is an example dev-transactor.properties file in the Datomic Pro distribution under config/samples

Not in the most recent download ...

do you have Datomic Pro Starter ?

OH, sorry. I found it

It is in a samples subdirectory.

if you’re using free, there is a free-transactor-template.properties file in config/samples

Thanks @marshall. When do I need to specify the transactor.properties file in a command? Do I need to start a transactor in a local dev environment?

@nando you want to specify the file when you launch your transactor. Yes, you will need to start a transactor in a local dev env.

Thanks @jaret

Is there a guide somewhere I’m missing that explains how to set up a dev environment?

@nando: http://docs.datomic.com/run-transactor.html

http://docs.datomic.com/console.html jumps into explaining how to start the console without mentioning that you need a transactor running

@nando The getting started section is where I went first. I didn’t make my way into the overview section until after the tutorial.

Ok, I’m more clear what I need to do now. Thanks again.

Using the pull api: Is there a way to turn {:foo {:id “1234”}} to {:foo “1234”} without doing any post processing?

If I want to use an aggregate result to track down another value on an entity, will I always need to use two queries?

(defn last-edited [conn eid]
    (d/q '[:find (max ?inst)
           :in $ ?e
           :where
           [?e _ _ ?tx _]
           [?tx :db/txInstant ?inst]]
         (d/db conn) eid))

;;(the following is not what I want because “Query variables not in aggregate expressions will group the results and appear intact in the result.”)
(defn last-edited [conn eid]
    (d/q '[:find (max ?inst) ?user
           :in $ ?e
           :where
           [?e _ _ ?tx _]
           [?tx :db/txInstant ?inst]
           [?tx :user/source ?user]]
         (d/db conn) eid))

alternatively I could just query the entire history, sort by txInstant, yadda yadda

@zentrope: no, pull will always do that for refs

@uwo: I believe you can use a subquery for this, but I don't know the right syntax off the top of my head. I think that you can put a second (d/q …) in the place of a where clause, though.

hmm! thanks I’ll look into that

@bhagany: I was willing to be totally surprised that it was possible. ;) Fortunately, clojure.walk solves all problems.