Fork me on GitHub

Is there a way to turn off datomic.process-monitor info logs from the repl?


I get plenty of these:


INFO datomic.process-monitor - {:MetricsReport {:lo 1, :hi 1, :sum 1, :count 1}, :AvailableMB 5970.0, :ObjectCacheCount 0, :event :metrics, :pid 18037, :tid 127}


The settings for metrics reporting are all configured with your logback.xml file


Are there any updated recommendations on how to sorting/pagination on large datasets? Most of the posts I’ve seen about this are 5 years old by now. My approach has been to do two queries. The first applies filters, and pulls just the ids and sort field. I sort in-process, and then do another query to fetch the collection of desired entities by id. This is still pretty slow (500ms for 30k entities).


If there is a single pretty-selective attribute you can use, you can use it with index-range to pull subsets then feed into queries


Another alternative is to use a secondary database as an index, either polling or watching the tx queue to update


Interesting. What do you mean by pretty-selective attribute? Could you give an example?


It’s usually the first clause in your :where


Suppose you want all things posted on a certain day and also a bunch of other stuff


So your first :where clauses assert that the thing is in the date range (because that is most selective) then a bunch of other clauses check other stuff before finally deciding if it’s in the result set


But there’s 100000 things in that date range and you only need the first 10 that match


So you don’t want those first few clauses to look at everything just to get extra results you won’t use


So you can either divide up the range in the query itself, or you can use d/datoms or d/index-seek to lazily fetch a subset of that range and feed those entity ids to the query (to test the other stuff) as input


If you get less than your desired limit in results, advance the range then repeat


The important thing is that this attribute test/range must by itself ensure that a thing may or may not be in the result set. Otherwise you may advance the range and end up with repeated results


Ie the attribute test if subsetted must produce non-overlapping result sets


Thanks, that’s helpful. 🙂


So, I’ve been working with on-prem Datomic for a while now, but now I have a client that is using the client API…with on-prem my testing is super easy since I can leverage datomock to make a db with sample data in it, transact/query/etc, and see the result. With client I have an external server dependency…how are others testing against that?

Lennart Buit19:01:42

I started using Datomic client memdb


hey all, playing around locally with datomic 0.9.5385 using the dev storage protocol. trying to transact a schema, which looks something like the following:

(def my-schema [{:db/ident       :user/id
                 :db/valueType   :db.type/uuid
                 :db/cardinality :db.cardinality/one}

                {:db/ident       :user/email
                 :db/valueType   :db.type/string
                 :db/cardinality :db.cardinality/one}

                {:db/ident       :user/first-name
                 :db/valueType   :db.type/string
                 :db/cardinality :db.cardinality/one}

                {:db/ident       :user/last-name
                 :db/valueType   :db.type/string
                 :db/cardinality :db.cardinality/one}])
when I call (d/transact my-conn my-schema) I get an error ":db.error/entity-missing-db-id Missing :db/id". I was under the impression that datomic assigned db/ids automatically once an entity was transacted. is that not the case?

Lennart Buit19:01:22

It suits me in sort of small-scoped integration testing

Lennart Buit19:01:05

Not sure how commonly used it is, or if there is anything better, but it at least works for me


@mss what version of datomic?


datomic-pro 0.9.5385 w the dev storage protocol


the implicit db/id was introduced in 5530


ah I see. well that explains that 😂


appreciate the help


whats the difference between Datomic Cloud and Utility Bastion per ? Or, more succintly, wth is Utility Bastion?


I'm trying to make some purchasing decisions today lol


the bastion is a node that you can connect to to give you access to the Datomic VPC in order to do local development


normally, the only things that can talk to the Datomic DB have to be within the same VPC


mainly for security reason


the bastion is an EC2 instance that you can connect to from your computer so that you can access the Datomic Cloud system without pushing your code to AWS

👍 5

Gotcha, that makes sense