Fork me on GitHub

Does anyone have any advice on managing application lifecycle events with Ions? I want to be able to start and stop Kafka consumers in a reliable way.


Is there an idiomatic way of managing services, like RDB connection pools etc.


Q: I had prod downtime with Datomic Cloud today. first time ever. I’m still trying to figure out what went wrong but killing the instances made it come back up. Trying to learn how to diagnose this…


the only thing I can see is the indexer memory is high but normally it drops down when it reaches its ceiling. after the restart it jumped straight back to the prev level. Is this normal or should it restart at a min level (similar to jvm free)?


other than that, I’m still combing through logs to try and find a signal that explains it


just found “Too many open files” warnings in the logs. I am using aws client lib in Ions. could this be some kind of leak?


pretty sure it’s a socket leak. probably due to aws client mis-use on my part. I wonder if there is a way to measure open “files” as a metric so I can monitor this


any advice you might have on how to find the leak would be much appreciate


another useful idea would be a way to cause cloud instances to be killed automatically when the “Too many open files” point is reached. suggestions for this would also be much appreciated


I will try and find the leak myself but suggestions are welcome


I’m assuming this is an aws client problem but is there any other reason that “Too many files” warnings can happen in cloud?


Some folks encountering this issue on the forums have been attempting to isolate by using a file leak detector on the box and isolating the box to a standalone query group.


I'd also put in that you have access to me with Datomic support. I'd be happy to help diagnose and troubleshoot. If you want to log a case shoot an e-mail to <mailto:[email protected]|[email protected]> or <mailto:[email protected]|[email protected]>


thanks @U1QJACBUM I’ve logged a ticket


I thought databases are values…

(let [db (get-db ...)]
      (d/as-of db #inst "2020")
      (d/as-of db #inst "2020")))
=> false


(let [conn (get-conn ...)
      db1 (d/db conn)
      db2 (d/db conn)]
  [(= db1 db2)
   (= (.-client db1) (.-client db2))
   (= (.-conn db1) (.-conn db2))
   (= (.-info db1) (.-info db2))])
=> [false true true true]


Which Datomic are you using? datomic-free, datomic-pro, client?


it confirms, although not explains the behavior


The point of db as a value is that data retrieval from it is (mostly) repeatable. I’m not sure what the value of db equality would be because what kind of equality you want depends on what you are doing. Eg suppose dbs could compare, what would you use that for?


maybe caching - if you asked for this value/query on the same db value, don't ask datomic we already know


Dbs have a unique id, basis-t, ishistory, and optional as-of-t, since-t, and filter. You could say that if these are all equal, the dbs are equal, but that means some “equivalent” dbs are not equal (eg if one uses as-of to travel back to the same t as another unfiltered later db)


i mean yeah - equals in the more general sense should return equals if all observable properties of a thing are definitely equal


Maybe caching, but how likely is it that you have an equal but not identical db in this caching context?


its more or less accepted that if you don't know if they are equal or not - or don't want to guarantee it - they aren't


but if all those things are equal you would know for sure they are equal so its odd


> how likely is it that you have an equal but not identical db in this caching context? lets say they query like graphql.


But the basis-t in a production db is constantly advancing


user_as_of(year: 2020) {

user_as_of(year: 2020) {
   friends {


you could get db/as-of 2020 for each of these


big caveat is that i'm stupidly new to this so i'm not sure what basis-t is exactly


Basis-t is the latest T the db contains


okay so if a db contains that the utility might go down somewhat


A db must necessarily contain that


i dunno how likely it is to do 2 (d/db ...) calls before a write happens


but it could be likely enough to warrant caching - idk


Depends on your system, but I’d say not likely enough to make caching useful


okay - but i think you'll agree that there is at least hypothetically a use


and datomic isn't widely or openly developed on so its kinda hard to get a feel for probabilities


Well I’ll put it this way, I’ve used datomic on-prem in production for over 5 years on massive 10bil datom+ databases. I think this equality makes sense in an abstract way, but I haven’t encountered any practical use for it. In practice, you’ll have execution contexts share an identical db anyway


also if you read the clojure docs through as to "what is a value" and then see this "value" be not-a-value it would throw you for a loop


I think there’s the opposite problem too, which is they may assume too much of db inequality or equality


Eg that equal dbs produce equal projections always (they may not due to non-determinism in how the projections are made) or that unequal dbs produce unequal values


> non-determinism in how the projections are made Wait, what


Eg pull expressions limits, calling impure functions in your queries, hash changes among versions, that sort of thing


Equal db guarantees that the datom sets you would get are the same, but most of what we do (query and pull) is projecting out of datoms and not part of the db per se, and the guarantees get weaker for all the usual reasons that (f x)=>y may not produce the exact same y for all time in all places


btw graphql and d/as-of is exactly the right context for my question


The as-of of a “fresh” db (the result of (d/db conn) ) is always nil.


and the basis-t of the db advances with each write


are you using an as-of db with graphql resolvers? My expectation is that a d/db is established in the resolver context at the beginning of the request and is the same through the entire request


I do “changelog” that returns a series of items from different as-of dbs


so no, in my case db is not in a context


I haven’t implemented it yet, but I think it might be possible for me to ensure identical as-of dbs for different items sharing the same tx…


Could you explicitly use T as a cache key? note the “effective T” of a database is (or (d/as-of-t db) (d/basis-t db)) iff since-t, isHistory, and filter are all unset and as-of-t >= basis-t


Strange there is no explanation for that, because there is even a talk by Rich — Database as a Value…

greg13:09:05 At the top of the file defining schema, authors made a note about "enum attributes" and "super-enum attributes". Sounds like a best practice. I read the definition posted there, in that file, but I can't get it. What is the difference between these two? Could you give me some examples?


In another repo, I've found two dataset files that refer to the same terminology: Looking at these files, it looks like there are only two differences: • super enums are just more numerous then simple enums • simple enum hold only name attribute while super-enum holds more of them Is that correct distinction between these two?


Super enums are "global" entities referenced by several types. For instance, :artist/country and :label/country can point to the same country entity. Each regular enums is only referenced by a single type, for instance artist/gender is only referenced by an artist.

👍 2

Hi, I know I can return a collection of values from a query with [?name …] , but what do I do if I have two such variables and want the concatenated result? Illustrated by this contrived example:

[:find [?friend-name ?sibling-name ...]  ;; Just one of the many alternatives I've tried
 :in $ ?person-id
 [?person :person-id ?person-id]
 [?person :friend ?friend]
 [?person :sibling ?sibling]
 [?friend :name ?friend-name]
 [?sibling :name ?sibling-name]]
In other words I’m trying to follow two many-relationships and return the same attribute from all those entities. Any help appreciated as I’m out of variations to try and I only seem to find documentation for the single variable variant.


:find ?friend-name ?sibling-name will return a set of [friend-name sibling-name] tuples, is that what you’re looking for?


Hmm, thanks, that was simple. Maybe I’ve been experimenting with bad data here. So, I will get a tuple of two collections that I can then concat afterwards, right?


you’ll get an array of tuples: [[f1 s1], [f1 s2], [f2 s2]... and so on


But all in all it will be the names of all the people that are either a sibling, friend or both of the person identified by person-id?


This will return all the tuples [friend-name sibling-name]


It's is maybe not what you want?


It’s always tricky to use a contrived example to try to illustrate. What I need is to follow two or more refs from an entity, where both refs are cardinality many. From the entities those links/edges leads to I need to extract an attribute.


What if friend and sibling both point to the same person? Do you want a query that excludes this possibility?


No, exclusion is not needed.


The refs essentially represents different reasons for targeting entities. Adding another example might make it worse, but here goes: A PC-technician gets the task to fix some computers. The task entity has refs to computer entities. Refs can be eg. :broken-down or :user-complaint, but all the tech needs is to get :serial from all the referenced computers to know which ones to work on. In this case a broken down computer could have a user complaint, but both leads to the same serial.


Could you use an or clause?

(d/q '[:find [?id ...]
       :in $ ?person
       (or [?person :person/sibling ?friend-or-sibling]
           [?person :person/friend ?friend-or-sibling])
       [?friend-or-sibling :person/id ?id]]
     db (d/entid pro-db [:person/id "1"]))


This will find those entities which satisifes either of the clauses (or both), and returns a vector of their value for some attribute (in this case :person/id )


Thanks, that looks like exactly what I was after. Reads better as well

👍 2