Fancy stuff like JSON queries and indexing may be difficult too. Same thing for triggers.

so, don't have 30 fields that are blank for 60 percent of the records?

that won't be a problem with Datomic

it might be in Postgres, in which case you will have one more argument for migrating 😉

truth.

interesting. So, the events coming in are json documents. I take it what you're saying is to avoid storing the events as JSON fields, but rather store them as proper records

like, dont have a table thats just an autoincrementing id and a json doc

it depends. Some portions of those documents may be just "raw data" that you never query against

in which case you would store them as strings or bytes in Datomic aniway

ah, I see.

Oh yes, also, you may have issues with ids that are unique at the table-level

you should make them unique globally if possible

interesting. Does datomic even have a concept of multiple tables?

or any kind of data segregation

other than schema

no, everything is an entity. I find it easier to think of the Datomic model as a graph

well, not even that. Everything is a Datom. Entities are a mental representation that is superimposed on Datoms

lol. If I could, I'd get you to write a whole blog post on how datomic is like a graph. The business side of my company got hold of the concept of graph based data models, and they sell it as part of the product pitch

If I could prepare a solid presentation on why datomic is a better fit for a graph model than relational, I'd be 80% the way there.

Basically, datoms are edges, and entities and scalars are the vertices. Datalog is pattern matching in the graph. Really, visually, it's literally what it does.

huh, interesting!

@tjtolton maybe my previous blog post can help 🙂 http://vvvvalvalval.github.io/posts/2016-07-24-datomic-web-app-a-practical-guide.html#business_logic

hadnt thought of it like that, but youre right

yessss, nice

The fact that querying is not remote lets you program graph traversal from your application code, in your favorite programming language

in that sense, the Entity objects are like cursors in the graph that is the DB.

Finally, regarding business logic, don't forget to mention Datalog rules (noun, not verb) as a means of business abstraction.

hahahaha

The last thing I would add is programmability. Generating Datalog is way easier than generating SQL, trust me.

its going to be a tough sell nomatter how I do it. Our senior data architects are SQL engineers through and through.

Tell them they can use Postgres as Datomic's storage service 😉

Hahahaha, I'm sure they'll tell me "we have bigger priorities than adding new things to our tech stack"

but hey, thats software

Tell them that of all their issues, having to learn a new tool is the only one that is guaranteed not to get worse

I might be able to spin that, yeah. Introducing it as a tool is the way to go.

I shouldn't say that I want to change the system, I just want to add a new tool.

I like that.

it depends how much you rely on stuff like ORMs I'd say.

So, you're probably going to double take when I say this, but that's not a problem, because our code base is all written in clojure so there are no objects.

Yes, that's right, I am in the strange position of fighting an uphill battle trying to sell datomic to a clojure shop.

strange indeed

Personally, Datomic was maybe the main reason I moved our stack to Clojure.

Clojure is pretty cool on its own, but Datomic gives it a lot of leverage.

Its a very strange thing. The company has used clojure since 2010

in production mind you

so a lot of the system was architected before some of clojure's interesting ecosystem had evolved around it

Well, one thing that is sure is that for them to add Datomic to the stack, the case for Datomic needs to be compelling, otherwise the additional complexity may just not be worth it. You should ask yourself this question honestly.

Maybe you can start with Postgresql, and watch out for situations where Datomic makes things much easier (N+1 problem, sparse data, graph-type queries, ...)

then importing the data into Datomic and demoing how it deals better with these issues should not be too hard, maybe a couple of days

I'll definitely be examining this question for a while before I try to make the sell

I'll be catching up on your blog 🙂

@tjtolton thanks 🙂 be critical!

Need to go to sleep now, but don't hesitate to let me know how it goes.

Thanks for the help @val_waeselynck! Goodnight.

working on a schema here, is it beneficial to have each entity type have its own uuid property? e.g. :user/uuid and :team/uuid vs plain :uuid? and what would be the benefit?

also for one-one relationships what’s the tradeoff in pulling everything into one entity vs splitting it into a (component) entity? e.g. a team has a set of credentials. so I might have the following properties: :team/id, :team/members, :team.creds/email, :team.creds/token. should those go into one entity or two separate ones?

@wei I’m no expert, but fwiw, I would use the same uuid attribute for entities that I expected to query across. So, using your example, if I thought I was going to do something like :where [?user-or-team :uuid ?uuid], I’d use the general one, but if it’s always like :where [?user :user/uuid ?uuid], then the more specific ones are fine.

For my own use case, I use a general attribute, with a semantically broad namespace

@wei for your second question, I would put all of those attributes in the same entity

I think components make the most sense for :cardinality/many references

When you pass a java.util.date into a query is it cast to a long so it can be compared against a :db/instant like so (> ?dbintant ?javautildate)?

@mbutler I believe they are compatible (= (type (java.util.Date.)) (type #inst “2016”)) => true

I find myself repeating (map (partial d/entity db)) a lot in my helper functions, e.g.

(defn all [db]
  (->> (d/q '[:find [?e ...]
              :where [?e :team/uuid]]
            db)
       (map (partial d/entity db))))

@wei Right, (> java.util.Date java.util.Date) is not valid clojure code as > needs integers. If you store a util date as a :db/instant it becomes a long so you can use < on it. However it also seems if you pass a util date into a query it is coerced into a long. Wanted to check I wasn’t incorrectly making a huge presumption.

@mbutler also found this old thread https://groups.google.com/forum/#!topic/datomic/iWFKMItXWkM

(> :db/instant java.util.Date) inside a query works which is super interesting and how I’ve been doing it 😄

Hi, Followed instructions using heroku datomic build as specified here https://elements.heroku.com/buildpacks/opengrail/heroku-buildpack-datomic but i keep getting error stating

Device "eth1" does not exist. 2016-12-14T18:56:43.386579+00:00 app[datomic.1]: sed: can't read /app/scripts/transactor.properties: No such file or directory 2016-12-14T18:56:43.398163+00:00 app[datomic.1]: Picked up JAVA_TOOL_OPTIONS: -Djava.rmi.server.useCodebaseOnly=true 2016-12-14T18:56:43.389335+00:00 app[datomic.1]: Launching with Java options -server -Xms256m -Xmx2g -Ddatomic.printConnectionInfo=false 2016-12-14T18:56:47.326896+00:00 app[datomic.1]: Critical failure, cannot continue: Error starting transactor 2016-12-14T18:56:47.327568+00:00 app[datomic.1]: java.lang.Exception: 'protocol' property not set 2016-12-14T18:56:47.327669+00:00 app[datomic.1]: at datomic.transactor$ensure_args.invokeSt

Any ideas? i have heroku postgress up and running

echoing in case anyone has a good answer:

could use the pull api inside the query, but that assumes you know the shape of the data you want up front and don’t want to leverage the cursor-like nature of entities

@wei, basically, no. 🙂

@wei well, that’s not strictly true. i wrote some transducer backed helpers:

(defn to-entity-for-db [db]
  (partial d/entity db))

(defn ids-as-entities
  ;; transducer
  ([db] (map (to-entity-for-db db)))
  ;; actually do the work
  ([db ids] (sequence (ids-as-entities db) ids)))

(defn datoms-as-entities
  ;; transducer
  ([db] (comp (map :e) (ids-as-entities db)))
  ;; actually do the work
  ([db datoms] (sequence (datoms-as-entities db) datoms)))

so your example would change to (ids-as-entities db (d/q '[...] db))

you can see i have one for use with d/datoms too

@djjolicoeur @robert-stuttaford ah, thanks for the ideas

@robert-stuttaford i’m finding your helpers useful, do you have a lib or gist of any more useful ones?

@wei https://gist.github.com/robert-stuttaford/39d43c011e498542bcf8

i’m toying with the idea of releasing a library with this stuff 🙂

thanks for sharing! I’ve also tried building helper layers in various forms, but am now of the opinion that small convenience functions work better than a substantial wrapper library. eventually I run into cases where those wrappers fail and I need to drop down into base api

the trick is not to wrap, i think, but to make useful shortcuts to compose the base api with

if you try to hide the base api away, you’re losing (i have this t-shirt 🙈 )