Fork me on GitHub
#datomic
<
2015-12-01
>
Ben Kamphaus15:12:31

@robert-stuttaford: re: the backup docs, ‘arbitrary’ means any t that exists in the database, which you can’t do. You can only use the t database values that have been specifically backed up - you can see which are there with list-backups.

robert-stuttaford15:12:04

thanks ben, we since determined that when greywolve found the list-backups command. thanks for answering even so!

Ben Kamphaus15:12:37

@domkm: not sure which exactly which api endpoint or use case you’re discussing, or what you mean by the referenced entity. Do you want a map of all attr/val pairs, the number, the reified entity that you get from e.g. datomic.api/entity? Do you want this returned by query, or any time you have a value in hand? You may be able to do what you want with pull or a query, as it seems like you might not like the default behavior in the entity api (trying to infer context from your problem description).

Ben Kamphaus15:12:41

@a.espolov: the generic reading of this error is that the peer can’t connect to the transactor. This could be a number of reasons: the peer doesn’t have permission to connect to the transactor (i.e. configuring open ports, etc. on AWS or wherever the transactor is), the transactor’s host isn’t specified correctly (or alt-host if e.g. it’s containerized), etc.

Ben Kamphaus15:12:21

@dmi3y: the console error you encountered is due to an incompatibiltiy b/t bundled version of console and Datomic 0.9.5327, discussed on this thread: https://groups.google.com/forum/#!topic/datomic/4BwCDxs6zKw — the next release will address this. In the mean time, the suggestion @dobladez passed along to download the most recent stand alone console is the recommended workaround.

dmi3y15:12:31

got it, thanks for clarification @bkamphaus

domkm17:12:34

@bkamphaus: Sorry for not being clear. I think your inference was correct. I want to use the entity API to walk entity relationships, as it's intended for. The automatic replacement of ident entities with keywords makes this difficult. I was asking if there is a way to turn off this automatic entity->keyword replacement.

domkm17:12:09

@bkamphaus: I suspected the answer was "no," so I wrote my own EntityMap to resolve it. It's working well except for one problem: I can't get Object#equals and IPersistentCollection#equiv to work correctly with the default datomic.query.EntityMap. It looks like the default EntityMap implements equals by first checking that the other object is an instance of datomic.query.EntityMap instead of checking that it implements datomic.Entity. This surprised me since Clojure favors programming to interfaces instead of concrete types. What do you think of changing the default EntityMap equals to check for the public datomic.Entity interface?

Ben Kamphaus17:12:51

@domkm: do you need laziness a la entity? If not, I would probably use [pull](http://docs.datomic.com/pull.html) and deal with the entity as a map (just the data) directly if the entity api’s ident behavior is not what you want (usually we have the opposite complaint for pull, as it does not return keywords). You could also build your own behavior for entities or ways of traversing or reifying data with pull. I will say that it's not really typical to use :db/ident for entities that represent e.g. nodes in a graph with either attributes or further connections that are meaningful. Can you describe this aspect of your data model a little more?

domkm17:12:28

@bkamphaus: Sure. I built a very basic type system and I'm using idents to specify entity types (not to be confused with value types). I want to walk the types to discover supertypes, attributes, etc. My EntityMap wrapper works perfectly for this except that equals is broken because datomic.query.EntityMap does a class check instead of an interface check.

domkm17:12:12

@bkamphaus: In terms of pull returning maps instead of ident keywords, I think that it makes the most sense for the behavior of the entity and pull APIs to be opposite of what they currently are, since the entity API is commonly used for walking references, or for everything to be like the pull api (entities with idents treated the same as all other entities). Ideally, All APIs would be configurable so users could choose the behavior that best suits their use.

jdubie17:12:51

Question about using d/db function vs using db-after. example is in this gist https://gist.github.com/jdubie/e7682a9c5cf7d5ecb60f

jdubie17:12:57

basically if you transact on a connection then run (d/db conn). will the resulting db always include the transaction?

Ben Kamphaus18:12:07

@jdubie: if you block until the transaction returns the db pulled from the conn should be at or later than the database value/t after the transaction, but, it may also include other things done to the database by transactions submitted by other peers (or even just threads) in the mean time. So the difference you should assess can be pointed the other direction - i.e., say it’s a deposit reporting the new balance back to a user. Do you want the balance immediately after the deposit, or the balance 100 milliseconds after the deposit, during which time either no or a small number of debits may have occurred?

jdubie18:12:06

awesome - that make sense. thanks @bkamphaus !

jdubie18:12:23

datomic rules - i’ve really enjoyed using it

Ben Kamphaus18:12:00

@domkm: I’m thinking on your use case. I understand what you’re running into and why you’re using idents now, and why you’re hitting the issues (traversing the graph implied by the type hierarchy, etc.) I’ll probably discuss more here and get back to you. I do also understand the complaint re: the class vs. interface check and how it’s impacting the way you’re approaching the problem at present.

currentoor20:12:06

Hello, I've heard DynamoDB is the recommended storage engine for ease of use and maintainability. Is this true?

tcrayford20:12:02

@currentoor: it's recommended you use whichever storage engine you are more familiar or comfortable with, but if there is no familiarity with any of the existing engines, dynamo is probably the one that's easiest

tcrayford20:12:15

it may be somewhat pricier than other storage engines though

currentoor21:12:51

What about performance constraints? Specifically, I'm worried about scaling postgres. Any quirks here?

tcrayford21:12:17

it depends, as with all scaling things. If you're actually worried about that, I'd spend some time doing performance testing and simulated load of the system

tcrayford21:12:58

dynamodb will likely at the very least have better availability than postgres, but if you know postgres well and don't know dynamodb well, then likely using postgres as the storage service will work better for you

currentoor21:12:23

but is it true that in postgres it stores all of datomic in one table?

currentoor21:12:36

wouldn't that be problematic?

Lambda/Sierra21:12:18

If you are running your app on AWS, DynamoDB will be the easiest and most scalable storage option.

Lambda/Sierra21:12:24

Datomic storage on SQL is supported largely for the benefit of organizations which already have substantial investment in infrastructure supporting SQL.

currentoor21:12:45

sounds good, thanks!

Ben Kamphaus23:12:33

@domkm: this topic has come up for a few people on our side as well, and the general consensus is the simplest possible solution is just to not use :db/ident for the name of the type system (using a unique string or something else as a unique/identity so you can use lookup refs). If you really want to stick with idents, pull is the way to go, you can make use of [nesting](http://docs.datomic.com/pull.html#nesting) and [recursion](http://docs.datomic.com/pull.html#recursive-specifications) to navigate between type entities. Recursion is fairly straight forward for traversal if you have consistent attributes for each type entity node - i.e., get the parent of the parent of the parent, (up to all parents), etc.

Ben Kamphaus23:12:43

I’d opt to not use idents for your types, personally. The point of idents is pretty much be substitutable for the entity. I.e., with enums where all you ever care about the entity when ref’d is its ident/identity.

domkm23:12:55

@bkamphaus: Thanks for bringing this up with your team. Using idents hasn't caused any problems for me except for the built-in EntityMap equality check. I should also add that another major motivating factor for me is that I want my code to be compatible with DataScript, which, unfortunately, lacks idents. Was the equals change from datomic.query.EntityMap to datomic.Entity discussed? If so and if it was rejected, may I ask why?