Fork me on GitHub
#datomic
<
2019-02-20
>
Brian Abbott00:02:11

Hi! I just wanted to report a small defect in the Datomic Documentation. On this page: https://docs.datomic.com/on-prem/getting-started/query-the-data.html - The link about 1/3 of the way down, titled "Seeing the History of the Data" points here: https://docs.datomic.com/on-prem/getting-started/seeing-the-history-of-the-data.html which, gives a 404 error. Maybe we could correct that, it would be nice to have the link!! 🙂

Brian Abbott00:02:24

At the very bottom is perhaps the correct link for the earlier one which is: https://docs.datomic.com/on-prem/getting-started/see-historic-data.html

Brian Abbott00:02:25

Has anyone considered authoring a book on Datomic? Conversely, would anyone be interested in reading one?

Brian Abbott00:02:38

Has Cognitect considered Open Sourcing the Datomics (prem, cloud, Ions)? I think it would go a long way to expediting adoption within the industry.

Brian Abbott00:02:13

Is there any path toward viewing/accessing the source outside of regular Open Source channels?

Joe Lane01:02:07

I’ve considered authoring a guerilla guide to ions and aws. I would be interested in reading one. I HIGHLY doubt cognitect will opensource datomic since it’s their product.

đź‘Ť 10
dpsutton02:02:53

> To those who think that Datomic ought to be open source: We don't see a viable economic model there. If you think otherwise, come up with the money to buy the IP and make a go of it. If you can't, then recognize your arguments for the hot air of entitlement they are. https://www.reddit.com/r/Clojure/comments/73yznc/on_whose_authority/do1olag/

dpsutton02:02:18

There's a bit of pepper in the response based on the circumstances that elicited the response. But the idea is there are mortgages to pay and mouths to feed. If you can point them towards a better model that will do these things well I believe they are all ears.

Brian Abbott07:02:44

Thank you for that link @dpsutton, I really appreciate it! As we talked in DM, it misses the motivation behind the suggestion. Its really not to make another step in some kind of idealistic march toward a kind of software socialism - most definitely not - personally, I march with Adam Smith. The reasoning is to produce an uptick in adoption and acceptance in Datomic. Many of the lists, blogs, conversations in the dev community focus on or around the Open Source Data Platforms with the OS part of it being merely the token cost only for entrance into the room - its funny whenever you ask most devs to cruise the source of a project with you to find an answer to a programming question - the responses make you wonder how OS ever happened in the first place - or highlight the spread we have in our minds between our words and our actions... Nevertheless, I feel that in order to make Datomic (semi-) mainstream (which would be nice for those banking on it) I feel that two things would need to happen - 1) the OS suggestion or, at least just part of the core engine - that pays that token cost, it gets to be written about, considered outside of the clojure comm., etc.. and 2) 1st tier support for client-apis for all major programming languages. If those two things happened, I believe that Datomic would see far greater adoption quite rapidly.

Chris08:02:46

@briancabbott much as I’m with you on the goal of promoting Datomic, I don’t think 1st tier support for other langs would pay off. Are there that many prospective Datomic users who aren’t interested in using Clojure or can’t change their client language?

Brian Abbott08:02:54

Yes. 🙂

Brian Abbott08:02:14

Its the key to being a first tier DB

Brian Abbott08:02:46

and really... JDBC (which we have) and, ODBC gets you a long way there but, having a totally supported native client API for every major platform opens the doors with the DB and Dev guys. OS opens the doors for the community. Then its what we make of it but, ATM we're out in the cold.

Chris09:02:25

Fair enough - it’s not been my experience, but I can only guess at what’s normal. And it’s true that Ops teams would prefer a client they can use more easily. (Telling Ops about Datalog reminds me a bit of this webcomic, actually, please excuse the profanity: http://howfuckedismydatabase.com/nosql/)

okocim15:02:56

is there any way to start a “system” inside the ec2 instances of a query group after deployment? I’m trying to reliably start some SQS message listeners without having to invoke a lambda from outside the system.

pvillegas1216:02:35

Looking at schema design https://docs.datomic.com/cloud/best.html#group-related-attributes, it looks like the best practice is to namespace attributes. However, is it not better to leverage the universal schema (specially with spec in mind)? Why would you prefer :release/name, :artist/name, and :movie/name, over something generic like :model/name where you would define one spec instead of 3 different ones? Want to hear the tradeoffs between both approaches (1. Namespace all attributes pertaining to an entity in your business domain (explosion of attributes), 2. Leverage the universal schema to have shared attributes through your entities)

ro618:02:47

I thought about this tension a lot when I came on board with Datomic and Spec. I think the wise choice is to use small, specific, sufficiently namespaced names for stuff in durable storage. There are mechanisms available for adding and changing abstractions over time, but un-abstracting (ie separating things you once tried to treat as functionally equal) is much harder, especially when your abstractions are reified in the data model. I think this topic deserves a book. The stuff that helped me sort it out was Zach Tellman's writing about abstraction in "Elements of Clojure" and "Data and Reality" by William Kent. Also, Rich's talks about Spec and namespaces make it pretty clear that the idea is to nail down enduring semantics and meaning at the attribute level. By being overly broad, something like :entity/name actually gives you less useful information.

osi19:02:04

“Data and Reality” sounds neat, thanks for the pointer @U8LN9KT2N

johnj16:02:45

I don't know spec yet but I do try to use generic attributes where I can, but curious how would you use spec here?

johnj16:02:11

spect would check the entity you are referring to? like, this model is release, artist, etc...

Dustin Getz16:02:13

:facebook/person-name has different validation rules than :linkedin/person-name. If they have the same semantics and validation rules, then use the same attribute. That way they can also share code that implements some semantics. Another example is :commonmark/markdown vs :github/markdown

Dustin Getz17:02:10

I think attributes that originate from the same system (which is pretty much most attributes) generally benefit from sharing semantics. For example, :amazon/product-title is probably useful on both books and electronics, but :amazon/isbn is not. If you’re just tagging stuff with a string for display to the user, i see benefit from having the same attribute, and see pointless cost from differing it. In the future you can & will change this stuff anyway

pvillegas1217:02:51

A downside I see from going universal schema with all attributes is as follows: I created a company entity which had address information and business information. All of these attributes are shared so they don’t have the :company/ prefix. When I’m going to query for all companies in datalog I have no way of specifying an attribute which is exclusively company based.

pvillegas1217:02:21

Another area of interest is that of refs. If I have an attribute that references let’s say an invoice, why would I not always want an :invoiceable/invoice attribute vs a :company/invoice or :inventory/invoice?

benoit18:02:43

@pvillegas12 It's a hard question and very much related to what computation your system makes in my opinion. It's about the abstractions in your system and how you decompose the computation. Choosing whether you share a set of attributes across entity types is a bit like choosing to define a clojure protocol that will be implemented by different types. To take the invoice example, there is no reason to multiple attributes like :company/invoice, :person/invoice... Or if you have the inverse relationship, you would have :invoice/company and :invoice/person. But the recipient of an invoice should be abstracted (maybe with something like :invoice/recipient) and then you can refer to any type of entity with it.

ro618:02:47

I thought about this tension a lot when I came on board with Datomic and Spec. I think the wise choice is to use small, specific, sufficiently namespaced names for stuff in durable storage. There are mechanisms available for adding and changing abstractions over time, but un-abstracting (ie separating things you once tried to treat as functionally equal) is much harder, especially when your abstractions are reified in the data model. I think this topic deserves a book. The stuff that helped me sort it out was Zach Tellman's writing about abstraction in "Elements of Clojure" and "Data and Reality" by William Kent. Also, Rich's talks about Spec and namespaces make it pretty clear that the idea is to nail down enduring semantics and meaning at the attribute level. By being overly broad, something like :entity/name actually gives you less useful information.

benoit19:02:26

I think it helps to think in terms of relationships and protocols rather than types.

ncg19:02:30

I would also recommend staying specific. Rules can help with implementing something like a generic notion of user, sourced from various specific user attributes in different namespaces.

ncg19:02:45

Attributes are the granularity at which you can define all your important semantics. And as ro6 says, generalizing later is easier than specializing later (an advantage of the universal relation over tables in the first place).

ncg19:02:23

(there are also indexing benefits to using specific namespaces, but that's an implementation detail)

Dustin Getz19:02:32

@pvillegas12 a company-specific entity needs only one company-specific attribute, it doesnt have to be all of them

pvillegas1219:02:45

@dustingetz in this case all attributes are shared!

lilactown19:02:57

some things might have similar attributes but semantically they are different

benoit19:02:03

I don't think it makes sense to go all shared or all type-specific. If you identified abstractions in your system then share attributes for those. If you haven't then don't share attributes. If you have 10 entity types sharing all the same attributes but you still need to be able to distinguish each type, nothing prevent you to have an attribute like :entity/type to indicate the type. But most often some entity types will be involved in relationships and others won't so you often have different attribute namespaces.

lilactown19:02:20

e.g. I would differentiate between a :person/name and a :company/name, because they are semantically different things

Dustin Getz19:02:12

Consider also :company/address vs :natzip4/address (http://www.zipinfo.com/products/natzip4/natzip4.htm) – natzip4 is far more semantic and yet flexible enough to be decoupled from company

benoit19:02:46

@lilactown It all depends on your system. If you manage invoice and you don't care wether your invoice is for a person or company then they might share the same attribute. It is hard to have discussions like this without actual system requirements 🙂

lilactown19:02:51

right. I think a safe default is to be as specific as possible

lilactown19:02:34

it's easy to assoc a new attribute to an entity that's more general. it's harder to sort your data and make it more specific after the fact

benoit19:02:28

I hear that advice often but that's not my experience. I encountered more systems fail because of an explosion of complexity due to special cases rather than a bad design. I still think it's better to have a bad plan than no plan at all.

benoit20:02:42

That said I have also seen systems where a rigid type system was put on top of datomic and prevented this kind of mix and match of namespaced attributes. Each entity could be of only one type. That made me sad. So you can definitely shoot yourself completely in the foot with bad abstractions. Especially if you artificially restrict power for no good reasons.