This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-06-09
Channels
- # aleph (4)
- # arachne (3)
- # beginners (41)
- # boot (300)
- # cider (17)
- # cljs-dev (37)
- # cljsjs (4)
- # cljsrn (5)
- # clojure (249)
- # clojure-boston (3)
- # clojure-czech (4)
- # clojure-dev (14)
- # clojure-greece (183)
- # clojure-nl (2)
- # clojure-russia (11)
- # clojure-spec (135)
- # clojure-uk (37)
- # clojurescript (56)
- # community-development (8)
- # cursive (22)
- # data-science (4)
- # datomic (150)
- # devcards (6)
- # emacs (5)
- # euroclojure (8)
- # funcool (18)
- # hoplon (29)
- # immutant (1)
- # jobs (1)
- # lambdaisland (3)
- # lein-figwheel (7)
- # leiningen (18)
- # mount (1)
- # om (81)
- # onyx (95)
- # planck (50)
- # proton (6)
- # re-frame (62)
- # reagent (2)
- # ring (1)
- # robots (1)
- # spacemacs (2)
- # specter (88)
- # test-check (32)
- # untangled (23)
- # yada (1)
just a initial commit, and by far not finished. But would like some input: https://github.com/peterromfeldhk/datomic-schema
btw throw exceptions and lazyiness is a bit painfull
how to be able to still catch ex-data, without dropping lazyness? is it possible?
(i was working as sys admin and devops before, so this is my first lib since i entered developent! Im super thankful for any input and help!)
need to make it more clear what i mean without constraints compared to other libs with example, but too tired now. will do over weekend
its mostly about that a entity is not a table, and you might be open that certain attribute could belong to one or another entity
github example
a user can have repos
but also an org can have repos
but when you lookup you might dont want to care if its a user or org entity, but you still wanna know the associated repos
so you have a entid(user or org) and want to get repos for it, without caring if its a user or org entity
you could still use or, but why constrain yourself?
peterromfeld: one comment: why use refs for enums? we're moving over to keywords because they're much easier to work with.
a enum is just a ref to a ident
we had start problems with datomic because we used ADI at first
and almost all devs got missleaded how datomic schema works
peterromfeld: i know. yet, as there is no type checking anyway and a ref attribute can take any value, we don't see why one not want to use keywords instead.
i dont know the difference of ref datoms to keyword datoms
but a ref to the same ident may be more effective then a new keyword for every entity
i dont know whats the difference between a ref to same datom, and a new datom for every keyword for a entity
in the end it may be the same, you have a ref datom or a keyword datom
but i read that ident enums more effective then keyword datoms
anyway, thank you for sharing your library. i hope that at some point, there will be a unified schema syntax for datomic
its not a lib yet, didnt published it yet 🙂
i would love that we could build it out a bit together
but i have to say, i have to deal much with afterdamage done from ADI
many still dont understand datomic
and its was very bad discussion to use for initial learning
better go forward a bit slower, but at least you understand what you working with
now we need to refactor soo much..
anyway, i would love if we could build out my initial idea (which is most likly very similiar to many others)
we've kind of succumbed to not creating any abstraction layer over datomic and letting it bleed into our application code in various ways.
you can still have a ds namespace which takes care of it
and if you decide to change you only need to change there and in subns
always keep ds stuff in one place! you never know if you wanna change... easier to refactor in one ns + subns then all over the place
even with good IDE
you dont know, if/when you leave that next people wanna move forward
you always develop legacy of tomorrow
and needto make it easy to throw away
sry im a bit drunk and too talk active 😉
anyways if you have inputs, just put it into issues 🙂 i welcome every input
why can some attributes be looked up and not others? (d/attribute (d/db conn) :db/add) => nil (d/attribute (d/db conn) :db/unique) => 42
@danielstockton: :db/add
is not an attribute. It is a special form in transaction data.
it looks like it is an attribute, at least there is an entity with this ident: 1 ([:db/ident :db/add] [:db/doc "Primitive assertion. All transactions eventually reduce to a collection of primitive assertions and retractions of facts, e.g. [:db/add fred :age 42]."]),
with entityid 1
i also wonder what 'installing' an attribute really does and why it isn't allowed to just use anything with a :db/ident as an attribute
(d/entid (d/db conn) :db/add) => 1
there seems to be something special that makes something an attribute but i don't really see what that is
@jannis: This blog post has an example of querying provenance data https://www.jayway.com/2012/06/27/finding-out-who-changed-what-with-datomic/
This example also queries for the transaction entity to identify a user who changed a value: https://github.com/Datomic/day-of-datomic/blob/master/tutorial/provenance.clj
@marshall: I'm specifically interested in this case: http://docs.datomic.com/transactions.html#sec-5-3
Where you have the db after and the datoms from the transaction and want to query the db starting with one or more matching datoms in the tx data.
Unfortunately, with the input datoms you don't seem to be able to use [?e :some/attr ?v2]
to match attributes. You have to do something like [?a :db/ident :some/attr]
to match the attribute separately. I guess I don't understand why that is yet.
@jannis: if you look at the datoms that are returned, they are things like:
[#datom[13194139534313 50 #inst "2016-06-09T14:13:57.997-00:00" 13194139534313 true]
where the attribute is a number (50 in this case)
That is the entity ID of the attribute definition in the schemaI suppose the reason is that the ?a
s are IDs of attribute entities in the db partition and not keywords.
I wonder what the most elegant way to query just a particular transaction is if one already has the tx-data available, except in a slightly less useful form.
so providing the database as a source allows you to look up the entity ID of the
?a
and resolve it to the :db/ident
keywordYou might want to look at the Log API if you’re looking for info about a specific transaction: http://docs.datomic.com/log.html
Well, I already have the info by receiving the transaction data from the log (via onyx-datomic).
When I do [?a :db/ident :command/data]
then ?v
will give me the ID of the entity that the :command/data
attribute points to in the transaction. Now my only remaining problem is how to query information about that entity. [?v :data/some-attr ?v2]
should give me - in ?v2
the value of :data/some-attr
in the entity ?v
, right?
I'm not an experienced user of datomic terminology, so I hope this description isn't too weird.
And that's what isn't working. It's almost as if it can't match the input datoms and the db in the same rule.
?v
comes from the tx datoms, just like in the example on http://docs.datomic.com/transactions.html#sec-5-3
ah, ok. Then in that case it would only be an entity ID if the added datom was a reference
QUERY [:find ?v :in $ [[?e ?a ?v ?tx ?added]] :where [?a :db/ident :command/data]]
> #object[java.util.HashSet 0x2071c184 [[285873023223181]]]
So ?v
is the entity ID of the entity refered to by the matching [?e :command/data ?v]
datom in my tx data.
Now what I really want though is query an attribute of that entity, e.g. by adding a rule like [?v :data/some-attr ?v2]
and than return ?v2
.
@jannis to your earlier question, you should be able to do [?e :some/attr ?v2]
to match attributes but it doesn't bind it to anything. If you want to bind it, you need the ?a symbol and it gets bound to the entityid of the attribute so you then need to join that with :db/ident to the actual keyword value
QUERY [:find ?v2 :in $ [[?e ?a ?v ?tx ?added]] :where [?a :db/ident :command/data] [?v :project.create/project ?v2]]
> #object[java.util.HashSet 0x7008a6b8 []]
vs. the following query against the db directly (which works):
boot.user => (datomic.api/q '[:find ?v2 :in $ :where [?c :command/data ?d] [?d :project.create/project ?v2]] (datomic.api/db (:conn (:datomic system))))
#{[1]}
you need to resolve the keyword attribute ID to an entity ID if you’re going to query it that way
since all query happens in process in Datomic there’s not the traditional client/server type of need to jam all the logic into a single query
you can run the first query to get the entity IDs of the intermediate entites, then a second query to populate the values from those
especially if it’s data that was just transacted, that will all be in the peer’s local cache
However, since it's such a simple chain of matching rules with only two levels of reference, this should easily be doable in a single query I'm thinking.
Odd. A separate query against db-after, after extracing the entity ID of the first reference, fails:
VALUE FOR ?d 285873023223262
QUERY [:find (pull ?e [*]) . :in $ ?d :where [?d :project.create/project ?e]]
> nil
But running the query against the latest db of the connection separately works:
boot.user => (datomic.api/q '[:find (pull ?e [*]) . :in $ ?d :where [?d :project.create/project ?e]] (datomic.api/db (:conn (:datomic system))) 285873023223262)
{:db/id 1, :db/ident :db/add, :db/doc "Primitive assertion. All transactions eventually reduce to a collection of primitive assertions and retractions of facts, e.g. [:db/add fred :age 42].", :project/name "New project 1"}
The onyx-datomic inject-db-calls
lifecycle helper must be injecting an old version of the db into the function that runs the first query (basis-t = 1095 vs. 1503 in the second case).
No wonder my query over db + tx datoms couldn't resolve references any further - they are probably missing from that old db.
There we go. Using inject-conn-calls
instead and then using the latest db for the queries makes it all work.
@marshall: I still can't get the single query to work but with the two separate queries you recommended it works.
@jannis: I usually prefer the single query approach, but you could also do a nested query if necessary: https://groups.google.com/d/msg/datomic/5849yVrza2M/31--4xcdxOMJ
not sure if this is the place to ask for help but i am having some trouble with connecting to transactor from a remote peer
it can happen when you're specifying a host name in the transactor.properties file which is not resolvable by clients.
@achan: Yes, the hostname value in the properties file should be something that the peers can resolve to the transactor machine.
(def uri "datomic:<sql://merckx?jdbc:postgresql://gmahg0ppav8v6d.cbv0gfpvcvqy.us-west-2.rds.amazonaws.com:5432/datomic?user=datomic&password=datomic0%22|sql://merckx?jdbc:postgresql://gmahg0ppav8v6d.cbv0gfpvcvqy.us-west-2.rds.amazonaws.com:5432/datomic?user=datomic&password=datomic0">)
got error: HornetQNotConnectedException HQ119007: Cannot connect to server(s). Tried with all available servers. org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory (ServerLocatorImpl.java:906)
how my peer know where the transactor is when in my uri for datomic specifies only RDS endpoint?
The peer reads the transactor location from storage. http://docs.datomic.com/deployment.html#getting-connected
Do you have a transactor up and running against that URI and does the machine you’re attempting to connect from have proper AWS credentials to access that RDS table?
and, as @hans mentioned, the address of the transactor needs to be correct in the transactor properties file.
One thing is after i built the EC2. It couldn't resolve the hostname. So i had to add "ip-10-16-25-24" to /etc/hosts file
host needs to be something that your remote peer can use to connect to the transactor
also, if you’re using the provided CF template and you’re trying to connect from a non-EC2 instance, you need to set
aws-ingress-cidrs=0.0.0.0/0
then you need to set the value of host to something you can reach from the client machine
@achan: you should be able to set inbound rules from the EC2 instance security group settings: http://docs.datomic.com/aws.html#aws-management-console