This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-01-22
Channels
- # beginners (10)
- # boot (134)
- # cider (29)
- # clara (2)
- # cljs-dev (10)
- # cljsjs (2)
- # cljsrn (13)
- # clojure (76)
- # clojure-india (4)
- # clojure-ireland (1)
- # clojure-russia (20)
- # clojure-spec (11)
- # clojure-uk (7)
- # clojurescript (88)
- # core-async (5)
- # core-logic (3)
- # cursive (17)
- # datascript (5)
- # datomic (123)
- # hoplon (8)
- # klipse (6)
- # lambdaisland (2)
- # leiningen (4)
- # luminus (2)
- # off-topic (23)
- # om (23)
- # om-next (1)
- # onyx (20)
- # pedestal (2)
- # perun (2)
- # re-frame (11)
- # ring-swagger (3)
- # untangled (12)
@dominicm interesting. do you know where I can find some info about it?
Wow. What are the use cases for this kind of thing?
Hi, I have a conceptual question. When I want to change an attribute, why do I have to db/retract
and then db/add
? Why isn't there a db/set
?
you don't have to retract for :db.cardinality/one
attribute
if you add a new value, the old value is automatically retracted
Anyone have any idea what would be the most efficient way to return, given a cardinality many attribute, only those entities for which there is actually more than one value?
The best I can think of is something like:
(d/q '[:find ?a ?b
:where
[?a :some/attribute ?b]
[?a :some/attribute ?c]
[(not= ?b ?c)]]
db)
Is there anything better?use (count)
and filter out those a count less than two?
When I try to run ensure-transactor
with an aws ddb configuration like this datomic ensure-transactor transactor.properties ensured-transactor.properties
, I get a bizarre error:
> java.io.FileNotFoundException: transactor.properties (No such file or directory)
I even opened permissions on the config file all the way up in case for some reason this was a permissions problem.
No cigar.
@ezmiller77 you need to run it from the root of the Datomic distribution. So something like bin/datomic ensure-transactor config/my.properties
Huh, interesting. That connects to a larger confusion i Have about where datomic is supposed to sit in my system.
So when running datomic commands they need to be run from within the root directory of the unzipped package?
Does that include commands that run, say, a datomic transactor?
Transactor is usually started via bin/transactor -yourJVMFlags path/to/properties.file
Do most people place a unique instance of datomic then directly in their project path?
None of this is strictly required, bit the included startup scripts that do things like set your classpath and such more or less assume that is what you're doing
Not really. You only need the distribution for running the transactor. You use the peer or client dependency via maven or lein or whatever for your actual app
Huh. Okay. I’m still trying to understand all these difference “services” if that’s the right word.
I’m just trying to build out a little project that would serve “notes”, like research notes, from a datomic server. Can you suggest which service I should try to use?
I was under the impression that in order to setup a peer server that used the file system locally, I need to configure a transactor, and then it seemed necessary to configure a storage service as well…
If you're running it all locally you can use the dev storage protocol for local disk storage
“clients” here refers to what exactly?
Right okay. Saw that doc.
Reviewing this might help too http://docs.datomic.com/architecture.html
Okay thanks.
@ezmiller77 regarding your tech choice question: I would advise you not to expect too much of the 'time travel' features of Datomic as a tool for versioning. In my experience, you usually don't want to query history in your application code, because you have very little control over history (example: imagine you want to import several dated revisions of a document at once. You could not do that if the way you store revisions is using Datomic history).
If I were you, I'd just store all the revisions on cheap storage like s3. If you really need to save space, you could store a base document along with a list of edits, and recombine them using a diffing library.
In summary: there are tons of very good reasons for choosing Datomic (one of them being that it's fundamentally more sound than most alternatives) but your versioning use case isn't one IMHO.
@zalky if you're after efficiency, you'll probably want to perform an AEVT traversal instead of a Datalog query
(->> (d/datoms db :aevt :my/attribute) seq (partition-by :e) (filter #(> (count %) 1)) (map #(-> % first :e)))
(Have not tried this code, but you get the idea)
@val_waeselynck I less interested in loading multiple version of a document at once, than I am in having access to those versions, and perhaps also in the diff that I think datomic provides…
I’m interested, though, in what you mean by one having “very little control over history”. Is it not possible to query the revision history of an entity?
In any case, what’s proving most frustrating about datomic is the challenge of setup and the inscrutability of the documentation.
I’ve been trying to get a setup working all day with very little progress. Quite frustrating.
Datomic does not give you diffs. You provide writes, and it gives you an aggregated view of those writes. It's the opposite from Git.
@val_waeselynck huh. Well I hope to have a sense of what you mean when I can get it running.
@ezmiller77 what kind of setup are you aiming for exactly? Setting up Datomic for production can be a bit of a hassle, but setting up for development should not be too hard
You can even start coding with just the in-memory storage
Yeah, the only thing I’ve been able to get running is the in memory peer-server.
I think the main thing that may be confusing me is the variety of services and combinations thereof, combined with a confusion about what a basic (even dev) project setup might look like in, say, leinigen with clojure (which is what I’m working with).
What I think I understand is that one runs some sort of datomic database using a peer-server or a client, and that process runs outside the project. Then, depending on what kind of db process one runs, you connect to it/interface with it in different ways.
@val_waeselynck , that's an interesting approach, I've never really tried traversing the index directly that way. I will give that a shot thanks!
I tried to get a dynamo-db local service running by: 1) installing dynamodb-local, 2) running ensure-transactor on a properties file setup for ddb-local, and then 3) running a transactor on that properties file. But then I started getting an error “Unable to load license key”.
That’s as far as I was able to get, and had to waste a bunch of time just to get there because it wasn’t apparent that the ensure-transactor needed to be run in the datomic root.
@ezmiller77 for dev work and exploration I'd definitely suggest using a dev transactor.
Put your starter license key in the dev transactor properties sample included under config/samples and start the transactor with something like bin/transactor config/samples/dev.properties
Then you can connect via peer with a uri like: datomic:<dev://localhost:4334/mydbname>
You could also start a peer server against the dev transactor with the approach here: http://docs.datomic.com/peer-server.html
Details on starting dev transactor can be found here http://docs.datomic.com/run-transactor.html
@marshall i will give it a shot. i guess i shied away from dev transactor for not knowing what that really means. what is it using as storage etc… but at this point if i can get up and running that’d be great.
Not for production work, but totally appropriate for development work. You can also backup and restore to and from dev just like any other storage
@marshall when i run with that file i also get this “Unable to load license key” message.
I put the key there that I have from my http://my.datomic.com page.
Oh no.
That must be it. Thanks!
Stupid me.
Nevertheless, much appreciated.
@marshall so I’ve run the transactor. the next step I think is to run a peer-server. i tried that using this: bin/run -m datomic.peer-server -p 8998 -a datemo,datemo -d datemo,datomic:
. Now I get an error regarding the db itself I think:
> Exception in thread "main" java.lang.RuntimeException: Could not find datemo in catalog
But isn’t this server the peer?
I mean this peer-server?
That makes sense I guess.
ahhh so the repl is kind of like running mysql
...
and the manually creating a db
bingo!
Much much thanks!
I might write up a blog post or something to sum up this process, maybe that can help fill in some of the blanks. I find it’s usually just missing steps especially where stuff isn’t intuitive. In this case, I think the potential for confusing is higher because there are so many different pieces (by design of course).
I watched the day-of-datomic training videos and one had this picture with the db services all decomposed, and that looked cool on a high level, but I realized that that was also why I hadn’t gotten anything running yet.
Well it also doesn't help that those videos were from before clients and peer server existed:)
Yes, that too.
Yeah, I partly got into this mess because I wanted to be able to play with the console to get a feel for the query language and the structure of the db visually. But discovered that that wouldnt work with the in memory db peer-server.
@val_waeselynck is there an easy test that you can think of whereby I could see the limitation vv history to which you were referring?
Try to edit history in any way :)
I’m not sure what that means…
I thought that in datomic you don’t edit history because you accumulate new information.
Sure, but that's not really the same as having several versions of a document.
But the fact that history cannot be edited is the reason that it should not be used to implement first class notions of your domain model
If that's not clear, you should go ahead and try implementing versions using history, the limitations of that approach will soon become evident to you
Well. That sounds ominous!
One situation where querying pasr dbs is especially problematic is when you need to evolve your schema
Admittedly, I can’t really follow the meaning of what you wrote. I think I’m not keyed into the lingo well enough yet. So that “first class notions of your domain model” sounds like a foreign language for the most part.
There has to be a mailing list thread where someone has expressed this more clearly :)
My assumption/hope was that I could simply provide text (which would just be markdown) and then be able to rewind somehow to see previous versions.
And have the power of a db to be able to query at the same time. Git gives me much of what I need, but it’s not queryable really.
My understanding was that with datomic you can go back in time, and also query a history.
Something perhaps like this example in the tutorial code in the docs:
(require '[clojure.pprint :as pp])
(def db-hist (client/history db))
(->> (<!! (client/q
conn
{:query '[:find ?tx ?sku ?val ?op
:where
[?inv :inv/count ?val ?tx ?op]
[?inv :inv/sku ?sku]]
:args [db-hist]}))
(sort-by first)
(pp/pprint))
=> ([13194139534399 "SKU-21" 7 true]
[13194139534399 "SKU-42" 100 true]
[13194139534399 "SKU-22" 7 true]
[13194139534400 "SKU-22" 7 false]
[13194139534402 "SKU-42" 1000 true]
[13194139534402 "SKU-42" 100 false])
The way to do that would be to simply store all the versions as different entities with a timestamp attribute
Hmmm.