Fork me on GitHub
#datomic
<
2017-05-19
>
favila00:05:57

@goomba ehat problem specifically are you having?

erichmond00:05:49

Quick question. When I am trying to restore a database from a remote dynamo table to a local dynamodb-local table, I keep getting these sorts of errors Caused by: java.util.concurrent.ExecutionException: clojure.lang.ExceptionInfo: :restore/read-failed Unable to read 58de91f6-6e59-4284-ad6b-94c262caf019 {:key "58de91f6-6e59-4284-ad6b-94c262caf019", :db/error :restore/read-failed}

erichmond00:05:09

2017-05-19 00:25:57.981 WARN  default    datomic.backup - {:message "error executing future", :pid 76, :tid 10}
java.util.concurrent.ExecutionException: clojure.lang.ExceptionInfo: :restore/read-failed Unable to read 58de91f6-6e59-4284-ad6b-94c262caf019 {:key "58de91f6-6e59-4284-ad6b-94c262caf019", :db/error :restore/read-failed}

erichmond00:05:47

I just have no idea what this error actually means, so it's hard to troubleshoot, any insight would be greatly appreciated

favila01:05:55

@goomba does your MySQL give SELECT rights to user datomic and table datomic? Seems like it may not

favila01:05:21

Transactor needs select insert update delete

favila01:05:30

Peers just need select

Lone Ranger01:05:59

It does, strangely

Lone Ranger01:05:27

I can login to the datomic db with mysql -udatomic -pdatomic datomic and select * from datomic_kv just fine

Lone Ranger02:05:39

It's quite the mystery to me

Lone Ranger12:05:47

okay what I've figured out so far is that the password parameter isn't being passed to MySQL for some reason

Lone Ranger12:05:27

when I do tail -f /var/log/mysql/error.log I see the messages 2017-05-19T12:33:51.825580Z 316 [Note] Access denied for user 'datomic'@'localhost' (using password: NO)

mgrbyte12:05:24

@goomba What works for us here is adding &useLocalSessionState=true - i forget the where and why this was needed.

Lone Ranger12:05:25

worth a shot!

Lone Ranger12:05:54

eh... what still seems to be the case strangely is that it's only picking up the first parameter I pass in for some reason

Lone Ranger12:05:59

it seems to ignore the rest of them

Lone Ranger12:05:37

datomic: is what I just ran and it gives the same error 😅

Lone Ranger12:05:30

If I switch the order of parameters around, it reads the password but not the username facepalm

mgrbyte12:05:21

goomba Is this the peer URI or in the transactor?

Lone Ranger12:05:55

this is a modification of what was given to me by the transactor that I'm trying to use to initialize the peer

Lone Ranger12:05:03

so, peer URI 😄

mgrbyte12:05:14

I wonder. In your URL you use "datomic" as the host, but the error is mentioning "localhost". Perhaps the hosts entries in the mysql auth tables are not matching?

Lone Ranger12:05:52

well it does say localhost:3306 in the string

Lone Ranger12:05:30

but I'm gonna glance at the grant table anyway

mgrbyte12:05:23

sorry. just saw your original message with the -m datomic.peer-server command

Lone Ranger12:05:44

marshall got me figured out

mgrbyte12:05:49

Have you tried enclosing the value to -d in double quotes?

Lone Ranger12:05:55

I wasn't escaping the ampersand

mgrbyte12:05:03

:thumbsup:

Lone Ranger13:05:36

Woohoo! Got it sorted! Need to escape that ampersand! Everyone can rest easy now 🙂 I may have gotten a little hint from cognitect sales team about being a shell scripting noob hehehe 😅

jfntn16:05:21

Has anyone tried to generate pull queries from s/keys specs?

miikka16:05:45

jfntn: nope, but the basic (s/keys :req [::foo] :opt [::bar]) → [::foo ::bar] would be pretty easy with spec-tools (https://github.com/metosin/spec-tools)

miikka16:05:55

user=> (into [] (:keys (st/spec (s/keys :req [::foo] :opt [::bar]))))
[:user/foo :user/bar]

jfntn17:05:07

@U1NDXLDUG actually tried that but I was getting nil keys, suspect it was because I was using 0.2 with alpha15 but not sure

jfntn17:05:39

And you'd use visit to traverse recursively right?

miikka18:05:12

Yeah, 0.2 needs alpha16, and yeah, visit for recursive traversing.

jfntn18:05:32

Sounds good, I’ll try again

Lone Ranger16:05:57

also would like to throw in the question queue has anyone used the datomic console with a sql based storage?

Lone Ranger16:05:18

struggling to figure out the correct command

Lone Ranger16:05:31

datomic: is what I'm trying but no luck so far

Lone Ranger16:05:07

okay wasn't sure if that was important or not

Lone Ranger16:05:44

yayyyy 😄 :thumbsup:

Lone Ranger16:05:12

okay so if I make an entity of some sort and I decide I want it to have more attributes later, that's doable yes?

Lone Ranger16:05:53

like this was presented in one of the examples

[{:db/ident        :story/title
   :db/valueType   :db.type/string
   :db/cardinality :db.cardinality/one}
  {:db/ident       :story/url
   :db/valueType   :db.type/string
   :db/cardinality :db.cardinality/one
   :db/unique      :db.unique/identity}
  {:db/ident       :story/slug
   :db/valueType   :db.type/string
   :db/cardinality :db.cardinality/one}]
but if later I wanted to make that entity also have, say, an author, that's doable yes?

favila16:05:33

these are not a single entity, these are three entities. This transaction creates the :story/title :story/url and :story/slug attributes @goomba

favila16:05:45

@goomba are you following along with a tutorial?

Lone Ranger16:05:11

ah, yeah I'm trying to follow along with the "Day of Datomic 2016" videos

Lone Ranger16:05:39

ok I see so my confusion was what an entity is

Lone Ranger17:05:02

wait wait, just for clarification... aren't those three schema that collectively form an entity?

Lone Ranger17:05:07

or do I have that backwards?

favila17:05:19

each schema attribute is itself also an entity

favila17:05:32

they don't collectively form anything

favila17:05:09

at the bottom, datomic is a set of assertions [entity attribute value transaction] (called "datoms")

favila17:05:27

an "entity" is just a number in the entity slot of a datom

Lone Ranger17:05:39

oh fascinating

favila17:05:52

the map view of a datom is gathering together all datoms with a common entity

Lone Ranger17:05:16

okay so if I wanted to (conceptually) add some information about a story, like an author, what would be the idiomatic way to do that?

Lone Ranger17:05:36

query by some attribute of the story and then do some kind of association with the entity id and the new information?

Lone Ranger17:05:01

by the way @favila I owe you at least one beer for all the help you've been! 🍻

favila17:05:03

first you need a reference to the story. This can be via direct entity id, or an attribute+value unique to the entity

favila17:05:51

e.g. [:db/ident :story/slug] or just :story/slug is the slug attribute entity.

favila17:05:24

so it looks like here :story/url is your unique attribute

favila17:05:33

so (d/transact conn [{:db/id [:story/url "the-url"] :story/author [:author/id #uuid"some-uuid"]}]) for example, would add a story author assertion to a story entity

favila17:05:09

the map form of the transaction is sugar for [:db/add [:story/url "the-url"] :story/author [:author/id #uuid"some-uuid"]]

Lone Ranger17:05:27

so in one shot (d/transact conn [{:db/id [:story/url "the-url"] :story/author [:author/id #uuid"some-uuid"]}]) does a logical lookup on the story that matches "the-url" on the attribute :story/url and associates it with the :story/author specified... magical

Lone Ranger17:05:51

now this presupposes that I've made a schema for :story/author first, yes?

Lone Ranger18:05:10

I was blind but now I see 😄 😄 😄

azimpel19:05:53

hi guys, im learning datomic using the official tutorial but unfortunately there is a query on the page http://docs.datomic.com/getting-started/see-historic-data.html which produces error can someone give me a hint what is wrong here? query: (<!! (client/transact conn {:tx-data [{:db/id commando-id :movie/genre "future governor"}]})) result: {:datomic.client-spi/request-id "b7ca85b3-b3d4-4f65-8f33-4b9d8b103fb5", :cognitect.anomalies/category :cognitect.anomalies/incorrect, :cognitect.anomalies/message ":db.error/invalid-lookup-ref Invalid list form: [[17592186045419]]", :dbs [{:database-id "591e005d-745c-492b-8f7b-ff406aaabda5", :t 1001, :next-t 1005, :history false}]}

favila19:05:07

@azimpel did you forget to ffirst the previous query result?

favila19:05:28

commando-id is [[123]] instead of 123

favila19:05:51

user=> (def commando-id 
            (ffirst (<!! (client/q conn {:query '[:find ?e 
                                                  :where [?e :movie/title "Commando"]] 
                                         :args [db]}))))
#user/commando-id
user=>

azimpel19:05:59

yes, you're the best

azimpel20:05:28

@favila , thank you, my dumb mistake

kenny20:05:44

Is it guaranteed that the first datom in :tx-data off of a transaction report is the datom with the tx entity?

favila20:05:53

I doubt it

favila20:05:10

just look at the tx of any datom

favila20:05:16

they should all be the same for a given transaction

favila20:05:47

(-> tx-result :tx-data first :tx)

tosh21:05:43

anyone knows what it could mean if (d/create-database db-uri) yields

IllegalArgumentExceptionInfo :db.error/read-transactor-location-failed Could not read transactor location from storage  datomic.error/arg (error.clj:57)

tosh21:05:50

issuing (d/get-database-names db-uri) with a version of the db-uri that has a * in place of the datomic db name gives me nil which isn't unsurprising because when i look at the datomic_kvs table in postgres it is still empty

favila21:05:55

@tosh transactor is either not running or not pointed at storage correctly

favila21:05:15

or there are permission problems etc

favila21:05:27

Investigate transactor logs

tosh21:05:00

ah of course, i haven't had a transactor running

tosh22:05:22

thanks @favila worked like a charm. my first error was not starting the transactor (for some reason i thought the peer will write directly into storage and that starting the transactor only makes sense later once the datomic db is created) my second error was that in the properties file of the transactor I had the name of the datomic db that I wanted to work with instead of the name of the postgres db that i use for storage => so many db names and cryptic db urls 🙂

jfntn22:05:42

Is there a way to use transact-async without blocking on the future later?

favila22:05:48

You can block with a deref. Is your goal to call a callback when it's done (push not pull)?

favila22:05:14

Sorry should be "you can block with a timeout"

jfntn22:05:53

@favila yes I’d like it if it were fully async, i.e. callback based

favila23:05:30

Only client API has that. You would have to emulate on peer

Lone Ranger23:05:33

is there a better/more-idiomatic way to do this?

(let [my-id (<!! (client/q conn {:query '[:find ?x
                                         :where
                                         [?x :story/title "Teach Yourself Programming in Ten Years"]
                                         ]
                                 :args  [db]}
                           ))]
  (println (<!! (client/transact conn {:tx-data
                                       [{:db/id              (ffirst my-id)
                                         :story/author-first "Peter"
                                         :story/author-last  "Norvig"}]}))))
i.e., inlining the query? I assume there is but when I try
(<!! (client/transact conn {:tx-data
                            [{:db/id              [:story/title "Teach Yourself Programming in Ten Years"]
                              :story/author-first "Peter"
                              :story/author-last  "Norvig"}]}))
I'm told that :story/title is not unique (which is true)

marshall23:05:41

@goomba unless you have a unique attribute (identity or value) you can include in the transaction map, the query for ID followed by transact is pretty standard

Lone Ranger23:05:21

gotcha! Well I'm cool with that as long as I'm not writing code like a barbarian over here. Well, not a complete barbarian

marshall23:05:58

It's often a good idea to have an externally unique identifier (ie domain ID of some kind) for this reason (and some other reasons)

marshall23:05:38

In this case something like the library of Congress identifier or a UPC might be appropriate

marshall23:05:04

Title prob not, since there are plenty of duplicate titles for different books

Lone Ranger23:05:20

mhm. Unfortunately my weak human memory was having trouble recalling the UPC exactly so I tried to fudge with whatever I could remember 😂

Lone Ranger23:05:53

Thanks again! 🎩's off!

Lone Ranger23:05:25

hey, how cool is our life that we get to do this? eh? eh?

Lone Ranger23:05:37

still coding at almost 8pm on a Friday cause it's the 🐝's knees!

Lone Ranger23:05:01

(if I was still stuck with JavaScript I would've hit the bar at 11am I think)

Lone Ranger23:05:40

(11am pacific time, and I work on the east coast)

Lone Ranger23:05:00

Is the backend of the site in datomic?

Lone Ranger23:05:17

This is a roundabout way of asking if datomic is appropriate for serving videos

Lone Ranger23:05:30

Oh wait just noticed they're using vimeo

Lone Ranger23:05:41

But still, "Is datomic appropriate for serving videos?"

marshall23:05:49

Definitely wouldn't store media in datomic

marshall23:05:03

Not intended as a blob store

Lone Ranger23:05:22

gotcha. not even close to my primary use case just curious

Lone Ranger23:05:32

we have polyglot persistence for that sort of thing so not a big deal

marshall23:05:49

Good for metadata for blobs stored in a secondary store like s3

Lone Ranger23:05:01

yeah I was thinking something along those lines

Lone Ranger23:05:53

blob store... I don't know where we come up with these words but another reason I love this industry 😂