Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2018-07-17

Channels

#datomic
<
2018-07-17
>
Rodin12:07:56

Hi, I'm trying to load about 0.5GB of data into datomic. Can anyone confirm that transact is not lazy, i.e. when passing it an ISeq that sequence will be reified into a list/and or the head of that sequence will be held onto?

jaret12:07:05

@rodinhart are you trying to transact that amount of data as a single transaction?

Rodin12:07:02

Well, I'd like to. The follow-up question, as expected, would be: how do I batch data that has references to earlier entities?

jaret12:07:00

That’s almost certainly far too much data for a single transaction. To batch you’ll want to build up batches with some kind of identifier on your entities. Like using lookup refs or unique identities that you create. The tempid’s map that is returned from transact can be used to map entities.

Rodin12:07:55

Are you confirming transact isn't lazy?

Rodin12:07:45

And are you saying if I give entities a temp id for :db/id, the return value of transact will give me a mapping from those temp ids to the actual ids in the db?

Rodin13:07:10

Ah, brilliant, very helpful.

Joe Lane17:07:51

Anybody know why after removing the backslashes for the ion deploy step from the :group tag and other tags the backslashes still occur in :rev and :uname?

stuarthalloway19:07:46

hi @U0CJ19XAM you should not need backslashes anywhere

Joe Lane19:07:33

clojure -A:dev -m datomic.ion.dev '{:op :push :uname "ch357"}'

Downloading: com/datomic/java-io/0.1.11/java-io-0.1.11.pom from 
(cognitect.s3-libs.s3/upload "datomic-code-f070a20d-8cb2-44f6-b83a-a47dd69ed035" [{:local-zip "target/datomic/apps/someapp/unrepro/ch357.zip", :s3-zip "datomic/apps/someapp/unrepro/ch357.zip"}] {:op :push, :uname "ch357"})
{:uname "ch357",
 :deploy-groups (someapp-compute),
 :dependency-conflicts
 {:deps #:com.cognitect{http-client #:mvn{:version "0.1.80"}},
  :doc
  "The :push operation overrode these dependencies to match versions already running in Datomic Cloud. To test locally, add these explicit deps to your deps.edn."},
 :deploy-command
 "clojure -Adev -m datomic.ion.dev '{:op :deploy, :group someapp-compute, :uname \"ch357\"}'",
 :doc "To deploy to someapp-compute, issue the :deploy-command"}
I have to pull backslashes off of the output of the above command still, I realize you all removed the backslashes from the other commands (thank you!), but these still seem to remain. 
clojure -Adev -m datomic.ion.dev '{:op :deploy, :group someapp-compute, :uname "ch357"}'

Joe Lane20:07:41

@U072WS7PE Just Tried it again from Vanilla Bash, same issue, now with :rev instead of :uname 

bash-3.2$ clojure -A:dev -m datomic.ion.dev '{:op :push}'

Downloading: com/datomic/java-io/0.1.11/java-io-0.1.11.pom from 
(cognitect.s3-libs.s3/upload "datomic-code-f070a20d-8cb2-44f6-b83a-a47dd69ed035" [{:local-zip "target/datomic/apps/someapp/stable/a510fee0af59e67a9ba99cfff20c935b7b02d517.zip", :s3-zip "datomic/apps/someapp/stable/a510fee0af59e67a9ba99cfff20c935b7b02d517.zip"}] {:op :push})
{:rev "a510fee0af59e67a9ba99cfff20c935b7b02d517",
 :deploy-groups (someapp-compute),
 :dependency-conflicts
 {:deps #:com.cognitect{http-client #:mvn{:version "0.1.80"}},
  :doc
  "The :push operation overrode these dependencies to match versions already running in Datomic Cloud. To test locally, add these explicit deps to your deps.edn."},
 :deploy-command
 "clojure -Adev -m datomic.ion.dev '{:op :deploy, :group someapp-compute, :rev \"a510fee0af59e67a9ba99cfff20c935b7b02d517\"}'",
 :doc "To deploy to someapp-compute, issue the :deploy-command"}

sho00:07:29

@U072WS7PE I have the same issue with @U0CJ19XAM. Even after the latest update, I still get unnecessary backslashes whenever I "push". For the other ops, this does not occur.

Joe Lane17:07:47

A better question may be, is there a way I can just invoke this library from the repl so I never have to go back to my terminal and mess with the deployment step in a different window?