Fork me on GitHub
#datomic
<
2018-07-02
>
Oliver George06:07:50

AWS novice question: Is there an easy way to start/stop my Datomic Cloud System?

Oliver George06:07:36

e.g. When casually exploring on the weekends it seems silly to leave dev cloud services running 24x7

rhansen09:07:09

@olivergeorge I've been setting desired capacity in my EC2 Auto Scaling groups to 0. This shuts down the EC2 instances and brings costs down to 0. Setting desired capacity to 1 brings them up again.

rhansen09:07:14

(solo topology)

Oliver George09:07:56

Thanks that makes sense. I guess I should be able to do that using the AWS CLI in a makefile too.

Ben Hammond10:07:41

I have a question about attribute aliases. Although I can transact multiple db/idents pointing at the same attribute entity the most recent one seems to get a 'most favoured status`, and the earlier ones seem to get hidden

Ben Hammond10:07:54

when I query

(sort-by second
         (d/q '[:find ?e ?a
                :in $
                :where [?e :db/ident ?a]]
              db))
it only shows me the 'most favoured' db/idents

Ben Hammond10:07:41

although previous ones continue to work, I cannot find out what they are; I just have to know

stuarthalloway10:07:32

hi @U793EL04V -- query against a history db to see retracted values

Ben Hammond10:07:44

so I have two db/idents pointing to the same attribute entity

Ben Hammond10:07:08

I want both to work (and they do)

stuarthalloway10:07:37

yes, and they can both be found in query via a history db

stuarthalloway10:07:52

db/ident is cardinality one

Ben Hammond10:07:02

ah right I see

favila18:07:32

If you just want the db id for a (possibly old/replaced) ident, use datomic.api/entid; to learn the "preferred" (current) ident use datomic.api/ident with the old ident as an arg

favila18:07:31

nevermind using ident doesn't work: just returns any keyword it's given

favila18:07:38

but entid does work

miridius13:07:53

Is there any way to have a single Ions-based application span over multiple AWS regions? With the old model of using Datomic (Cloud) and running my app separately, I could have a single Datomic instance in say US East 1, but then have multiple instances of my app (including the Datomic peer library) running all over the world, which would then each be able to cache read results from the DB locally and serve geographically nearby client requests faster

miridius14:07:01

Another, more noobie question: can I develop an Ions app using an in memory datomic DB (i.e. "dev" storage) on my own machine for local dev, and then push it to the cloud afterwards?

stuarthalloway15:07:48

@miridius no cross-region support at this time

stuarthalloway15:07:45

Ions do not currently have any local storage, but the :server-type :ion is designed for local dev: see https://docs.datomic.com/cloud/ions/ions-reference.html#server-type-ion and discussed in the video starting at https://www.youtube.com/watch?v=3BRO-Xb32Ic&amp;t=668s

miridius20:07:53

@stuarthalloway yeah I've seen that video already, it's great! That's what got me really excited about Ions. I can imagine that using the ion server type is useful in plenty of local development situations. But what about when you don't have an internet connection (or a crappy one)? Or a more likely scenario, what if you have a large test suite that you want to run frequently, and if those test read/write to the DB it's going to make the suite a lot slower.

miridius20:07:57

Or maybe I'm just thinking about things the wrong way and haven't got my head around the Ions approach yet

miridius22:07:25

So after thinking more about this I realized 2 things: the Datomic free edition can be used for offline, in memory development/testing, and if I export my tests as lambdas I could run them in the compute cluster directly. Both of those options solve the test suite latency dilemma, and the first one also solves the developing without an internet connection issue (happens more than you might think!). It made me wonder though, if it would be possible to include test execution as a step in the Ions deployment Step Program so that the code is only deployed if tests pass?

steveb8n02:07:41

I noticed that the datomic.client.api is a protocol so Iā€™m currently testing a testing impl of that protocol which uses Datomic-Free instead of the cloud connection. This should provide the local dev/test flow we all want.

steveb8n02:07:19

there are a few behaviour differences that need to be adapted and some that need to be avoided but so far itā€™s working well

Oliver George02:07:04

Sounds interesting.

steveb8n02:07:40

just hit a road-block with d/q. itā€™s the only fn not behind a protocol. Iā€™ll update as I try and get around it

steveb8n03:07:12

ok, it works. Iā€™ll post a gist

steveb8n03:07:34

lots of work to do but it proves the concept. thereā€™s one important question remaining which Iā€™ll ask Stu in a new thread

šŸ‘ 4
eraad15:07:18

Hi, IĀ“m getting the following error when trying to push code for a Datomic Ion: $ clojure -A:dev -m datomic.ion.dev ā€˜{:op :push :uname ā€œhelloā€}ā€™ {:command-failed ā€œ{:op :push :uname \ā€œhello\ā€œ}ā€, :causes ({:message ā€œShell command failedā€, :class ExceptionInfo, :data {:args (ā€œclojureā€ ā€œ-Sdescribeā€ ā€œā€), :result {:exit 1, :out ā€œInvalid option: -Sdescribe\nā€, :err ā€œā€}}})}

eraad15:07:33

Any lights on whats the error related to?

stuarthalloway15:07:53

that is telling you that clojure -Sdescribe failed -- maybe you do not have Clojure command line tools installed?

eraad15:07:24

Ok. IĀ“m sorry for such a newbee error. Will check that out

eraad15:07:11

@stuarthalloway Did brew upgrade clojure and everything worked as expected, thanks!

šŸ‘ 4
sparkofreason17:07:32

Using the Datomic Cloud client, I'm seeing errors like "Attempting to call unbound fn: #'datomic.client.api.sync/client". Any thoughts on why that might occur?

marshall17:07:44

@dave.dixon what version of client?

marshall17:07:07

and what version of Datomic Cloud?

marshall17:07:50

And when are you getting that error?

sparkofreason17:07:10

That one occurred in a call to datomic.client.api/client

marshall17:07:43

youā€™ve required the client API namespace?

sparkofreason17:07:36

Yes. It's sporadic, and not always the same error. I've also seen something to the effect of "method db not found on connection", and another when transacting where the message is just "datomic.client.impl.shared.Connection". I need to check my logging to ensure that I'm getting the full chain of causes, perhaps.

sparkofreason17:07:27

SystemCFTVersion 397

marshall17:07:43

yeah, if you could get the full error/stacktrace from cloudwatch logs that may help

marshall17:07:15

@dave.dixon I suspect this is related to a code loading race condition issue; can you upgrade to the latest Cloud release/template (v 402)

stuarthalloway17:07:26

@dave.dixon are you rolling instances (e.g ion deploy?)

sparkofreason17:07:22

@stuarthalloway Don't think so, no ions, just running the DB in the vanilla production topology.

stuarthalloway17:07:13

@dave.dixon any chance you are racing with your own code?

sparkofreason17:07:11

@marshall I'll give it a try ASAP. That was my guess too, We're running an onyx cluster processing messages from Kafka, and these error seem to crop up early in the lifetime, when the onyx job starts and there's messages waiting to be processed.

sparkofreason17:07:24

@stuarthalloway What sort of scenario are you thinking of? It's possible, I suppose, and I'll take another look at the code. but I believe it's wired up so that all of the Datomic interactions are scoped to a single onyx task, clients and connections not being shared amongst different threads.

stuarthalloway17:07:29

@dave.dixon if your code calls require from multiple threads you could be hitting the equivalent of https://dev.clojure.org/jira/browse/CLJ-2026

stuarthalloway18:07:42

Clojure's ns macro marks the namespace loaded at the moment the ns form completes, so another thread requiring the namespace thinks it is loaded when it may still be load*ing*

stuarthalloway18:07:51

symptom is symbols not present, seemingly at random

sparkofreason18:07:39

That sounds like a strong candidate.

sparkofreason18:07:18

@stuarthalloway Is the workaround to try and require the client namespace as early as possible?

stuarthalloway18:07:15

@dave.dixon the workaround is a concurrency mechanism that blocks the system on initialization, e.g. a a delay or future

miridius22:07:25

So after thinking more about this I realized 2 things: the Datomic free edition can be used for offline, in memory development/testing, and if I export my tests as lambdas I could run them in the compute cluster directly. Both of those options solve the test suite latency dilemma, and the first one also solves the developing without an internet connection issue (happens more than you might think!). It made me wonder though, if it would be possible to include test execution as a step in the Ions deployment Step Program so that the code is only deployed if tests pass?