Fork me on GitHub
#datomic
<
2018-07-23
>
Oliver George07:07:25

This is the result of an experiment to automate the API Gateway setup required for Web Service Ions via the aws cli (so each new deployment isn't a manual setup task). I'm interested in any feedback (approach, assumptions, implementation...) https://gist.github.com/olivergeorge/cc0ca9a945cb372d35d97e45573656ee (updated to tidy up)

👍 4
steveb8n12:07:00

I did something similar here https://github.com/hlprmnky/ion-appsync-example/blob/master/src-pages/cf/node_pages.clj although not for Ions, instead for CLJS lambdas intended to eventually be the host pages for an Ion backed SPA

steveb8n12:07:02

Using Crucible and Cloudformation is a pretty nice experience for doing infra as code IMO. I like where all these ideas are taking us

Oliver George12:07:03

Thanks I'll check it out. Crucible is new to me. Still getting familiar with the AWS landscape.

eoliphant14:07:27

Good deal, I’ve been looking at this myself.

eoliphant15:07:38

We’re more of a terraform shop, but similar idea

cjsauer16:07:12

condensation is also a fun option for writing infra as code in Clojure. I've had good success working this library into certain deployment workflows. https://github.com/comoyo/condensation

Oliver George01:07:29

Now that I look at the cloudformation documentation it does seem like generating a cloudformation template is ultimately a simpler solution.

billyr11:07:34

Is excision CPU bound? Does the way I partition transactions matter? I'm excising 150k entities and wondering how long it'll take

marshall12:07:53

That’s a HUGE excision. Excision is not intended for size control and should be used only when necessary (i.e. legally required to remove something). As a rule of thumb, if you can’t type out the excision transaction manually it’s probably too big to run at once.

👍 4
val_waeselynck11:07:53

@bill_rubin I think it's not so much about size, more a matter of how much it disrupts your indexes.

val_waeselynck11:07:50

how long it takes probably depends a lot on your Transactor's hardware, your storage and your network performance characteristics

henrik11:07:04

Does anyone know where I can find the options available for the :headers field for a web ion?

billyr11:07:53

@val_waeselynck Thanks. It's on a single host and the transactor has been maxing out the cpu overnight so I'm assuming that's the limiting factor. I guess I'll just recreate the db

billyr11:07:34

@val_waeselynck Yea that's what I'm doing haha, thanks!

henrik13:07:26

For a web ion, how can I capture the path, like the /dev/hello/world part of ?

marshall13:07:08

i believe it is in the :uri key

marshall13:07:19

you’d need to parse it yourself

henrik13:07:52

Excellent, thank you @marshall

eoliphant15:07:31

Also @henrik, since what’s coming in is basically ring-compatible, you can drop right into one of the various and sundry routing/middleware libs if your use case is non-trivial. I just swapped out some custom code for reitit literally last night

luchini16:07:49

I’m trying a simple retraction on Datomic Cloud and getting a weird error. The retraction is: [[:db/retract :person/email :db/unique :db.unique/identity]] (yes, the dataset I’m working on does not guarantee unique emails for some bizarre reason).

marshall16:07:20

@luchini that’s a known issue. we’re working on a fix

luchini16:07:24

Datomic gives me a nth not supported on this type: Db anomaly

luchini16:07:24

Great @marshall! Thanks a lot. Do you know if it would work in the very same transaction where I’m in fact asserting duplicate emails, or do I need to keep two separate transactions?

marshall16:07:42

you’d need to retract the :db/unique first

luchini16:07:59

What about the opposite scenario? (trying to prepare for the future). When I manage to implement a transaction that fixes the dataset in the live system, I’ll need to make sure that I’m updating all duplicate emails in the same transaction I’m adding the :db/unique back in.

luchini16:07:13

Is that possible?

marshall16:07:03

“If there are values present for that attribute, they must be unique in the set of current database assertions.”

marshall16:07:20

so you’d have to update things to make them unique then issue the schema change transaction

eoliphant17:07:33

hey to the datomic folks, I dropped in a couple feature requests. I’d talked to stu about supporting deploying into existing vpc’s, based on that discussion I know part of the desire is to keep things as contained as possible from a support perspective. But, I think this will be pretty important in the context of datomic cloud’s inevitable massive growth 🙂 Say in our case, we’ve gone from a kind of cheesy, ad-hoc couple vpc’s across two accounts (prod v non-prod) to a 1-1 account/vpc approach for dev,test,etc complemented by shared vpcs for management, ingress etc. with IPSec vpn’s wiring them together. It’s kind of hard to shoehorn extra dedicated datomic VPC’s, per env into this. ions can complicate this further. Since the code is effectively global, I’d like to give each of my devs their own solo system, then have a common ‘dev’ system that’s updated via CI or something. It’d potentially be more manageable to support multiple systems/vpc. Just a few thoughts 😉 In the meantime, ions, are fricking amazing 😉

jjfine20:07:48

anyone have any tips on how to write tests for queries that use the :db/txInstant field? i'm having trouble fixturing test data without doing a (Thread/sleep ..) between calls to transact

matthavener13:07:08

You can pass a instant to your transaction to arbitrarily set the time of the transaction. All that datomic cares about is that the txInstants are monotonic

kenny20:07:58

When testing my Ion endpoint via the Method Test UI in the AWS Console, my response body is base64 encoded. Is there a way to get the UI to display the decoded version?

parentheian22:07:14

is :db/index not supported in datomic cloud? when I try to do (dc/pull db '[*] :db/index) I get #:db{:id nil}