Documentation suggestion: document the :next-t key of Database values in clients https://docs.datomic.com/client-api/datomic.client.api.html#var-db It's pretty useful for working with the Log API in particular

a stupid question: what is the clojure way of maintaining a certain relationship between datomic entities? Say there are three entities in the db, $PUBLIC_KEY $MESSAGE and $SIGNATURE such that the latter enjoys a mathematical relationship with regard to the first two. can datomic know about such a relationship or does it have to be programmed independently of datomic?

What is the story around integration testing and unit testing with datomic cloud? I would like to be able to run tests without internet connection. The only reference I found was this: https://forum.datomic.com/t/integration-testing/465

for those of you who use ions / api gateway, do you ionize a handler for every REST end point? for example, a mostly 1-1 map: (/example/auth -> my-auth-ion, /example/colors -> my-colors-ion). my lambdas go cold after a while (to be expected), and so each visit to the web app is painfully slow as each supporting lambda heats up. just wondering in practice how you all deal with this? is this acceptable in your production applications? does anyone use one big lambda and then dispatch internally?

in something like nodejs a cold start is barely noticeable, but as i fetch various assets i'm looking at 4-6 seconds per resource if it hasn't been touched in a while.

I’m enjoying using one lambda, then dispatching with multimethods.

Then it stays warm.

I can always scale my lambda concurrency up, then set a timeout where if it takes more than 200ms (due to cold start when scaling up during a burst), retry and hit the other warm instance.

so you have a single /{proxy+} integration?

No, not neccessarily.

Some apps I do, some I dont.

But for the most part, a lot of the endpoints, even with different routes, all end up calling the same lambda.

So /some/route/foo calls my-ion-lambda and /some/route/bar ALSO calls my-ion-lambda

Internally in my ion I can route based on the path.

but then my-ion-lambda stays warm.

ahhh i see. so do you dispatch on the url path?

using something like bidi?

yes, I'm doing the same. Just like a traditional Clojure webapp setup.

I can. One of my apps uses AppSync + GraphQL so there I dispatch based on graphql stuffs.

I know pedestal is coming out with an Ion based interceptor chain.

interesting

However, I’ve also succeeded in calling an ion lambda directly from the web without fronting it with API Gateway.

There's some tricks to getting things like CORS to work with API Gateway, but once you wrap your head around it it's pretty simple. I'm actually going to write a quick post at some point about the little stuff I ran into and how to fix it.

Ill say it again. Call a lambda. From the Browser.

yeah i ended up writing some custom wrappers to support cors and various mime types

browser -> json -> ion -> json -> browser

No web layer.

ah, so you're not apigw/ionizing

Ha, I still need to learn from @lanejo01 how he's doing that. Basically the way the AWS SDKs do?

Here is a scratch project. Its a bit under documented and not totally cleaned up. https://github.com/MageMasher/ion-web

@joshkh In some cases.

However after listening to rich’s talk at clojure nyc (theres a video on youtube) he made a comment that makes curious about sticking with apigw

I’ve been toying with wrapping calls to ions with reagent atoms 🙂

you mean what he said about going direct from APIGW to your code w/o Lambda in between?

i think my brain was warped by the datomic ions tutorial. it's just an odd place to leave the user: a single greedy end point pointing to a ring handler. i'm fairly new to AWS though, so maybe it's just me.

by "user", you mean the developer?

er yeah, trying to multitask in a meeting. 😉

by the way, thanks for the discussion. finding best-practices or "real life" example setups of ions is tough. i'm planning to blog a few of my trials and tribulations along the way. might spare a few people some trouble.

yeah, it's early days. The community sharing is a big help.

(it's mostly me not understanding what are probably basic principles but i'm sure i'm not alone)

i'm used to whacking stuff on heroku or dokku and calling it a day

you have to keep in mind that API GW can be configured as an edge network and Lambdas probably have similar elastic scaling properties (though I'm not sure if they're bound to one AWS region), so even though it looks like you're piping everything through one "place", it's not really a bottleneck.

haha, me to. Heroku was always my goto for prototyping.

Ions isn't quite as ironed out (yet), but it's getting close, and with a much more powerful model

agreed, and i'm certainly not complaining. it's a huge win for us to be able to push clojure code.

yeah, and as usual (in my experience), the difficulties come more from facts about how the JVM works (eg classloading and such), not Clojure proper

yup. or in my case AWS configuration and infrastructure.

right, all the non-Clojure-itself stuff, ha

even still, it's much less AWS stuff than other deployment solutions

for instance i still get worried about co-authoring our ions project. if another developer is working on some newly exposed ions in their code base while i do the same, and we're both pushing and deploying, then are we removing each other's ions?

(based on what's in ions-config.edn)

if you're pushing to the same system, I think so. I think the answer is for each developer to have their own Solo system for dev purposes

so multiple compute stacks sharing a storage stack?

I was thinking an entire storage+compute stack for each

but maybe what you're saying would be cheaper and more efficient, if it's possible

i guess so long as you're all in communication about schema changes that could work

anyway that wasn't a huge concern. just an example of my unknowns. 🙂

right. I think if devs are doing a good job with schema attribute namespacing, and your project is organized sensibly, you're much more likely to have clean "schema merges" when you go to integrate than you would with eg relational schema, or even schemaless. This is all speculation on my part though, since I haven't gone through any of that yet.

Honestly though, Datomic's notion of schema, and the possibility that it has properties like that, is the main difference-making reason that I'm betting on it

we found that our schema changed quite a bit during the very start of the project. in a normal db we'd "take the good parts" and maybe start a fresh one, but we still haven't solved data dumps / loads in Cloud.

but we've also found that having such a factual and flexible schema means we can continue working with the same instance and chalk up changes to historical events

which is great.

anywho, gotta run. thanks for the chat!

I see. Again, I haven't gotten far enough to speak from experience, but my sense is that following the rules for non-breaking growth that Rich has been laying out across the Clojure ecosystem allows you to live with your past decisions/data without having to do massive structural migrations along the way. I think that makes us devs uncomfortable because we like to clean up our messes (eg cleaning up Git history, refactoring, schema migration, etc...), but that discomfort with messes is totally trumped by the pain of breakage at integration points.

Hey I'm trying to create a datomic cloud from awsmarketplace and after some waiting on CloudFormation I get this error The following resource(s) failed to create: [DhcpOptions, EnsureEc2Vpc]. Then it auto-rollback to ROLLBACK_COMPLETE status

Is your account EC2-VPC? I think I ran into that for that reason

Yes. @eoliphant

Any tips?

it’s not clear to me if you are ec2-vpc, or just vpc https://docs.datomic.com/cloud/operation/account-setup.html#ec2-vpc-only