This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-12-17
Channels
- # adventofcode (23)
- # announcements (2)
- # aws (11)
- # babashka (181)
- # beginners (59)
- # chestnut (2)
- # clj-kondo (9)
- # clojure (90)
- # clojure-brasil (2)
- # clojure-europe (18)
- # clojure-italy (24)
- # clojure-nl (9)
- # clojure-spec (3)
- # clojure-uk (28)
- # clojured (4)
- # clojuredesign-podcast (3)
- # clojurescript (12)
- # community-development (49)
- # core-async (49)
- # cryogen (5)
- # cursive (16)
- # data-science (1)
- # datascript (7)
- # datomic (54)
- # defnpodcast (4)
- # events (2)
- # figwheel-main (14)
- # fulcro (139)
- # graphql (1)
- # jobs-discuss (6)
- # kaocha (1)
- # luminus (2)
- # malli (3)
- # music (1)
- # off-topic (34)
- # pathom (24)
- # re-frame (13)
- # reitit (5)
- # shadow-cljs (8)
- # test-check (6)
if we follow the datomic ion tutorial we're interacting/changing with live database, correct? so we should delete it afterwards in order to start from fresh slate (since there's no way to rollback changes)?
also, does starting/stopping a datomic-gateway
effect billing? (so i shouldn't forget to stop it after developing?) there's not a lot of info on implications of these commands in docs
@alidcastano Yes, the ion tutorial transacts data to a live database. If you want to start over, deleting and recreating the database shouldn't cause any problems. The API gateways are billed based on the number of requests, so you should only see a cost if you are using it a lot (mine is $3.50/million requests, not going to break the bank from development use).
so it doesn't matter whether we leave it running or not, just the amount of requests we do during development
APIGateway is billed by the request, however, @alidcastano is talking about what used to be known as the bastion server
.
Did they change the name on me and I didn't notice?
@alidcastano The datomic-gateway
is an ec2 machine which you will be billed for like normal unless you shut it off.
https://docs.datomic.com/cloud/operation/bastion.html
is now called the datomic-gateway
then?
@U0CJ19XAM so starting/stopping is required for ion development correct?
That is my understanding. https://docs.datomic.com/cloud/getting-started/get-connected.html#access-gateway https://docs.datomic.com/cloud/whatis/architecture.html#security
Oh yeah, it's called "Access Gateway" in https://docs.datomic.com/cloud/operation/operation.html now Thankfully, if you forget to turn it off it's still not going to cost you much - looks like mine cost $3.74 to run the EC2 instance for an entire month
oh yeah that's not bad at all. my first time interacting with aws I once left a service running that costs me a few hundred dollars so that has scarred me 😆
thanks for looking into it @U054BUGT4
Ouch!
@alidcastano "developing", no, there is no need to bounce the machine (except maybe with some of the new analytics stuff, i'm not 100% on that...). If you want to deploy an ion for development (like a transaction function) the push/deploy commands from the cli go against the primary node. The access gateway is an evolution of the bastion
machine which is used for secure access into the vpc. It now does more than secure access but if you're just getting started you can treat it like a "jump box".
I am not an expert, but I think generally you should not need to bounce the datomic gateway for anything with analytics - it picks up changes in the metaschema dynamically afaik (but I could be wrong)
latest info is here: https://docs.datomic.com/cloud/operation/howto.html#datomic-gateway
@alidcastano I usually scale down the auto-scaling group when not actively developing in order to save money. This section might be useful to you. https://docs.datomic.com/cloud/operation/planning.html#turning-down
what's the recommended testing approach for ions? any repos or articles that demonstrating it?
(not referring to repl development, but unit tests ideally with some sort of rollback per test)
What exactly do you want to unit test? You can find code in the helpers here (https://github.com/cognitect-labs/day-of-datomic-cloud) that you can adapt to creating/destroying databases, but you may not need it.
at least that's what im used* to doing with graphql + postgres. is not how it's done with datomic?
that would be the equivalent setup, yea, but I haven't even gotten that far yet. I just like to wrap my head around how the dev/test flow works so I can have more confidence in what im doing.
Well, an "ion" is an aws lambda function which acts as a proxy for your clojure code, so, you can just invoke your clojure code in a test and not involve invoking actual ions.
ah have to learn the terminology better, thanks. yes, I mean testing the actual the clojure code. what I have in mind is an example setup that shows how I can test API with an the in memory client database (has same API ions use) and rollback changes per test hopefully that makes sense in context of datomic, otherwise I'll have to dig more into the code before commenting further 😅
I also want to know, what we're doing right now is creating a database then destroying it after each test, but its awfully slow and "seems wrong"... 😄
Eh, it's not wrong, but you might be able to optimize. You can use d/with-db
to try several tests after creating a db with some common fixture data.
Is there any alternative? Like, to run local tests , the need to connect to a cloud instance, create a database with a unique name that does not conflict with anyone else running tests at that time seems strange...
How do I do an integration test, for example, for a CRUD application without working with the database?
do only peer database have an in-memory alternative? (so that approach can't be used with ions / client api)
can :db/tupleAttrs
refer to idents with :db.type/tuple?
(ie tuple ident containing tuples)
Any idea how to get past this issue with stack creation: "Embedded stack arn:aws:cloudformation:us-east-1:...:stack/vcl-Storage.../... was not successfully created: The following resource(s) failed to create: [InternetGateway, Vpc, AvailabilityZones]."? This is a solo instance.
Q: I just went live with a prod system today. I think I still have a slow memory leak but it there’s so much headroom that its stable anyway. Is it ok to occasionally kill one of the instances to flush this out? I was thinking of doing this each night until I can find the problem
Ah. We had that same problem. We still create a client on all function calls but have a
(def default-http-client
(delay (aws/default-http-client)))
statically declared. That is then passed to the client function via :http-client @default-http-client
. That solved the memory leak for us.The behavior you described sounded very similar. It would take ~1 day before an OOM would occur.