Fork me on GitHub
#datomic
<
2019-05-31
>
joshkh11:05:18

we noticed a behaviour today with ion/get-params that has potential to knock down live services. get-params throws an exception if a key has more than 10 (or maybe it's 9) values:

(ion/get-params {:path (str "/datomic-shared/")})
ExceptionInfo Too many results  clojure.core/ex-info (core.clj:4739)
is that by design? one can innocently add a key to Parameter Store, and then any deployed services that call get-params at run time will fail.

grzm13:05:52

I believe this is on Cognitect's radar: the underlying issue is that the AWS call returns SSM parameters in batches of 10, along with a token that indicates whether there are more to fetch. ion/get-params doesn't check to see if there are more. They're not the first library to make this mistake: I found the same thing with Omniconf https://github.com/grammarly/omniconf . I believe it's fixed in Omniconf's 0.3.3-SNAPSHOT. Another workaround is to call the AWS api directly (@U0ENYLGTA’s AWS client library makes this a breeze: https://github.com/cognitect-labs/aws-api)

grzm13:05:02

Here's some code that can get you started on doing that:

hadils14:05:09

Here's source code that works for me. It gets all the params.

joshkh14:05:57

cheers to both of you. it just caught me by surprise how easy it was to tip over our API. yikes. i'll stick to aws-api.

souenzzo17:05:53

Is possible to use cast without datomic ions? I'm with a on-prem app, that we are working to move it to ions.

lilactown17:05:41

you might be able to create your own cast impl in the meantime

johnj17:05:26

When you have an entity or attribute that can be shared with other entities, what's considered best practice? to create a reference to the entity or attribute in order to have all the attributes of an entity with only one namespace or, just the shoved that entity/attr directly into an entity?

benoit18:05:43

Entities can and should have attributes from different namespaces. Namespaces scope a set of attributes, not "entity types".

👍 4
kenny18:05:37

Is there a way to make Datomic Cloud actually clean up the resources it creates? It a big pain to chase down everything it does. It also makes recreating a stack with the same name impossible.

kenny18:05:06

The CF stack does not delete an IAM policy (AdminPolicy) it creates.

grzm18:05:57

There are a number of other resources that are kept around as well.

grzm18:05:09

There are scripts out there that make this easier to handle in an automated fashion.

kenny18:05:28

It's super annoying that Datomic doesn't provide this.

grzm18:05:36

I sympathize. Though I try to set aside the things I find really annoying to those I'm not able handle with a script.

kenny18:05:53

That will block future stacks from being created with the same name with a CREATE_FAILURE due to resource already exists.

grzm18:05:49

I've been seeing these errors in lambda logs for Kinesis ions.

{:cognitect.anomalies/category :cognitect.anomalies/unavailable, :cognitect.anomalies/message "Connection refused", :clojio/throwable :java.net.ConnectException, :clojio/socket-error :connect, :clojio/at 1559312137252, :clojio/remote "10.213.43.136", :clojio/queue :queued-input, :datomic.ion.lambda.handler/retries 2}
They're not very informative. I'm guessing it's due to the lambda not being able to connect to the application on the server instance. I've been bouncing the applications to make them go away, but that's unsatisfying. Any strategies out there for digging into these?

anovick19:05:07

Hi! i'm interested in Datomic, but haven't had a chance to try it out yet. I would like to know more about the available operations permitted on data on the clients in a system with Datomic on the backend: # Read access on clients: Does a client have read access to an in-memory Datomic locally? e.g. being able to query without requiring network request to the peristent Datomic instance on the backend. # Write access on clients: Is it true that a client doesn't have write access to an in-memory Datomic? I'm thinking that this is the case since I've seen Datomic described as Strongly Consistent as opposed to Eventually Consistent which would have enabled clients to maintain own transaction log and ultimately synchronize with the backend when reconnected).

Dustin Getz20:05:51

writes serialize through transactor and thus require network; reads require network but can often deliver local performance (depending on your topology, Ions vs Cloud vs Onprem etc)

anovick20:05:21

@dustingetz thank you for the reply 🙂 so on an Onprem config the performance would be local but Ions/Cloud would depend on availability of AWS servers in the vicinity of the client

Dustin Getz20:05:19

you will always need the transactor and storage to be reachable