Fork me on GitHub
John Leidegren08:04:25

I'd like some input on general EAV data design. I've started using tuples to create unique identities so that I can enforce a uniqueness constraint over a set of refs within an entity. It looks something like this (tx map):

{:db/id 17592186045416
 :list-of-things [{:some-unique-identity [17592186045416 17592186045418]
                   :some-ref 17592186045418
                   :meta "foo"}
                  {:unique-identity [17592186045416 17592186045420]
                   :some-ref 17592186045420
                   :meta "bar"}]}
So, what I'm trying to do, is to prevent there from being two or more :some-ref for the same thing in this set of :list-of-things. Is this nuts or does this make sense? I'm worried I just invented some convoluted way of doing something which should be modelled different? If find tuple identities to be incredibly useful because I get this upsert behavior each time but I don't see how I can avoid the potential data race that would otherwise occur. Any suggestions here, would be much appreciated.

✔️ 4
John Leidegren11:04:36

I think I figured this out. --- These unique identity tuples are needed because I created entities to group attributes but that's already provided by namespaces. I could just has well let the namespaces encode the topology and let the grouping of things be based on that. --- These "topological" identities wouldn't be needed if I went for bigger, fatter entities, over excessive container like entities. These intermediate entities that encode a tree like structure are just causing me pain. And I will do away with them.

David Pham11:04:09

Does Datomic support Java11?


Is it possible to ssh in to the non-bastion instance in a Solo topology for Datomic Cloud?


I'm seeing some weird behavior in which a freshly restarted instance seems to get hung whenever I try to deploy (which fails).


Oddly, the CPU utilization, as visible on the CloudWatch dashboard, jump to 45% and stays there after I try a deploy, whereas before it's near 0%.


Once I start the deploy, neither an existing socks proxy nor a new one allows connections through to Datomic, whereas before the deploy it works fine.


I can datomic solo reset to get back to a working state…but if I try to deploy, I get back into the hung state.


I'd like to ssh in to see what process is using so much CPU.


I'm fairly perplexed by all of this. It's on a fresh AWS account and Datomic Cloud instance, and I've had success with Ions and the Solo topology before…


One more clue: the CodeDeploy deployment fails on the final ValidateService event. The script output has about 100 lines of [stdout]Received 000. I think this means that the Datomic health check request is failing to get a response at all, let alone a 200.


@jeff.terrell sometimes I jump into the Datomic nodes to do a jstack dump.


By default, the bastion cannot SSH to the nodes because of a security group


there is a security group that is called datomic-something-nodes


you need to modify that SG to allow the bastion instance in on port 22


Ah, security groups, right! I assumed that the compute node would already be configured to accept traffic from the bastion. But yeah, maybe not on port 22, right. Thanks!


Protip: you can add an entry to that security group, referring to the bastions security group symbolically


instead of hardcoding an IP address or CIDR block


then, you need the ssh keypair for the nodes, which you had when you created the stack


so what I do is I add the bastion's key from ~/.ssh/datomic-whatever to my ssh agent:


ssh-add ~/.ssh/datomic-whatever


then add the node keypair: ssh-add ~/wherever/nodekey


then I ssh to the bastion with -A, which forwards the local ssh-agent


then from there you can ssh to the node in question


sudo su datomic to become the datomic user


Ah, fantastic tips, thanks! I would have been stumbling around trying to scp the appropriate private keys over to the bastion.


so that you can run jstack on the pid, or poke around


My pleasure. Whatever you do end up finding, see if there is some signal of it in CloudWatch Logs or CodeDeploy or wherever

👍 4

if there's not, maybe worth a support ticket?


OK. I haven't seen any clue in those places yet. I'll be sure to follow up as needed to be sure others don't run into this.


When I got into the system, I learned that the CPU utilization was because of bin/run-with-restart being called to start some Datomic-related process over and over, which was failing every time. When I ran the command manually, it tells me:


> The specified log group does not exist. (Service: AWSLogs; Status Code: 400; Error Code: ResourceNotFoundException; Request ID: c58f56ce-3b7f-4be1-bbd3-463a950018c7)


…followed by a stack trace, and an anomaly map:

jeff.terrell18:04:18 #error {
 :cause "Unable to ensure system keyfiles"
 :data {:result [#:cognitect.anomalies{:category :cognitect.anomalies/incorrect} #:cognitect.anomalies{:category :cognitect.anomalies/incorrect}], :arg {:system "pp-app", :prefix "s3://"}}
 [{:type clojure.lang.ExceptionInfo
   :message "Unable to ensure system keyfiles"
   :data {:result [#:cognitect.anomalies{:category :cognitect.anomalies/incorrect} #:cognitect.anomalies{:category :cognitect.anomalies/incorrect}], :arg {:system "pp-app", :prefix "s3://"}}
   :at [$ensure_system_keyfiles_BANG_ invokeStatic "cluster_node.clj" 336]}]
 :trace ,,,}


I'm thinking this is not because of something I did wrong (though I'm happy to be corrected on that point). Opening a support ticket…