This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-12-03
Channels
- # adventofcode (91)
- # announcements (7)
- # aws (3)
- # babashka (69)
- # beginners (46)
- # calva (30)
- # cider (12)
- # clj-kondo (88)
- # cljs-dev (11)
- # cljsrn (1)
- # clojure (195)
- # clojure-dev (21)
- # clojure-europe (2)
- # clojure-italy (13)
- # clojure-nl (56)
- # clojure-spec (4)
- # clojure-sweden (6)
- # clojure-uk (27)
- # clojurescript (179)
- # core-async (2)
- # cryogen (1)
- # cursive (2)
- # data-science (1)
- # datomic (57)
- # fulcro (15)
- # graalvm (9)
- # instaparse (6)
- # joker (18)
- # juxt (9)
- # leiningen (6)
- # off-topic (20)
- # other-languages (10)
- # pathom (5)
- # re-frame (20)
- # reitit (2)
- # rewrite-clj (5)
- # shadow-cljs (78)
- # sql (34)
- # tools-deps (128)
- # uncomplicate (16)
- # vim (6)
Is it possible to run Datomic Cloud on Fargate?
You can connect fargate clients to datomic cloud though, @briancabbott
Sorry, that is what I mean. Is there somewhere that I could find some documentation on how to do that?
Does anyone here on this channel have experience doing it?
Yep, there’s a few things to take care of: right subnet && right IAM policy on the fargate task role
when I create fargate service I need to create subnets for it. But in order to connect to datomic a service must be in the datomic subnet (as the documentation says). If I create a service and put in it datomic subnets then the service fails to even start. Can you give some insights where to look for the problem?
And confirm that the “nodes” security group is augmented to include ingress from your new subnets
Are you doing peering @U5JRN2PU0 or same vpc?
I wasn't able to connect datomic with fargate in the same VPC, so now I'm trying peering
what do you mean by "Confirm that the datomic subnets and the fargate subnets are mutually routable" ?
I've managed to do connecting within the same VPC from different subnets, thanks you!
nice -- what did you have to do @U5JRN2PU0?
I just allowed ingress traffic in <system>*-nodes security group from my services subnets' cidrs
hi, I have some questions about datomic analytics.. what is the "active workers" that we see at the presto server gui ? I have 3 instances for my query-group but I still get 1 active worker and 0 worker parallelism
it will use parallelism on that instance, but analytics support doesnt currently use multiple presto workers
ahn, cool. I am still trying to proper configure the query-group for my analytics needs. i) I noticed that on the cloud watch dashboards no action was happening on the query-group[I run the x-ray table from metabase on a 100M datoms database] and the bastion server had 100% cpu and this single worker node was the bastion server indeed.
but if this is the case, I dont understand the purpose of the query-group
itself. I thought those instances were performing the hard-work.
some of the work, yes (the datomic DB work) but some of the work (the SQL processing) happens on the gateway instance
however, I found an error on the sync -q <query-group-name>
config and fixed it [it was not recognizing the parameter and was taking the system name instead]. But now, the S3 bucket for analytics/
has two folders, one for my system-name
and another with query-group-name
. Is it right?
I have a very minor feature request for the datomic team. When you issue a push request the response contains the deploy command (e.g. clojure -A:ion-dev '{:op :deploy, :group elided, :rev \"0876503319a40bafffc8525f0597b1355b94b587\"}'
). The rev entry contains escaped quotes so I always have to paste this into a shell and then cursor over to the slashes and remove them. Any chance we can get a version in a future release that doesn't include the slashes? The status-command output does not require doing the above.
@iagwanderson seems fine - what was the thing you changed in the script?
but how do I tell the bastion to use the folder of the query-group-name
? Or I shouldn't?
I'm poking around and I deleted the folder system-name
and everything stop working 😃 Presto complains that no catalog was found. It seems it was always looking at the system-name
folder
“Analytics Endpoint Provide the name of a query group if you’d like analytic queries to go to a different endpoint. Defaults to system name.”
that will cause the access gateway to direct its analytics queries to the query group you specify
I cannot find this option when I click in update
in the cloudformation. It should be done when I first create the primary compute group?
it is available either way Are you running a split stack? you can’t do it with a master stack system
great! thanks for the help. I think would be nice to have these infos in the documentation Analytics Support -> Configuration. It seems I only needed to add the -q <query-group-name>
. I don't know, I am using this stack for over a month now and probably missed some instructions in the docs too.
@marshall just to recap the question about the usefulness of the query-group
with analytics-support in mind. Makes sense to say that for the analytics support, the access-gateway
should be priority when we talk about machine-sizing rather than the query-group
instances itself? As I understand, the presto can only make few operations before it start the SQL-processing of the data in memory. Maybe an access-gateway
more optimized for memory-intensive tasks should be a good call?
Hey all, I've got a datomic ion I'm trying to deploy and I keep getting a runtime error: "Syntax error (ClassNotFoundException) compiling new at (core.clj:79:38).
com.fasterxml.jackson.core.exc.InputCoercionException". This class was added in 2.10 (http://fasterxml.github.io/jackson-core/javadoc/2.10/com/fasterxml/jackson/core/exc/InputCoercionException.html). In my deps.edn file I specify com.fasterxml.jackson.core/jackson-core {:mvn/version "2.10.1"}
in my :deps map. However, when I push with clj -A:ion-dev '{:op :push}'
I get a dependency conflict warning for jackson-core listing com.fasterxml.jackson.core/jackson-core #:mvn{:version "2.9.8"}
as the version being used. This leads me to believe my specified version isn't taking. Any ideas as to how I specify/force the runtime version of a library in my datomic instance?
I was able to change my deps.edn file version of the jackson libs to 2.9.8 and it appears to be unbreaking. I'll just have to watch for version issues when pushing. I'm still interested in knowing if there's a way to control the deployed versions of the ion so that it uses the latest jars in the cloud.
Ahh. Good 'ol jackson.
Datomic's dependencies can't be overridden unfortunately. https://docs.datomic.com/cloud/ions/ions-reference.html#dependency-conflicts