This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # announcements (7)
- # babashka (1)
- # beginners (25)
- # calva (7)
- # cider (15)
- # clj-kondo (13)
- # cljdoc (14)
- # clojure (151)
- # clojure-europe (4)
- # clojure-hamburg (2)
- # clojure-italy (22)
- # clojure-nl (57)
- # clojure-spec (12)
- # clojure-uk (6)
- # clojuredesign-podcast (5)
- # clojurescript (12)
- # core-async (8)
- # cursive (26)
- # datascript (9)
- # datomic (92)
- # emacs (4)
- # fulcro (7)
- # graalvm (1)
- # graphql (2)
- # instaparse (3)
- # jobs (1)
- # jvm (2)
- # kaocha (6)
- # nrepl (3)
- # off-topic (5)
- # re-frame (45)
- # reagent (5)
- # reitit (18)
- # ring (1)
- # shadow-cljs (89)
- # slack-help (9)
- # spacemacs (2)
- # sql (54)
- # tools-deps (75)
- # vim (28)
- # xtdb (17)
- # yada (31)
is there a way (without using the excellent tx-function examples shared by @andrew.sinclair) to retract multiple values from a cardinality many ref attribute? E.g. in (d/transact conn [ [:db/retract [:app/id "app_2"] :children [:component/id "Component 1"]]] I'd like to remove multiple children (components) at once.
Each retraction performed like this (with a list-form
:db/retract) translates directly into a single datom, so you would need to create a
:db/retract for each child you want to retract
Another question: similar to the example above, is it possible to remove a child's parent? I tried (d/transact conn [ [:db/retract [:component/id "Component 1"] :children [:app/id "app1"]]]) and I get an error: :cause ":db.error/not-an-entity Unable to resolve entity: :_children"
in this case you would want to retract from the parent entity, so something like:
[:db/retract [:app/id "app_1"] :app/children [:component/id "Component 1"]]
How to get all entities with a missing field? For example all customers that does not have :customer/address
Is there a way to rename a Datomic Cloud application? I need to change my dev environment to prod and I put dev in the name.
also for the codedeploy application name you have to recreate the compute stack, you cannot alter this parameter in cloudformation (I tried it last week)
Hello! I’m wondering if anyone can help me with a query that is looking for missing values that is timing out — I have the following function, which looks for
user entities that have an
id, but are missing the name.
My goal is to assemble those incomplete users, so I can do a scan to fetch their missing information. But the query for getting the
count times out.
I’m sure there’s a better way to write the query? THANK YOU!!!
(I’m running this via datomic proxy, if that matters…)
(defn count-uninitialized-users  " count users with missing names " (ffirst (let [conn (get-conn)] (d/q '[:find (count ?e) :where [?e :user/id] [(missing? $ ?e :user/name)]] (d/db conn))))) Execution error (ExceptionInfo) at datomic.client.api.async/ares (async.clj:58). processing clause: [?e :user/id ?id], message: java.util.concurrent.TimeoutException: Query canceled: timeout elapsed
I'm aware of three different strategies to deal with timeouts: First, there is a timeout option on the query object (see https://docs.datomic.com/on-prem/clojure/index.html#datomic.api/query) Second, I have found that using the async api can alleviate the problem (i think async helps when the result set is very large but I'm not certain) Third, switching to the index api rather than the query api is a last resort.
@genekim You can pass a
:timeout to the query if you think it’s an issue of it just being a long-running job https://docs.datomic.com/client-api/datomic.client.api.html
you’ll need to use the arity-1 version of the
Yep!!! Alas, still same result.
@jeroenvandijk — the approximate count of entities is 115K with names, and 1MM without names…
ah i see it is, never mind
@genekim make sure to pass
db to your functions, so that you can assemble together many functions that operate on the same
db basis. This is useful for things like reporting routines, or really anything where you need to take multiple looks at the database and ensure you're seeing the same thing
plus, it has the benefit of centralizing the d/db calls and cleaning up the function bodies
Thank you!!! People kept suggesting that to me, but I can’t say I actually understood why until your comment. That’s awesome!!!
@ghadi Ohhhhh…. Got it… I’ll look in the music-brainz code samples to see how it should look. Thank you!
I am building a new production instance of Datomic Cloud and want to downgrade the dev instance to Solo. My attempt at just upgrading failed. Any ideas?
harmless typo in the first sentence here: "reslts" 🙂 https://docs.datomic.com/cloud/query/query-data-reference.html#predicate-example
Has anyone deployed an application on AWS Lambda that uses Datomic Cloud? I'm curious how Datomic Cloud would handle many short-lived connections.
We have some lambdas doing that in production. The connection overhead is far overshadowed by the Clojure runtime boot up time and the VPC ENI provisioning time.
AWS is rolling out a fix for the ENI problem but the Clojure startup time is still a barrier for any customer-facing use cases (we just use lambda for background processing right now)
Thanks for the info. Couple other questions. Are you AOT'ing your code? About how many lambdas are running in parallel?
Yup we are AOTing the code, however, bootstrapping the Clojure runtime on a lambda incurs a ~1.9s hit for cold starts. We’ve tested this in Java lambdas that only depend on Clojure and just import the clojure.core ns.
That's totally fine for our application. Anything under 10s is probably ok actually. What is your actual startup time with all your production deps?
Hmm, we will have a significantly higher level of parallelism. Should be somewhat easy to test, I suppose.
Because of the VPC issue, it takes about 18s. However, we’ve tested in other availability zones that have the VPC fix and that goes down to ~8s.
Is the VPC issue only a problem when sources outside the VPC need to contact the Lambda?
The fix is likely only a few weeks away for us east 1. Can follow it here https://aws.amazon.com/blogs/compute/announcing-improved-vpc-networking-for-aws-lambda-functions/
Another option that we’ve considered is just exposing datomic over API gateway using ions w/ http direct to expose a REST-like interface for datomic operations and securing it with an IAM authorizer.
We have an internal library that we’ve soft open-sourced (use at your own risk): https://github.com/StediInc/lambda . Its pretty basic, just uses
clojure.core/compile and juxt.pack
We use lambda for pretty much everything so it has some extras in there for making it easy to manage multiple lambda entrypoints in a service.
Nice, will write it down to take a look at when we're ready to investigate more. Why'd you guys write your own lib instead of using one of the existing AWS Lambda clj libs?
None of them worked quite the way we wanted. We use quite a bit of middleware internally and wanted an interface more conducive to the pattern established by ring. Additionally, we do quite a bit with cloudformation and we have a library to make connecting lambdas to that easier. We wanted better control of the output artifacts so we could line that up better.
Looking at the org, I see cdk-clj. We use Pulumi at our org for deployment. It sounds like cdk-clj leverages jsii to work with a TypeScript API from the JVM. Is that correct? I'd be very interested in exploring that with Pulumi.
Pretty sure it was built specifically to make CDK cross-language compatible. I’m not aware of any other projects using jsii.
I'm not familiar with jsii. Does that mean the libraries need to be actually written in a jsii compatible way? Or is there declarative spec files the library needs to have?
I believe so, although this is stretching my knowledge at this point since we just did it to target CDK. The source library has to be in typescript and has some restrictions: https://github.com/aws/jsii/blob/master/docs/typescript-restrictions.md.
Our implementation requires the jsii bundle manifests that get produced in the JVM target config. No idea how hard it would be to build an existing project with jsii.
Not sure. Pulumi supports a few different languages now but I don't think they use jsii.
Not a high priority for us but definitely super interesting stuff. I'll bring it up with the Pulumi team to see if they have any input. Thanks for all this info - been super useful.
Ok. One of the services we're interested in running on Lambda needs some native libs. Seems like other people have used native libs on the jvm on Lambda.
You should be able to build the native lib on a docker image like https://hub.docker.com/r/lambci/lambda/ and then package it into a lambda layer https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html
If you have a pre-built binary thats compatible you can just put that on a lambda layer directly and ignore the docker step.
https://hub.docker.com/_/amazonlinux is what you’d want to build on, not the ci image I posted above.
That seems like a better path than manually extracting the native libs from the classpath. Thanks! What do you guys use for logging in Lambda? I've used CloudWatch logs in the past and can hardly stand it after using Datadog logs for a while.
i.e. Do you guys ship the CW Lambda logs to somewhere else? Do you have some other log viewer?
We use cloudwatch logs for lambda logging but we’ve been moving to using traces instead with Amazon Xray and then just linking the logs from Xray.
We also post all the cloudwatch logs to an elasticsearch instance. Although haven’t really used that since we’ve gotten Xray setup properly.
Do you have some wrapper around the Xray api? We use opentracing right now and have a
defn wrapper that can be used to instrument certain functions.
Yeah we just have a lightweight wrapper around the Xray java API. Nothing fancy. We are leveraging the annotation and metadata features pretty heavily, not sure how those interplay with opentracing. We’re pretty much all-in on AWS so we have tried to use their supplied libraries to get as much as we can out of their services.
Specifically this https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java.html. They will auto instrument calls to all AWS services (which we use a lot of) and to anything using the apache http client.
We started using the cognitect aws-api so we probably will miss out on the auto-instrumentation 😞
We did too although we are reverting back to the aws sdk. We like the interface for the aws-api lib by cognitect but its missing a lot of goodies. Luckily, the aws sdk v2 exposes a data class interface that allows for the same aws-api payloads to be marshalled into request classes. We are working on an internal lib that is basically a 1-1 replacement with the cognitect aws-api library.