This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-08-14
Channels
- # beginners (116)
- # boot (6)
- # cider (16)
- # cljdoc (5)
- # cljsrn (3)
- # clojure (156)
- # clojure-conj (5)
- # clojure-finland (1)
- # clojure-italy (46)
- # clojure-mexico (36)
- # clojure-russia (2)
- # clojure-sanfrancisco (1)
- # clojure-spec (10)
- # clojure-uk (69)
- # clojurescript (71)
- # core-async (2)
- # cursive (18)
- # datomic (91)
- # duct (2)
- # editors (1)
- # emacs (5)
- # figwheel-main (11)
- # fulcro (31)
- # graphql (8)
- # hispano (1)
- # jobs (1)
- # jobs-discuss (2)
- # lambdaisland (1)
- # london-clojurians (1)
- # nyc (1)
- # off-topic (104)
- # onyx (34)
- # parinfer (3)
- # pedestal (2)
- # precept (17)
- # protorepl (2)
- # re-frame (13)
- # reagent (4)
- # ring-swagger (52)
- # shadow-cljs (92)
- # spacemacs (17)
- # sql (14)
- # tools-deps (6)
- # unrepl (4)
- # vim (39)
@stuarthalloway any chance you could comment on this discussion about Ion Lambdas? https://clojurians.slack.com/archives/CC0A7PUHF/p1533611391000066
@steveb8n what is the performance requirement?
it’s arbitrary i.e. I’m determining what is acceptable but multiple 15 second waits is not gonna work
hence I’m considering options to work-around it but would prefer to avoid extra complexity
who is calling the lambdas? events don't care...
api-gateway has independent timeout and retry capability, so why tie yourself to lambda?
clarifying to ensure we are talking about same thing: re-frame client, xhr request to API-GW which invokes Ion Lambda.
so if API gateway has a 5 second timeout you will never see a 15 second delay from lambda
how many endpoints are we talking about? how often are they invoked?
how many 9s are you promising?
The Ion architecture seems to suggest 1 lambda per read or write fn so many for a SPA app
categorically:
1. Lambdas will become less latent
2. There are other options for wiring API Gateway that don't go through Lambda at all, and to the extent that #1 is a problem for applications we can investigate and implement them
API gateway timeout is not (by itself) downtime, because API gateway should automate retry as well
interesting. it’s been suggested by others that the cold-start is likely due to ENI init for Lambdas in VPCs which is a well known problem. on that basis, not using Lambdas would help
I would like to see an actual app, under actual load, with API gateway timeout and retry configured, that has a problem meeting a particular SLA. I say this not in the "I don't believe you" sense, but in the "particular example we can use to make Datomic better" sense
full disclosure - I’m giving the e.g. of API-GW but in my case I’m using AppSync and not API-GW. same end result
I know less about AppSync, does it have timeout/retry config?
I understand this could be a problem, I just want to grab it hard by the specifics and not some generalized concern. We are committed to doing the best thing possible under AWS.
but FWIW graphql as a service without code is great. Fits really well with Ions as well. Hence I’m really happy with this stack
Good to hear, and when you have the concrete perf problem in hand we will jump on it!
I appreciate your position also. Others are interested also. What form would be best to present the perf profile to you/team? is a video of client good enough?
even just some prose about the objectives and numbers here for starters
last point: my arbitrary requirement is that 15 secs is too long for users but 2-3 secs is fine. now that Clojure (tools.deps) starts in 2-3 secs, this would be “arbitrarially” acceptable for my use case
ok. I’ll see if I can get a few others on board to form up a description of stack/UX/requirements
not looking for more than the kind of prose you are producing now
thank you for using Datomic!
What is the proper way to restart Datomic Cloud? All of a sudden it stopped working for us and simply restarting the EC2 instance did not work
Also curious if there’s a best practice. When DynamoDB throttles, the transactor dies and does NOT come back. At that point I stop the entire CloudFormation and start it again. Now I err on the side of too much DynamoDB capacity, unfortunately.
@U8ZA3QZTJ Datomic Cloud does not have transactors. Are you talking about On-Prem?
Oops I got confused. It’s cloud from my perspective but it’s not Datomic Cloud. My comment above isn’t relevant, then.
@U8ZA3QZTJ if you run transactors in an ASG (as our template does) you should have HA in On-Prem also: https://docs.datomic.com/on-prem/ha.html
also @U8ZA3QZTJ Cloud puts much less pressure on DynamoDB, so that would be less of (or not) a problem there
can I confirm that retracting a :db.type/ref
attribute of an entity that's isComponent true
will only retract the entity's immediate sub-components, not recursively their sub-components ?
@U0CKDHF4L retract is primitive and does exactly what you tell it. retractEntity works per https://docs.datomic.com/cloud/transactions/transaction-functions.html#sec-1-1, "components of the given entity are also recursively retracted."
ah so :db/retract will not recursively retract the component's subtree
(or retracting the entity itself)
- in Cloud - I just tried and I get only one-level of retraction which is what I'd expect and what I want
Is there any sort of counter or time for a given entity that is updated whenever there is a transaction with that entity?
Not for a certain attribute, but for every attribute used with the entity
something like (last-modified eid)
@alqvist Not ‘built-in’. You can use a combination of query and the log to determine that or you can track it explicity yourself
@marshall Ok, thanks. Probably will add another attribute since I am unsure of the performance implications
I have an interesting issue. I am connecting to my local datomic transactor using this connection string: "<postgresql://localhost:5432/datomic?user=...&password=...>". When I uberjar the application and run it, I get this exception: java.sql.SQLException: No suitable driver. Any ideas? For my edification, where is the driver specified for the peer in the first place (i.e. why does it even work in the REPL)?
After Datomic Cloud just stopped we checked the logs to see what went wrong; we did not find any alerts. The log just ends suddenly, with normal messages regarding Datomic's compute stack. What should we look for?
How do you distinguish stopped from working?
@stuarthalloway our application could no longer connect to Datomic - we started seeing
clojure.lang.ExceptionInfo: Connect Timeout
cognitect.anomalies/category: :cognitect.anomalies/unavailable
cognitect.anomalies/message: "Connect Timeout"
We saw the same error both on our dev machines and also in the logs of a deployed version (uses the same dev database). The load is fairly low as we are in development and there are only several users. It has been working for months before that.
are you connecting through the socks proxy?
for simplicity we have just exposed the port and we are connecting directly
@stuarthalloway so what would be the difference in the logs in that case?
This is in my pom:
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.1.1</version>
</dependency>
And both the transactor and peer work fine if the peer is a REPL.So, the inclusion of the jar in the project should be sufficient, though? If that is the case I'll do some poking around in my uberjar to make sure everything was included.
yeah, should be. java -cp your-uberjar.jar clojure.main -e org.postgresql.Driver
<-- does that error out?
Nope, it produces "org.postgresql.Driver".
If multiple sql driver implementations are on the classpath will that cause any issues (e.g. postgres and the ms driver)?
unlikely, but there may be interactions with ServiceLoaders (a JVM mechanism): What does this return:
unzip -l your-uberjar.jar | grep 'META-INF/services'
These are the entries in META-INF/services: com.fasterxml.jackson.core.JsonFactory io.undertow.protocols.alpn.ALPNProvider io.undertow.predicate.PredicateBuilder io.undertow.attribute.ExchangeAttributeBuilder io.undertow.server.handlers.builder.HandlerBuilder io.undertow.client.ClientProvider org.xnio.XnioProvider javax.servlet.ServletContainerInitializer io.undertow.servlet.ServletExtension javax.websocket.ContainerProvider io.undertow.websockets.jsr.WebsocketClientSslProvider javax.websocket.server.ServerEndpointConfig$Configurator java.sql.Driver javax.validation.spi.ValidationProvider org.apache.commons.logging.LogFactory javax.json.spi.JsonProvider com.fasterxml.jackson.core.ObjectCodec
The java.sql.Driver's contents are "org.h2.Driver"
Should I be seeing the postgres classname in the java.sql.Driver file?
I am not 100% sure if how Datomic loads drivers. I'll defer to someone more knowledgeable.
Interesting, when I build the uberjar with lein vs. mvn I get the other drivers in my java.sql.Driver file. I'm guessing it's my mvn assembly configuration that's the problem. I'll poke on that. Thanks!
Closing the loop on this, it looks like it's the way the maven assembly plugin merges (or doesn't merge) the services file. The solution is to do something along these lines https://maven.apache.org/plugins/maven-assembly-plugin/examples/single/using-container-descriptor-handlers.html or use the shade plugin. Thanks again for the help.
has anyone experienced a StackOverflowError while trying to redirect the (cast/…) functionality? I thought I followed the guide, but I’m seeing the following error when I try to call cast/dev
in the repl:
StackOverflowError
clojure.lang.RT.seq (RT.java:530)
clojure.core/seq--5124 (core.clj:137)
clojure.core/drop/step--5646 (core.clj:2919)
clojure.core/drop/fn--5649 (core.clj:2924)
clojure.lang.LazySeq.sval (LazySeq.java:40)
clojure.lang.LazySeq.seq (LazySeq.java:49)
clojure.lang.RT.seq (RT.java:528)
clojure.core/seq--5124 (core.clj:137)
clojure.core/take/fn--5630 (core.clj:2876)
clojure.lang.LazySeq.sval (LazySeq.java:40)
clojure.lang.LazySeq.seq (LazySeq.java:49)
clojure.lang.RT.seq (RT.java:528)
With Datomic on-prem, is it possible to connect to a DynamoDB table owned by a different AWS account? If so, what does the URI look like for that?
Maybe someone can help me stretch the datalog part of my brain?
I'm trying to write a generic query that returns a particular attribute called :global/uuid
for any referenced entities that are directly related. And for silly reasons I need to act exactly like a normal pull:
({:global/uuid #uuid"d7becc50-b43a-4873-805d-890a612174ea",
:person/email "gNJf4D4OJv6I4oc",
; Example A:
:person/manages [#:db{:id 1697645953286222}]})
Except that {:db/id} is replaced with an entity's :global/uuid
:
({:global/uuid #uuid"d7becc50-b43a-4873-805d-890a612174ea",
:person/email "gNJf4D4OJv6I4oc",
; Example B:
:person/manages [#:global{:uuid #uuid"e9c98197-9cc0-45fd-8dce-59356d780dbd"}]})
I could easily get the value back if the reference was a component, but that's not the behaviour I want in the schema.
I could use two pulls, but that returns :global/uuid in its own collection and I want it to be in the value of the :person/manages
key.
I could use Map Specifications, but the query needs to be generic. So wild card support like: (pull ?a [* {* {:global/uuid}}])
which I don't think exists.What part of the query are you trying to make generic here, the pull spec, or the conditions?
in my case every entity in datomic has a :global/uuid, and i want to always return the uuids for entities referenced by whichever entity is being pulled.
i managed to do it using spec and a function to build queries from that spec but it feels like an abomination
(assoc :find (vector (list 'pull '?root (conj without-refs map-spec))))
where map-spec
is a map specification built from a collection of attributes defined in a spec somewhere