This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-09-20
Channels
- # 100-days-of-code (2)
- # aleph (53)
- # architecture (2)
- # aws (3)
- # beginners (230)
- # boot (15)
- # calva (3)
- # cider (19)
- # cljs-dev (1)
- # clojure (139)
- # clojure-conj (3)
- # clojure-italy (47)
- # clojure-nl (19)
- # clojure-spec (26)
- # clojure-uk (98)
- # clojurescript (152)
- # clojutre (4)
- # core-async (22)
- # cursive (5)
- # datomic (48)
- # emacs (11)
- # events (1)
- # figwheel-main (219)
- # fulcro (15)
- # instaparse (3)
- # jobs (4)
- # jobs-rus (1)
- # leiningen (30)
- # luminus (8)
- # off-topic (67)
- # onyx (5)
- # pedestal (16)
- # re-frame (1)
- # reagent (4)
- # reitit (31)
- # ring (8)
- # ring-swagger (3)
- # shadow-cljs (115)
- # specter (4)
- # videos (1)
- # vim (20)
- # yada (15)
Is anyone else experiencing intermittent Dynamo DB errors? My datomic instance will run for a week or so and then will crash with a message saying Critical failure, cannot continue: Heartbeat failed
.
Is there a recommended restart protocol to recover from these errors?
Hi @U06B77GQ6! Are your running in the HA configuration? That will recover automatically.
Also, make sure your DDB provisioning matches your needs, you can start by looking at https://docs.datomic.com/on-prem/capacity.html#dynamodb.
Hi @U072WS7PE! No, I am not using HA. I’m still in alpha testing for my app and we have a tiny amount of traffic and data (the entire database backed up is 336k w/o compression). I’ll take a look at those docs. Maybe we are pushing the defaults?
Thanks for the quick response. 🙂
if you are still early days I would look at Datomic Cloud -- it is less subject to this problem, and your overall AWS cost will almost certainly go down if you use ions.
Yeah, I’d like to transition there eventually. It would require rewriting my whole backend though.
I haven’t had any stability issues with the dev
storage. Would it be a terrible idea to just setup frequent backups to S3 for the time being?
Probably 😂. I would recommend just turning up the DDB knobs.
Will do. Thanks for the advice @U072WS7PE. 🙂
http://datomic.com works for me
ah, its an http-only redirect
(map :e (seq (d/datoms db :eavt)))
is a thing you could do
Probably pour it into a set
in case anyone else needs the same, :eavt needs to be in a map (seq (d/datoms (d/db conn) {:index :eavt}))
thanks nosesmith you are right but I wanted to know aside from go blocks blocking threads, in a system heavily dependent on go blocks should I be concerned about separating critical sections' thread pool from others ?
nothing critical should be in a go block
the number of threads used for go blocks is small, and doesn't grow, they are a method of coordination, not a utility for execution / parallelism
But can you elaborate a little more about "nothing critical should be in a go block" and "they are a method of coordination, not a utility for execution / parallelism"
Your comment kinda scared me, cause quite a few of my applications are actually a network of channels with async/go blocks doing almost all of the work in between them
they are for lightweight coordination of state between tasks, anything that uses blocking IO or is CPU intensive should not be in a go block
async/thread is OK for blocking IO or CPU intensive tasks
it returns a channel that a go block can park on
what happens is that when resource-intensive tasks are in go blocks, they can starve the channel operations, since the number of threads for go blocks are limited
interesting, I didn’t realize the api varied thusly
it also seems to max out at 1,000 values. hmmm...
(count (map #(nth % 3) (seq (d/datoms (d/db conn) {:index :eavt}))))
=> 1000
yup, that did it. the api docs refer to the top of the namespace for further docs. is the client open source?
@joshkh @U09R86PA4 Datomic client is Apache 2 license, license file and source code are present in the .jar
a query would probably overwhelm the instance, but you could also do [:find ?e :where [?e]]
ExceptionInfo Insufficient binding of db clause: [?e] would cause full scan clojure.core/ex-info (core.clj:4739)
that's what i'm worried about. basically, all i want is to clone a (cloud) database, and i don't care about history.