This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-02-06
Channels
- # adventofcode (1)
- # aleph (9)
- # announcements (22)
- # beginners (59)
- # boot (8)
- # calva (1)
- # cljdoc (7)
- # cljs-dev (10)
- # cljsrn (9)
- # clojars (10)
- # clojure (23)
- # clojure-dev (6)
- # clojure-europe (3)
- # clojure-italy (26)
- # clojure-nl (3)
- # clojure-seattle (1)
- # clojure-spec (46)
- # clojure-uk (85)
- # clojurescript (97)
- # core-async (13)
- # cursive (3)
- # data-science (10)
- # datomic (156)
- # duct (34)
- # emacs (37)
- # figwheel (3)
- # figwheel-main (9)
- # fulcro (59)
- # hyperfiddle (4)
- # immutant (1)
- # jackdaw (3)
- # jobs (1)
- # off-topic (112)
- # parinfer (1)
- # qlkit (2)
- # re-frame (1)
- # reagent (35)
- # ring-swagger (2)
- # shadow-cljs (104)
- # spacemacs (9)
- # speculative (12)
- # tools-deps (30)
- # yada (10)
Hey guys! Sorry for a stupid question: how should I omit certain :db/id
s in my query? Tried with (not [?e :db/id ?currId])
but that clearly doesn't work.
Hi Everyone... we are currently running Datomic on-prem version 5067 and management is wondering what it would take to upgrade to the latest version? Any idea? Any Incompatibilities? (we use postgres as a DB).
Normally you can just stop the transactor, update and start the updated transactor, and then update all the clients, but some versions have had special cases. 5067 is pretty old so better check https://docs.datomic.com/on-prem/changes.html
and what would be a good way to do this? do a Datomic DB dump... install new version and import?
With dynamodb stop,update,start worked fine, but i'd do a backup before just to be sure
Weāve upgraded for 5 years (without changing storage) and have rarely had to do anything other than restart the txor with the newer version
Ion question: Can you deploy multiple projects to a single query group? We have several workloads that peak at different times and so could efficiently share the same compute resources. But they are different projects.
@jmiller Each query group can only be the target of a single Code Deploy application, so you could serve multiple applications from a single QG, but the code for all of them would need to be in the same Ion repository
Thanks. That was how it looked but I wanted to make sure.
Iām running into some unexpected behavior using CAS, I was hoping someone might be able to point me in the right direction. Iām trying to transact a payload:
[[:db/add "account" :account/email (:email identity)]
[:db/cas "account" :account/organization nil "org"]
[:db/add "org" :organization/name (:name args)]
[:db/add "org" :organization/id id]]
Where :account/organization
is cardinality one and :account/email
is unique identity. Iād expect that if I ran this transaction twice, the :db/cas
would fail as the organization is no longer nil, however, I am able to transact this many times without failure. Feels like Iām missing something fundamental but I cannot see it.tempids
Those string tempids identify a new entity each time the transaction runs
:account/email maybe is not set to :identity ?
Ah, so if I break it into two steps: Upsert the account and then upsert the organization it should work?
You shouldnāt have to do that
If I upsert the account first and then transact:
[[:db/cas [:account/email (:email identity)] :account/organization nil "org"]
[:db/add "org" :organization/name (:name args)]
[:db/add "org" :organization/id id]]
It behaves as expected.It could be a bug
Is the first version dangling a bunch of entities or something? Is there any evidence that an upsert didnāt resolve?
And in the second form @ 3:44?
{:db/id "account" :account/email (:email identity)}
{:db/id "org"
:organization/name (:name args)
:organization/id id}
[:db/cas "account" :account/organization nil "org"]
Fwiw it seems to only be a problem when there is a tempid in the entity position of :db/cas
I also tried hacking it with a boolean attribute to rule out weirdness with two entities and it still doesnāt work. This is on datomic cloud btw.
interesting. iām guessing there is an issue with resolving the tempid passed to a db fn
In my experience db fns do not get their tempids resolved (at least for strings this seems impossible, since you don't know what type the db fn expects: a real string or a tempid), and it is impossible to resolve a tempid within a transaction. So in practice you can't use tempids with tx fns unless you can expand to something where it doesn't matter
A larger question is when is tempid resolution performed? It seems impossible in the most general case to know that a tempid resolves to an eid because of an indexed/upserting attribute+value. You can't see those assertions until all expansion is done, and you can't expand everything without running the tx fns.
The docs do say an entity id specifically https://docs.datomic.com/cloud/transactions/transaction-functions.html#sec-1-2
Tx fns need to be written defensively. If you need a real eid (eg to read from the vs just expanding to something template-like) use d/entid on input to coerce it, check for nils and strings and throw if you canāt make sense of it
Cas should be able to look at the valueType of the argument to know of a string is a ref or scalar
āIn my experience db fns do not get their tempids resolved (at least for strings this seems impossible, since you donāt know what type the db fn expects: a real string or a tempid), and it is impossible to resolve a tempid within a transaction. So in practice you canāt use tempids with tx fns unless you can expand to something where it doesnāt matterā This doesnāt entirely follow for me, can you say more
[:db.fn/cas "some-string" :a "b" "c"]
--- is "some-string" a tempid placeholder or a literal string
without fancy type info how can datomic know? so datomic just invokes the fn as-is, with a string
(with tempid records--the old tempid method--at least it had a type so conceivably datomic could do deep walks of all tx fn input and replace tempids with eids)
second problem: tempids that resolve to existing eids need to do so by datomic detecting a [:db/add "tempid" :unique-attr "unique-value"]
pattern in the tx data
however, you can only look for those patterns after all tx expansion is done (including those by tx fns)
so how can you substitute a tempid for a real id in a tx fn call if you can't know what substitutions to make until all tx fns are evaluated?
there might be hacky half-measure ways through this (double expansion for eg, or doing substitution eagerly during expansion instead of after, or implicit restrictions on tx fn behavior) but datomic doesn't do that
so bottom line, tx fns get as arguments exactly what was present as parameters (no transformation) and it's up to the tx fn to interpret it and throw if it can't do it's work
d/entid can help with coercing lookup ref and ident sugar to real eids and you should use it in tx fn bodies where you want to allow such sugar (and check for nil)
and if your tx fn needs a real eid e.g. because it wants to do a db read and check some invariant, it can't accept a tempid
You lost me in the first part, i think the āfancy type infoā is perfectly well defined, e.g. this cas is on a :db.valueType/long which is known because :account/balance has schema [[:db.fn/cas 42 :account/balance 100 110]]
the function knows the type of its arguments
i dont see why
why does the caller need to know that, i would implement cas to pass the entity name to d/entity or some such
suppose create-fancy-thing expands to [:db/add "foo" :thing-type :fancy][:db/add "foo" :upserting-thing-id 123]
suppose :upserting-thing-id exists already, so "foo"
should resolve to an existing eid
i dont see why it matters, :db.fn/cas knows āfooā can be treated as a ref, and in proper Clojure form allows us to shoot foot off if malformed
i see
so the effect has to be delayed, which means txfns need an expansion phase and an effect phase
you'll have to expand again, this time with the tempid mapping available (e.g. smuggled in through d/entid)
by not pure you mean (launch-missles!) or something different
just that calling with same arguments+db does not produce exactly the same value every single time
This is all onprem anyway, Cloud doesnāt even have this, right?
another approach: with a fancy enough type system you could discover all the places in the un-expanded tx that had tempids, then you could make a flow graph
i think cloud has a closed set of txfns? NOt sure
āclasspath transaction functionsā you are right
āTransaction functions must be pure functions, i.e. free of side effects.ā
so squuid is disallowed (and not a thing anymore anyway)
i think they mean the former
Thanks for the deep dive
the tx fn can't produce the same value for the same input (or at least can't without severely limiting its usefulness)
Datomic free uses io.netty/netty-all 4.0.39.Final
. We are using other dependencies that require Netty 4.1. If I try excluding io.netty/netty-all
from Datomic, I get an exception:
Execution error (ClassNotFoundException) at java.net.URLClassLoader/findClass (URLClassLoader.java:382).
io.netty.handler.codec.http.HttpRequest
Is there a way to make Datomic work with a newer version of Netty?Seems like this has occurred before: https://clojurians-log.clojureverse.org/datomic/2016-07-04. Any solutions?
I updated to org.apache.activemq/artemis-core-client {:mvn/version "1.5.0"}
and it appears to work.
do ions have any story for application initialization? e.g. ion-starter just does a lazy initialization of the db in get-connection ā presumably thereās no lifecycle to hook into for application initialization like this?
In my experience db fns do not get their tempids resolved (at least for strings this seems impossible, since you don't know what type the db fn expects: a real string or a tempid), and it is impossible to resolve a tempid within a transaction. So in practice you can't use tempids with tx fns unless you can expand to something where it doesn't matter
https://github.com/Datomic/ion-event-example -- would anyone in here mind cloning and running clojure deps
to verify it's not just me this is broken for?
@devn clojure deps
what are you expecting to happen when you run that command, and what are you getting?
org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for com.datomic:ion:jar:0.9.26
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.datomic:ion:pom:0.9.26 from/to datomic-cloud (): Unable to load AWS credentials from any provider in the chain
Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain
Doesnt matter what they are, as that repo is publicly readable, but they need to be available
i have creds in ~/.aws/config
-- two profiles, with no default. i will try to s3 ls
that bucket.
actually, turned out that it wasn't a default
missing, but instead that i hadn't updated my session credentials. my bad! thanks again.
āIn my experience db fns do not get their tempids resolved (at least for strings this seems impossible, since you donāt know what type the db fn expects: a real string or a tempid), and it is impossible to resolve a tempid within a transaction. So in practice you canāt use tempids with tx fns unless you can expand to something where it doesnāt matterā This doesnāt entirely follow for me, can you say more