This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2015-10-27
Channels
- # beginners (22)
- # boot (652)
- # boulder-clojurians (1)
- # cider (19)
- # cljs-dev (3)
- # clojure (158)
- # clojure-dev (8)
- # clojure-nl (1)
- # clojure-poland (5)
- # clojure-russia (27)
- # clojure-sg (3)
- # clojure-za (4)
- # clojurescript (44)
- # community-development (2)
- # core-async (17)
- # core-logic (10)
- # css (1)
- # cursive (35)
- # data-science (5)
- # datascript (1)
- # datomic (90)
- # editors-rus (3)
- # events (3)
- # hoplon (90)
- # ldnclj (19)
- # lein-figwheel (2)
- # leiningen (1)
- # om (225)
- # reagent (1)
- # uncomplicate (27)
@bkamphaus: could you paste in an example map … I can’t get it working
conn-map {:protocol :sql
:db-name "customer"
:sql-driver-params "ssl=true;sslfactory=org.postgresql.ssl.NonValidatingFactory"
:jdbc-url (env :jdbc-database-url) }
conn (d/connect conn-map)
db (d/db conn)
Exception in thread "main" java.lang.IllegalArgumentException: :db.error/invalid-sql-connection Must supply jdbc url in uri, or DataSource or Callable<Connection> in protocolObject arg to Peer.connect, compiling:(web.clj:57:9)
Quick question will a datomic console count towards the peer process?
i believe it does
Does the speed of a cut querye (i.e. [:find ?e . :where …]
) depend on the number of possible results
;; there are 1366 of these
user> (time (d/q '[:find ?e . :where [?e :vulnerability/title]] db))
"Elapsed time: 1.272812 msecs"
17592186931182
;; there are 676457 of these
user> (time (d/q '[:find ?e . :where [?e :artifact-version/number]] db))
"Elapsed time: 461.450483 msecs”
17592187088881
I understand the problems with using time
in a repl, I get similar results using criterium
@max: As a general rule, the time cost of a query is proportional to the number of Datoms matched by each clause.
The :find ?e .
syntax is just a convenience for when you only expect one result.
Is datomic.db.DbId
an implementation detail or can I rely on tempid
always returning an instance of DbId
?
Not sure I follow. tempid
returns a tempid, which is different from an entity id. A temp id has to be resolved to the actual id assigned in the db with resolve-tempid
(in clojure) - http://docs.datomic.com/clojure/index.html#datomic.api/resolve-tempid
Oh you mean type
I don't think you should consider that guaranteed
instance of, yeah.
I want to be able to test if something is a tempid and then if that tempid is in the reserved space of -1..-1000000
Which is (and (instance? datomic.db.DbId x) (<= -1 (:idx x) -1000000)))
, assuming that part of the API can be relied on.
@alexmiller: If this were to change, would it be documented in the release notes?
I'd defer to @bkamphaus for any official answer but my personal opinion is that you are relying on implementation details that may change without warning (and may or may not be documented when they do)
Hmm, okay. Thanks @alexmiller. @bkamphaus, what do you think? Is there an official way to test if something is a tempid and to retrieve its index?
@domkm I’ll look into it. The standing answer for the mean time (and in general the default answer) consistent with @alexmiller ’s gut take is to consider undocumented specifics like types or members to be implementation details and not promised.
What’s the goal of the test?
Note that you can use with
( http://docs.datomic.com/clojure/#datomic.api/with ) to determine what will happen when you apply tx-data to a db without having to transact durably, which includes using resolve-tempid
with :tempids
in the returned map.
@bkamphaus: I am writing a function that composes mutators (transactor functions) together using d/with
and d/resolve-tempid
. It's based on this (https://github.com/webnf/webnf/blob/master/datomic%2Fsrc%2Fwebnf%2Fdatomic.clj#L181-L244) function. There are problems with aliasing tempids if any mutators create sub-mutations with reserved tempids but no problems with unreserved ones since those are unique (right?). I want to be able to test if a tempid is in the reserved range in order to throw an error instead of transacting invalid data.
@domkm still parsing this. What’s the difference in unique behavior you’re discussing? Potential for collision with transactor functions also generating tempids? All unique tempids in a single transaction should resolve to unique entity ids, except in the event of upserts (dup values for unique identity attrs).
@bkamphaus: In the linked function, if the same reserved negative tempid is used in multiple composed txs, it will result in aliased data. In other words, you can end up with an entity that is an invalid collection of facts (like an entity with both a :user/name
and an :org/name
). In terms of unique tempids, could using with
in a transaction introduce duplicate tempids?
hi @bkamphaus … update on the connection saga
(System/setProperty "datomic.sqlUser" user-value)
(System/setProperty "datomic.sqlPassword" password-value)
(System/setProperty "datomic.sqlDriverParams"
"ssl=true;sslfactory=org.postgresql.ssl.NonValidatingFactory”)
conn-map {:protocol :sql
:db-name "datomic"
:sql-driver-params "ssl=true;sslfactory=org.postgresql.ssl.NonValidatingFactory"
:sql-url simple-jdbc}
created! (d/create-database conn-map)
Exception in thread "main" java.util.concurrent.ExecutionException: org.postgresql.util.PSQLException: The server requested password-based authentication, but no password was provided., compiling:(web.clj:65:9)
@raymcdermott: I don’t believe the peer looks for user and pw system properties (those are documented for txor only), it will require that username and password be set in the URI params or e.g. on a data source provided in the map.
with data source, last time I tested and built an example was in Java, but if you stick with the map route maybe this will be helpful.
Java example config (parameters elided, but described in comments).
Object driver = java.sql.DriverManager.getDriver(sqlUrl);
Object driverClass = driver.getClass();
String name = driverClass.getName();
setURL() //JDBC portion of Datomic URI
setValidationQuery() // Use provider default or one you use in transactor properties file
setValidationInterval() // this should match the transactor properties file heartbeat-interval-msec
setTestOnBorrow() // set to true
setInitialSize(2)
.setDriverClassName() // ^ from code block above
.setUsername // username
.setPassword // password
.setConnectionProperties // anything else, i.e. what would be in transactor properties optional sql params.
For my local toy config, though, I just URL as string arg to connect
(though no SSL), e.g.
datomic:
@raymcdermott: have you tried everything hard-coded in one URI as string?
i.e. datomic:
the peer needs to connect to postgres IP
it will look up transactor endpoint from storage (transactor writes its location to storage as part of heartbeat)
ha! I think this may have been working before… I think the error is coming from the data in postgres
Thinking about it, I think I need to update the hostname in the transactor properties file so that it no longer uses the default local host
@raymcdermott: yeah, the transactor properties file hostname needs to be the address the peers can reach the transactor at.
I get it now… fingers crossed - now that I know what connections are going back and forth - I should get it started up
I will write a small blog once it’s all done (with appropriate attributions for support!!)
@raymcdermott: good luck Feel free to drop any additional questions here as needed.
@bkamphaus: Could you clarify for me under what circumstances multiple invocations of (tempid :db.part/user)
could possibly return duplicate tempids? There is no guarantee of uniqueness between transactor invocations and peer invocations, right? Inside a transactor function, can using tempid
inside and outside of with
potentially cause duplicates?
Note that I am asking about the unary tempid
function that returns a random-ish tempid. I understand the binary tempid
function.
@domkm sorry, my comment above was the opposite side of the uniqueness constraint. If you specify (tempid :db.part/user)
on multiple entities making assertions e.g. in a map with the same value for an attribute set as unique identity, then those will resolve as upserts on the same entity id.
Re: the collision issue re: user reserved (which you saw you understand, but just for clarification/verifying), I mean the issue described on group here: https://groups.google.com/forum/#!searchin/datomic/transactor$20function$20tempid/datomic/xRWXX0coMcI/wrBY-YMzbE8J
@bkamphaus: all is hooked up now… next to put some actual data and run a few queries
@bkamphaus: Thanks for the link. Magnar Sveen's last response answers my question about whether unreserved tempid conflicts can occur between a peer and the transactor and, as I suspected, they can. I don't think it answers the question about conflicts in a transactor function that uses with
. I was thinking that with
might cause the uniqueness-within-a-transaction guarantee of tempid
to be violated because with
is sort of like a transact
(probably shares a lot of the same code). I haven't been able to clearly describe this question so let me go put together an example.