Fork me on GitHub
#datomic
<
2017-06-05
>
nadejde06:06:24

Hi everyone! I’m having a problem with datomic client. I’m playing with a small app to get some play time with datomic. But the moment I try to add the datomic client dependency my project breaks this is my project file:

clojure
(defproject rail-cards-api "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :dependencies [[org.clojure/clojure "1.8.0"]
                 [metosin/compojure-api "1.1.10"]
                 [buddy/buddy-core "1.2.0"]
                 [com.datomic/clj-client "0.8.606"]
                 [org.clojure/core.async "0.3.443"]]
  :ring {:handler rail-cards-api.handler/app}
  :uberjar-name "server.jar"
  :profiles {:dev [:project/dev :profiles/dev]
             :project/dev {:dependencies [[javax.servlet/javax.servlet-api "3.1.0"]
                                  [cheshire "5.5.0"]
                                  [ring/ring-mock "0.3.0"]]
                   :plugins [[lein-ring "0.10.0"]]}
             :profiles/dev {}})
and this is the error i get:
bash
lein ring server
2017-06-04 23:44:35.300:INFO::main: Logging initialized @1619ms
Exception in thread "main" java.lang.NoClassDefFoundError: org/eclipse/jetty/http/HttpParser$ProxyHandler, compiling:(ring/adapter/jetty.clj:27:11)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6875)
	at clojure.lang.Compiler.analyze(Compiler.java:6669)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6856)
	at clojure.lang.Compiler.analyze(Compiler.java:6669)
	at clojure.lang.Compiler.analyze(Compiler.java:6625)
		at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:6001)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6868)
	at clojure.lang.Compiler.analyze(Compiler.java:6669)
	at clojure.lang.Compiler.analyze(Compiler.java:6625)
	at clojure.lang.Compiler$IfExpr$Parser.parse(Compiler.java:2797)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6868)
	at clojure.lang.Compiler.analyze(Compiler.java:6669)
	at clojure.lang.Compiler.analyzeSeq(Compiler.java:6856)
	at clojure.lang.Compiler.analyze(Compiler.java:6669)
it seems to have a problem with the ring plugin? This happens the seccond I add the [com.datomic/clj-client “0.8.606”] dependency. Tests still run fine though… Am I doing it wrong?

nadejde06:06:03

I’m thinking that perhaps I need to include the datomic dependency differently? The one on Maven seems to be quite old.

isaac06:06:24

I have got this error early time, both ring & clj-client depends on jetty, you need fix the conflicts of dependencies. use lein deps :tree show the confilict.

nadejde06:06:38

Thank you @isaac will do that. New to clojure didn’t think to check the dependency tree:)

nadejde07:06:34

Got it to build like this:

(defproject rail-cards-api "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :dependencies [[org.clojure/clojure "1.8.0"]
                 [metosin/compojure-api "1.1.10"]
                 [buddy/buddy-core "1.2.0"]
                 [com.datomic/clj-client "0.8.606" :exclusions [org.eclipse.jetty/jetty-client
                                                                org.eclipse.jetty/jetty-http
                                                                org.eclipse.jetty/jetty-util]]
                 [org.clojure/core.async "0.3.443"]
                 [org.eclipse.jetty/jetty-util "9.2.21.v20170120"]
                 [org.eclipse.jetty/jetty-http "9.2.21.v20170120"]
                 [org.eclipse.jetty/jetty-util "9.2.21.v20170120"]]
  :ring {:handler rail-cards-api.handler/app}
  :uberjar-name "server.jar"
  :profiles {:dev [:project/dev :profiles/dev]
             :project/dev {:dependencies [[javax.servlet/javax.servlet-api "3.1.0"]
                                          [cheshire "5.7.1"]
                                          [ring/ring-mock "0.3.0"]]
                   :plugins [[lein-ring "0.12.0"]]}
             :profiles/dev {}})

nadejde07:06:41

is there a way to apply the exclusions for datomic client just in the dev profile while loading them in a production scenario? The conflict seems to be with lein-ring plugin. I will probably not be using that in a production environment right?

nadejde07:06:33

I guess what I’m asking is if a dependency added in the profiles overrides the one in the project?

favila15:06:52

dependencies declared higher (closer to your project) in the dep tree override ones declared lower @nadejde

favila15:06:17

"Dependency mediation" describes how deps are chosen

nadejde15:06:27

Hi. Thank you. Ended up with this:

(defproject rail-cards-api "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :dependencies [[org.clojure/clojure "1.8.0"]
                 [metosin/compojure-api "1.1.10"]
                 [buddy/buddy-core "1.2.0"]
                 [com.datomic/clj-client "0.8.606" ]
                 [org.clojure/core.async "0.3.443"]]
  :ring {:handler rail-cards-api.handler/app}
  :uberjar-name "server.jar"
  :profiles {:dev [:project/dev :profiles/dev]
             :project/dev {:dependencies [[javax.servlet/javax.servlet-api "3.1.0"]
                                          [cheshire "5.7.1"]
                                          [ring/ring-mock "0.3.0"]
                                          [com.datomic/clj-client "0.8.606" :exclusions [org.eclipse.jetty/jetty-client
                                                                                         org.eclipse.jetty/jetty-http
                                                                                         org.eclipse.jetty/jetty-util]]]
                   :plugins [[lein-ring "0.12.0"]]}
             :profiles/dev {}})
seems to work

favila15:06:45

lein also has a "pedantic" mode which rejects ambiguous deps

mgrbyte15:06:58

Is there standard way to model hierarchical keywords? I'm thinking along the lines of: {:type my-type :item some-entity} , where my-type can be derived as per clojure.core/derive. The use case is to query all entities that have a sub-type.

robert-stuttaford15:06:59

basically, though, Datomic’s data model is so flexible, you can do what you describe quite comfortably

robert-stuttaford15:06:49

there was a talk given at 2013 Conj about this but i can’t find it

mgrbyte15:06:59

I meant keywords not "collections"

robert-stuttaford15:06:27

don’t think i’ve considered that before. generally speaking, i find it’s better to be explicit about what you’re asking for. there’s a running joke (at least in the parts of the Clojure community i subscribe to) that there’s no good use for derive yet

mgrbyte15:06:54

context: I'm modelling interactions between two or more biological entities, which can have an ontological type

mgrbyte15:06:12

In pure clojure, using clojure.core/derive would work as per https://clojure.org/reference/multimethods

mgrbyte15:06:53

or a java class hierarchy would work with an embeded isa? in the query

robert-stuttaford15:06:05

this is interesting 🙂

mgrbyte15:06:28

I was wondering if anyone has had the need for such a construct....

robert-stuttaford15:06:46

my gut says that derive shenanigans probably won’t work, because attrs are actually just db/idents that resolve to integer ids for the schema that they represent

mgrbyte15:06:38

yep, that makes sense. i.e idents are "concrete" keywords

mgrbyte15:06:29

I'll have to think on this some more, perhaps there's an alternate way to achieve the same thing: datalog query for: find all entities who's type t matches (ancestor t) or (child t) (made up notation)

robert-stuttaford15:06:49

you could model these relationships explicitly, and write Datalog rules that encompass these lookups

robert-stuttaford15:06:55

and then use those wherever

robert-stuttaford15:06:21

https://github.com/Datomic/mbrainz-sample has nice examples of rules and even recursive rules

mgrbyte15:06:34

I could do it with string prefix matching, since the (namespaced) keywords will follow the same pattern. e.g: :interaction.type.genetic/supressing :interaction.type.genetic.supresssing/sub

mgrbyte15:06:24

datalog clause: [(= (namespace ?a) "interation.type.genetic.supressing)]

mgrbyte15:06:57

not sure about the efficiency of this however

robert-stuttaford15:06:52

efficiency all depends on how often you do it and how much data you do it on

mgrbyte16:06:42

often: not very often, possible to cache results. volume: ~1/2 a million entities

mgrbyte16:06:30

somewhere in the region of 200 distinct (but ontological) types

mgrbyte16:06:33

anyhow, thanks for sounding me out @robert-stuttaford 🙂

nadejde19:06:54

Hi. Does anyone have any ideea what version of ring is compatible with [com.datomic/clj-client “0.8.606”]?

nadejde19:06:26

I get some conflict in jetty dependency. datomic works with 9.2.21.v20170120 but ring does not

nadejde19:06:58

If i try with a higher version ring works but datomic client does not:(

nadejde19:06:00

how do I deal with this?

rustam.gilaztdinov22:06:45

Hello! I want to use docker-compose for docker with postgres storage and guess what? This isn’t working 😃 My steps: 1. dockerfile with postgres 2. dockerfile with datomic-transcator, which depends on 1 3. dockerfile with datomic-peer, which depends on 2 4. dockerfile with datomic-console This is my docker-compose.yml, 2 first steps

version: "2"
services:
  db:
    build:
      context: ./postgres
      dockerfile: Dockerfile
    environment:
      - POSTGRES_USER=datomic
      - POSTGRES_PASSWORD=datomic
      - POSTGRES_DB=datomic
    ports:
      - "5432:5432"
  datomic-transactor:
    build:
      context: ./datomic-transactor
      dockerfile: Dockerfile
    ports:
      - "4336:4336"
      - "4335:4335"
      - "4334:4334"
    volumes:
      - "/data"
      - "/log"
    depends_on:
      - db
In step 1 I add sql-snippets from bin/sql to image and execute it In step 2, in config/sql.templates I add this
protocol=sql
host=0.0.0.0 (???) 
port=4334
alt-host=datomic-transactor
license-key=...
sql-url=jdbc: (db, not localhost!!, see docker-compose, where service name is db and this is default name for network)
sql-user=datomic
sql-password=datomic
When I execute docker-compose build, this happening
Starting datomic:sql://<DB-NAME>?jdbc:, you may need to change the user and password parameters to work with your jdbc driver ...
System started datomic:sql://<DB-NAME>?jdbc:, you may need to change the user and password parameters to work with your jdbc driver
Critical failure, cannot continue: Lifecycle thread failed
java.util.concurrent.ExecutionException: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

rustam.gilaztdinov22:06:35

When I go to container with postgres, everything work fine, I can execute psql -h localhost -d datomic -U datomic

rustam.gilaztdinov22:06:10

I search all across github, but nobody share solution =( All work with dev properties

favila23:06:24

can you connect to the postgresql container from outside the doctor container? could the postgresql user restrict permissions by ip?

favila23:06:20

can you run a transactor from outside?

rustam.gilaztdinov23:06:36

Nope, my build fails

rustam.gilaztdinov23:06:55

And container doesn’t creates

favila23:06:24

This is really a docker question. I don't know why the transactor container can't connect to the postgres container. My advice is only general: simplify the problem. Verify that a simple container (no transactor) can connect, work from there

rustam.gilaztdinov23:06:51

Now I can’t run just datomic with postgres storage local, not in Docker My steps — launch bin/transctor with sql properties, then bin/replor bin/shell, where I define db-uri

datomic % uri = "datomic:"
datomic % Peer.createDatabase(uri);
/ Error: // Uncaught Exception: bsh.TargetError: Method Invocation Peer.createDatabase : at Line: 2 : in file: <unknown file> : Peer .createDatabase ( uri )

Target exception: java.util.concurrent.ExecutionException: org.postgresql.util.PSQLException:

favila23:06:04

database is "demos"?

favila23:06:33

did you get that backwards?

favila23:06:03

"datomic:"

rustam.gilaztdinov23:06:42

Yes, database, which I want to create is “demos” I try your approach, still same error

favila23:06:03

missing username and password

favila23:06:07

what is the exception precisely?

favila23:06:10

the connection uri = I would expect on the peer for a db named demos, given your transactor startup, is datomic:

favila23:06:35

you may still get an error about not being able to connect to the transactor, but not a postgresql connection error

rustam.gilaztdinov23:06:33

no, this is psql error

// Error: // Uncaught Exception: bsh.TargetError: Method Invocation Peer.createDatabase : at Line: 2 : in file: <unknown file> : Peer .createDatabase ( uri )

Target exception: java.util.concurrent.ExecutionException: org.postgresql.util.PSQLException: The connection attempt failed.

java.util.concurrent.ExecutionException: org.postgresql.util.PSQLException: The connection attempt failed.
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at clojure.core$deref_future.invokeStatic(core.clj:2290)
	at clojure.core$future_call$reify__9377.deref(core.clj:6849)
	at clojure.core$deref.invokeStatic(core.clj:2310)
	at clojure.core$deref.invoke(core.clj:2296)

Caused by: org.postgresql.util.PSQLException: The connection attempt failed.
	at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:233)
	at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64)
	at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:144)
	at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29)
	at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21)
	at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31)
	at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
	at org.postgresql.Driver.makeConnection(Driver.java:410)
	at org.postgresql.Driver.connect(Driver.java:280)

favila23:06:21

for the new uri?