Fork me on GitHub
#datomic
<
2018-07-27
>
rhansen00:07:49

After following the tutorial I just get connection refused when trying out api gateway with a ring handler šŸ˜•

rhansen00:07:55

anyone experience that?

kenny03:07:20

Is there a way in code to tell if an Ion is deployed or not? Curious how others are handling configuration in dev vs prod.

steveb8n03:07:43

@kenny you could try invoking it using AWS CLI. that would verify itā€™s deployed. but it doesnā€™t seem like this is really your question. can you elaborate?

steveb8n03:07:02

Iā€™m curious because Iā€™ll be setting up the same environments in the coming weeks

kenny03:07:30

My application configuration depends on the environment it is running in (dev/qa/prod). I donā€™t see anyway to parameterize the deployment like that.

jaret12:07:54

We have a release in the works to deliver params for deployment. Weā€™re currently working with AWS to get it out. I donā€™t have a timeline, but weā€™re going to be delivering parameterized deployment.

steveb8n03:07:26

Iā€™m wondering about that also. hence my earlier question about CI setup. If you look at the Ion lambdas there are env variables there so that seems like a good way to do this but not sure how to populate those from code.

stuarthalloway13:07:44

@U0510KXTU stay tuned šŸ™‚

šŸ™‚ 12
eoliphant02:07:05

thatā€™s what Iā€™m using at the moment. I tagged my Datomic Cloud deployment with an ENV tag, and use that to ā€˜rootā€™ my stuff in parameter store

steveb8n04:07:35

but it would be good to have an example of how to use it with Ions / deploy

Oliver George05:07:04

@stuarthalloway I think the Datomic Ion tutorial could be improved by including a sample apigateway cloudformation template in the repo. It'd provide a more complete picture of how to deploy an app and potentially save some pain for potential users who are less familiar with AWS.

šŸ’Æ 12
rhansen09:07:33

Has anyone seen this in their ion logs? "Message": "java.lang.StackOverflowError, compiling:(riddley/walk.clj:29:37)",

steveb8n13:07:55

There's a good chance this is the same issue I had this week. Solo has low memory and classpath problems get masked by these stackoverflow errors.

steveb8n13:07:54

The problem is not in the walk NS. Most likely O E of the libs you see in the push result warnings is the case. Try commenting them out one by one. Not great but worked for me

stuarthalloway13:07:30

instead of commenting them out, change your local deps to match the warnings and see what happens locally

rhansen23:07:05

Ok. So I altered my deps.edn to match the warnings, spin up a local repl, and require the namespace which contain the ion functions. Everything works perfectly.

rhansen23:07:55

My deps.edn currently looks like:

{:paths ["backend" "resources"]
 :deps
 {org.clojure/clojure {:mvn/version "1.9.0"}
  com.datomic/ion {:mvn/version "0.9.16"}
  ring/ring-core {:mvn/version "1.7.0-RC1"}
  ring/ring-json {:mvn/version "0.4.0"}
  email-validator {:mvn/version "0.1"}
  clj-mailgun {:mvn/version "0.2.0"}
  crypto-password {:mvn/version "0.2.0"}
  danlentz/clj-uuid {:mvn/version "0.1.7"}
  clojure.java-time {:mvn/version "0.3.2"}
  commons-codec/commons-codec #:mvn{:version "1.10"},
  com.fasterxml.jackson.core/jackson-core #:mvn{:version "2.9.5"}}

 :mvn/repos {"datomic-cloud" {:url ""}}

 :aliases
 {:dev {:extra-deps {com.datomic/client-cloud {:mvn/version "0.8.54"}
                     com.datomic/ion-dev {:mvn/version "0.9.175"}
                     ring/ring-jetty-adapter {:mvn/version "1.7.0-RC1"}
                     org.eclipse.jetty/jetty-server {:mvn/version "9.4.9.v20180320"}
                     org.eclipse.jetty/jetty-client {:mvn/version "9.4.9.v20180320"}}}
  :test {:extra-paths ["tests"]
         :extra-deps {expectations {:mvn/version "2.2.0-rc3"}}}}}

rhansen18:07:28

So, that worked... But I now have no idea how to send e-mails šŸ˜¢

rhansen18:07:33

But i'll work it out šŸ™‚

rhansen09:07:20

I'm trying to setup my ring app for api-gateway integration, and my datomic node just crashes. Can't re-produce it localy šŸ˜•

tlima16:07:35

I know this is the Clojurians workspace, but anyone here knows if it's possible to use the client (not the peer) in Java? And, if so, where I can find some documentation on that?

marshall16:07:31

There is currently no Java client library

tlima17:07:25

Thanks, @marshall. I guess there are no docs around the API used by the clients, except for (maybe) the Clojure client's source code, right?

marshall17:07:02

that is currently correct; we plan to publicize the client wire protocol once itā€™s completely finalized

tlima17:07:20

Any ETA for that?

marshall17:07:32

unfortunately not

tlima17:07:59

I see. Thanks.

tlima17:07:47

One more thing, @marshall: is the Java peer library still supported and updated?

marshall17:07:38

the API has remained pretty stable, so the java peer itself hasnā€™t changed much, but On-Prem is definitely still updated and supported

mtbkapp17:07:40

I'm curious about the 2^20 limit for schema attributes as described: https://docs.datomic.com/cloud/schema/schema-limits.html . Is that per database or per cluster (transactor, storage, peers)?

marshall17:07:47

@mtbkapp Cloud or On-Prem (Cloud doesnā€™t have transactors or peers)

Mark Addleman18:07:20

Hey folks - Weā€™re using Datomic Cloud in production topology. Our database is about 450m datoms and weā€™ve started to receive ā€œBusy rebuilding indexā€ responses from the transactor. Some googling led me to an earlier #datomic conversation that indicated we are pushing too much data into Datomic. How would I go about debugging this problem? Is this a transactor resource problem or too few DyanmoDB resources or something else entirely?

mtbkapp18:07:32

@marshall both. I forgot about the difference in Cloud.

marshall18:07:58

per database in both I belive @mtbkapp - however, are you likely to reach that? that would be a rather large DB if so

octahedrion18:07:26

hey @marshall I had a problem I hoped I could get your advice on: I have an attribute that I wish I'd added a :db/unique :db.unique/identity to, but it's too late as there are multiple values in the current set of database assertions

mtbkapp18:07:40

right, I was think about a case where users can create "custom fields" for entities. I think there are at least two ways to do this. 1) using datomic attributes directly (with additional constraints for security and what not). 2) modeling them using datomic entities, which turns it into something like a meta data model inside the datomic meta data model.

octahedrion18:07:28

retracting those multiple values doesn't work, but I think I found another way: I renamed the attribute :old-attribute-name and asserted the attribute anew with the unique constraint, and thereafter one has only has the small inconvenience of having to specify the attribute in one's queries (to prevent assertions for the old one appearing) I have to assert the latest values of the old attribute on the new one

octahedrion18:07:08

but what I wondered was, when asserting those values of the old attribute to the new one, is it possible to retain their txInstants ?

octahedrion18:07:37

as though they had been asserted back when the old ones had been ?

marshall19:07:47

@octo221 retracting the values until it is unique should work. is this on Cloud?

marshall19:07:37

if so, This may be related to a known issue (where you canā€™t retract :db/unique on an attr). we are working on a fix for that

tlima19:07:21

(@marshall, is this a good channel for Java support or I should reach out other ways?)

marshall19:07:34

@t.augusto here is good; the Datomic forum is also good

marshall19:07:39

also, the Datomic support channel (http://support.cognitect.com) if you have a paid license

octahedrion19:07:38

@marshall ah - I tried retracting until it was unique but it didn't work - I'll try again tomorrow

octahedrion19:07:10

yes it's on Cloud

marshall19:07:11

@octo221 OnPrem or Cloud?

marshall19:07:17

yeah, itā€™s possible thatā€™s a bug.

marshall19:07:36

if you canā€™t get it to work, let me know, but hopefully the fix for retracting unique also fixes that

marshall19:07:41

(i believe it will)

octahedrion19:07:46

ok thank you!

tlima19:07:50

Ok, @marshall, here it goes: I managed to create a docker composition, with the transactor (pro starter) + console + app (my java app). The console is working fine, but I canā€™t get the app to connect to the transactor. I changed the logback.xml to redirect the log messages to stdout, so that it shows in my regular docker logs. The snippet (on itā€™s way) is what it looks like in the transactor and app when the app tries to connect.

marshall19:07:29

Error communicating with HOST 0.0.0.0 or ALT_HOST transactor on PORT 4334

marshall19:07:41

^ indicates that your peer canā€™t reach the transactor

tlima19:07:57

I can telnet from the app to the transactor on that exact port, with no issues

marshall19:07:02

you need to set the values for HOST and/or ALT_HOST to addresses that the peer instance can resolve as the transactor

tlima19:07:38

well, ALT_HOST=transactor, which is exactly the name of that service inside the compositionā€¦ šŸ˜•

marshall19:07:44

you shouldnā€™t be able to telnet to that port

tlima19:07:18

Telnet will just establish the connection, no matter the protocolā€¦ right?!

marshall19:07:25

erm. possibly

marshall19:07:28

i guess i havent tried that

tlima19:07:13

Iā€™m used to telnet to validate reachability, ACLs, etc šŸ™‚

marshall19:07:16

yeah, i suppose you should at least see something there

marshall19:07:20

for reachability

tlima19:07:49

The odd part for me is the AMQ214013: Failed to decode packet message

marshall19:07:36

your application is using what version of Datomic? and what transactor version?

tlima19:07:44

Per the transactor log: transactor_1 | 2018-07-27 19:53:46.327 INFO default datomic.lifecycle - {:tid 24, :username "HgWS7pDswz79iMxZZKUEGfhle+9ggGa4PSSdxeZlhBY=", :port 4334, :rev 4852, :host "0.0.0.0", :pid 1, :event :transactor/heartbeat, :version "0.9.5703", :timestamp 1532721226326, :encrypt-channel true} My pom.xml: <dependency><groupId>com.datomic</groupId><artifactId>datomic-pro</artifactId><version>0.9.5703</version></dependency>

marshall19:07:55

can you get the rest of the transactor log that you elided as well?

marshall19:07:04

as well as a few lines prior

marshall19:07:04

using dev storage?

marshall19:07:36

also, you said the console works fine - are you able to connect it and see a DB, etc?

tlima20:07:43

Yes, @marshall, dev storage

tlima20:07:46

Yeah, console works

tlima20:07:36

(can you see the screenshot, despite of this slackbot warning about ā€œno storage space leftā€?)

marshall20:07:58

so that suggests that your peer config is the issue somehow

marshall20:07:14

is it possible you have a conflicting dependency that is overriding something like netty?

marshall20:07:44

in your app specifically

tlima20:07:25

Here is how Iā€™m trying to connect:

public Connection getDatabaseConnection() {
        String uri = "datomic:

        Peer.createDatabase(uri);

        return Peer.connect(uri);
    }

tlima20:07:24

Iā€™d need to check for those conflictsā€¦ Maybe spring boot is also pulling netty, not sure.

marshall20:07:03

Iā€™ll have to do some more looking - may not be able to discuss with the team until Monday

tlima20:07:08

Ok, @marshall. Iā€™ll try it a bit more, here, and ping you back on monday.

marshall20:07:42

If you have Clojure on the docker system running your app, Iā€™d be interested if you could connect from a REPL

marshall20:07:53

or with a simple clojure app using the same URI

tlima20:07:09

Iā€™m just trying to evaluate if it fits our project needs, before getting deeper into budget approvals and stuff.

tlima20:07:29

My goal is to, unfortunately, create an API layer on top of it. Our app language is not supported (peer library nor client) so I donā€™t see much of an alternative.

tlima20:07:47

(btw, @marshall, I just tried the enforcer:enforce command (from the maven-enforcer-plugin) and it reports no other conflict than datomic-pros own dependency on both com.google.guava:guava:18.0 (direct) and com.google.guava:guava:19.0 (through Artemis related packages))

tlima20:07:29

I will try adding Clojure to the mix and experiment with the REPL. Just not really fluent on Clojure šŸ˜•

marshall20:07:54

to try creating a DB and connecting via a REPL

sparkofreason20:07:09

Is there any difference in the query results obtained using or vs, or-join, or is or-join more of a hint for the query compiler?

tlima20:07:15

@marshall, JFYI, the REPL worked.