Fork me on GitHub
#datomic
<
2018-03-07
>
Desmond03:03:34

so I have been testing my queries by passing in sample datoms as vectors of vectors. Now I want to use pull syntax in the query and i'm getting the error: clojure.lang.PersistentVector cannot be cast to datomic.Database. what's going on here? how can I do this?

favila04:03:49

You need an actual database

favila04:03:54

Consider using the mem (in memory) database

Desmond04:03:46

Thanks for clarifying. Yeah i just separated the main query from the pulling part. I wasn't sure the best approach to this separation. I need to keep the association between the attributes from the pull and the attributes from the main query so I'm doing a pull for each entity. Works fine on the small dataset I just tested it with but i'm not sure how the pull-per-entity strategy will perform with more data. Any ideas?

fmnoise14:03:25

hi there! for some reason I can't open https://www.datomic.com/training.html

marshall14:03:10

@fmnoise Are you looking for the Day of Datomic videos?

marshall14:03:14

They’ve moved to the docs ^

marshall14:03:36

also, can you tell me where you found that link ?

fmnoise15:03:39

thanks, that works!

fmnoise15:03:55

I had the link in bookmarks

mgrbyte15:03:20

@marshall / @stuarthalloway could you clarify that the claims in the doc string for datomic.api/tx-range should hold pls? I've found that only using t values work for me (and not transaction ids) - thanks!

marshall15:03:15

@mgrbyte in the peer API?

marshall15:03:40

ok i’ll have a look

mgrbyte15:03:46

thanks :thumbsup:

mgrbyte15:03:12

on latest version of datomic-pro, memory db for context

mgrbyte15:03:15

I'm using tx->t atm, so not an issue, just wanted to know if the doc string was telling me the truth/if I'd found a bug

unlisted15:03:44

Who has experience migrating from Datomic on-prem to cloud (AWS)? Anyone? Anyone?...

marshall15:03:23

@mgrbyte ^ seems to work

mgrbyte15:03:45

hmm. not sure what's going on then. I have code which works with (d/tx-range (d/log conn) (d/tx->t tx) nil) but not with (d/tx-range (d/log conn) tx nil)

mgrbyte15:03:00

also, in your example it could be that equality assertion is comparing true for two empty seqs?

marshall15:03:21

it will work in that case, but i also checked that they contained something

marshall15:03:17

user=> (let [t-val 1
      tx-id (d/t->tx t-val) 
      using-tx (seq (d/tx-range (d/log conn) tx-id nil)) 
      using-t (seq (d/tx-range (d/log conn) t-val nil))
      _ (println (count using-tx))]
  (= using-tx using-t))
5
true

mgrbyte15:03:37

thanks for looking. when I have time, I'll see if I can extract out the code I have that's failing into a gist/repo

donmullen16:03:04

I have a simple sample project up and running on AWS Elastic Beanstalk and would like to access Datomic Cloud. I’m relatively new to AWS — how straight-forward will it be to give my server access? Any pointers into sample configurations for beanstalk and/or docs before I dive in this afternoon? Being new to AWS - how realistic is it for me to ramp up and get connected today? AWS docs are pretty dense! I liked how straight-foward it was to get datomic cloud up an running locally - would be great to have a similar guide for a sample app running in elastic beanstalk (where you can just upload Procfile/uberjar to get going). Will cross post to #aws.

marshall16:03:39

@donmullen if by “my server” you mean your EB app, you should have a look at https://docs.datomic.com/cloud/operation/client-applications.html#vpc-peering

marshall16:03:54

to set up VPC peering between the VPC where your EB app is running and the Datomic Cloud VPC

donmullen17:03:55

Yes - my elastic beanstalk app. So VPC peering is the way to go - I’m assuming there isn’t a way to tell EB to use the Datomic Cloud Application VPC. Thanks @marshall.

marshall17:03:27

@donmullen I think you could do that, but the Datomic Cloud VPC may not have the settings you desire for your app (i.e. public IPs / NAT gateway /etc)

marshall17:03:02

for just trying it out, that may be the quickest path, though

donmullen17:03:44

@marshall - I think I have first two of three steps completed to do the peering. Stepping through Associating a VPC with a Privated Hosted Zone now.

donmullen19:03:02

@marshall - The only thing I think I may have gotten wrong is setting the routing table information. There was already a 0.0.0.0/0 entry with target igw-f8675881 for the EB and Datomic routes - where evidently igw is the internet gateway. I replaced the `0.0.0.0/0' with the IP of the other VPC peer IP. I’m thinking now that I should leave the 0's - and add an entry (as the server is no longer coming up). Trying that now. Thoughts?

donmullen19:03:29

OK - so going back to 0.0.0.0/0 seems to have resolved the EB app problem. I added an entry to the routing for each - selecting the vpc-peer-connection as the target. Is that correct? In testing a call to datomic.client.api/client - I’m now getting Exception in thread "main" java.lang.NoClassDefFoundError: org/eclipse/jetty/util/thread/NonBlockingThread. Switching back to local to make sure that works still. Any thoughts on this error from client call, @marshall?

marshall19:03:05

you are getting that error or you’re not ?

marshall19:03:57

the NoClassDefFoundError usually points to a deps conflict and/or a missing dep

donmullen19:03:35

@marshall - yes - getting that error (not -> now). Indeed thinking it’s a dep conflict of some sort. I seem to recall something about a jetty exclusion? Trying [com.datomic/client-cloud "0.8.50" :exclusions [org.eclipse.jetty/jetty-io]] now.

donmullen20:03:38

OK - progress. deps resolved - now back to getting vpc peer connection correct. Now I’m getting

Exception in thread "main" clojure.lang.ExceptionInfo: Unable to connect to system: {:cognitect.anomalies/category :cognitect.anomalies/unavailable, :cognitect.anomalies/message "Connection refused"} {:config {:server-type :cloud, :region "us-east-1", :system "nycdvp", :query-group "nycdvp", :endpoint "", :proxy-port 8182, :endpoint-map {:headers {"host" ""}, :scheme "http", :server-name "", :server-port 8182}}} 
which implies that I don’t have the vpc peers set up correctly.

marshall20:03:25

i would take out proxy-port from the config map

marshall20:03:30

you’re not using a proxy anymore

marshall20:03:20

And you’re using a security group with the correct permissions?

marshall20:03:35

in particular, one that has ingress allowed in the Datomic entry group

donmullen20:03:22

Removing the proxy-port - now getting Connect Timeout instead of Connection refused :

Exception in thread "main" clojure.lang.ExceptionInfo: Unable to connect to system: {:cognitect.anomalies/category :cognitect.anomalies/unavailable, :cognitect.anomalies/message "Connect Timeout"} {:config {:server-type :cloud, :region "us-east-1", :system "nycdvp", :query-group "nycdvp", :endpoint "", :endpoint-map {:headers {"host" ""}, :scheme "http", :server-name "", :server-port 8182}}}

redinger20:03:26

@donmullen Does your datomic entry security group allow ingress on port 8182 from your eb security group?

redinger20:03:23

And does your datomic route table have an entry to route back to the eb route table?

donmullen20:03:50

@redinger - see routing above - I did add entries both ways.

redinger20:03:10

Yep, that’s the right place

donmullen20:03:26

But it doesn’t accept sg-66193f12 - which the EB default security group….

donmullen20:03:57

@redinger - at least for Custom TCP rule - what should Type and Protocol be?

redinger20:03:30

The peering is to vpc-90b1b3e8. You should use the security group for that vpc - sg-c212b2b4

redinger20:03:54

Type is Custom TCP Rule, Protocol is TCP

donmullen20:03:44

@redinger - ok - so that sg-c212b2b4 also wasn’t in the drop-down list - but I put it in anyway and trying that. Any idea why that value not in the list of selectable items?

redinger20:03:00

@donmullen Hmm, not sure. Maybe their auto suggest doesn’t work across VPCs.

donmullen21:03:12

Could be — progress though!

donmullen21:03:21

Exception in thread "main" clojure.lang.ExceptionInfo: Forbidden to read keyfile at . Make sure that your endpoint is correct, and that your ambient AWS credentials allow you to GetObject on the keyfile. {:cognitect.anomalies/category :cognitect.anomalies/forbidden, :cognitect.anomalies/message "Forbidden to read keyfile at . Make sure that your endpoint is correct, and that your ambient AWS credentials allow you to GetObject on the keyfile."}

redinger21:03:54

So now the IAM policy for that EB needs to grant access to S3 for those keys

donmullen21:03:25

okay dokie - feel like I’m getting close… no? most of this aws stuff is new to me - realize that technically this isn’t datomic support - but guess it’s good to over issues that people new to aws have that need resolution in order to get connected to datomic cloud. help is much appreciated.

redinger21:03:04

Yeah, I think once the IAM permissions are sorted, this should be working. This feedback is super useful, since we need to find the balance in our documentation about how to consume this stuff.

donmullen21:03:01

@marshall @redinger Success! Thanks much!

INFO [fulcro.easy-server:181] : Web server () started successfully. Config of http-kit options: {:port 5000}
18-03-07 21:16:01 ip-172-31-44-80 INFO [aws.server-main:23] - Testing Datomic
18-03-07 21:16:10 ip-172-31-44-80 INFO [aws.server-main:26] - created client
18-03-07 21:16:11 ip-172-31-44-80 INFO [aws.server-main:28] - created database
18-03-07 21:16:21 ip-172-31-44-80 INFO [aws.server-main:30] - connected to database
18-03-07 21:16:22 ip-172-31-44-80 INFO [aws.server-main:32] - deleted database

Oliver George23:03:47

Hello. I'm interested in seeing how people prepare datomic queries. Specifically, adding more constraints depending on, say, api query params. Google and I ended up with what feels like an elegant hack at best: https://gist.github.com/olivergeorge/e086543f519d5e0559c1179545e20c68

Oliver George23:03:56

Perhaps there are other approaches I'm not seeing.