Fork me on GitHub
#datomic
<
2019-11-03
>
Aleed18:11:38

for aws, the atomic solo 1 monthly estimate is $118 whereas the production 1 is $21, is this correct? I thought solo was more affordable/suitable for getting started - did this change?

marshall18:11:09

This is an error in marketplace

marshall18:11:22

The estimates should be reversed

marshall18:11:33

We are working with aws to correct

Jon Walch22:11:14

Would you mind letting me know when this is corrected? I'd like to stand up a Solo 1 ASAP

marshall23:11:28

You can use solo as it is

marshall23:11:52

The price you are charged is correct

marshall23:11:09

Its just the display on the marketplace site that is wrong

Jon Walch23:11:30

The CloudFormation template for Solo is also making reference to the instance sizing in production 1

Jon Walch02:11:37

I just went through the process for setting up a Solo 1. If I look at my EC2 instances, my bastion is a t3.nano and then I have two i3.large instances? Is this correct?

marshall12:11:58

That is not correct

marshall12:11:01

that is a production topology

marshall12:11:15

What version did you launch from Marketplace?

marshall12:11:05

@UNVU1Q6G1 Can you follow the directions here https://docs.datomic.com/cloud/operation/new-system.html and launch a stack from our Releases page instead of from Marketplace I will contact AWS today and look into getting the listing corrected

marshall12:11:28

@UNVU1Q6G1 I believe this may be an issue with the “535-8812.1” release. Can you try the one without the .1 (just 535-8812) ?

Aleed18:11:52

so does that mean i should solo 1 despite the large estimate, since the actual amount will be reversed?

Aleed18:11:33

ok, thanks for clarifying !

joshkh18:11:40

i'm curious -- what makes the following two queries different enough to return empty vs. non-empty results?

; find entities whose :user/name attribute has a value of "user123"
(d/q '{:find  [?e]
       :in    [$ ?v]
       :where [[?e :user/name ?v]]}
     db "user123")
=> [[12345678912345]]

; find entities with any attribute (unbound) that has a value of "user123"
(d/q '{:find  [?e]
       :in    [$ ?v]
       :where [[?e _ ?v]]}
     db "user123")
=> []

favila18:11:23

That is pretty alarming

favila18:11:49

Maybe :user/name is not indexed?

favila18:11:55

Nonetheless, my expectation is the second query would not be empty; it might be so slow it never terminates, but not empty

joshkh18:11:07

is there a way to find out? perhaps it's just me? i can definitely reproduce it.

favila18:11:30

To me this looks like a bug

favila18:11:06

it’s a pathological case, you’d never want a query like [?e _ ?v] as the first clause with ?v bound, but it should still work

favila18:11:23

experiment with binding ?a in various ways, see if it gives results

joshkh18:11:02

> Nonetheless, my expectation is the second query would not be empty; it might be so slow it never terminates, but not empty i did wonder if this would trigger a full-db scan alert, however an empty set (returned instantly) made me scratch my head.

favila18:11:09

[0 :db.install/attribute ?a] [?e ?a ?v] or filter down further

joshkh18:11:21

i did try binding ?a with the same result

favila18:11:06

[?a :db/ident :user/name] [?e ?a ?v]?

favila18:11:17

same result meaning empty set?

joshkh18:11:00

[?a :db/ident :user/name] [?e ?a ?v] works as expected

joshkh18:11:20

simply binding ?a (and not using it) returns an empty set

favila18:11:47

I meant forcing ?a to be bound to every attribute explicitly

favila18:11:01

It could be the query planner refuses to even try to match by ?v if it doesn’t know ?a

favila18:11:25

there’s no index it can use effectively after all

favila18:11:46

I would still want an error not a silent empty set

joshkh18:11:50

interesting, this works!

(d/q '{:find  [?e ?b]
       :in    [$ ?v]
       :where [
               [?a :db/ident ?b]
               [?e ?a ?v]]}
     (client/db) "user123")
=> [[12345678912345]]

joshkh18:11:30

(by the way, i would never use queries like these in production. this only stemmed from some hacky experimentations.)

joshkh18:11:47

however, my concern is that an empty set can be dangerously misleading

dmarjenburgh08:11:24

FYI, I reproduced this with the same results and it’s not what I expected. Adding the [_ :db/ident ?a] or [?a :db/ident] clause works (and is considerably slower).

👍 4
dmarjenburgh08:11:26

Adding a predicate clause [(= ?v ?name)] [?e _ ?name] will warn you of a full db scan

joshkh19:11:20

also curious -- is it normal to see what look like duplicate references in :db.alter/attribute?

{:db/id              0
 :db.alter/attribute [#:db{:id 99, :ident :user/name}
                      #:db{:id 99, :ident :user/name}]}

marshall12:11:31

That is an issue that was resolved in the most recent release

Aleed21:11:17

hello i'm requesting datomic in a new boot app, and receiving a Could not locate datomic/client/impl/pro__init.class error. is there a way I can go about in resolving this? here's my require code: (:require [datomic.client.api :as d]) here's my dependency: :dependencies '[[org.clojure/clojure "1.10.0"] [com.datomic/client-cloud "0.8.78"]]

Jon Walch21:11:27

your require and deps look fine to me

Aleed22:11:35

figured out it's not a dependency error it's a connection error

Jon Walch21:11:37

I'm trying to read an edn file from my code base and then transact it, If I copy it verbatim and paste it in as the tx-data it works fine, however if I try to read it as a resource, slurp, and edn/read-string it, I get the following when I try to transact it

.Exceptions$IllegalArgumentExceptionInfo
   :message :db.error/not-a-data-function Unable to resolve data function: #:db{:doc "User first name", :ident :user/first-name, :valueType :db.type/string, :cardinality :db.cardinality/one}
   :data #:db{:error :db.error/not-a-data-function}
   :at [datomic.error$arg invokeStatic error.clj 57]}]
I think its because edn/read-str is using the Map namespace syntax (https://clojure.org/reference/reader#_maps), is there a way to force it not to?

Jon Walch22:11:55

I'm just going to declare my txs in code instead of in edn

Alex Miller (Clojure team)22:11:08

whether you use that syntax, or not, the map in memory is identical

Alex Miller (Clojure team)23:11:26

the error makes it sound like you've got an attribute definition where datomic expects a function, which seems like something else

Aleed23:11:36

if we're using datomic cloud, what setup is recommended for dev and staging? i'd rather not create two more cloud instances, so it OK to use datomic free for these scenarios?

Jon Walch02:11:13

You can but the API is different

Jon Walch02:11:48

I tried fiddling with https://github.com/ComputeSoftware/datomic-client-memdb/ but it didn't work quite right for me

Aleed02:11:27

so is the expected solution to create/pay for separate cloud instances?

Aleed02:11:59

or I guess wrap both cloud/on-premise APIs to make them the same

favila14:11:08

datomic cloud does not have a local-dev story

favila14:11:37

you are expected to have something running in the cloud, even for test runners

kenny15:11:28

@UNVU1Q6G1 What did work? @UPH6EL9DH We use datomic-client-memdb for running unit tests & gen tests on CI. We also have a dev system always running which lets you connect locally to run integration tests. The Datomic client for this dev system is created by specifying a "prefix" which will get added to all DBs created. We just implemented a simple wrapper around datomic.client.api.protocols/Client.

Jon Walch18:11:05

@U083D6HK9 It was working perfectly for transacting. When I was trying to do (d/db conn) where conn is a datomic-client-memdb LocalConnection, it was telling me that the type couldn't be cast. Let me see if I can repro

Jon Walch18:11:14

java.lang.ClassCastException: class compute.datomic_client_memdb.core.LocalConnection cannot be cast to class datomic.Connection (compute.datomic_client_memdb.core.LocalConnection is in unnamed module of loader clojure.lang.DynamicClassLoader @1e420b95; datomic.Connection is in unnamed module of loader 'app'

kenny18:11:34

Can you send the full code you’re using to do that @UNVU1Q6G1 ?

Jon Walch19:11:02

Spoke with @U083D6HK9 in a DM, it was user error on my part 😄

Aleed19:11:35

@U083D6HK9 so to be clear you're not running two cloud instances, just one instance but for dev you prefix all databases created?

kenny19:11:45

@UPH6EL9DH yes — one instance with multiple devs using the same instance. Each dev makes their own prefix.

Aleed19:11:57

@U083D6HK9 would you be able to share some of the wrapper code? 🙂 also, even with a prefix, are you not concerned at all about mixing production and dev database?

kenny19:11:39

I can see how coupled to other code our wrapper is when I get back in front of a computer. Might be able to paste some code here. Oh, I guess we run two systems then. One for production. Dev and QA environments both use the single Datomic dev system.

Aleed19:11:41

@U083D6HK9 that'd be great thanks; and yea, that seems like best solution, but for a side project jumps cost for 30$ monthly to 60$ which can get pretty steep

kenny19:11:48

I think you’d honestly be fine running prod and dev on the same system. Make it so prod uses no prefix.

kenny19:11:15

We run separate topologies so we get high availability. Dev is just running solo.

kenny20:11:12

@UPH6EL9DH It's essentially this:

(defrecord DatomicClient [client db-prefix return-anomalies?]
  datomic-protos/Client
  (list-databases [_ arg-map]
    (let [dbs (d/list-databases client arg-map)]
      (into (list)
            (comp
              (filter (fn [db-name]
                        (if db-prefix
                          (str/starts-with? db-name (db-prefix-str db-prefix))
                          (not (str/starts-with? db-name "__")))))
              (map (fn [db-name]
                     (str/replace-first db-name (db-prefix-str db-prefix) ""))))
            dbs)))
  (connect [_ arg-map]
    (d/connect client {:db-name (prefix-db-name db-prefix (:db-name arg-map))}))
  (create-database [_ arg-map]
    (d/create-database client {:db-name (prefix-db-name db-prefix (:db-name arg-map))}))
  (delete-database [_ arg-map]
    (d/delete-database client {:db-name (prefix-db-name db-prefix (:db-name arg-map))})))

Aleed23:11:20

@U083D6HK9 great, thanks for sharing 👍

vnctaing23:11:00

I’ve some issue installing Datomic Pro Starter Edition I created a file ~/.lein/credentials.clj

{#"my\.datomic\.com" {:username "…."
                      :password "…."}}
then generated ~/.lein/credentials.clj.gpg gpg --default-recipient-self -e ~/.lein/credentials.clj > ~/.lein/credentials.clj.gpg added to my project.clj
:repositories {"" {:url ""
                                 :creds :gpg}}
:dependencies [[com.datomic/client-pro "0.9.5927"]]
but when i run lein deps I get
Could not find artifact com.datomic:client-pro:jar:0.9.5927 in central ()
Could not find artifact com.datomic:client-pro:jar:0.9.5927 in clojars ()
Could not find artifact com.datomic:client-pro:jar:0.9.5927 in  ()
This could be due to a typo in :dependencies, file system permissions, or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.

marshall12:11:54

the client-pro version is not the same as the datomic version

👍 4
marshall12:11:36

client library is in Maven: https://search.maven.org/search?q=a:client-pro%26 latest version is 0.9.37