Fork me on GitHub
#datomic
<
2021-12-16
>
anders09:12:27

any hopes of adding support for aws c5/c6 ec2 instances to datomic-pro on-prem?

jaret13:12:04

In Datomic on-prem you can roll your own CFT. The CFT we provide is purely for development convenience.

Ivan Fedorov12:12:14

Is this the right place to ask about ions? I was wondering if there are configuration options other than listed in the https://docs.datomic.com/cloud/ions/ions-reference.html#parameters. E.g. can I upload my own config-staging.edn?

Drew Verlee06:12:06

are you talking about the configuration the ions use for functionality like :allow or :lamdas ? what were you looking to add?

Benjamin08:12:55

You can put it in resources/config-staging.edn and retrieve it with (io/resource "config-staging.edn")

Ivan Fedorov12:12:44

@U02CV2P4J6S yes, I somehow overlooked that. I can check ions env map and then dance from there, thanks! I initially was thinking about bringing in an outside file and placing it in the project root directory.

Ivan Fedorov12:12:59

@U0DJ4T5U1 I’m trying to understand how Ions work and if I can bring an outside configuration file for my app and place it into a project root. I’m not the dev-ops in my team, and I don’t have understand of how Ions nodes are composed. I think I was looking for this https://docs.datomic.com/cloud/whatis/architecture.html#nodes But I don’t yet understand if I can ssh into any node.

Drew Verlee18:12:37

@U0A5V8ZR6 why ssh into a node? The general workflow i have used in my very limited case has been that i don't ssh to the nodes. For the clojure part, i work locally. For integrations like to web api gateway, i have to read lot of docs and look at logs.

xceno14:12:00

I can't seem to figure out how to pass a "blank" as an argument into a query. Here's what I'd like to do:

(d/q '[:find (pull ?edge *)
         :in $ ?direction ?uuid
         :where
         [?e :my.graph.node/id ?uuid]
         [?edge ?direction ?e]]
    db direction-kw uuid)
In this query I'd like to set direction-kw to either some keyword or _ (a https://docs.datomic.com/cloud/query/query-data-reference.html#blanks) I tried to set direction-kw to nil and '_ but that only ends in exceptions. What am I doing wrong?

favila14:12:21

_ is a syntatic construct, not a value

favila14:12:11

to have an “optional” binding, you need to use either a sentinel value and rules for sentinel vs non-sentinel, or you need to construct the query to include or omit the optional binding

favila14:12:02

construction example

(let [base {:query {:find '[(pull ?edge *)]
                    :in '[$ ?uuid]
                    :where '[[?e :my.graph.node/id ?uuid]]}
            :args [db uuid]}
      full (cond-> base
                   (some? direction-kw)
                   (-> (update-in [:query :in] conj '?direction)
                       (update-in [:query :where] conj '[?edge _ ?e])
                       (update :args conj direction-kw)))]
  (d/query full))

favila14:12:00

Actually, before I go further, I think you may not want this. How do you know that [?edge _ ?e] is matching what you consider to be a “direction” attribute?

favila14:12:27

I think you should not do anything I said and instead enumerate all direction attributes

favila14:12:54

(d/q '[:find (pull ?edge *)
         :in $ [?direction ...] ?uuid
         :where
         [?e :my.graph.node/id ?uuid]
         [?edge ?direction ?e]]
    db [direction-kw] uuid)

favila14:12:47

and the “blank” case is

(d/q '[:find (pull ?edge *)
         :in $ [?direction ...] ?uuid
         :where
         [?e :my.graph.node/id ?uuid]
         [?edge ?direction ?e]]
    db [:left :right] uuid)

favila14:12:03

note only the input changed. (I’m making up direction kws, I don’t know what your set is)

xceno15:12:34

Ohh I didn't think of binding a collection like this at all. Thank you, that actually makes way more sense for what I want to do!

borkdude14:12:28

Which dependency is supposed to provide datomic.api and datomic.function if you're using dev-local? https://github.com/fulcrologic/fulcro-rad-datomic/blob/develop/src/main/com/fulcrologic/rad/database_adapters/datomic.clj#L13-L14

souenzzo14:12:04

https://forum.datomic.com/t/requesting-feedback-on-dev-local-getting-started/1608/5

An easy to use experience, like com.datomic/datomic-free "0.9.5697" for beginners/learners/demos/tutorials. No one will create a account, download stuff, install, configure, etc… just for learn a new technology (we are clojure devs. we learn at REPL).

borkdude14:12:35

if datomic-free is still required then I'm at a dead end

borkdude14:12:14

unless #datomic can provide one that is linked against a newer version of clojure

souenzzo14:12:26

you can create a account at https://my.datomic.com/, do credentials setup and etc to access datomic-pro

stuarthalloway15:12:36

Hi @U04V15CAJ! What are you trying to do?

borkdude15:12:01

Hey Stuart! I'm helping out @U6VPZS1EK to compile his app to native with #graalvm. We are running into an issue with the locking macro which was fixed in clojure 1.10.2. But datomic-free is still AOT-ed against an older version of clojure so we're still running into the "unbalanced monitor" error from the graalvm bytecode verifier.

borkdude15:12:17

We might be able to do this without datomic though, I have a meeting with him today so I'll report back here after that.

Daniel Jomphe15:12:43

Hi Michiel! Just to be sure, you mentioned dev-local in your original question. Dev-local is for Datomic Cloud only, not for Datomic On-Prem. From the look of your most recent messages (you use datomic-free), dev-local is not applicable to your situation. Let's hope Cognitect can update datomic-free with a more recent AOT.

stuarthalloway17:12:10

@U04V15CAJ We do not support any version of Datomic on #graalvm. I do not know what the issues would be (you are much experienced there) but I suspect that there would be many. Am I wrong about that?

borkdude17:12:32

@U072WS7PE It depends. I do not know what is in datomic :)

borkdude17:12:59

but this was the first issue I bumped into

borkdude17:12:26

there are various other datalog-like databases that work fine with graalvm native, datalevin being one of them

stuarthalloway17:12:01

Lots of stuff, written before there was #graalvm to worry about. It would be interesting to know if dev-local would work in #graalvm as a proxy for the problems you would encounter, even if dev-local is not suitable for the current use case.

borkdude17:12:53

alright, I can try to make that work. do you by any chance know against which clojure version dev-local is compiled?

stuarthalloway17:12:08

I will have to check

stuarthalloway17:12:14

Something recent for sure.

borkdude17:12:04

if it's newer or equal to 1.10.2 then it's good

borkdude19:12:26

@U072WS7PE I did a little bit of trying with the below program:

(ns devlocal.main
  (:require [datomic.client.api :as d]
            [datomic.dev-local.impl])
  (:gen-class))

(def movie-schema [{:db/ident :movie/title
                    :db/valueType :db.type/string
                    :db/cardinality :db.cardinality/one
                    :db/doc "The title of the movie"}

                   {:db/ident :movie/genre
                    :db/valueType :db.type/string
                    :db/cardinality :db.cardinality/one
                    :db/doc "The genre of the movie"}

                   {:db/ident :movie/release-year
                    :db/valueType :db.type/long
                    :db/cardinality :db.cardinality/one
                    :db/doc "The year the movie was released in theaters"}])

(def first-movies [{:movie/title "The Goonies"
                    :movie/genre "action/adventure"
                    :movie/release-year 1985}
                   {:movie/title "Commando"
                    :movie/genre "thriller/action"
                    :movie/release-year 1985}
                   {:movie/title "Repo Man"
                    :movie/genre "punk dystopia"
                    :movie/release-year 1984}])

(defn -main [& _args]
  (let [client (d/client {:server-type :dev-local
                          :system "dev"
                          :storage-dir :mem})
        _ (d/create-database client {:db-name "movies"})
        conn (d/connect client {:db-name "movies"})
        _ (d/transact conn {:tx-data movie-schema})
        _ (d/transact conn {:tx-data first-movies})
        db (d/db conn)
        all-titles-q '[:find ?movie-title
                       :where [_ :movie/title ?movie-title]]
        results (d/q all-titles-q db)]
    (prn results)
    (shutdown-agents)))
I needed to require [datomic.dev-local.impl] to work around the dynaload stuff, Then I got into some problem around cognitect.caster.Caster.thread where a Thread is initialized at the top level which is not allowed/possible in an image.

borkdude19:12:24

I'll leave it at this, since I don't have the sources to do further digging or make changes to do further experimentation. Usually the above error can be worked around by using a delay around the top level value.

stuarthalloway19:12:00

Thanks! Is there a good list of standard gotchas such as the top-level thread issue?

borkdude19:12:04

For completeness, here is how to repro:

mkdir -p classes
clojure -M -e "(compile 'devlocal.main)"
$GRAALVM_HOME/bin/native-image -cp classes:$(clojure -Spath) --initialize-at-build-time=. --no-server devlocal.main --no-fallback
We try to capture such caveats here: https://github.com/clj-easy/graal-docs

stuarthalloway19:12:43

Also @U6VPZS1EK we would be interested to know your motivations for targeting #graalvm. This is nowhere on our priority list but understanding why it is important could help move it up.

borkdude19:12:32

Some classes are restricted like Random, Security (probably related to Random) and Threads to be initialized at build time (which makes sense because they aren't random anymore).

borkdude19:12:03

Also stuff like (def x (System/getProperty "user.dir")) should be wrapped inside a delay to prevent them from becoming constants

borkdude19:12:06

This is related to --initialize-at-build-time=. which is necessary for clojure, since run time initialization of clojure compiled classes doesn't work (because of what happens in static initializers in such classes: loading from the classpath etc)

genekim20:12:28

Hello, @U072WS7PE and team! and you, too, @U04V15CAJ! 🙂 I hope y’all are doing well — I promise to write up a couple of paragraphs in the next day or two on what I’ve done with Datomic Cloud, and my aspirations that are behind my request for @U04V15CAJ’s help. I think some of the progress we made today might obviate some of this (we had a pairing session this morning, grinding through a couple of other GraalVM native-image issues) , but maybe you’ll find my use case of using Datomic Cloud in Google Cloud Run interesting? Catch y’all soon, and thanks for all the great info above! 🎉

❤️ 1
borkdude12:12:27

@U6VPZS1EK @U072WS7PE I completely forgot about this: https://github.com/babashka/babashka/pull/505/files But someone seems to have made the datomic-pro client work with babashka (which is compiled with graalvm). That means it should also work in the above setup. I'm not sure if there are differences to the pro client that would make sense to you, e.g. the top level thread difference? This was with [com.datomic/client-pro "0.9.57"]