Fork me on GitHub
#babashka
<
2022-06-23
>
jcf15:06:37

Has anyone succesfully loaded Malli up inside Babashka? I’m told clojure.lang.LazilyPersistentVector is an unresolvable classname when requiring malli.core🧵

jcf15:06:45

----- Error --------------------------------------------------------------------
Type:     java.lang.Exception
Message:  Unable to resolve classname: clojure.lang.LazilyPersistentVector
Location: malli/impl/util.cljc:2:3

----- Context ------------------------------------------------------------------
1: (ns malli.impl.util
2:   #?(:clj (:import (clojure.lang MapEntry LazilyPersistentVector)
     ^--- Unable to resolve classname: clojure.lang.LazilyPersistentVector
3:                    (java.util.concurrent TimeoutException TimeUnit FutureTask))))
4:
5: (def ^:const +max-size+ #?(:clj Long/MAX_VALUE, :cljs (.-MAX_VALUE js/Number)))
6:
7: (defn -tagged [k v] #?(:clj (MapEntry. k v), :cljs (MapEntry. k v nil)))

----- Stack trace --------------------------------------------------------------
malli.impl.util                           - malli/impl/util.cljc:2:3
malli.impl.regex                          - malli/impl/regex.cljc:37:3
malli.core                                - malli/core.cljc:4:3

borkdude15:06:57

@U06FTAZV3 That is something we can fix

jcf15:06:22

@U04V15CAJ that is the best news! What can I do to help? excited

Alex Miller (Clojure team)15:06:57

(LazilyPersistentVector is an internal implementation detail that no one should be using anyways .... but I digress)

🙈 2
borkdude15:06:16

oooh, good point :)

borkdude15:06:03

@U064X3EF3 Do you also consider clojure.lang.PersistentVector an implementation detail - where is the line?

Alex Miller (Clojure team)15:06:29

there are a series of fuzzy lines :)

🙂 2
Alex Miller (Clojure team)15:06:15

like I would consider IPersistentVector to be a useful SPI-level interface that people can implement to do useful things and that is occasionally useful as a polymorphic target

Alex Miller (Clojure team)15:06:25

PV is a concrete implementation detail so past that line, and generally something that you should not need to use directly and LazilyPersistentVector is implementation details even farther past that that I would not recommend anyone using

borkdude15:06:43

I agree fully with this, but many libs don't which is why I've included some of those in bb anyway...

borkdude15:06:07

similarly for things like clojure.lang.RT/iterator or so, which is why I made an ask for some of these things to expose it as a function, but it's an uphill battle to ask everyone to abide by this

borkdude16:06:59

@U06FTAZV3 FWIW, schema now works in bb :)

❤️ 1
🙇 1
borkdude16:06:12

also clojure.spec

borkdude16:06:02

@U06FTAZV3 With some local hacking, I was able to make malli work (a bit):

(ns malli
  (:require [babashka.deps :as deps]))

(deps/add-deps '{:deps {metosin/malli {:local/root "/tmp/malli"} #_{:mvn/version "0.8.8"}}})

(require '[malli.core :as malli])

(prn (malli/validate [:map [:a [:int]]] {:a 1}))
(prn (malli/explain [:map [:a [:int]]] {:a "foo"}))
$ clojure -M:babashka/dev ~/Dropbox/temp/malli.clj
true
{:schema [:map [:a :int]], :value {:a "foo"}, :errors ({:path [:a], :in [:a], :schema :int, :value "foo"})}
cc @U055NJ5CC

❤️ 2
🎉 1
jcf16:06:56

That’s really cool! Thank you, @U04V15CAJ!

borkdude16:06:31

I do need to make SCI work with mutable deftypes to make m/Cached work, which is basically the biggest blocker to make malli working properly

borkdude16:06:49

But this is something I've wanted to do anyway, so perhaps soon-ish...

ikitommi16:06:28

malli.impl uses it for perf reasons as I think many clojure libs (internally) when performance matters.

ikitommi16:06:22

happy to help with the bb-support, just not sure how I could help.

borkdude16:06:16

@U055NJ5CC I'll let you know when I need something

Alex Miller (Clojure team)17:06:20

but is it really that much better than using vec which calls it?

Alex Miller (Clojure team)17:06:51

enough better to be worth using internal implementation details?

richiardiandrea17:06:11

Oh malli support is great news, we are using clojure.spec with babashka to convert/conform (I know I know) csv files and it is a bit on the slow side

borkdude18:06:01

@U0C8489U6 Could just be SCI being slow there. Could be worth comparing with JVM with both malli and spec to give a fair comparison between the two.

👍 1
ikitommi19:06:17

@U064X3EF3 have benchmarked many of the clojure public & non-public APIs. As a single invocation, it might not matter, but using the fastest possible impl in all the possible (internal) places might accumulate an order(s) of magnitude difference, which starts to matter. Some quick benchmarks here: https://www.metosin.fi/blog/high-performance-schemas-in-clojurescript-with-malli-1-2/

👍 1
ikitommi19:06:26

that said, in most real life use cases, the order(s) of mag diffs in a simple data transformation might not matter. But, still fun to play the perf game.

Alex Miller (Clojure team)19:06:49

Only fun until your code is all broken by depending on internal details…

borkdude19:06:40

And fun because in bb those things will actually be slower if you're not using the core fns ;)

😮 1
richiardiandrea19:06:38

This leaves me a bit confused on what to do 😄 but yeah, I think/hope seeking performance makes sense for my use case cause we are transforming a (potentially large) csv file using spec

borkdude19:06:26

@U0C8489U6 With large files / throughput just use the JVM I'd say, either spec or malli.

👍 2
richiardiandrea19:06:36

it definitely seems like malli plus babashka would be a winner but I'll try to profile as soon as I have some time

borkdude19:06:58

Or if your data is really big, maybe take a look at what @UDRJMEFSN has made with his data science libraries which are optimized for column based processing

👍 1
borkdude19:06:17

(I learned this on @U8A5NMMGD’s podcast recently)

❤️ 2
ikitommi20:06:24

> Only fun until your code is all broken by depending on internal details… > based on history, Clojure has been very good at not breaking, even internals. Also happy to fix the libs if that would happen.. That specific fast path was learned from one of the @UDRJMEFSN fast data library btw :man-bowing:

borkdude20:06:25

All hail @UDRJMEFSN!

🚀 1
💯 1
chrisn15:06:27

I am honored 🙂!!

chrisn15:06:33

And I am working on faster versions of persistent datastructures (hashmap hashset, vector) along with a solid portion of clojure.core right now. Just not ready to release it but getting there.

👍 1
❤️ 1
😮 1
Ben Sless16:06:56

are you using inlining for those reimplementations of clojure.core?

chrisn12:06:49

Not yet in general. I use macros for assoc-in and get-in. Other than that no, just extremely carefully written Java.

Ben Sless12:06:42

There are opportunities for update-in, assoc and merge, feel free to https://github.com/bsless/clj-fast

borkdude12:06:17

OK, back on topic, bb + malli. Got a fork of malli (where I made minor changes for bb) working where all of malli.core now works according to the tests:

$ clojure -M:babashka/dev test:bb

Testing malli.core-test

Ran 31 tests containing 1231 assertions.
0 failures, 0 errors.

🎉 3
❤️ 1
ikitommi13:06:46

That's brilliant. Will you do a PR for malli about those?

borkdude13:06:15

Will do. Trying to minimize the change by also making borkdude.dynaload compatible with bb ;)

👍 1
richiardiandrea16:06:42

This is great news

borkdude16:06:05

@U0C8489U6 If you want to try:

bash <(curl -s ) --version 0.8.157-SNAPSHOT --dir /tmp
for installing bb master in /tmp. Then in a script:
(ns malli
  (:require [babashka.deps :as deps]))

(deps/add-deps '{:deps {metosin/malli
                        {:git/url ""
                         :git/sha "d44a6cedaf3e95370e511dc1d3762fdd15bf4d0c"}
                        #_{:local/root "/tmp/malli"}}})

(require '[malli.core :as malli])

(prn (malli/validate [:map [:a [:int]]] {:a 1}))
(prn (malli/explain [:map [:a [:int]]] {:a "foo"}))

❤️ 1