Fork me on GitHub
#clojure
<
2023-03-08
>
Drew Verlee00:03:09

Does anyone an example where they used the clojure transducer function eduction?

Ben Sless05:03:45

I use it where performance matters and I know I'm looking at a single pass, for example https://github.com/tonsky/datascript/blob/master/src/datascript/db.cljc#L614

👍 2
borkdude08:03:15

I've used it to compose filtering results from sql query result sets at some job (where it was also a single pass)

👍 2
Drew Verlee16:03:28

i see, i'll look at it again, I'm not sure I understand the single pass aspect yet.

borkdude16:03:31

@U0DJ4T5U1 when realizing an eduction computations aren't cached, but executed every time

👀 2
Ben Sless16:03:50

The way to think of an eduction is a (r)eduction. it's literally a thin wrapper for the purpose of reducing over something. If you're going to reduce on it later, it's great. If you want a sequence you're going to consume more than once, less great.

👀 2
Patrick Brown02:03:15

If anyone would like to discuss relying heavily on metadata attached to clojure vars, hit me up. I’ve been building a system around it, the more I used Clojure’s metadata facilities, the more I liked it. It’s really enabling a plain data first approach, but I don’t know what I don’t know. I’m curious if you have tried it, in what capacity and what was your overall impression/experience?

dgb2308:03:35

I have two uses for meta data in the current project I'm writing: • malli lets you put function schemas in metadata which I find very convenient • reitit lets you generate routes at runtime, which I use to create specialised routes and handlers based on db transactions. I plan to put some meta data into the handlers such as the transaction time and maybe more to be able to inspect them. If you want inspiration for meta data usage, then I suggest you read Software Design for Flexibility by Sussman and Hanson.

2
p-himik11:03:43

Metadata is slightly dangerous though: • Not all core functions preserve it • Wrapping objects hides it, extracting data destroys it • Not immediately visible when logging, printing, using tools like Portal • Not all types can have it So one has to be more careful with it than with regular data. I prefer avoiding it altogether if other options are not much worse.

👍 4
💯 2
dgb2312:03:38

Thanks for the heads up. I assume putting meta data on functions is a robust thing as quite a bunch of tools and libs depend on it. Is that correct?

p-himik12:03:45

Putting it onto anything is robust, assuming that the put operation is successful. But doing something to that "anything" might have some unintended consequences. In the case of functions, if we're talking about defn then the metadata is actually not on the function itself but rather on the var that holds it. And it's exceedingly rare that you have to do something to a var as we usually operate on values.

🙏 4
2
Patrick Brown21:03:17

@U2FRKM4TW Just for clarity, are there gotchas and worries associated with putting metadata directly on vars? The documentation said meta was immutable, so I translated that as dependable. What are the functions that do not preserve meta? On a related note, I have definitely run into the portal issue. That was a most unpleasant surprise.

p-himik21:03:36

Metadata is immutable, but things you put it on might not be. The opposite is true for vars - they are mutable, while the things you put into them might be immutable.

(def ^:should-be-true x)
=> #'dev/x
(:should-be-true (meta #'x))
=> true
;; Metadata can be changed by interning a symbol with metadata.
(intern *ns* (with-meta 'x {:should-be-true false}))
=> #'dev/x
(:should-be-true (meta #'x))
=> false
;; Or by directly setting it on a var.
(.setMeta #'x {:should-be-true 7})
=> nil
(:should-be-true (meta #'x))
=> 7

Patrick Brown21:03:22

Ok, I get that. That’s expected behavior in my eyes.

p-himik21:03:23

Oh, and there are more direct alter-meta! and reset-meta! which I completely forgot about. :) In any case, yeah - using meta on vars should be OK. I imagine there might be some nasty instrumentation or what have you libraries that lose your own metadata, but I haven't encountered them.

Patrick Brown21:03:48

Cool, while I’ve got you and you’re opinionated. Do you have a strong opinion on using meta as a global (AS OUTLINED: not really immutable) registry and the values stored in var meta as arguments to fns?

p-himik21:03:04

Can you give an example?

Patrick Brown21:03:38

I have datomic queries that have :returns and :args keys on meta, I then supply a map with :i-want and :i-have keys to a generic query fns which finds the match and executes the query that fits the needs. Attaching vars to the meta the queries can be exectued anywhere on the classpath. This is an example of a generic system that extends to a staggering amount of functionality in my system. Honestly, it’s really cool and fun, but for obvious reasons it feels scary.

p-himik21:03:38

If you iterate over every single var, then why not have an explicit registry? If you can have an explicit registry, then why not store those :returns and :args values right there along the actual queries?

Patrick Brown21:03:54

Because then I’d have to manage two things, the query and the registry. Both are tied to a namespace in the since of needing to be required, not cyclical, etc. Instead I just write one datastructure, the meta for the query and let the registry be created on the fly through a pure function and the query executed regardless of location, In this way the registry always reflects what’s available and coordination is baked in. It’s kind of like how some test runners find and execute tests

Patrick Brown21:03:30

It’s honestly how I manage systems in integrant, building out core.async mixers and pub/sub, I’m doing a lot of things with it. It’s kept the code base small. Then I turned around and said, yikes, I’ve really shoved a bunch of stuff in this.

p-himik21:03:11

If there's anything I've embraced after some experience with Python, is that explicit is better than implicit. :) Tests are different because if something test-related breaks then it happens in an environment that's not visible to users in any way. I don't see requiring an extra ns as a great obstacle (after all, we require clojure.string to use str/join opposed to having Python's ",".join(arr)). Re-frame has something similar, only there it's just one namespace. All the things that you register with re-frame are then referred to by their IDs. But perhaps less reliable then wars - all the "unresolved"-like errors move to run time. It greatly reminds me of C/C++ and recently Swift - there, you don't have to require something for it to be available. You just have to load it somewhere, and then it's there. Bloody impossible to find anything unless you're using a fully language-aware IDE that's completely in sync with your cmake config or whatever it might be. Perhaps if I see the actual code it will make me change my mind, but so far I'm far from being convinced that it's a good approach.

Patrick Brown21:03:37

Fair. I’ve got some other reasons, but it’s some proper valid points. I really need to have a critical look at the tradeoffs. On one hand I can store and deliver most of my application from datomic serialized. On the other hand, my code looks like a bunch of maps and vars are not exactly trustworthy. You’ve brought up some good insight. I appreciate it.

👍 2
gleisonsilva20:03:00

Hello! Anyone knows if there's something in Clojure similar to Quarkus? (in other words, some kind of framework that is target/taylored to compile to native using Graal (besides other little things that Quarkus offers)...

respatialized20:03:16

https://github.com/nikvdp/bbb I haven’t used this project and it’s mostly focused on CLIs, but its aim is making it easier to package things for Graal from Clojure. Not sure if that matches your needs (not very familiar with Quarkus either)

respatialized20:03:52

there is also a #CAJN79WNT channel which may have more info

borkdude20:03:57

@U2CVD69EC http-kit is very small and works fine with native-image

borkdude20:03:16

but depending what dependencies you pull in, YMMV

Danilo Oliveira12:03:43

can Aleph + ring run on graalvm?

borkdude12:03:07

haven't tried

Danilo Oliveira12:03:35

nice!!! thanks a lot!

Matthew Davidson (kingmob)06:04:38

@U035WLPF552 Aleph/Manifold don't officially support graal yet, but we would like to eventually. Let us know what issues you run into.

Kevin Lyter23:03:38

Nooby deps.edn question: I created a new application w/ https://github.com/seancorfield/clj-new. clojure is installed from AUR. Running and testing that works fine. I added a dependency, just com.taoensso/timbre {:mvn/version "6.1.0"}. clojure -T:build shows the jar downloading. Then when I test it though, it doesn't find the jar. > Execution error (FileNotFoundException) at lyterk.accounting/eval1939$loading (accounting.clj:1). > Could not locate com/taoensso__init.class, com/taoensso.clj or com/taoensso.cljc on classpath. It's definitely in ~/.m2, I haven't changed any defaults like the classpath or anything. I haven't used Clojure since the boot days, so it's been quite a bit, but this feels like something that should be working without a ton of debugging

hiredman23:03:03

looks like you are trying to use the maven coordinate as a namespace name

hiredman23:03:32

maven coordinates (that name artifacts in a maven repo) and namespace names are disjoint

hiredman23:03:57

you should look at the docs (or source) for the timbre library to find out what the names of the namespaces in it are

Kevin Lyter23:03:33

Makes sense. I have tried this with other libraries, and I don't think it's that alone. That disjointness did always mess me up though. Fixed it to (:require [taoensso.timbre :refer [log]])) which still results in Could not locate taoensso/timbre__init.class, taoensso/timbre.clj or taoensso/timbre.cljc on classpath.

Kevin Lyter23:03:36

There in 6.1.0 I do have the jar with -rw-rw-rw- 47706 27-Feb-2023 16:18:12 taoensso/timbre.cljc

hiredman23:03:11

have you restarted your repl since adding the dependency to your deps.edn?

hiredman23:03:56

how did you start your repl, what does you deps.edn look like, and what is the output of clojure -Sdescribe

Kevin Lyter23:03:40

Atm my testing procedure is just clojure -T:build test I think that environment is self-contained, I assume there isn't any state carried from one invocation to the next

Kevin Lyter23:03:55

> {:paths ["src" "resources"] > :deps {org.clojure/clojure {:mvn/version "1.11.1"}} > :aliases > {:run-m {:main-opts ["-m" "lyterk.accounting"]} > :run-x {:ns-default lyterk.accounting > :exec-fn greet > :exec-args {:name "Clojure"}} > :build {:deps {io.github.seancorfield/build-clj > {:git/tag "v0.8.2" :git/sha "0ffdb4c" > ;; since we're building an app uberjar, we do not > ;; need deps-deploy for http://clojars.org deployment: > :deps/root "slim"} > com.taoensso/timbre {:mvn/version "6.1.0"} > clojurewerkz/money {:mvn/version "1.11.0"} > org.clojure/data.csv {:mvn/version "1.0.1"}} > :ns-default build} > :test {:extra-paths ["test"] > :extra-deps {org.clojure/test.check {:mvn/version "1.1.1"} > io.github.cognitect-labs/test-runner > {:git/tag "v0.5.0" :git/sha "48c3c67"}}}}}

hiredman23:03:09

how is test defined in your build.clj?

hiredman23:03:07

you added the logging stuff as deps for the :build alias

hiredman23:03:34

they should be just under the toplevel :deps key

Kevin Lyter23:03:53

I totally missed that. Makes sense though

hiredman23:03:55

the :build alias is for dependencies of your build.clj

Kevin Lyter23:03:10

Tysm for your help