Fork me on GitHub
#clojure
<
2021-04-11
>
Joshua Suskalo05:04:59

I'm currently testing out a way that I can remove the necessity to use gen-class in a library I'm writing. I have to have a class be referred to in a catch block for my code to perform the desired behavior, but to prevent breakage if the proxy-name implementation ever changes, I would prefer to not have the name of the proxy class directly embedded in the code. As a result, I'm considering writing code like the following:

(try whatever
     (catch #=(symbol (proxy-name Exception [some.interface.Foo])) e
       ...))

Joshua Suskalo05:04:20

Is this use of read-time evaluation considered a problem, or is this one circumstance where it would be acceptable?

vemv06:04:04

Note that #=( isn't a public API (as e.g. grepping https://github.com/clojure/clojure-site would reveal)

hiredman05:04:57

That sounds horrendous. Why do you need a custom exception type?

hiredman05:04:08

You are controlling both the throw and the catch, so like don't do that

Joshua Suskalo05:04:08

Custom control flow operations. It extends Error so that catch blocks in most java code won't interfere.

hiredman05:04:25

Sounds terrible

3
☝️ 6
Joshua Suskalo05:04:46

Eh, it's what I have to deal with to implement dynamic extent non-local return.

Joshua Suskalo05:04:29

The entire purpose of the Error class is that it isn't caught by sane code outside of the runtime.

jumar05:04:06

Your assumption might be violated more often than you think. Many people catch Throwable at edges to catch things like AssertionError

Joshua Suskalo05:04:40

While it may be that there is code that violates this, and perhaps more code than I would like, the places where it will matter (clojure code binding a context, passing a closure to java to run, and then signaling to the context from inside the closure) are infrequent enough that I'm comfortable just documenting them and moving on with my life.

Joshua Suskalo05:04:35

So it's more or less safe.

Joshua Suskalo05:04:52

All I'm trying to do at this point is remove the need for gen-class

Joshua Suskalo05:04:53

Also I guess it just doesn't quite work either, because the reader doesn't seem to like this code anyway

Joshua Suskalo05:04:13

I guess unfortunately I need to just expand this to a macro

quadron11:04:16

say i have a set of independent actions, what is the simplest representation of asynchronously executing them?

jjttjj12:04:32

Just wrapping each one in future will execute each one on a different thread. Do you need to do anything with the results or care about order of execution?

✔️ 3
quadron14:04:02

no, results don't matter

quadron14:04:53

i also found the

consume-async
in manifold library

quadron14:04:21

was wondering if there are special constructs in core async for colls of actions

quadron14:04:33

since colls are treated specially in clojure that is

jjttjj14:04:36

consume-async in core.async would just roughly be

(defn consume-async [f ch]
  (go-loop []
    (when-let [x (<! ch)]
      (f x)
      (recur))))

✔️ 3
jjttjj14:04:34

you can take items from a collection and put them on a channel with core.async/onto-chan!

pinkfrog13:04:54

I would like to refactor by moving all clj files under xx.yy name space to xx.yy.zz. How to achieve that?

vemv16:04:45

clj-refactor is supposed to do that but it's not super fast beyond a certain project size 😔 for these cases I simply create a folder (`yy` this case ), rename files accordingly and then perform a search/replace for updating the ns forms did so last week, it was a pretty meaty case and yet it still took me < 5m

fubar14:04:06

Are there any current Clojure newsletters? It looks like The REPL and and Clojure Weekly have been deprecated which is a bummer. https://www.therepl.net/newsletters/ https://us19.campaign-archive.com/home/?u=f5dea183eae58baf7428a4425&amp;id=ef5512dc35

chepprey15:04:29

Eric Normand's https://purelyfunctional.tv/ has a newsletter signup. I'm a subscriber. All of Eric's stuff is great.

borkdude14:04:49

Perhaps a new initiative could be organized as a collaborative repo where people can make PRs and the newsletter is built semi-automatically, this would probably make it more sustainable and less dependent on one person. I know other languages like Rust and Haskell have something like this.

pinkfrog14:04:54

@borkdude For clj-kondo.lint-as/def-catch-all, what does it check?

borkdude14:04:09

it disables all linting but just registers the fact that (deffrob foobar ....) introduces a var foobar.

pinkfrog14:04:10

I’ve another question, how can I supprese warning on a variable in a clj file? Can I do that inline in that file?

pinkfrog14:04:39

For example, I have a core.clj where I import other variables for external use.

borkdude14:04:00

Let's take it to #clj-kondo

colliderwriter16:04:32

I have a question about Kaocha tests which i suspect is really a question about classloaders. I was trying to config Kaocha in my multi-project repo. which means that some defaults must be altered. Specifically, the test-pathskey must point to ${subproject}/test rather than test. After making this change, Kaocha was not finding any tests. I had my test file tree arranged to mirror the source tree. Therefore, the first component I wanted to test was not in the root test directory, but rather in test/components/first-component.cljc That didn't work either, so I copied it to test just to see if I could get a test to run. Eventually I figured out that the filenames should be snake case rather than kebab case. That worked, but I still had the kebab case files in my tree and to my surprise it found all three. I then tried moving the snake case file to the components directory as per my original intent, and Kaocha stopped finding it. I was about to file a Kaocha bug requesting better documentation but I decided I wasn't sure what the rules were. 1. Does the snake case file have to be in the test root? (Is that a Kaocha bug or a rule of classloading?) 2. Once Kaocha finds a snake case file, why can it then find all the kebab case files, even in different directories?

Noah Bogart16:04:52

Clojure files have to be snake case, right?

colliderwriter16:04:53

i just checked my source tree and i don't have any filenames with multiple segments, have i lost track of a fundamental rule? maybe :sheepish

colliderwriter16:04:04

that said, my questions still stand

andy.fingerhut16:04:54

I don't know about Koacha's rules for finding source files, but Clojure/Java require must have snake case, or it will not find the files.

andy.fingerhut16:04:40

If Koacha sometimes finds files with kebab case, then keeping such files around, rather than renaming or deleting them, is going to cause confusion for you on what Clojure/Java is loading, versus what Koacha is finding.

andy.fingerhut16:04:01

If the classpath you have set up in your Leiningen project.clj or Clojure CLI tools deps.edn file includes a directory like test, and you have a test namespace named components.first-component, then that should be in a file named test/components/first_component.clj, or Clojure/Java require will not find it.

andy.fingerhut16:04:29

(unless you confusingly have a file named components/first_component.clj in another of your classpath directories, e.g. src. Avoiding such name conflicts is why you will often see test namespaces have test as part of their name, so their namespace names do not conflict with production source code file names, e.g. components.first-component for a production code namespace, and components.first-component-test for a test namespace.

colliderwriter16:04:58

i hold up my hands in surrender re: snake case. I'll change the kebab case test files to snake case. I'm genuinely surprised that i have zero multi-segment filenames in this (pretty large) code base. Apparently my code is so well organized that it's not necessary? Obviously whatever-test namespaces were going to highlight this.

andy.fingerhut16:04:01

Are you saying your projects have no Clojure namespaces with . in their names?

andy.fingerhut16:04:35

That is legal, but unusual.

colliderwriter16:04:24

No, but the dot-based namespace hierarchy maps precisely to the slash-based filesystem hierarchy

andy.fingerhut16:04:47

dots in Clojure namespaces are multi-segment namespaces

andy.fingerhut16:04:25

That is what is meant by a multi-segment namespace, unless I have missed the meaning of that term completely somehow.

andy.fingerhut16:04:26

When you say "multi-segment filenames", do you mean "has a dash or underscores in their names" ?

colliderwriter16:04:02

yes, but there are no multi-segment filenames. There is no foo-componentthus foo_component.clj, everything is arranged like components/foo.clj

colliderwriter16:04:52

yes to your last question

andy.fingerhut17:04:18

I will defer to others here who may know the terminology better than I do, but I am pretty sure that most Clojure developers use "multi-segment namespace" to mean something like foo.bar, with two segments, versus namespace foo-bar, which has one (no .). It has nothing to do with underscore/dash in the name.

👆 3
colliderwriter17:04:34

i was trying to draw a distinction between namespace and leaf filenames. I think I have my answers though. Thanks everyone for indulging my self-induced myopia

dpsutton18:04:01

i'm working on a clojure.main/repl that will put all repl forms into a graph database. What would be a good api for using this? Currently it's just (:grepl/root +) would show all repl forms that used + somewhere in them or (:grepl/parent +) to show all one-level-up forms that used +. Trying to think how you kinda interact with the repl's provided features

dpsutton18:04:01

grepl.repl=> (let [a 1 b 2] (+ a b))
3
grepl.repl=> (:grepl/parent b)
(+ a b)
nil
grepl.repl=> (:grepl/root b)
(let [a 1 b 2] (+ a b))
nil

seancorfield18:04:27

Since (:some/kw b) has meaning already, wouldn’t (grepl/parent b) be less confusing as an API?

seancorfield18:04:00

Also, given (let [a 1 b 2] (+ a b)) I think I would expect [a 1 b 2] to also be a parent to b?

dpsutton18:04:53

To me that’s a sibling in the tree

seancorfield18:04:36

OK, so bindings are not considered “use”. Fair enough.

dpsutton18:04:04

And about using a symbol, I didn’t want to do that in case someone had that as an alias. Using the keyword made sure it’s always kinda ok to hijack evaluation

seancorfield18:04:02

What about:

grepl.repl=> (def b 2)
#'grepl.repl/b
grepl.repl=> (let [a 1 b b] (+ a b))
3
grepl.repl=> (:grepl/parent b)
???

dpsutton18:04:09

grepl.repl=> (def b 2)

#'grepl.repl/b
grepl.repl=> (let [a 1 b b] (+ a b))

3
grepl.repl=> (:grepl/parent b)
(+ a b)
(def b 2)
nil
grepl.repl=> 

dpsutton18:04:31

grepl.repl=> (:grepl/root b)
(let [a 1 b b] (+ a b))
(let [a 1 b 2] (+ a b))
(def b 2)
nil
grepl.repl=> 

seancorfield18:04:54

So the “use” of the global b in the let binding isn’t registered here?

seancorfield18:04:38

(I’m trying to establish a “mental model” of what you’re trying to do here — and somewhat failing)

dpsutton18:04:44

correct. a "use" is where a node is exactly what you typed in. it's doing a tree-seq on the form, stuffing each value into a graph database

dpsutton18:04:12

and when you look for "parent", it find's all the nodes that match it exactly and then "goes up" one level to give context, as otherwise it would just return exactly what you typed in

dpsutton18:04:51

grepl.repl=> (pprint (->tx-data '(let [a 1 b b] (+ a b))))
({:grepl/form "(let [a 1 b b] (+ a b))", :grepl/root -1, :db/id -1}
 {:grepl/form "let", :grepl/root -1, :db/id -2}
 {:grepl/form "[a 1 b b]", :grepl/root -1, :db/id -3}
 {:grepl/form "(+ a b)", :grepl/root -1, :db/id -4}
 {:grepl/form "+", :grepl/root -1, :db/id -5}
 {:grepl/form "a", :grepl/root -1, :db/id -6}
 {:grepl/form "b", :grepl/root -1, :db/id -7}
 [:db/add -1 :grepl/parent -1]
 [:db/add -1 :grepl/root -1]
 [:db/add -2 :grepl/parent -1]
 [:db/add -2 :grepl/root -1]
 [:db/add -3 :grepl/parent -1]
 [:db/add -3 :grepl/root -1]
 [:db/add -4 :grepl/parent -1]
 [:db/add -4 :grepl/root -1]
 [:db/add -5 :grepl/parent -4]
 [:db/add -5 :grepl/root -1]
 [:db/add -6 :grepl/parent -4]
 [:db/add -6 :grepl/root -1]
 [:db/add -7 :grepl/parent -4]
 [:db/add -7 :grepl/root -1])

seancorfield18:04:02

Right, but I would expect (let [a 1 b b] ..) to register as a use of the global b.

dpsutton18:04:18

oh i see. i wonder why it didn't descend into that

dpsutton18:04:44

ah, classic (seq? [a 1 b 2]) is false

dpsutton18:04:42

switching to seqable? as the branch test in the tree-seq fixes it

dpsutton18:04:45

grepl.repl=> (:grepl/parent b)
(+ a b)
[a 1 b b]
(def b 2)
nil
grepl.repl=> 

seancorfield19:04:34

OK, that’s much more intuitive for me now!

dpsutton19:04:00

thanks. and i further updated it so it enumerates maps better.

dpsutton19:04:16

need to refine the api as well. i've got "substring" type matching as well

seancorfield19:04:23

Kind of amazing how little code something like that is… 👀

dpsutton19:04:59

Ha. The graph db does all the work. It’s really just putting the forms and the pointers in there

dpsutton19:04:24

Some thought how to provide a help menu and how to return items. Might want values not just pprint

dpsutton19:04:52

Also, I’ve found that the pr str of regex aren’t readable so that’s why there’s a try catch in there

dpsutton19:04:07

Probably need to harden more as well in general

dpsutton19:04:30

Also if I can figure out how to return values can make them datafiable so you can walk up the form as much as you like

yuhan18:04:45

How do I get a ns-qualified symbol in a macro? (symbol (str (ns-name *ns*)) (str sym)) works but doesn't seem right.

potetm19:04:12

well. it depends what exactly you’re trying to do

yuhan19:04:42

I tried that but it doesn't qualify the symbol

yuhan19:04:09

Context: I'm trying to write a def macro which does expression-based caching of the definition body:

(defonce cache (atom {}))

(defmacro defcached
  [sym & args]
  (swap! cache assoc (symbol (str (ns-name *ns*)) (str sym)) args)
  `(def ~sym ~@args))

potetm19:04:15

it does, but it also resolves it

phronmophobic19:04:26

`~sym
That doesn't produce a fully qualified symbol for me either

phronmophobic19:04:51

I think you @UCPS050BV’s (symbol (str (ns-name *ns*)) (str sym)) should work

phronmophobic19:04:46

I might also suggest using [*ns* sym] as the cache key rather than converting it to a fully qualified symbol

phronmophobic19:04:21

or even (swap! cache assoc-in [*ns* sym] args)

yuhan19:04:36

oh right, that makes a lot more sense!

yuhan19:04:57

Are there any issues with using namespace objects as map keys? Or should I convert them to symbols to be safe

potetm19:04:11

I would actually suggest using a caching lib.

phronmophobic19:04:52

specifically, clojure.core.cache.wrapped from that lib

yuhan19:04:03

I had a look at that but it seemed too complex a solution for my use-case - I just wanted to scratch an itch with REPL development

phronmophobic19:04:31

if you want a per function cache, you can use the built in memoize

yuhan19:04:08

ie. having a

(def big-data (expensive-operation :some :args))
and wanting to only re-evaluate it when the arguments change

yuhan19:04:07

I guess memoize would work in that case :thinking_face: Thinking of a case where it wouldn't

phronmophobic19:04:07

for that, I typically have in my repl:

(def expensive-operation-memo (memoize expensive-operation))
so you can do:
(def big-data (expensive-operation-memo :some :args))

phronmophobic19:04:25

so that I can still redo the calculation if I really want to

yuhan19:04:07

Ah, just remembered why I wanted to stay away from memoize - I don't want previous evaluations taking up memory in the memoization map

yuhan19:04:53

Calling expensive-operation on a new set of arguments means I don't want to revisit the old args

potetm19:04:27

Are you looking for def?

potetm19:04:46

really though, clojure already has a cached, ns-symbol -> value lookup: the namespace. 🙂

yuhan19:04:40

Hmm, I'll make it more concrete - let's say I have

(def im
  (img/load-image "resources/A1.png")) 
with a 500Mb image file that takes half a minute to load. Re-evaling the form / reloading the namespace would cause the image to be read every time

phronmophobic19:04:29

caching as you were doing before seems reasonable. You may also be able to get away with defonce

phronmophobic19:04:44

(defonce im (img/load-image "resources/A1.png"))

phronmophobic19:04:55

which doesn't work if you want to reload it if the args change

yuhan19:04:10

Yeah, I want to reload it when I change A1 to A2.png

phronmophobic19:04:25

yea, your cache macro seems like a good fit :thumbsup:

yuhan19:04:06

Great, thanks for the advice!

potetm19:04:10

does it actually solve that problem?

potetm19:04:23

(defcached foo
           (do (println "sleeping...")
               (Thread/sleep 1000)
               (println "done")
               :foo))

potetm19:04:31

Loading dev/user/cert.clj... 
sleeping...
done
Loaded
Loading dev/user/cert.clj... 
sleeping...
done
Loaded

yuhan19:04:43

That was only the first half of the impl, I'm testing the rest of it now

potetm19:04:45

alternatively: defonce + alter-var-root for when you want to redefine

yuhan19:04:46

Yup, seems to work 🙂

(defonce definition-cache (atom {}))

(defmacro defcached
  "Behaves just like clojure.core/def, but subsequently
  is only reevaluated if the body expression has changed
  (according to Clojure equality semantics)

  Purely compile-time syntactical check,
  does not trigger if eg. a dependency has changed."
  [sym & args]
  (let [k [(ns-name *ns*) sym]
        old-definition (get @definition-cache k)]
    (if (= args old-definition)
      `(var ~sym)
      (do
        (swap! definition-cache assoc k args)
        `(def ~sym ~@args)))))

(defcached big-data
  "wow"
  (do (println "sleeping...")
      (Thread/sleep 1000)
      (println "done")
      (range)))

@definition-cache
;; => {[sandbox.defcached big-data]
;;     ("wow"
;;      (do (println "sleeping...") (Thread/sleep 1000) (println "done") (range)))}

potetm19:04:58

yeah that makes more sense now. I missed the part where you were going to check the body form before re-evaluating.