Fork me on GitHub

Hi! I have a question about using datatypes in Clojure. I’m thinking about this situation: I have a special two-element vector that can identify some entitiy; for example, [:user/id 123] can refer to a specific user, and [:car/id 567] can point to a specific car. Of course not every two-element vector is such a special reference. So I’m thinking it makes sense to somehow be able to mark these special two-element vectors as being a reference to another entity. I could then in all kinds of processing (`map`, reduce, prewalk, etc) identify them and act accordingly. There are external APIs that expect to be able to use these things as vectors though. So that got my OO-mind to think: maybe I should create a subclass of PersistentVector that add a bit of API and of course the ability to identify these things. But that doesn’t seem to be how it was designed in Clojure ( What would be some idiomatic ways of dealing with this in Clojure? Thanks for your inspiration!


I do something similar to implement custom nested transformations in what ultimately becomes a json map. I use qualified keywords to signify my customizations, since json lacks qualified keys, a vector starting with one must be "mine". This might not be applicable for you though. You could also process them with a multimethod with a :default implemention that just passes through the value unchanged unless you have a custom method for it

Ben Sless13:09:13

Why not use a protocol and extend via metadata?


@UK0810AQ2 If I understand correctly (“extend via metadata”) because some operations drop metadata…


What about something like this?

(deftype EntityReference [id-attr id-value]
  (seq [this] (seq (vector (.id-attr this) (.id-value this))))


(partial implementation, but as a solution direction)


Why do you need to mark the references?


I heard a desire but not a convincing rationale


All the discussion so far was downstream of that decision


Partially the rationale is me wanting to learn about deftype/defrecord etc. But also I’m thinking if I have a big nested datastructure with these things, I could easily automatically expand them using a tree walker and defmulti polymorphism for example.


So then the tree walker would need a way to distinguish between these special vectors that need extra processing, and normal vectors.


another way a tree walker could operate is by attribute context:

  :patient [:patient/id 123] ;; tree walker could dispatch on :patient


I would caution against custom types, they will make data more opaque unless you implement a couple dozen methods


Yeah I can see that risk, thanks for your advice!


I would just create some functions that convert data to please the external APIs and have another format that pleases myself


Would it be considered “bad” to define a new record and then make it conform to IPersistentVector (if that’s even possible)?


That would be a bit weird


You could also use metadata on vectors but in my experience this is brittle since some operations drop metadata


Yeah exactly, that option I already rejected.

Edward Hughes13:09:49

Would anyone have a clue why a dependency would be unable to load its own namespaces? I have a project that relies on a library I had to fork and have tried hosting both from github via jitpack, and installed into my local .m2 (opening the installed .jar for a peek shows that the ns is, in fact, there) but my repl complains that it doesn't exist when trying to load-file on the ns that requires the library. That is to say, library.core is on the path, but library.dir.required-ns is not.

Joshua Suskalo13:09:34

you're manually calling load-file from the repl?

Edward Hughes13:09:04

I'm using the chlorine shortcut on atom

Joshua Suskalo13:09:03

Ah, I see. Not entirely sure why that'd be failing if it is indeed on your classpath, which basically only requires that your chlorine instance is using the correct build tool to determine what your dependencies are.

Joshua Suskalo13:09:02

I will point out that atom is basically deprecated at this point though. If you're comfortable with it, that's ok, but as long as you're aware that it's not really going to be maintained in any official capacity. Though I suspect you are since you're using it.

Edward Hughes13:09:24

Yeah, the odd thing I can't seem to find any documentation on is that it loads the dependency just fine, but cannot resolve the dependency's internal require statement. I'm using a community maintained build of atom pending graduation to Emacs since it's good enough:tm:. There doesn't seem to be a similar problem with any of my other projects, and this is my first attempt at using my own library, so I figured it was how I deployed it/set up the project.clj.

Joshua Suskalo13:09:28

Ah, if it's your own library and it's the first time you've built one, there's a possibility that you have the paths inside the jar not matching the namespace names. What's the actual structure inside the jar look like?

Edward Hughes13:09:46

As it should be, pic related. The odd thing is that I get no complaints when I try to require the namespace inside the interndir directly Would the fact it's a :refer :all be relevant?

Joshua Suskalo13:09:10

and the namespace that you're requiring is krak.intern.client ?

Joshua Suskalo13:09:14

or are you trying to like require krak.intern :refer :all

Edward Hughes13:09:13

I'm requiring krak.client in my project, the main ns of the lib, which requires krak.intern.client via a :refer :all

Joshua Suskalo13:09:10

Ok, so requiring krak.intern.client as long as your krak jar is on the classpath should work fine. If it's not then I question if the jar is actually on your classpath.

Edward Hughes13:09:16

however, if I do this in the project under development, no problems at all, and it seemingly can find the ns

Joshua Suskalo13:09:03

so are you actually getting errors when you load the ns, or are you just getting underlines with warnings?

Edward Hughes13:09:07

so krak.client is found, krak.intern.client can be found, but not from krak.client

Edward Hughes13:09:26

I get errors, both using the cholrine repl commands and by trying to load it in the terminal running the project via lein

Joshua Suskalo13:09:29

and you have the dependency declared in leiningen?

Joshua Suskalo13:09:33

in your project.clj

Joshua Suskalo13:09:47

Sorry, just trying to run through everything to make sure

Edward Hughes13:09:52

yep, initially it was with the com.github.user/ prefix to resolve it via jitpack, but same issue whether I try to call it in remotely or have it installed into my local repo

Joshua Suskalo13:09:38

Yeah this is confusing then. The last major thought I have on this if lein can't work with it is that your client namespace has a compiler error, which sometimes results in this "cannot require" exception.

Edward Hughes13:09:57

hmm. That's unlikely because I was doing a bunch of prototyping for the project inside the library project, but I'll give that a look

Edward Hughes13:09:02

works fine, even when loading the client ns from another ns

Joshua Suskalo13:09:27

Hmm, then I'm about out of ideas, I'm sorry.

Edward Hughes13:09:39

np, thanks for the assist. Hopefully someone else might have run into this before.

Edward Hughes15:09:06

On the off chance someone else has a stab at pondering what might be going wrong, moving krak.intern.client out into the top level src dir doesn't seem to help.


Can I make a new zipper from node? I want to walk the full sub-tree of some nodes to gather information on how to edit the node. Not sure how I should go about it...

Joshua Suskalo14:09:01

couldn't you just call zip/node and then build a new zipper from that?

Joshua Suskalo14:09:26

You could even make it so that it builds the same type of zipper by fetching the metadata off the one you get passed and using that to construct the zipper on the produced node.

Joshua Suskalo14:09:30

That would make it generic.

Joshua Suskalo14:09:41

(require '[ :as zip])

(defn sub-zipper [loc]
  (let [{::zip/keys [branch? children make-node]} (meta loc)]
    (->> loc
        (zip/zipper branch? children make-node))))


> couldn't you just call zip/node and then build a new zipper from that? I'm a total noob with zippers. I think this is exactly what I want to do!

Joshua Suskalo15:09:52

awesome, glad I could help!


And a beautiful little sub-zipper to boot. I like how it is general.


In Calva I have something quite zipper-like called a Token Cursor (that runs over the Clojure code in the editor). I think cloning out what would be equivalent to a sub-zipper is the most common thing happening in Calva. 😃

Joshua Suskalo15:09:50

Yeah, it's really nice to have the general library for working with zippers. I've never actually had a usecase for them myself but I really like them conceptually.

Joshua Suskalo15:09:17

It definitely feels like rewrite-clj and similar should be able to provide a zipper interface

Joshua Suskalo15:09:04

Oh, ofc they have

Joshua Suskalo15:09:58

it's too bad that it's a customized version. Oh well.


rewrite-clj is wonderful. I'm mostly using it by proxy of other tools, like clj-kondo, cljfmt, zprint, etcetera, though. Since Calva is mostly TypeScript and has its own cursor thing.

Joshua Suskalo15:09:13

I wish cljfmt wasn't quite so opinionated about configuration

Joshua Suskalo15:09:59

It's really frustrating as someone who writes a lot of macros that I can't distribute any kind of formatting information to anyone but cider users directly as a part of the dependency.

Joshua Suskalo15:09:25

I've wanted to work on a cider-compliant cli formatter using the clj-kondo analysis to grab var meta, but I've been too busy with other projects.

Joshua Suskalo15:09:43

plus cljfmt having the really bad semantics around formatting multiple things with the same symbol name.


Calva could pick up cljfmt config from the dependency, if that's what you mean. Right, @U02EMBDU2JU? You can file an issue on Calva describing it a bit, if you like. (If this is what you meant, haha.)

Joshua Suskalo15:09:00

If that's something that calva can support that'd be helpful and I'd start trying to distribute that. The problem is that cljfmt on its own has no way of "combining configs" as far as I can tell a-la clj-kondo, and ofc it also has no facility for importing configs via maven or git dependencies.

Joshua Suskalo15:09:37

Plus the problem of "this symbol has this formatting in this ns, regardless of where the symbol came from" is also an issue for this sort of thing.


@pez What are you talking about, exactly? It sounds like rewrite-clj but it might just be about


@pez If you want to use zippers with XML/HTML you can try the Hiccup zipper from Hickory


Thanks, @borkdude and @simongray. It is indeed HTML and I am using the Hiccup zipper. I think my question is more general, though. I have this Hiccup zipper (which is a zipper as well) over the AST and can edit the HTML very nicely. But for some nodes I need to know about things in the sub-tree of that particular node (it's children, I guess it is) before I know how to edit the node. Intuitively I want to walk the subtree, using z/next, but I think that risks walking out of the sub-tree. So I want to make a new zipper with my current node (element) as the root, walk that and collect the info I need. But maybe I am just holding the thing wrong...

Joshua Suskalo14:09:49

@pez I put a response on your original message in a thread that includes a function to do what I think you want.

🙏 1
❤️ 1

Is there any good way to check which env vars clojure is trying to access (trying to catch them all)? (overloading System is an option right?)

Joshua Suskalo15:09:15

That'd be hard. The JVM reads the entire environment when it starts to populate the internal map, and tools like environ in clj read the entire jvm system environment to turn it into a clojure map.

Joshua Suskalo15:09:32

So you'd just get tools saying "everything is used" if you weren't doing really smart static analysis.

Joshua Suskalo15:09:55

and I'm not aware of any tools like that for clojure at this time.


still most things inside app if they are trying to read particular env var would go to System/getenv ? so if I monkey patch it, I might be able to see right?


Just in case - if it's just for some debugging, a proper debugger would be able to put a breakpoint inside System.getenv(String). But note that there's also System.getenv().

👍 1

I already spotted, System/getenv is not a regular fn... trying to go monkeypatch route, funnily cannot eval symbol of System/getenv either way, doesnt behave like normal namespace


The problem is I am trying to go the other way, some env variable is missing and I am getting bizzare outputs, so trying to find exactly which env is causing that

Joshua Suskalo15:09:17

Yes, System/getenv is in fact a static method on the System class inside the JVM, so it can't be monkeypatched with Clojure.

😭 1

is there a way I can shadow System/getenv with my function? I dont even need it to work, other than print me args passed


Here's an involved hack to hack System/getenv and setenv:

👀 1

probably grepping for System/getenv is easier to find all the spots where your program accesses the environment?


not really, its probably some lib in dependency... anyways thanks a lot, reading the PR


well, you can grep dependencies too


assuming the source code is written in clojure


I would say to use bpftrace to intercept syscalls, but getenv doesn't issue a syscall


It is possible to do this using a Java agent, and something like bytebuddy, but it's far from trivial.

Ivar Refsdal10:09:20

I was able to hook into java.lang.System/getenv using java agent + javassist: The code is not pretty (reflection + getting classloader from thread), but it works.

😄 2

very interesting :)

🧡 1

I’m being asked at work how to expand the loop inside this macro. I’m very leery of nested macros. What’s the best way to do it?

(defmacro foo [xs]
    `(let [a# (atom 0)]
       (doseq [x# ~xs]      ;; <-- how to expand this loop?
         (swap! a# + x#))


What does expanding a loop mean?


is “unrolling” a better word?


inlining the loop so that the let-block has a bunch of swap statements instead of a loop around one of them


sure, unrolling


not possible in the general case, to do that you would need to know the value of evaluating whatever xs is bound to, and of course at macro expansion time it is not evaluated


for example in (let [x [1 2 3]] (foo x)) when foo is expanded, xs is bound to the symbol x


and there is no way to know in foo what value x will be bound to at runtime


similar (fn [x] (foo x))


is it possible if we assume xs is always a vector of integers


unevaluated I mean


there are two cases where you can kind of getaway with this, passing a literal is the easy one


so like, if you had a vector x and wanted to turn it into a seq of lists containing a single element, the number one, how would you do it?


(lists of just the number 1 for each element in the vector)


~@(for [x xs] (list 1))


not in a macro, just in general, how would you write code that did that


(for [x xs] '(1))


so what if instead of replacing with a constant list like that, you wanted a list where the first element was the symbol swap!, the second was a#, the third was +, and the fourth was the element in the vector?


like, syntax quote, and the unquoting bits are very useful, but not required for writing a macro, and can make things less clear, specifically when you are doing acrobatics quoting and unquoting at different levels


so it might be clearer to write a nice clojure function that can take some input and produces a data structure as an output, using all the functions you already use in clojure to construct the output data structure, and then whoops, the output datastructure happens to be valid clojure code, so you just swap the defn for defmacro and you are good


I guess the hard part is that a# is not accessible from inside the loop function


but I understand what you’re saying that it’s easier to have a function produce the swap statement list


if you take the whole macro and write it without using syntax quote(and the features it provides: unquoting, splicing, auto gensyms) then you'll have a better feel for using syntax quote


syntax quote is a helpful tool for writing macros, but it has limitations, and if you are not familiar with writing macros without using syntax quote, then the limitations of syntax quote are your limitations. it seems like maybe in this case your stumbling block is auto gensyms, if you just didn't use them, then you would be fine


this is good advice, I’m going to play with that, thanks


(defmacro foo-noloop [xs]
    (let [a-sym (gensym "a")]
        (list 'clojure.core/let [a-sym (list 'clojure.core/atom 0)])
        (for [x xs] (list 'clojure.core/swap! a-sym 'clojure.core/+ x))
        [(list 'clojure.core/deref a-sym)])))


this worked, I guess it’s just hard to follow but easy to write


(defmacro foo-noloop [xs]
    (let [a-sym (gensym "a")]
        (list `let [a-sym (list `atom 0)])
        (for [x xs] (list `swap! a-sym `+ x))
        [(list `deref a-sym)])))


I guess I can use syntax-quote to qualify the clojure.core symbols


(defmacro foo [xs]
  `(-> 0
     ~@(for [x xs]
         `(+ ~x))))

(macroexpand-1 '(foo [1 2 3]))
;; yields
(clojure.core/-> 0
  (clojure.core/+ 1)
  (clojure.core/+ 2)
  (clojure.core/+ 3))

🙏 1

I guess I'm not sure what you are actually after, so deleted my previous comments


but I guess the thing you were missing was the unquote splicing ~@?


no need for concat/list at all


@U05224H0W thanks! the problem is that I can’t use a gensym in a nested syntax quote


oh I see what you did, by removing the need for a named accumulator


the example is contrived but it was all I was given


Oh, this works just fine too:

(defmacro foo-noloop [xs]
  (let [a-sym (gensym "a")]
    `(let [~a-sym (atom 0)]
       ~@(for [x xs]
           `(swap! ~a-sym + ~x))


I’m looking for anyone up for a micro-pairing session on clojure (I can set up a Zoom session)… I have a few awesome advisers already but I might exasperate them with the basic nature of my issues. I’m still struggling with the basics of interacting with data structures (i.e. building up new ones and so on). So feel free to private message me if you’d be willing to move this slow boat along. By ‘micro-pairing’ I mean 10-15 minutes (I’m always up for longer sessions but I think this is a good strategy).

Drew Verlee01:09:40

Additionally, i would just ask the questions here in slack.


Great reminder… I will do that as well! Someone awesome hopped on with me and helped me get rolling along a lot better, and now my questions here will be a tad better.