I do something similar to implement custom nested transformations in what ultimately becomes a json map. I use qualified keywords to signify my customizations, since json lacks qualified keys, a vector starting with one must be "mine". This might not be applicable for you though. You could also process them with a multimethod with a :default implemention that just passes through the value unchanged unless you have a custom method for it

Why not use a protocol and extend via metadata?

@UK0810AQ2 If I understand correctly (“extend via metadata”) because some operations drop metadata…

What about something like this?

(deftype EntityReference [id-attr id-value]
  clojure.lang.IPersistentVector
  
  (seq [this] (seq (vector (.id-attr this) (.id-value this))))
  )

(partial implementation, but as a solution direction)

Why do you need to mark the references?

I heard a desire but not a convincing rationale

All the discussion so far was downstream of that decision

Partially the rationale is me wanting to learn about deftype/defrecord etc. But also I’m thinking if I have a big nested datastructure with these things, I could easily automatically expand them using a tree walker and defmulti polymorphism for example.

So then the tree walker would need a way to distinguish between these special vectors that need extra processing, and normal vectors.

another way a tree walker could operate is by attribute context:

{...
  :patient [:patient/id 123] ;; tree walker could dispatch on :patient
 ...}

I would caution against custom types, they will make data more opaque unless you implement a couple dozen methods

Yeah I can see that risk, thanks for your advice!

you're manually calling load-file from the repl?

I'm using the chlorine shortcut on atom

Ah, I see. Not entirely sure why that'd be failing if it is indeed on your classpath, which basically only requires that your chlorine instance is using the correct build tool to determine what your dependencies are.

I will point out that atom is basically deprecated at this point though. If you're comfortable with it, that's ok, but as long as you're aware that it's not really going to be maintained in any official capacity. Though I suspect you are since you're using it.

Yeah, the odd thing I can't seem to find any documentation on is that it loads the dependency just fine, but cannot resolve the dependency's internal require statement. I'm using a community maintained build of atom pending graduation to Emacs since it's good enough:tm:. There doesn't seem to be a similar problem with any of my other projects, and this is my first attempt at using my own library, so I figured it was how I deployed it/set up the project.clj.

Ah, if it's your own library and it's the first time you've built one, there's a possibility that you have the paths inside the jar not matching the namespace names. What's the actual structure inside the jar look like?

As it should be, pic related. The odd thing is that I get no complaints when I try to require the namespace inside the interndir directly Would the fact it's a :refer :all be relevant?

and the namespace that you're requiring is krak.intern.client ?

or are you trying to like require krak.intern :refer :all

I'm requiring krak.client in my project, the main ns of the lib, which requires krak.intern.client via a :refer :all

Ok, so requiring krak.intern.client as long as your krak jar is on the classpath should work fine. If it's not then I question if the jar is actually on your classpath.

however, if I do this in the project under development, no problems at all, and it seemingly can find the ns

so are you actually getting errors when you load the ns, or are you just getting underlines with warnings?

so krak.client is found, krak.intern.client can be found, but not from krak.client

I get errors, both using the cholrine repl commands and by trying to load it in the terminal running the project via lein

and you have the dependency declared in leiningen?

in your project.clj

Sorry, just trying to run through everything to make sure

yep, initially it was with the com.github.user/ prefix to resolve it via jitpack, but same issue whether I try to call it in remotely or have it installed into my local repo

Yeah this is confusing then. The last major thought I have on this if lein can't work with it is that your client namespace has a compiler error, which sometimes results in this "cannot require" exception.

hmm. That's unlikely because I was doing a bunch of prototyping for the project inside the library project, but I'll give that a look

works fine, even when loading the client ns from another ns

Hmm, then I'm about out of ideas, I'm sorry.

np, thanks for the assist. Hopefully someone else might have run into this before.

On the off chance someone else has a stab at pondering what might be going wrong, moving krak.intern.client out into the top level src dir doesn't seem to help.

couldn't you just call zip/node and then build a new zipper from that?

You could even make it so that it builds the same type of zipper by fetching the metadata off the one you get passed and using that to construct the zipper on the produced node.

That would make it generic.

(require '[clojure.zip :as zip])

(defn sub-zipper [loc]
  (let [{::zip/keys [branch? children make-node]} (meta loc)]
    (->> loc
        zip/node
        (zip/zipper branch? children make-node))))

> couldn't you just call zip/node and then build a new zipper from that? I'm a total noob with zippers. I think this is exactly what I want to do!

awesome, glad I could help!

And a beautiful little sub-zipper to boot. I like how it is general.

thanks!

In Calva I have something quite zipper-like called a Token Cursor (that runs over the Clojure code in the editor). I think cloning out what would be equivalent to a sub-zipper is the most common thing happening in Calva. 😃

Yeah, it's really nice to have the general library for working with zippers. I've never actually had a usecase for them myself but I really like them conceptually.

It definitely feels like rewrite-clj and similar should be able to provide a zipper interface

Oh, ofc they have

it's too bad that it's a customized version. Oh well.

rewrite-clj is wonderful. I'm mostly using it by proxy of other tools, like clj-kondo, cljfmt, zprint, etcetera, though. Since Calva is mostly TypeScript and has its own cursor thing.

I wish cljfmt wasn't quite so opinionated about configuration

It's really frustrating as someone who writes a lot of macros that I can't distribute any kind of formatting information to anyone but cider users directly as a part of the dependency.

I've wanted to work on a cider-compliant cli formatter using the clj-kondo analysis to grab var meta, but I've been too busy with other projects.

plus cljfmt having the really bad semantics around formatting multiple things with the same symbol name.

Calva could pick up cljfmt config from the dependency, if that's what you mean. Right, @U02EMBDU2JU? You can file an issue on Calva describing it a bit, if you like. (If this is what you meant, haha.)

If that's something that calva can support that'd be helpful and I'd start trying to distribute that. The problem is that cljfmt on its own has no way of "combining configs" as far as I can tell a-la clj-kondo, and ofc it also has no facility for importing configs via maven or git dependencies.

Plus the problem of "this symbol has this formatting in this ns, regardless of where the symbol came from" is also an issue for this sort of thing.

That'd be hard. The JVM reads the entire environment when it starts to populate the internal map, and tools like environ in clj read the entire jvm system environment to turn it into a clojure map.

So you'd just get tools saying "everything is used" if you weren't doing really smart static analysis.

and I'm not aware of any tools like that for clojure at this time.

still most things inside app if they are trying to read particular env var would go to System/getenv ? so if I monkey patch it, I might be able to see right?

Just in case - if it's just for some debugging, a proper debugger would be able to put a breakpoint inside System.getenv(String). But note that there's also System.getenv().

I already spotted, System/getenv is not a regular fn... trying to go monkeypatch route, funnily cannot eval symbol of System/getenv either way, doesnt behave like normal namespace

The problem is I am trying to go the other way, some env variable is missing and I am getting bizzare outputs, so trying to find exactly which env is causing that

Yes, System/getenv is in fact a static method on the System class inside the JVM, so it can't be monkeypatched with Clojure.

is there a way I can shadow System/getenv with my function? I dont even need it to work, other than print me args passed

Here's an involved hack to hack System/getenv and setenv: https://github.com/lambdaisland/launchpad/pull/3/files

probably grepping for System/getenv is easier to find all the spots where your program accesses the environment?

not really, its probably some lib in dependency... anyways thanks a lot, reading the PR

well, you can grep dependencies too

https://github.com/borkdude/grasp

specifically this: https://github.com/borkdude/grasp#grasp-a-classpath

assuming the source code is written in clojure

I would say to use bpftrace to intercept syscalls, but getenv doesn't issue a syscall

It is possible to do this using a Java agent, and something like bytebuddy, but it's far from trivial.

https://tersesystems.com/blog/2016/01/19/redefining-java-dot-lang-dot-system/

I was able to hook into java.lang.System/getenv using java agent + javassist: https://github.com/ivarref/hookd/blob/main/agentuser/test/agentuser/core_test.clj#L8 The code is not pretty (reflection + getting classloader from thread), but it works.

very interesting :)

What does expanding a loop mean?

is “unrolling” a better word?

inlining the loop so that the let-block has a bunch of swap statements instead of a loop around one of them

sure, unrolling

not possible in the general case, to do that you would need to know the value of evaluating whatever xs is bound to, and of course at macro expansion time it is not evaluated

for example in (let [x [1 2 3]] (foo x)) when foo is expanded, xs is bound to the symbol x

and there is no way to know in foo what value x will be bound to at runtime

similar (fn [x] (foo x))

is it possible if we assume xs is always a vector of integers

unevaluated I mean

sure

there are two cases where you can kind of getaway with this, passing a literal is the easy one

so like, if you had a vector x and wanted to turn it into a seq of lists containing a single element, the number one, how would you do it?

(lists of just the number 1 for each element in the vector)

~@(for [x xs] (list 1))

not in a macro, just in general, how would you write code that did that

(for [x xs] '(1))

so what if instead of replacing with a constant list like that, you wanted a list where the first element was the symbol swap!, the second was a#, the third was +, and the fourth was the element in the vector?

like, syntax quote, and the unquoting bits are very useful, but not required for writing a macro, and can make things less clear, specifically when you are doing acrobatics quoting and unquoting at different levels

so it might be clearer to write a nice clojure function that can take some input and produces a data structure as an output, using all the functions you already use in clojure to construct the output data structure, and then whoops, the output datastructure happens to be valid clojure code, so you just swap the defn for defmacro and you are good

I guess the hard part is that a# is not accessible from inside the loop function

but I understand what you’re saying that it’s easier to have a function produce the swap statement list

if you take the whole macro and write it without using syntax quote(and the features it provides: unquoting, splicing, auto gensyms) then you'll have a better feel for using syntax quote

syntax quote is a helpful tool for writing macros, but it has limitations, and if you are not familiar with writing macros without using syntax quote, then the limitations of syntax quote are your limitations. it seems like maybe in this case your stumbling block is auto gensyms, if you just didn't use them, then you would be fine

this is good advice, I’m going to play with that, thanks

(defmacro foo-noloop [xs]
    (let [a-sym (gensym "a")]
      (concat
        (list 'clojure.core/let [a-sym (list 'clojure.core/atom 0)])
        (for [x xs] (list 'clojure.core/swap! a-sym 'clojure.core/+ x))
        [(list 'clojure.core/deref a-sym)])))

this worked, I guess it’s just hard to follow but easy to write

(defmacro foo-noloop [xs]
    (let [a-sym (gensym "a")]
      (concat
        (list `let [a-sym (list `atom 0)])
        (for [x xs] (list `swap! a-sym `+ x))
        [(list `deref a-sym)])))

I guess I can use syntax-quote to qualify the clojure.core symbols

(defmacro foo [xs]
  `(-> 0
     ~@(for [x xs]
         `(+ ~x))))

(macroexpand-1 '(foo [1 2 3]))
;; yields
(clojure.core/-> 0
  (clojure.core/+ 1)
  (clojure.core/+ 2)
  (clojure.core/+ 3))

I guess I'm not sure what you are actually after, so deleted my previous comments

but I guess the thing you were missing was the unquote splicing ~@? https://clojuredocs.org/clojure.core/unquote-splicing

no need for concat/list at all

@U05224H0W thanks! the problem is that I can’t use a gensym in a nested syntax quote

oh I see what you did, by removing the need for a named accumulator

the example is contrived but it was all I was given

Oh, this works just fine too:

(defmacro foo-noloop [xs]
  (let [a-sym (gensym "a")]
    `(let [~a-sym (atom 0)]
       ~@(for [x xs]
           `(swap! ~a-sym + ~x))
       @~a-sym)))

Additionally, i would just ask the questions here in slack.

Great reminder… I will do that as well! Someone awesome hopped on with me and helped me get rolling along a lot better, and now my questions here will be a tad better.