clojure.set/rename-keys can do the direct renames, or if it's just qualifying, it could probably be done with update-keys

simplified examples:

(let [m {:id 5
         :name "Tim"}]
  [(set/rename-keys m {:id :new-id :name :new-name})
   (update-keys m #(keyword "user" (name %)))])
=> [{:new-id 5, :new-name "Tim"} #:user{:id 5, :name "Tim"}]

Nice, those are pretty close to what I want. For some keys I want to transform the data. So it sounds like I'd do a combination of: • rename all the keys to be what I expect • update-keys to do data transforms

and with the stringifying, you could thread that:

(let [m {:id 5
         :avatar-url "ava.tar"
         :url "u.rl"}]
  (-> m 
      (update :id str)
      (update-keys #(keyword "user" (name %)))))
=> #:user{:id "5", :avatar-url "ava.tar", :url "u.rl"}

you'd want update-vals to update the values in the map, but update-vals will do every val in the map, so if you want to transform values selectively, it'll need to be more specific. Unless you happen to just be calling str on things, because str on a string just returns the string, so it'd be possible to update-vals with str without changing string values

consider #CFFTD7R6Z , especially if this gets more complex

Awesome, really appreciate the tips!

Maybe check out #portal: https://github.com/djblue/portal

@U8LB00QMD That looks like a cool project. Could you perhaps point to how this could tie in to what I was describing?

It's what I most often hear folks using to augment the REPL experience

Yea I can tell from a glance at the project page and introductory video. However it seems mainly about exploring data within REPL sessions. On the other hand, I'm describing some persistence of metadata about the sort of functions (language feature) that I'm using (at the REPL) across sessions

Quite different beasts

Guess something like this doesn't exist yet, and I could perhaps take a stab at implementing some proof of concept of it sometime.

Yeah you described a rather specific set of features that's probably not been implemented yet. I just mentioned Portal because it seems like the closest thing, and might serve as inspiration

I don’t think anything like this exists, but it’s a very cool idea. If your editor supports plain clojure.main REPLs, you could build something like this by customizing the R step of the REPL. With nREPL, since you can’t swap in your own R, you’d have to write your own version of the interruptible-eval middleware and use that, but editor support for that sort of thing varies, I think.

I’ve thought about making something along these lines (maintaining a queryable REPL history). I think implementing something like that on top of a Datalog database would be cool, but I’m not sure whether any of them supports storing S-expressions as is.

There’s a little bit of prior art here, I think: https://github.com/dpsutton/grepl

clj and lein repl both track REPL input in a history file so you could probably write something that could query over that (by reading it as EDN or Clojure forms).

@U4ZDX466T Thanks for your comments! Re: "tooling integration" Unfortunately I'm not very knowledgeable on the technical side of the tooling technologies (editors, REPLs). I mainly use Cursive when programming Clojure, but I believe that's closed source? perhaps it's difficult to make an integration with it w.r.t the features I listed (auto-completing customization via white-list / black-list, have it sort things via frequency of usage, have it send notifications about suggested functions to use, etc.) I have used VSCode's (with Calva) a long time ago (3 years maybe?) and at that time it lagged behind in static analysis capabilities which was the reason I ditched it. However, it is open-source and therefore can be worked with to develop stuff like these, so I'm considering going that route. I may be required to alter something in the REPL technology too, but I don't have a clue about that yet. Also, I like the project your linked, it is very relevant and I'll consider learning from it :)

Calva uses clojure-lsp and clj-kondo these days for static analysis so it's pretty much best-in-class now.

@U04V70XH6 That's interesting to know! However I may wish to have more extensive data stored about the code sent than just the history of it being sent (Having a counter, a date, belonging to a white-list / black-list etc.)

@U04V70XH6 Re: "Calva" I believe clj-kondo existed even back when I tried it 3+ years ago, I also remember trying to use it with a VSCode extension (maybe it was Calva) but at the time that interaction didn't work even a single time. It did work through the CLI I believe but that wasn't useful enough on its own. Did these tools (clj-kondo and Calva) really improve that drastically in recent years?

Yes, night and day.

Interesting! I'll give it a try again :)

Yes, here you have the java decompilation :

public final class dev$power extends AFunction
{
    public static final Object const__2;
    public static final Var const__3;
    public static final Keyword const__7;
    public static final Var const__8;
    
    public static Object invokeStatic(Object b, Object p) {
        while (!Util.equiv(p, 0L)) {
            Object o;
            if (Util.equiv(p, 1L)) {
                o = b;
            }
            else {
                final Object invoke = ((IFn)dev$power.const__3.getRawRoot()).invoke(p);
                if (invoke != null) {
                    if (invoke != Boolean.FALSE) {
                        final Object multiply = Numbers.multiply(b, b);
                        p = Numbers.divide(p, 2L);
                        b = multiply;
                        continue;
                    }
                }
                final Keyword const__7 = dev$power.const__7;
                if (const__7 != null) {
                    if (const__7 != Boolean.FALSE) {
                        o = Numbers.multiply(b, ((IFn)dev$power.const__8.getRawRoot()).invoke(b, Numbers.dec(p)));
                        return o;
                    }
                }
                o = null;
            }
            return o;
        }
        return dev$power.const__2;
    }
    
    @Override
    public Object invoke(final Object b, final Object p2) {
        return invokeStatic(b, p2);
    }
    
    static {
        const__2 = 1L;
        const__3 = RT.var("clojure.core", "even?");
        const__7 = RT.keyword(null, "else");
        const__8 = RT.var("dev", "power");
    }
}

public static java.lang.Object invokeStatic(java.lang.Object b, java.lang.Object p);
        Flags: PUBLIC, STATIC
        Code:
                  linenumber      2
               0: aload_1         /* p */
               1: lconst_0       
                  linenumber      2
               2: invokestatic    clojure/lang/Util.equiv:(Ljava/lang/Object;J)Z
               5: ifeq            15
               8: getstatic       dev$power.const__2:Ljava/lang/Object;
              11: goto            115
              14: athrow         
                  linenumber      2
              15: aload_1         /* p */
              16: lconst_1       
                  linenumber      5
              17: invokestatic    clojure/lang/Util.equiv:(Ljava/lang/Object;J)Z
              20: ifeq            28
              23: aload_0         /* b */
              24: goto            115
              27: athrow         
                  linenumber      2
              28: getstatic       dev$power.const__3:Lclojure/lang/Var;
              31: invokevirtual   clojure/lang/Var.getRawRoot:()Ljava/lang/Object;
                  linenumber      8
              34: checkcast       Lclojure/lang/IFn;
              37: aload_1         /* p */
                  linenumber      8
              38: invokeinterface clojure/lang/IFn.invoke:(Ljava/lang/Object;)Ljava/lang/Object;
              43: dup            
              44: ifnull          73
              47: getstatic       java/lang/Boolean.FALSE:Ljava/lang/Boolean;
              50: if_acmpeq       74
              53: aload_0         /* b */
              54: aload_0         /* b */
                  linenumber      9
              55: invokestatic    clojure/lang/Numbers.multiply:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Number;
              58: aload_1         /* p */
              59: ldc2_w          2
                  linenumber      9
              62: invokestatic    clojure/lang/Numbers.divide:(Ljava/lang/Object;J)Ljava/lang/Number;
              65: astore_1        /* p */
              66: astore_0        /* b */
              67: goto            0
              70: nop            
              71: nop            
              72: athrow         
              73: pop            
                  linenumber      2
              74: getstatic       dev$power.const__7:Lclojure/lang/Keyword;
              77: dup            
              78: ifnull          113
              81: getstatic       java/lang/Boolean.FALSE:Ljava/lang/Boolean;
              84: if_acmpeq       114
              87: aload_0         /* b */
              88: getstatic       dev$power.const__8:Lclojure/lang/Var;
              91: invokevirtual   clojure/lang/Var.getRawRoot:()Ljava/lang/Object;
                  linenumber      12
              94: checkcast       Lclojure/lang/IFn;
              97: aload_0         /* b */
              98: aload_1         /* p */
                  linenumber      12
              99: invokestatic    clojure/lang/Numbers.dec:(Ljava/lang/Object;)Ljava/lang/Number;
                  linenumber      12
             102: invokeinterface clojure/lang/IFn.invoke:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
                  linenumber      12
             107: invokestatic    clojure/lang/Numbers.multiply:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Number;
             110: goto            115
             113: pop            
             114: aconst_null    
             115: areturn        

Yes, I think line 67 "goto 0" is the (recur ....) call.

To me that is really interesting, because in some other languages that I use regularly, Scala for example, either all the calls are compiled as tail calls, or none are. I might be mistaken, but that is how I understand it. In Common Lisp, of course it depends on the compiler, but SBCL for example will compile tail calls if possible and leave others as simple calls whenever the programmer requests a certain level of optimization.

I'm preparing a beginner course in clojure programming for first year engineering students. This will be an interesting discussion point when contrasting various functional languages.

This was asked quite recently. lmk if this thread helps: https://clojurians.slack.com/archives/C053AK3F9/p1694640884780829

@UE1N3HAJH thank you, that is really handy 🙂

your function will be executed before swap! it is conj that can be executed multiple times but the arguments to it will be the same.

Got it, then I have nothing to worry about that's good to know

> I hope I am not asking too many questions in this channel! Nope. Ask away :)

not with destructuring but you can always do something like this:

(defn foo [{:keys [a b]
            :or {b "b"}
            :as m}]
  (let [m (assoc m :b b)]
    m))

Yeah, that works. It would have been cool to be able to do it all at the beginning, but to be honest maybe that would have been to messy anyway. Thank you! 🙂

"Clojure way" ™ is to avoid mutations

(defn foo [{:keys [a b]
            :or {b "b"}
            :as m}]
  (let [m' (assoc m :b b)]
    m'))

if i need the :or defaults to exist in the :as map, i don't destructure the arg, i destructure in a let block below:

(defn foo [m]
  (let [m-defaults {:b "b"}
        {:keys [a b] :as m} (conj m-defaults m)]
    ...))

Nice solution, thank you!

I don't know much about stateflow, but I think you may be able to use with-redefs to wrap your function so you can test if it was called in the context of the test. Something like:

(def called-fns (atom {}))
(t/deftest fn-call-test
 (t/testing "context"
  (with-redefs 
   [my-fn (fn my-fn-wrapped [arg]
             (swap! called-fns assoc :my-fn true)
(my-fn arg)
)]
; do integration test stuff ...
(t/is (:my-fn @called-fns) "my-fn should be called")
)))

like maze exploration, it usually needs to get the index of a value in the matrix, or using index to traverse the matrix.

interop (`.indexOf`) is an option, or map-indexed or keep-indexed to search, and get and get-in are for 'traversing', depending on precisely what's desired

I'm not 100% certain about this, but I don't think it's strictly accurate to speak of a Clojure "runtime." Certainly running Clojure involves importing lots of Java classes, but those are still Java/JVM objects, not another intermediate layer on top.

There is no runtime per se, the compiled classes is the whole thing

The repl is literally the functions read eval and print running in a loop

Read is string -> Clojure data, eval evaluates Clojure data returning more data, and print is Clojure data -> strings

Eval analyzes, macroexpands, and compiles

compilation and runtime are not as distinct steps either

@U064X3EF3 then calls the compilation unit?

compilation unit is a term of art in compiles to describe the frontier of what source is visible to the compiler

so a "compilation unit" isn't actually a thing to call, in some programming languages it might refer to a source file

in clojure the compilation unit is a single top level expression

ok, I thought it was the compiler output, not the input

https://en.wikipedia.org/wiki/Translation_unit_%28programming%29

in clojure, generally you feed eval a single expression, that expression is compiled to jvm byte code, then that jvm byte code is executed

What I meant to ask above of @U064X3EF3 was to clarify that the eval step also executed what it compiled.

for larger projects you can do things like "aot compilation" which looks similar to file at a time compilation, but is not that

aot compilation saves the generated byte code, so you can run your clojure program from the bytecode instead of from the clojure source

the clojure runtime isn't so much running, it is inert code unless called

like part of the clojure runtime is the static method seq on the class clojure.lang.RT

and some clojure code, when compiled to bytecode and then executed, will reference and possibly call that method

@U064X3EF3 Why do the docs mention both on-the-fly compilation and Ahead-of-time compilation? Is it a choice for the developer which compilation type is used, or do the tools determine which one is best automatically?

which docs?

https://clojure.org/reference/compilation

yeah, "on-the-fly" only exists when comparing against aot compilation

there is the normal clojure modal of compiling to byte code and executing that bytecode as needed

and there is the aot model, which looks deceptively like batch compilation which you might be familiar with from java, c, haskell, rust, etc

from the aot model perspective how clojure works looks like "on-the-fly" but outside of looking at it from that perspective it is just how clojure works, no one really refers to it as anything

the way aot works is basically exactly the same as the normal way clojure works, it just writes the generated bytecode to disk too

but it is still interleaving compilation and code execution like a repl session would, so you have (launch-the-missiles) as a top level form, when aot compiling you will launch the missiles

so from a users perspective there is one execution model

I think that docs page could use a fair bit of improving, it is a mix of the general clojure compilation model and aot related stuff, and if you just want to understand the clojure compilation model, the aot stuff will just confuse you

@U0NCTKEV8 It confuses me that's for sure

the model is you type in something, clojure reads that in as data structure, then evals the data structure, producing another data structure, then prints the data structure out

where eval is implemented by compiling the data structure to jvm byte code and executing it

there are other things clojure.core/compile, load-file, require, ways to launch programs without using a repl, etc but those are all extensions of that model that try to add useful additional things (line numbers in errors, etc) without removing anything from the basic model

that all being said, a clojure program is not just the repl

there is no single "repl" for a clojure program

and there is no single clojure repl, there are two in widespread use the repl built into clojure and nrepl

if you try to look at the compiler code eval is tricky, because for each ast node there is an eval path and a emit path, which is confusing. I would put it more like, for each form you type at the repl : • the LispReader.java will read the string into a nested structure of lists, vectors, maps, symbols etc, called forms • then the parse path in Compiler.java will walk those forms and make Expressions, like (if ...) will make a IfExpr node, etc (this makes the ast) • then the emit path of Compiler will call emit on each of those, creating in memory classes and their bytecode, which don't get written to disk unless you specify it • then those classes are loaded and executed, by calling invoke etc which will generate an output object • then that object is printed to the console

Does every Clojure program have a REPL running?

no

so it's possible to just have a Clojure code get compiled to a program without a REPL yes?

but every Clojure program can call the reader, and eval with the resulting form, so the compiler is always available

Is the REPL used during compilation?

no, the compilation system doesn't require any repl

compilation is used during eval, eval is the E in repl

user=> (defn f [x] x)
#'user/f
user=> (type f)
user$f
user=> f
#object[user$f 0x3eb631b8 "user$f@3eb631b8"]
user=>

at the repl I typed in (defn f [x] x) which created a function and bound it to f

a repl is just a function you run if you want to interactively call the read->eval(compile,execute)->print

the function is clojure code compiled to bytecode, and a jvm class

the type, the name, of that jvm class is given by the type function there

user$f

So can we say that by running CLJ code with the clj tool, we have this tool *R*ead all namespaces in the files, *E*valuate them (compile them to bytecode), *P*rint nothing and (not) *L*oop? So basically just a single cycle of interaction yes?

no

and this is a key point, it reads form by form in the namespaces and evaluates each as it goes

which is why it behaves the same way as if you were typing the code into a repl

oh

and it doesn't "compile" them, it evalutes them, which is compile+execute

so the differences are that compiling the initial code (from a file) is using Ahead-of-time compilation, and saves the bytecode to disk, whereas interacting with the REPL is using on-the-fly compilation, and saves the bytecode only to memory?

No, reading from a file is the same form-by-form process as the REPL.

and reading the file can happen with or without aot compilation

aot compilation is just like a switch that you turn on and off in the compiler to write the code it generates to disk

everything else is the same

AOT is a "mode" for the "compile" portion of the "eval... yeah, what he said ☝️:skin-tone-2:

(where everything is subject to some provisos about gen-class, etc)

OK so the underlying mechanism (specific compilation) is just a detail in the evaluation stage?

I could choose to save the compiled bytecode to disk or to memory? whichever I prefer?

And both options are on-the-fly compilation?

Typically, folks only use AOT compilation mode as the "last step" when building a deployable artifact. And the sole purpose of it is to make it faster for the application to start up.

Well it makes sense... how would I compile "ahead of time" if I keep interacting with the REPL and sending arbitrary code to it

generally the interface to aot compilation that clojure provides is a pain, so it isn't used directly, but via tooling

You can build an uberjar -- a deployable artifact -- that is all source code and run it via clojure.main:

java -cp path/to/the.jar clojure.main -m my.entry.point

java -jar path/to/the.jar 

Some people use AOT compilation to help speed up starting their app via the REPL. See https://clojure.org/guides/dev_startup_time

(I did it for a while and then decided it really wasn't worth it -- even with as large a codebase as we have at work)

All of this tooling technicalities are confusing for me. I may have learned a thing or two from this discussion. I'll save the rest for another day to hopefully unravel the remaining mysteries eventually.

For the vast majority of Clojure-related work, the subtleties of this stuff just don't matter. Many Clojure devs live productive working lives without caring about these details 🙂

@U04V70XH6 Yea, but when I'm interested in developing some dev tool, understanding behind-the-scene workings is sometimes necessary (if my dev tool has to do with the REPL)

referencing that idea I had raised earlier (keeping track of function usage)

https://clojurians.slack.com/archives/C053AK3F9/p1695272317695309

Sure, but that would operate at the source level -- and for it to be widely useful, I suspect it would need to be nREPL middleware (as well as perhaps something for the "regular" built-in REPL).

Thanks @U04V70XH6 for the thoughtful replies. Very much appreicate it 🙂 have a great day

I don't see the class in the jar file

Hmm -- okay, maybe they have a bug -- looks like they're actively hacking!

in the build file it looks like JsonType is in a separate build declaration (or whatever they're called in that build system) - it's not immediately apparent to me how that relates to what goes into the output jar file

Understood. Thank you @U013JFLRFS8!

From a look it doesn't seem to be included in the main build declaration: https://github.com/google/cel-java/blob/1520cfe1370ea4e390c87633204a7f6b72a152a0/bundle/src/main/java/dev/cel/bundle/BUILD.bazel#L31

Is it something like this?

(transduce xf save! init-data)

My confusion comes from that there isn't really any result of the transduction, it's all side-effects, so unsure if this is possible.

you can use transduce and just ignore the accumulation value.

another option is: (run! save! (eduction xf1 xf2 xf3 init-data))

Thanks!

The run! option seems clearer to me. Using transduce for side effects seems like it would be a surprise later on. You'd also need to make save! a reducing function, I think, whereas with run! it's just a command to execute.

@U7RJTCH6J quick follow up question: (eduction xf1 xf2 xf3 init-data) - this is returning a type of Eduction. When/How is this evaluated by run!?

Oh right the implementation uses reduce

Right, the eduction won't start applying xforms. each time the eduction is passed to a function that expects a seqable or reducible, then it will process the xforms.

and as you said, run! uses reduce under the hood.

I found that during my reduction, I was wanting to refer to previous values or do small little side-effect tasks like logging and the continue with the reduction. I wrote this function to be able to do it

(defn passthrough [f]
    (fn [xf]
        (fn
            ([] (xf))
            ([result] (xf result))
            ([result input]
             (f input)
             (xf result input)))))

It would be interesting to hear what others think, but seems totally reasonable to me 👍

I think separating who, what, when, why, and how is a good idea, but I tend to put transducers in the when/how category so adding more when/how steps is reasonable.

There's a lot of similarities between transducers, middleware (eg. ring), and interceptors (eg. pedestal) which also are about managing when/how (eg. side effects).

I haven't heard of interceptors. When you say middleware are you referring to the web definition of it or a broader definition of it?

I guess middleware is an overloaded term. I was more specifically referring to https://github.com/ring-clojure/ring, but there are other libraries that have a similar implementation like https://nrepl.org/nrepl/design/middleware.html.

What are your thoughts on this part of what I said earlier? > I was wanting to refer to previous values Hmm now that I think of it that makes way less sense because the shape of the data is arbitrarily changing

Oh and specifically I mean - is there a way to refer to values produced by previous steps of a reduction while you're in the reduction? I assume no

I believe referencing previous values is already available via https://github.com/cgrand/xforms window, window-by-time, and reductions.

Will look at that! Thanks again, this was extremely helpful. This is the first time I was really writing transduction code but I've been reading/watching as many resources as I can find

it's not optimal but you could use jsoup to parse and reformat the generated html

(println (.toString (org.jsoup.Jsoup/parse "... html ...")))

thank you @U02F0C62TC1, I'll have a look into it.

Did you try clojure.pprint/pprint

@U051GFP2V Yes, I have tried:

(-> [:div
       [:h1 "Hello"]
       [:p "World"]]
      (hc/html)
      (str)
      (clojure.pprint/pprint))
"<div><h1>Hello</h1><p>World</p></div>"
=> nil

Oh, right. I thought you were asking to pretty print the hiccup data. e.g.

(require 'clojure.pprint)
(-> [:div [:h1 "Hello"] [:p "World"]
     [:ul (for [n (range 5)]
            [:li [:ol (for [m (range (rand-int 10))]
                        [:p n ">" m])]])]]
      clojure.pprint/pprint)

;; [:div
;;  [:h1 "Hello"]
;;  [:p "World"]
;;  [:ul
;;   ([:li [:ol ([:p 0 ">" 0] [:p 0 ">" 1] [:p 0 ">" 2])]]
;;    [:li [:ol ([:p 1 ">" 0] [:p 1 ">" 1])]]
;;    [:li [:ol ([:p 2 ">" 0] [:p 2 ">" 1] [:p 2 ">" 2] [:p 2 ">" 3])]]
;;    [:li [:ol ([:p 3 ">" 0])]]
;;    [:li
;;     [:ol
;;      ([:p 4 ">" 0]
;;       [:p 4 ">" 1]
;;       [:p 4 ">" 2]
;;       [:p 4 ">" 3]
;;       [:p 4 ">" 4])]])]]