They way you're describing the spec it looks like you want data that looks like this:

{:id 1 :source "email" :text {:text "abc" :message/source "email"}}

(defmulti message-post :source)  ;; if you're using non namespaced keywords for the dispatch function
(defmethod message-post "email" [_] (s/keys :req-un [:customer/id :message/source :email/text] :opt-un ...)
(defmethod message-post "sms" [_] (s/keys :req-un [:customer/id :message/source :sms/text] :opt-un ...)

(s/def ::messages-post (s/multi-spec message-post :source)

{id 1 :source "email" :text "abc"}

namespaces are not clojure maps

I'm not sure if I was right about the eldoc. I try again

namespaces hold a number of different mappings, are are ConcurrentHashMaps under the hood

the main mapping is from symbols to vars

when clojure code is compiled it looks up the var, and that looked up var is what is used in the compiled code

at runtime the code doesn't look things up in the map

imho trying to explain a repl confuses people because they tend to compare it with other languages shells and consoles. I think the important thing to explain is a language and a system designed to build a program interactively while you run it by adding and removing from it without the need to restart it

for

(defn f [x] (+ x x))

(def f (let [g (resolve '+)] (fn [x] ((deref g) x x))))

the var resolution happens once, and every time f is invoked it just rederefs the known var

the performance of looking things up in namespaces then only matters when compiling and when loading code, but not when running code

🤯

what about all the code that is loaded in a big project? Will it not bloat the repl?

functions are compiled to classes, and is just the JVM loading classes

that question is not really answerable

code takes space, all code does, for any language or runtime

so loading more code takes more space

the repl is, in one sense, the ability to interact with your running program. if you don't worry about loading the project itself, why would it be a problem "in the repl" (which is the same program but you interact with it directly)

loading less code takes less space

the repl has nothing to do with code taking space

yea the program is loading code. The repl doesn't have anything to do with that. It is just a interaction layer on top of the program.

https://jackrusher.com/strange-loop-2022/ especially at the various points when he mentions "interactive" programming or runtime. That might be a more helpful style of answer to "what is a REPL?".

what would you say when the one says: "This dotnet project takes minutes to compile, it must be so much code!" "Doesn't Clojure break when it is holding all that code in memory?"

why would it?

because it is so much! I cannot imagine it working

when machine code runs, it is loaded in memory, do you ever ask yourself if it is too much?

the clojure runtime doesn't actually keep a reference to the clojure source

it is the "same" amount of byte code, doesn't matter if you compiled java or clojure

it compiles it to jvm bytecode loads it, and throws the source away

you can aot compile clojure, which does the same process but saves the bytecode to disk, and then you can just load the bytecode and not use the source at all

even if you were to maintain a reference to the source code strings loaded in memory, it should be cheap anyway, source code doesn't take a lot of memory

the primary thing that impacts is start up speed, either way it is being compiled to bytecode and the jvm is executing the bytecode

so we start the jvm, we load clojure core and then we load our clojure sources as needed. And they byte comple when we load, same as jvm or dotnet.. And then we have the byte compiled classes in a lookup?

user=> (defn f [x] x)
#'user/f
user=> (println (nope/disassemble (fn [] (f 10))))
// Compiled from NO_SOURCE_FILE (version unknown : 52.0, super bit)
public final class user$eval366$fn__367 extends clojure.lang.AFunction {
  
  // Field descriptor #13 Lclojure/lang/Var;
  public static final clojure.lang.Var const__0;
  
  // Field descriptor #24 Ljava/lang/Object;
  public static final java.lang.Object const__1;
  
  // Method descriptor #7 ()V
  // Stack: 1, Locals: 1
  public user$eval366$fn__367();
    0  aload_0 [this]
    1  invokespecial clojure.lang.AFunction() [9]
    4  return
      Line numbers:
        [pc: 0, line: 1]
  
  // Method descriptor #11 ()Ljava/lang/Object;
  // Stack: 3, Locals: 1
  public java.lang.Object invoke();
     0  getstatic user$eval366$fn__367.const__0 : clojure.lang.Var [15]
     3  invokevirtual clojure.lang.Var.getRawRoot() : java.lang.Object [20]
     6  checkcast clojure.lang.IFn [22]
     9  getstatic user$eval366$fn__367.const__1 : java.lang.Object [26]
    12  aconst_null
    13  astore_0 [this]
    14  invokeinterface clojure.lang.IFn.invoke(java.lang.Object) : java.lang.Object [29] [nargs: 2]
    19  areturn
      Line numbers:
        [pc: 0, line: 1]
        [pc: 6, line: 1]
        [pc: 12, line: 1]
      Local variable table:
        [pc: 0, pc: 19] local: this index: 0 type: java.lang.Object
  
  // Method descriptor #7 ()V
  // Stack: 2, Locals: 0
  public static {};
     0  ldc <String "user"> [33]
     2  ldc <String "f"> [35]
     4  invokestatic clojure.lang.RT.var(java.lang.String, java.lang.String) : clojure.lang.Var [41]
     7  checkcast clojure.lang.Var [17]
    10  putstatic user$eval366$fn__367.const__0 : clojure.lang.Var [15]
    13  ldc2_w <Long 10> [42]
    16  invokestatic java.lang.Long.valueOf(long) : java.lang.Long [49]
    19  putstatic user$eval366$fn__367.const__1 : java.lang.Object [26]
    22  return
      Line numbers:
        [pc: 0, line: 1]

}
nil
user=> 

I think this could be also a little easier to read :

(defn foo [a b]
  (+ a b))

(defn bar []
  (foo 4 5))

// Decompiling class: dev$foo

public final class dev$foo extends AFunction
{
    public static Object invokeStatic(final Object a, final Object b) {
        final Number add = Numbers.add(a, b);
        return add;
    }
    
    @Override
    public Object invoke(final Object a, final Object b) {
        return invokeStatic(a, b);
    }
}


// Decompiling class: dev$bar

public final class dev$bar extends AFunction
{
    public static final Var const__0;
    public static final Object const__1;
    public static final Object const__2;
    
    public static Object invokeStatic() {
        final Object invoke = ((IFn)dev$bar.const__0.getRawRoot()).invoke(dev$bar.const__1, dev$bar.const__2);
        return invoke;
    }
    
    @Override
    public Object invoke() {
        return invokeStatic();
    }
    
    static {
        const__0 = RT.var("dev", "foo");
        const__1 = 4L;
        const__2 = 5L;
    }
}

he says the NASA project is not big enough. Only 13600 lines clj files

the important parts related to vars are : • when the class gets initialized (when the dev/bar fn instance is created it will load a reference to the dev/foo var

const__0 = RT.var("dev", "foo");

final Object invoke = ((IFn)dev$bar.const__0.getRawRoot()).invoke(dev$bar.const__1, dev$bar.const__2);

thanks, will check. Still hard for me to convincingly make my friend relaxed. I think he just fundamentally assumes that big projects bloat .... something. Maybe because of the suffering he had from csharp tooling and a big project... I remember rider trying to analyze code 😅

sounds like the central concept has not been communicated

everything has already been compiled because the repl is the loaded, running program

Ah do we compile clojure sources on the classpath? Say I have src/foo.clj I thought we only load - and thereby compile - when I require or load ? Unless I am requiring foo of course.

another thing to keep in mind is that projects typically don't have to get as big (in line count) in clojure as in e.g. C# to accomplish a similar thing it's also easy to just decide "this can't possibly work" and then dismiss every example of it working as "eventually it won't work", and sure, eventually you'll run out of memory to hold a running system; C# and Java can also throw OOMs

It's a funny question your friend asked you. But that's basically why we have a lot of RAM in our computers nowadays.

When you start the REPL, Clojure will compile the code, load it, and initialize it, this is why REPL start takes a while. AOT compile can be used to pre-compile it. A Java app work the same way (but is always AOT). But when you start a Java app, it similarly loads the code and initializes it. That code will take up some space in memory. It's not that much though, even for very large code base.

The REPL doesn't really change anything. It just allows you to load additional code into the app at runtime.

> "This dotnet project takes minutes to compile, it must be so much code!" "Doesn't Clojure break when it is holding all that code in memory?" Clojure will compile code in a streaming fashion, as needed. Though it can also do it all ahead of time using AOT. It doesn't hold the source code in memory. So when a ns is loaded, the file is taken from the JAR, and each form one by one are compiled and loaded. The source file is not maintained in-memory. The difference is between AOT compilation, and streaming. AOT means, the compiler loops over all files, and one by one compiles them, and saves the compiled output back into files. Then when you start the app, those compiled files are loaded into memory by JVM. Clojure, when not AOT, will do this compile -> load as needed, in a streaming fashion.

This also allows the REPL, because you can stream source code to the REPL, and it will compile and load it on-demand.

It gets a bit more complicated underneath. Because here it compiles to bytecode, which is an IR (intermediate representation). The JVM will then interpret this bytecode to execute it, using an interpreter. And it will also start to profile it as it gets executed. As it gathers heuristics on it, it will take the bytecode and JIT compile it into optimized machine code (Just in time compilation). But it will not forget the bytecode once it does that, because if the code paths change dramatically, it will deoptimize, throw away the machine code and go back to interpreting, or recompile it into a different machine code.

This is kind of why in general, JIT runtimes like the JVM eat up more memory. Not only do you maintain bytecode + machine code in memory, but also a bunch of profiling information, and all of the code needed for the JIT and GC themselves.

And all that is why you'll struggle getting an app in Java/Clojure that can run with less than 100MB, because all this stuff takes up a minimum amount of memory on its own. But if you use Graal to do a native image, that stuff won't be there, it's all pre-compiled directly into static machine code, and that's all that gets loaded. Now, there's still going to be more than a small C program, because it will bring about a whole GC with it, but it will have no uneeded code, because it's all tree shaken, statically, and pre-compiled to machine code. So it takes considerably less memory.

those two maps share one sponsorid, so I think the count of distinct sponsorid is maybe a bit misleading - it seems like you want any groups by sponsorid that have more than one unique value of concatting a and b (and I assume the desire is to get all the members of that group) so, I think you'd have to first group by sponsorid, and then filter each group

that worked, thanks @U013JFLRFS8