Fork me on GitHub
#clojure
<
2021-07-20
>
Jim Newton07:07:54

I have defined a macro and used often in my program, called defn-memoized. The macro essentially takes three things, a public-name, internal-name, and function body which includes the arglist. The macro, defn-memoized defines a function named by the internal-name using the given function body, and defines a global dynamic variable using the public name. The value of the dynamic variable is a memoized version of the internal function. The advantage is that within a dynamic scope, using binding, I can rebind the memoized function, and after the binding form finishes, clojure releases the cache for the function. I heavily use this in test suites which run 1000s of randomly generated tests-- mitigating the risk of filling up the memoization cache. That works well. What does not work is that I have no way of doing this for function which are implemented by defmulti/`defmethod` .

(defmacro defn-memoized
  [[public-name internal-name] docstring & body]
  (assert (string? docstring))
  `(let []
     (declare ~public-name) ;; so that the internal function can call the public function if necessary
     (defn ~internal-name ~@body)
     (def ~(with-meta public-name {:dynamic true}) ~docstring (gc-friendly-memoize ~internal-name))
     ))

(defn call-with-compile-env [thunk]
  (binding [rte/compile (gc-friendly-memoize rte-to-dfa)
            canonicalize-pattern-once (gc-friendly-memoize -canonicalize-pattern-once)
            gns/check-disjoint (gc-friendly-memoize gns/-check-disjoint)
            ]
    (thunk)))

(defmacro with-compile-env [[] & body]
  `(call-with-compile-env (fn [] ~@body)))

Jim Newton07:07:25

Why does this work for normal functions, but not for multi methods? well I can easily change a defn to a defn-memoize without touching any call-site, and without touching any ns definition require. I don't see how to do this for methods. because the defmulti, defmethod, and call-sites all need to match.

Jim Newton08:07:23

TLDR; how can I memoize a function defined by defmulti in a way which is invisible to the call-site?

Ed08:07:48

For testing purposes, I think with-redefs is more useful than binding. That will bind in more than the current thread, and work for all vars not just dynamic ones.

Ed08:07:07

(defmulti foo :tag)
  (defmethod foo :foo [_] :foo)
  [(foo {:tag :foo})
   (with-redefs [foo (fn [_] :bar)]
     (foo {:tag :foo}))] ;; => [:foo :bar]

Ed08:07:55

dynamic binding has a performance cost, so (around 1.2, I think) clojure started enforcing that you marked vars that needed to by bound dynamically with some metadata (`^:dynamic`) to hint to the compiler to not inline that lookup (for example)

Ed08:07:41

but I think in general people often put a defmulti behind a defn anyway in the same way people use protocols.

Jim Newton08:07:42

that means you can't make it invisible at the call site because the name at the call site necessarily differs from the name given at defmulti and defmethod ? right?

Jim Newton08:07:53

wrt with-redefs vs binding for testing purposes, don't I want rebinding ONLY in the current thread and threads the current thread starts, which is what binding promises to do if I'm not mistaken. I don't want two different tests running in two different threads to see the same rebinding.

Ed09:07:22

I guess that depends on what else is going on while you're running your tests. Sure, if you're trying to run multiple things concurrently, and they never need to coordinate this across threads, then maybe binding fits. But it's a test suite, so I think it's more usual to coordinate the tests so that they don't run concurrently with other processes that would be interfered with. The call site is dereferencing a var and running whatever it finds there, right? I'm not sure I understand the problem.

Jim Newton09:07:32

the problem: if you put a defmulti behind a defn then the call-site must correspond to the defn not to the defmethod ... thus you can't take an existing program which has many calls to functions defined by defmulti and refactor it so that the function is memoized ... i.e. you'd be forced to invent a new name for the defn and change all the call sites to use that new name

Ed09:07:35

I think I often consider the dispatching facilities like defmulti and defprotocol to be internal things that are only exposed as extension points for "advanced" uses of a lib. So for uses of the lib, it should only have call sites into the public function and if some additional functionality needs to be added to handle a new case, that's an extension point

Ed09:07:32

yes ... so I think often libs are designed like that

Jim Newton09:07:51

this is very different from the use case of my defn-memoized macro. I can take any function defined by defn and change defn to def-memoized without touching any call-site

Ed09:07:50

can't you just extend your macro to put either a defn or a defmulti in the private one? You've still created a public fn which is the the main thing that's getting called, right?

Ed09:07:50

something like

(defmacro defmulti-memoized
    [[public-name internal-name] dispatch-fn docstring]
    (assert (string? docstring))
    `(let []
       (declare ~public-name) ;; so that the internal function can call the public function if necessary
       (defmulti ~internal-name ~dispatch-fn)
       (def ~(with-meta public-name {:dynamic true}) ~docstring (gc-friendly-memoize ~internal-name))))
???

Ed09:07:40

you'd still need to add the methods after that, but that's a multi-method 😉

Jim Newton09:07:49

yes, and all the methods area already named with the public name.

Jim Newton09:07:21

perhaps I need to also define defmethod-memoized ?

Jim Newton09:07:24

Or I suppose refactor all the defmethod calls to use the internal name.

Jim Newton09:07:45

BTW the terminology seems bizarre to me. according to the documention defmulti defines a multimethod, and defmethod installs a method onto a multimethod.

Ed09:07:05

so you're trying to memoise just one method on a mulitmethod?

Jim Newton09:07:29

in CL defgeneric defines a generic function and defmethod installs a method onto the generic function.

Jim Newton09:07:28

no I want the multimethod to be memoized.

Ed09:07:42

you can't call a method directly without going through the dispatch function ... the multi in defmulti is just short for multimethod ... right?

Jim Newton09:07:25

a multimethod is not a special kind of method as the English language prefix implies.

Jim Newton09:07:40

just terminology, and naming is the hardest problem in computer science. I realize that

Ed09:07:29

> no I want the multimethod to be memoized. then I think that the bit of code I posted above will probably do it.... right?

Ed09:07:48

> a multimethod is not a special kind of method as the English language prefix implies. true, but it's also not a cl generic function or a method (as in java) ... which both kindof inform the need to find a new name 😉 ... shrug ... agreed ... naming is hard

Jim Newton09:07:46

BTW in the documentation of defmethod the name multifn is used instead of multimethod

Ed09:07:08

ok ... then I'm probably getting the name wrong 😉 ... conceptually, the body of a defmethod is one branch of the dynamically constructed switch statement inside a defmulti ... right? ... so as long as you register all the methods before calling the memoized fn it should work ... right?

Jim Newton09:07:18

just that as far as clojure is concerned the defmethod and defmulti must have the internal name. and the public name is used at the call site and is just a dynamic variable whose value is the value memoize (or gc-friendly-memoize) returns when given the internal name of the defmulti.

Jim Newton09:07:33

I think it will work. I'm trying it out.

Ed09:07:07

I don't think clojure exposes names for the methods. You can get at the implementations by calling methods

Ed09:07:44

or did I misunderstand what you mean?

Jim Newton10:07:48

sorry, that's not what I mean. I mean that when you type defmethod you must then give the name of some multimethod. and that name must all match the name at the call-site, i.e. where the function is called in the application.

Jim Newton10:07:53

I want the call site to name a memoized function. therefore the macro must insert the internal name into the defmethod call and the same name into the defmulti call.

Jim Newton10:07:00

(defmulti-memoized [public-foo internal-foo] ...)
(defmethod-memoized public-foo ...)

(defn xyzzy [x y z]
   (public-foo ...))

Jim Newton10:07:11

defmulti-memoized expands to (defmulti internal-foo ...) and defmethod-memoized expands to (defmethod internal-foo ...)

Ed10:07:27

I'm not sure I understand the need for defmethod-memoized ... surely that's just caching the same thing twice?

Ed10:07:48

ah ... I see ... you mean defmethod-memoized is just mapping the name to the one generated by defmulti-memoized ... sorry ... I'm being slow

Ed10:07:00

if you have to provide the internal-foo when you're defining the memoized multimethod, I'm not sure I see the difference between a search and replace for defmethod foo -> defmethod-memoized foo and defmethod foo -> defmethod foo-internal

Ed10:07:43

if that's what you're doing ... ????

Jim Newton10:07:33

yes that's basically what the macro does. good point. if I have to edit all the method definitions anyway, I can just as easily change the function name as change defmethod to defmethod-memoized I suppose.

Jim Newton10:07:10

BTW, I'm curious why what I've written doesn't work.

(defmacro defmulti-memoized
  [[public-name internal-name] docstring dispatch-fn]
  "Define a multimethod on an internal name, and a memoized function implemented
   as a dynamic variable.  Methods must be defined using defmethod-memoized, using
   the public-name."
  (assert (string? docstring))
  `(let []
     (declare ~public-name) ;; so that the methods can call the public function if necessary
     (defmulti ~internal-name ~dispatch-fn)
     (def ~(with-meta public-name {:dynamic true :internal-name internal-name :multi true})
       ~docstring
       (gc-friendly-memoize ~internal-name))))

(defmacro defmethod-memoized
  [public-name dispatch-val & fn-tail]
  (assert ((every-pred :dynamic :multi :internal-name) (meta public-name))
          (format "%s does not name a multimethod defined by defmulti-memoized" public-name))
  `(defmethod ~(:internal-name (meta public-name)) ~dispatch-val ~@fn-tail))

Jim Newton10:07:50

defmethod-memoized (as I have written it) tries to read the meta data established by defmulti-memoized in order to compute the internal name given the public name.

Jim Newton10:07:06

but meta returns nil

Jim Newton10:07:54

it seems that (def ^:dynamic xyzzy) does not put meta data on the symbol, just passes metadata to def 😞

Jim Newton10:07:27

BTW, perhaps an advantage of using the public name in defmethod-memoized is that it allows the internal name to be as cryptic as I like. And I believe it expresses the intent better. However, my idea might not work if I can't retrieve the meta data to compute the internal name from the public name.

Ed10:07:57

(def ^:dynamic thing ^{:other :thing} {})
(meta thing)   ;; => {:other :thing}
(meta #'thing) ;; => {:dynamic true, :line 10 ....}
are you looking for a var-quote?

Ed10:07:02

I think I would keep going with the search and replace, and just rename the defmulti / defmethods to a specific internal name (eg foo*) and then have the public fn foo implement the memoization, rather than having the macro creating names ...

Jim Newton10:07:18

yes, with defn it is easy to encapsulate, but with defmethod/defmulti it seems difficult.

Jim Newton10:07:05

not very lispy though. the lispy is is to not make the user follow a bunch of programming rules which are automatable. but one has to choose his battles I suppose.

Ed10:07:20

well ... I'd also probably keep going, and have two explicit names for the defn's too

😭 2
Jim Newton10:07:20

another nice thing about macros, as you already know, is that I may not know at first what the final implementation needs to be. if the logic is centralized into one place I'm free to change it later without a huge refactoring.

Ed10:07:21

if you want to hide it you could look at alter-var-root ... and you might be able to get a single fn that memoizes both a defmulti and a defn and attaches the original implementation as metadata?

Ed11:07:20

I don't think that property is specific to macros .... I think that's achievable with functions too

Ed11:07:30

I think macros are for when you care about syntax

Ed11:07:23

I think I would prefer to work with the code that has explicit vars for the memoized and non-memoized versions, with the non-memoized ones considered "private" and not for normal consumption ...

Ed11:07:08

but I'm not working on your app ... so I'm pretty sure my opinion doesn't matter much 😉

Ed11:07:29

(defn- memoize-var [v]
    (alter-meta! #'foo assoc :orig/impl (deref v))
    (alter-var-root #'foo (fn [f] (m/memoizer f (c/soft-cache-factory {})))))
  (memoize-var #'foo)

Ed11:07:10

the problem with macros is that they're only available at compile-time ... so they're not really composable at run-time without compiling more code. So once you're in macroland it's really hard to get back out again ... In my experience it's best to do all the actual work in functions, and only write a very thin macro over the top to handle the syntax part

Ed11:07:28

and I guess by "syntax part" I mean quoting and wrapping things in fn 😉

zendevil.eth09:07:56

I’m trying to write the following in clj (http://docs.web3j.io/latest/transactions/transactions_and_smart_contracts/):

Function function = new Function<>(
             "functionName",  // function we're calling
             Arrays.asList(new Type(value), ...),  // Parameters to pass as Solidity Types
             Arrays.asList(new TypeReference<Type>() {}, ...));
and so far I have this:
222 (Function. "buy"
223   (java.util.Arrays/asList (into-array [str]))      
224   (java.util.Arrays/asList (into-array [clojure.reflect/TypeReference])))
225              
But I don’t think this is correct. I have many questions. What does new Type(value) mean? What goes into value? What is new TypeReference<Type>() {}, what does it mean to have () {} after new TypeReference<Type>, what is this function doing?

pyry09:07:22

Off topic? From skimming the docs I get the impression that the value is the amount of ether you wish to deposit, but I'm obviously not an expert on the library. Might want to post your question on a channel more closely aligned with your interests.

zendevil.eth09:07:49

my question is more about the interop; the value part I might do with trial and error

zendevil.eth09:07:25

more specifically the new TypeReference<Type>() {} part

zendevil.eth09:07:28

Currently I’m getting:

No matching ctor found for interface java.util.function.Function

pyry09:07:37

Right. That comes from (Function. ...)

pyry09:07:27

Function is a functional interface, it doesn't have a constructor.

pyry10:07:12

Use instead sth like (reify Function (apply [_ t] ... your function body here))

pyry10:07:15

.. which would be the case if you're supposed to use j.u.f.Function like in the exception you pasted. However I think you really should be using org.web3j.abi.datatypes.Function instead.

pyry10:07:52

Check the ABI specs (Github repo essentially) linked to from the docs you gave.

Ed13:07:33

the new TypeReference<Type> () {} means create an anonymous subclass of TypeReference with no initialisation code ... see https://docs.oracle.com/javase/tutorial/java/javaOO/anonymousclasses.html ... I think that the Function in that code block refers to org.web3j.abi.datatypes.Function which looks like it does have a constructor

zendevil.eth05:07:04

I think I can simply do:

(java.util.Arrays/asList (into-array [java.lang.String]))
rather than:
(java.util.Arrays/asList (into-array [(Type'. java.lang.String)]))
?

zendevil.eth05:07:55

How do I convert the `new TypeReference<Type> () {}` to clojure?

zendevil.eth05:07:08

I tried:

237 (java.util.Arrays/asList (into-array [(proxy 
238                                         [TypeReference] [])]))
but this gives: “Missing Type Parameter” from here: https://github.com/web3j/web3j/blob/781ce5019b3f9655c20236644a4940dcfde0ac88/abi/src/main/java/org/web3j/abi/TypeReference.java#L49

Ed10:07:18

Clojure vectors implement java.util.List so you don't need to make an array to turn it into a list : (instance? java.util.List [1 2 3])

Ed10:07:39

It looks like that TypeReference abstract class is using the java reflection api to get around type erasure : https://github.com/web3j/web3j/blob/781ce5019b3f9655c20236644a4940dcfde0ac88/abi/src/main/java/org/web3j/abi/TypeReference.java#L25

Ed10:07:30

so I think you're well into the weeds of java interop ... you'll probably need to at least implement this : https://docs.oracle.com/javase/7/docs/api/java/lang/reflect/ParameterizedType.html with your proxy

Ed10:07:48

you're probably better off just writing a parameterised java class and using that

zendevil.eth11:07:04

yeah I’m just writing the thing in java and calling from my clojure app

Jim Newton11:07:10

Does anyone have an idea of what this error message means? It happens when I evaluate (def rte-inhabited? nil)

Show: Project-Only All 
  Hide: Clojure Java REPL Tooling Duplicates  (18 frames hidden)

2. Unhandled clojure.lang.Compiler$CompilerException
   Error compiling src/clojure_rte/rte_construct.clj at (50:1)
   #:clojure.error{:phase :compile-syntax-check,
                   :line 50,
                   :column 1,
                   :source
                   "/Users/jnewton/Repos/clojure-rte/src/clojure_rte/rte_construct.clj",
                   :symbol def}
             Compiler.java: 7114  clojure.lang.Compiler/analyzeSeq
             Compiler.java: 6789  clojure.lang.Compiler/analyze
             Compiler.java: 6745  clojure.lang.Compiler/analyze
             Compiler.java: 7180  clojure.lang.Compiler/eval
             Compiler.java: 7131  clojure.lang.Compiler/eval
                  core.clj: 3214  clojure.core/eval
                  core.clj: 3210  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper
                  AFn.java:  144  clojure.lang.AFn/applyTo
                  core.clj:  665  clojure.core/apply
                  core.clj: 1973  clojure.core/with-bindings*
                  core.clj: 1973  clojure.core/with-bindings*
               RestFn.java:  425  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn
                  main.clj:  414  clojure.main/repl/read-eval-print/fn
                  main.clj:  414  clojure.main/repl/read-eval-print
                  main.clj:  435  clojure.main/repl/fn
                  main.clj:  435  clojure.main/repl
                  main.clj:  345  clojure.main/repl
               RestFn.java: 1523  clojure.lang.RestFn/invoke
    interruptible_eval.clj:   84  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:   56  nrepl.middleware.interruptible-eval/evaluate
    interruptible_eval.clj:  152  nrepl.middleware.interruptible-eval/interruptible-eval/fn/fn
                  AFn.java:   22  clojure.lang.AFn/run
               session.clj:  202  nrepl.middleware.session/session-exec/main-loop/fn
               session.clj:  201  nrepl.middleware.session/session-exec/main-loop
                  AFn.java:   22  clojure.lang.AFn/run
               Thread.java:  832  java.lang.Thread/run

1. Caused by java.lang.NullPointerException
   Cannot invoke "clojure.lang.IPersistentMap.valAt(Object)" because
   the return value of "clojure.lang.Var.meta()" is null

                  Var.java:  248  clojure.lang.Var/isMacro
             Compiler.java: 7496  clojure.lang.Compiler/lookupVar
             Compiler.java: 7501  clojure.lang.Compiler/lookupVar
             Compiler.java:  544  clojure.lang.Compiler$DefExpr$Parser/parse
             Compiler.java: 7106  clojure.lang.Compiler/analyzeSeq
             Compiler.java: 6789  clojure.lang.Compiler/analyze
             Compiler.java: 6745  clojure.lang.Compiler/analyze
             Compiler.java: 7180  clojure.lang.Compiler/eval
             Compiler.java: 7131  clojure.lang.Compiler/eval
                  core.clj: 3214  clojure.core/eval
                  core.clj: 3210  clojure.core/eval
    interruptible_eval.clj:   87  nrepl.middleware.interruptible-eval/evaluate/fn/fn
                  AFn.java:  152  clojure.lang.AFn/applyToHelper

p-himik11:07:41

Feels like it has something to do with how your editor is evaluating things.

Jim Newton11:07:39

maybe cider is confused. I'll restart. but this will be a headache because I know the code isn't working yet 😞

p-himik12:07:09

Do you call reset-meta! anywhere? Perhaps you used nil as its second argument, something like (reset-meta! #'rte-inhabited? nil).

Jim Newton12:07:38

yes I did. I was trying to figure out the difference between reset-meta! and something-else-meta!

p-himik13:07:45

Well there you have it. :) Metadata cannot be nil. If you want to reset it, use an empty map instead.

Jim Newton14:07:51

if metadata cannot be nil, then why does reset-meta! allow such?

Jim Newton14:07:21

that's curious

p-himik14:07:19

Just a lack of the check. You can find a lot of such things in Clojure. Or rather, not find. :)

p-himik14:07:20

Although a bit strange that it's not mentioned in any of the docstrings.

✔️ 2
gammarray12:07:38

Does anyone know if there’s an epub or mobi version of the Clojure docs out there? My searches are coming up short.

gammarray14:07:43

Figured as much. Just rooted an old nook reader and thought it might be a nice way to study clojure offline.

indy15:07:17

If you're using Cursive, it can download Clojure docs and show them on the Quick documentation action.

gammarray15:07:58

Good to know, thx!

dharrigan12:07:59

If you're interestd in a forthcoming book about pretty much all of clojure and the standard library, you wouldn'tgo far wrong with this

Stuart15:07:01

I have this, its a great reference.

dharrigan12:07:05

That may help you?

richiardiandrea17:07:40

Hi there, very newbie question here - is require composing namespaces with same names ... can I have multiple my-ns.foo with different vars in them? This is very handy when you want to "add vars" for development with namespaces stored in src/dev - for instance - for some reason I have always thought that would work but it does not seem to

richiardiandrea18:07:51

Ok, well you basically end up needing to differentiate using the namespace name (say my-ns.dev.foo) while you could have just used the classpath mechanism (`-A:dev`) for it and let the runtime compose. I guess you would end up with what Java is doing (and conflicts) so I can see why this was chosen not to be the case.

hiredman17:07:25

there is a single namespace registry per clojure runtime, and it is a flat map (no trees) of namespace names to namespace objects

hiredman17:07:38

you can run multiple clojure runtimes in a single jvm, which a lot of building tooling will often do, but it is an involved process, and communicating between the runtimes is usually painful

indy17:07:35

Mind elaborating what multiple Clojure runtimes mean? Can these supposed runtimes really be isolated from each other?

indy17:07:40

By multiple runtimes, does it mean multiple ns maps?

hiredman18:07:20

you do some classloader magic to isolate the loaded classes, which causes the jvm to treat them as distinct and different types

hiredman18:07:37

so you can load multiple copies of the clojure.lang.RT class

hiredman18:07:50

on the jvm it often looks like what identifies a type is its name, but it is actually the pair of the name and the classloader that loaded in the class, and in most cases classloaders will delegate loading to their parent, so you usually get the same class (same name and loader) for a given class, but if you break that delegation chain you can load a class multiple times with the same name and the jvm will treat it as a different type

indy18:07:14

Interesting, is that classloader magic of breaking the delegation chain considered a hack or is it something legitimate?

indy18:07:48

Curious because this sounds like you can do what docker does with the OS but with Clojure and the JVM.

hiredman18:07:27

it has been discussed a little bit in #tools-deps as functionality people might want for some parts of the new tools.build stuff, I wouldn't be surprised if that could be added as a library there

hiredman18:07:40

I wrote something like it for work, and I think @U04V70XH6 may whip it into shape for using tools.build to run tests for our monorepo's subprojects

hiredman19:07:01

I do know about a million years ago at some point when I asked rhickey on irc about running multiple clojure runtimes in a single process, he said something like "same as java with classloaders"

seancorfield19:07:28

I think I'm happier with running tests in a subprocess, at least for now, but Polylith does use that approach for running tests directly in-process with independent dependencies for each "project" so it's definitely something to keep in our back pocket in case we change our minds on that.

seancorfield19:07:36

Boot's pods were very nice but they added caching/"pod pooling" behind the scenes and that was very buggy (partly because they would asynchronously clean pods up for reuse) and those bugs were a big reason we moved from Boot to the CLI/`deps.edn`.

seancorfield19:07:40

It would be great if t.d.a provided "something" to set up isolated Clojure instances out of the box but I don't think that will happen any time soon (if at all).

hiredman20:07:58

it does seem like the kind of complicated feature that could be added as a library so core will avoid taking it on

5
kristof02:07:25

@U04V70XH6 can you elaborate more on "isolated clojure instances" and what this would be good for? It sounds interesting. Like a kind of clojure container but there's only one JVM and garbage collector running.

kristof02:07:42

and if the instances need to communicate then they're not really isolated, are they? What's the surface area of the connecting points?

seancorfield03:07:52

@UJHMA11DJ Pretty much exactly what @U0NCTKEV8 was talking about: running code in multiple classloaders that are isolated. Boot used to do this with its "pods" but it was a bit fussy in places -- and part of it is because the initial instance has to set up the isolated instances and "communicate" with them to get the environment set up.

seancorfield03:07:25

In Boot's case, you usually wanted to pass parameters into the process and get results out -- and that all has to be serialized in a way that avoids any classes that unique to each classloader (because a HashMap in one classloader is not going to look like a HashMap in another classloader -- because they are different classes... they just happen to have the same name, but their classloader is part of their identity).

hiredman03:07:19

clojurebot (the ircbot in a clojure irc channel) has some really horrendous code for doing this as part of it's sandboxed eval https://github.com/hiredman/clojurebot/blob/master/clojurebot-eval/src/clojurebot/sandbox.clj#L223

hiredman03:07:12

So I can use a different newer version of clojure in the sandbox without being forced to update the version of clojure the sandbox is built with

kristof03:07:43

@U04V70XH6 That led to some interesting reading, thanks. What would have needed to be fixed for you to stay happy with Boot's pods?

seancorfield06:07:52

@UJHMA11DJ Sorry, missed that Q. Two things: the fileset abstraction caused quite a performance overhead (and could also be problematic when doing dev work since source files often appeared to be in the "shadow" file locations instead of their actual locations); the async refresh of pod contexts was just plain buggy and caused us all sorts of problems in big, complex builds where pods got reused. I think if Boot's fileset abstraction was "good intentions" but not necessary and caused more problems than it solved. The whole pod pooling/reuse thing was definitely valuable -- pods were expensive to stand up on their own so cleaning them up and reusing them helped performance but it also caused weird async bugs.

richiardiandrea18:07:51
replied to a thread:no

Ok, well you basically end up needing to differentiate using the namespace name (say my-ns.dev.foo) while you could have just used the classpath mechanism (`-A:dev`) for it and let the runtime compose. I guess you would end up with what Java is doing (and conflicts) so I can see why this was chosen not to be the case.

j3vs19:07:15

trying to wrap my head around whether there's a way to write a macro to do something like

(with-uuid [uuid]
  ;; do something with uuid)
that would be equivalent to
(let [uuid (java.util.UUID/randomUUID)]
  ;;do something with uuid)
I tried something like
(defmacro with-uuid
  [bindings & body]
  `(let [uuid# (str (UUID/randomUUID))
         bindings# (conj ~bindings uuid#)]
     (let* bindings# ~@body)))
but using it complains about not being able to resolve the symbol uuid

p-himik19:07:11

With that #, uuid# becomes uuid_%something_generated%. There is no uuid binding.

p-himik19:07:37

If you want uuid and for it to shadow any other uuid at the scopes above, just remove the #.

j3vs19:07:26

the uuid# is the value of the uuid, rather than the symbol, defining the macro doesnt appear to fail, its running

(With-uuid [uuid] (println uuid))
that complains about not being able to resolve uuid

Russell Mull19:07:55

I'd recommend looking atone of the with-* macros in clojure.core to understand how it's done.

p-himik19:07:46

Ah, I see what you wanted to do. Try this:

(defmacro with-uuid
  [bindings & body]
  (let [b (gensym "uuid")
        bindings (conj bindings b)]
    `(let [~b (str (UUID/randomUUID))]
       (let* ~bindings ~@body))))

p-himik19:07:11

You must compute the bindings vector at macro expansion time, but your code tries to do that at run time.

seancorfield19:07:53

@U0ERTS466 If you want the name from that vector to be the actual symbol used in the expansion, you want an anaphoric macro like this:

dev=> (defmacro with-uuid [[sym] & body] `(let [~sym (java.util.UUID/randomUUID)] ~@body))
#'dev/with-uuid
dev=> (with-uuid [x] (println x))
#uuid "30832455-a7b9-403d-98c3-f19d17177445"
nil
dev=> (with-uuid [y] (println y))
#uuid "58872d97-4944-4819-9854-5bcf4b4c6093"
nil

👍 4
seancorfield19:07:59

Just be aware that is "odd" syntax, so you'll find linters will likely complain about the symbol being undefined in the body of your macro: normally a binding has a symbol and an expression (making it easier to lint such macros "as if" they were clojure.core/with-open for example (which has [sym expr] as its binding form).

👍 2
seancorfield19:07:58

https://en.wikipedia.org/wiki/Anaphoric_macro -- background reading on what I meant there.

dpsutton21:07:51

What's going on here?

(= (conj clojure.lang.PersistentQueue/EMPTY 3)
   (conj (clojure.lang.PersistentQueue/EMPTY) 3))
I'm confused why EMPTY can be invoked. But calling ((conj clojure.lang.PersistentQueue/EMPTY 3)) gives the expected Queue cannot be cast to IFn

noisesmith13:07:41

same thing that happens here:

user=> (= Math/PI (Math/PI))
true

👍 2
Alex Miller (Clojure team)21:07:20

the first one is just syntax sugar for the second one, both are handled basically the same by the compiler

Alex Miller (Clojure team)21:07:39

or rather they are both syntax sugar for the same . form

hiredman21:07:40

there is ambiguity in clojure syntax around field accesses and no arg methods invokes

👍 2
emccue21:07:11

public final class Thing {
    public static final int x = 4;

    public static int x() { return 5; }
}

emccue21:07:29

so what would happen with (Thing/x) would change if the method was added?

Alex Miller (Clojure team)21:07:43

yes, methods are preferred

didibus03:07:55

I guess its just a convenience thing, otherwise you'd be forced to use .- all the time like in ClojureScript

Alex Miller (Clojure team)21:07:50

use .- to force field access

Alex Miller (Clojure team)21:07:24

fortunately, this overlap is pretty unusual in Java

ghadi21:07:48

java records have this overlap routinely now

ghadi21:07:05

(but the field is private, I think)

emccue21:07:17

i don't think so - i thought the field was public

Alex Miller (Clojure team)21:07:18

well that's not an overlap here :)

emccue21:07:12

maybe i'm wrong about that though

ghadi21:07:50

❯ javap -p JrueHoliday.class
Compiled from "JrueHoliday.java"
final class JrueHoliday extends java.lang.Record {
  private final int x;
  JrueHoliday(int);
  public final java.lang.String toString();
  public final int hashCode();
  public final boolean equals(java.lang.Object);
  public int x();
}
yeah the field is private, method is public

👍 2
emccue23:07:55

Any convenient way to serialize objects in transit based on their type?

emccue23:07:05

specifically ^{:type ::thing} {:a 1}

vemv02:07:17

side-note, metadata is cute but it seems quite highly likely to get lost in translation as it jumps across module boundaries clojure.core has some degree of care for metadata not to get easily lost, but still can, especially in face of non-clojure-core defns plus you'd have to spec data and metadata. it seems awkward to do some double work?

emccue23:07:59

(def User [:and [:map
                 [:user/email string?]
                 [:user/password-hash string?]]
           [:fn {:error/message "should have user metadata"}
            (fn [o] (= (type o) ::user))]])

;; ----------------------------------------------------------------------------
(malli/=> create [:=>
                  [:cat [:map
                         [:email string?]
                         [:password-hash string?]]]
                  User])
(defn create
  [{:keys [email password-hash]}]
  ^{:type ::user}
  {:user/email         email
   :user/password-hash password-hash})

;; ----------------------------------------------------------------------------
(malli/=> email [:=> [:cat User] string?])
(def email :user/email)

;; ----------------------------------------------------------------------------
(malli/=> password-hash [:=> [:cat User] string?])
(def password-hash :user/password-hash)

emccue23:07:41

trying to figure out the "right" way to make domain models with malli type stuff

lilactown23:07:17

type seems like the wrong way to do this IMO

emccue23:07:46

okay, but a record is less than ideal too, considering how much has trended towards namespaced keywords