its a bit out of your hands if you find another namespace with the same name

(require 'foo.bar) will require only one of those. You’d have to start spelunking in the jar to enumerate files i think

There isn't a notation that combines a dep coordinate with a name within that dep?

No

Coordinates don't exist at run time. They are merely ways to create a class path. That's paths you hard essentially

(To jars) typo

And within a clojure runtime a namespace is global, there is no hierarchy of namespaces, and there is only ever a single namespace with a given name

and this isn’t a Clojure thing but how the jvm works

(about the coordinates and jars thing i was talking about earlier)

So you could do it like this:

(defn resource-path->filenames [resource-path]
  (->> (-> (Thread/currentThread)
           (.getContextClassLoader)
           (.getResources resource-path))
       (enumeration-seq)
       (distinct)
       (mapv str)))

(comment
  (->> (resource-path->filenames "clojure/string.clj")
       (filter your-criterion)
       (first)
       (slurp)
       (load-string)))

wait, wouldn't your distinct there break it? The question is to try to get multiple different namespaces with the same name. Running a distinct would lose that information. You'd want to load them before then.

Yeah, no, that is just effect loading files that create/mutate the namespace object

There is still one namespace object for a given namespace name

In any case, I'm not asking how I could get such a reference from some files on my disk, but whether there exists a community standard for such a notation.

No, and it doesn't make sense with how projects are organized

The community standard is to make unique project names for libraries and to sometimes use reverse domain name namespace names to prevent conflicts.

That is to say: the community standard is not to conflict

Reverse domain name notation works where it's used. Not the version-specific part, but at least project specific.

@U0NCTKEV8 why, because you could alter your classpath to point to different versions of the same name in the same project?

because generally you run some tool which makes all the dependencies available before actually running your code

For version-specific part: the clojure community doesn't make breaking changes. Use the latest version and it's close to a guarantee that unless it's wrapping something that had a massive breaking change, it won't have had one. Your code will work fine, and so will your libraries that depend on that code.

sort of stage 1 gather dependencies then state 2 launch some process with all the dependencies

if your code was littered with dependency information, potentially with runtime dependencies, then you have to run your code to determine the dependencies of your code, and that kind of staged approach doesn't work well

and you don't have a central place where dependency information is, so checking for things like version conflicts, failing early if a dependency cannot be found, etc, don't work as well

by "dependnecy information with runtime dependencies" I mean if say the version of the library wasn't a literal string, but the result of some computation

I'm not sure I totally understand. We have dep coordinates that can reliably get us specific library versions. Those libraries have name hierarchies. In theory, couldn't we combine the dep coordinate with a coordinate in the namespace graph to reliably point to a name?

in theory sure, but in practice all the existing tooling operates differently, and it opens up other unsolved issues (how do you deal with two requests for different versions?), and that is before you even get to how you would need to architect your classloaders (how the jvm loads code) to make it work, which in some cases it just wouldn't work

From an external perspective, yes. However the way the JVM works doesn't allow this to be the way you refer to things at runtime, because "versions" and "libraries" don't exist at runtime. You could potentially build a very large and very complicated system atop the JVM in order to support something like this by way of building custom specialized classloader hierarchies, but this would cause a problem because it bloats the size of the application you're distributing because you're no longer distributing 100 dependencies, you're distributing 1000. It's very rare that two libraries will use the same version of a third library.

Also it wouldn't work without forking clojure because of the namespace map, of which there's only one and it doesn't support "versioned namespaces"

Thanks for the input, everyone. I still think such a notation could have some potential uses - at least on the documentation side - but I now better understand the runtime limitations that doesn't make it a particularly useful idea outside of that.

(let [bar 10]
  (list 'foo bar))

=> (foo 10)

(let [bar 10]
  `(foo ~bar))

=> (your-namespace/foo 10)

Hum, ya, but the thing is I'm in a macro, but I don't want things fully qualified, but I still want to substitute some things. If I use list then I have to quote each symbol one by one, or if I use syntax quote than I get things fully qualified

Oh, this thread has what I want: https://clojureverse.org/t/a-macro-between-quote-and-syntax-quote/4466/13

(let [bar 10] 
  `(~'foo ~bar))
=> (foo 10)

Meaning while, (re-find #"^abc$" "abc\n\n") doesn't match.

I found the problem. $ doesn't mean the end of the string, it means the end of the line. The solution I found is to use (re-find #"\Aabc\z" "abc\n") ;; no match anymore

I think maybe it depends on the mode

based on the documentation at https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html

I was thinking maybe (?m) would change the behavior of ^ and $, but you found something else that seems to work too

See MULTILINE in that page

Its a bit confusing

> In multiline mode the expressions ^ and $ match just after or just before, respectively, a line terminator or the end of the input sequence. By default these expressions only match at the beginning and the end of the entire input sequence.

I guess that it is due to java's https://docs.oracle.com/javase/7/docs/api/java/util/regex/Matcher.html#find() having its own special way of doing the matching.

What's the expression? If you've got a regex now you have two poblems? 😛

I guess the scheduling should survive application restarts, right?

Yes.

My current approach is to use a database to store the pending tasks. But I am not sure if there is any existing frameworks (better lightweight) for that purpose

I have some feature like that planned for a Clojure app, too, but never done it until now. In the Java world I would use Quartz, and there seems to be a Clojure Wrapper: https://github.com/michaelklishin/quartzite, but the linked webpage containing all the docs and tutorials seem to be dead There were no commits on that repo for a long time, too. Can be a sign of maturity, but it means that it probably runs on a very old Quartz Version

Quartz has a lot of advantages, it already takes care of persisting the schedules in the database, has actions to re-schedule if a job run fails, has locking mechanisms to share scheduling over several application instances.

We use quartz at Metabase. We just swap in a more modern Java lib underneath and it's fine. Works great

Probably fine to use Quartz directly through interop

@U11BV7MTK so you use quartzite with an up to date quartz Version?

Yeah. Would also like to learn more about the metabase story with quartz

If you use AWS, you can also us: https://docs.aws.amazon.com/lambda/latest/dg/services-cloudwatchevents.html

Cron expressions that trigger a Lambda

Looks that way https://github.com/metabase/metabase/blob/7eb7cc034d11980ca1d39e24ad93c536c2b5b97c/deps.edn#L22

Ok, but, since there was some talks recently about the value of wrappers, look at this code: https://github.com/michaelklishin/quartzite/blob/master/src/clojure/clojurewerkz/quartzite/scheduler.clj

Do you really need a wrapper?

Maybe there's still value, just saying, looking at the code, most wrapped function are 1 line of code

There's also #goose which was released lately and might be useful

@U0K064KQV what’s the talk on wrapper?

@U0K064KQV tend to agree here. Some wrappers try and transfer the wrapped libraries into more idiomatic Clojure concepts, but this seems like it is a very literal 1:1 translation

for me, java wrapper gives better auto completion support or otherwise I have to look up the java doc to guess and write the code.

The wrapper isn't necessary. I only like it because it adds autocomplete and apropos support

could easily do with interop

but we already have established stuff so on it goes

Didn't mean a talk talk, meant there was some chatter in some of the forum topics on how some Clojure wrapper libs are incomplete or outdated and no longer maintained. So some people were saying because of that Clojure is missing a lot of libs, but others were saying that using a Java lib directly is the better idiom anyways, unless the wrapper adds substantial value, and its probably because most wrappers don't add value that they probably all end up abandoned by their maintainer anyways.

But fair point on the auto-complete

yeah. if the api of the underlying lib changed we might consider abandoning the clojure wrapper. it adds an extra layer on top of the actual lib. and then two sets of documents

luckily its pretty 1-1 with the underlying lib. there are a few macros in there as well for the builder stuff that can be a bit annoying in clojure

cron job works miracles too

https://github.com/contribsys/faktory

there may be a cleaner way but:

(let [get-sym (gensym)]
  `(let [~get-sym (fn ...)]
      ~@(mapcat get-sym ...)))

Unfortunately, it doesn't seem to work. That mapcat returns an empty list - it seems to be calling get-sym , the symbol (defined in the outer let), not the function (defined in the inner let)

Replace get-sym# with an explicit call to gensym outside of the syntax quote.

Bind it within a let, use that binding in place of get-sym#.

@U2FRKM4TW isn't that what @U02F0C62TC1 shown? If so, it didn't work well for me.

No, completely different. I'm telling about having a let outside of any quoting.

Oh, wait, I'm blind.

This is exactly what I tried:

(defmacro x [binding-syms & body]
  (let [gx (gensym)]
    `(let [~gx (fn [sym#] [sym# 1])
           ~@(mapcat gx binding-syms)]
       ~@body)))

(x {'a 1 'b 2}
   [a b])
;;=> 
1. Caused by java.lang.RuntimeException
   Unable to resolve symbol: a in this context

I was completely skipping the first line in rolt's code because I thought it was defmacro. :D sigh

I assume it's because with unquote I escape the syntax quote context and then it simply doesn't see the binding established within it (to bind gx to the function)

(defmacro x [binding-syms & body]
  (let [gx (gensym)]
    `(let [~gx (fn [sym#] [sym# 1])
           ~@(mapcat (fn [[sym val]] [sym (list gx val)]) binding-syms)]
       ~@body)))
=> #'dev/x

(macroexpand-1 '(x {a 1 b 2} [a b]))
=>
(clojure.core/let
 [G__99470 (clojure.core/fn [sym__99454__auto__] [sym__99454__auto__ 1]) a (G__99470 1) b (G__99470 2)]
 [a b])

(x {a 1 b 2} [a b])
=> [[1 1] [2 1]]

Two mistakes in your code: • Lack of destructuring of the elements of binding-syms (when you iterate over a map, each item is a kv-pair) • Passing {'a 1} instead of {a 1}. It results in {(quote a) 1} after reading.

Unless I'm misunderstanding your initial question. I see that in your expected result you have a (get-sym 'a) but you pass to your macro {a 1}. I was assuming 1 was just a placeholder for 'a, but given the name get-sym, you probably meant to write it that way and 1 was supposed to be used in some completely different place?..

In that case, this should work:

(defmacro x [binding-syms & body]
  (let [gx (gensym)]
    `(let [~gx (fn [sym#] [sym# 1])
           ~@(mapcat (fn [[sym _val]] [sym (list gx (list 'quote sym))]) binding-syms)]
       ~@body)))
=> #'dev/x

(macroexpand-1 '(x {a 1 b 2} [a b]))
=>
(clojure.core/let
 [G__99715 (clojure.core/fn [sym__99699__auto__] [sym__99699__auto__ 1]) a (G__99715 (quote a)) b (G__99715 (quote b))]
 [a b])

(x {a 1 b 2} [a b])
=> [[a 1] [b 1]

Thanks a lot - it almost works. The tricky thing for me is (list 'quote sym). Anyway, this is what I'm trying to do:

(defmacro x [binding-syms & body]
  (let [gx (gensym)]
    `(let [~gx (fn [sym#] [sym# (get ~binding-syms sym#)])
           ~@(mapcat (fn [[sym _value]] [sym (list gx (list 'quote sym))])
                     binding-syms)]
       ~@body)))

(x {a 1 b 2}
   [a b])
;; macroexpands to: 
(let [G__30658 (fn [sym__30643__auto__] [sym__30643__auto__
                                         (get
                                          {a 1, b 2}
                                          sym__30643__auto__)])
      a (G__30658 'a)
      b (G__30658 'b)]
  [a b])
;; .. and that fails with
1. Caused by java.lang.RuntimeException
   Unable to resolve symbol: a in this context

Are you planning to do more stuff with it, e.g. by expanding what gx does? Because if not, there's a much simpler way to simply turn a map into let bindings.

This is the original function I have right now: https://github.com/jumarko/clojure-experiments/blob/develop/src/clojure_experiments/debugging.clj#L357-L359 But I wanted to tweak it to generate nicer code. Right now, it expands like this (https://github.com/jumarko/clojure-experiments/blob/develop/src/clojure_experiments/debugging.clj#L409)

(let*
  [x
   (get (peek (deref my-locals)) 'x)
   y
   (get (peek (deref my-locals)) 'y)
   z
   (get (peek (deref my-locals)) 'z)
   a
   (get (peek (deref my-locals)) 'a)
   b
   (get (peek (deref my-locals)) 'b)
   c
   (get (peek (deref my-locals)) 'c)]
  [x y z a b c])

In this case, why does gx function return [sym# (get ...)] instead of just (get ...)? In any case, if you really need that vector with a symbol, here's the macro:

(defmacro x [binding-syms & body]
  `(let [~@(mapcat (fn [[sym val]] [sym [(list 'quote sym) val]]) binding-syms)]
     ~@body))

(defmacro x [binding-syms & body]
  `(let [~@(mapcat identity binding-syms)]
     ~@body))

The helper function returns a tuple [sym sym-value]. sym-value is looked up in a map stored in the var. It actually seems to work after I fixed a few inconsistencies:

(defmacro expand-locals
  "Establish  bindings saved in given var as local symbols via `let`."
  [bindings-var-sym & body]
  (let [atom-var (deref (ns-resolve *ns* bindings-var-sym))
        binding-syms (keys (peek @atom-var))
        ;; For details about this implementation see:
        ;; 
        get-sym (gensym "get-sym-val-")]
    ;; get the latest values of locals from bindings-var-sym
    `(let [~get-sym (fn [sym#] [sym# (get (peek @~bindings-var-sym) sym#)])
           ~@(mapcat (fn [sym] [sym (list get-sym (list 'quote sym))])
                     binding-syms)]
       ~@body)))

(defmacro exl [& body]
  `(expand-locals my-locals ~@body))

(defn my-function [x y z]
  (let [a (* x y z)
        b (inc a)
        c (/ b 10)]
    (locals)
    (->> (range b)
         (map inc)
         (filter odd?))))

(my-function 5 6 7)

(macroexpand '(exl [x y z a b c]))
;;=>
(let*
 [get-sym-val-30936
  (clojure.core/fn
   [sym__30912__auto__]
   [sym__30912__auto__
    (clojure.core/get
     (clojure.core/peek @clojure-experiments.debugging/my-locals)
     sym__30912__auto__)])
  x
  (get-sym-val-30936 'x)
  y
  (get-sym-val-30936 'y)
  z
  (get-sym-val-30936 'z)
  a
  (get-sym-val-30936 'a)
  b
  (get-sym-val-30936 'b)
  c
  (get-sym-val-30936 'c)]
 [x y z a b c])

So many thanks @U2FRKM4TW! The key part for me to make this work was (list get-sym (list 'quote sym))

And of course, you were right - I had an extra vector layer around it which didn't work. This is hopefully the final version 🙂 https://github.com/jumarko/clojure-experiments/pull/24/files#diff-f2edafb7344c15faa9918535bca855291266590852dff41b6fc2fc630ccd6afeR364-R366

Just to be certain - you use the same atom twice in different contexts. During macro expansion - to figure out what keys to extract. During run time - to get the actual values of those keys. So if that atom gets new keys in run time, you won't know about it. But maybe it's fine. One tiny thing - you execute peek + deref on each binding. Might be worth it extracting as well, so that the get-sym function has only get inside it, and then you won't need the function at all - you will be able to simply use get right in that mapcat's function.

Yeah, good pointers! • Fetching the keys twice is fine - and I don't know how would construct the let otherwise (I need those symbol names) • Good remark about peek + deref. I'll look at that.

Yeah, you can't have a let without knowing what names to use beforehand. It means that you gotta specify some value for every key that you will need, even if you don't have an actual value yet. And then you might need to differentiate between "set to something" and "not set just yet". But that's just me going on a tangent. :)

Ok, one more question 🙂 Let's say I wanna simplify and generalize expand-locals by making it accept a function (`get-binding-fn`) instead of a var symbol:

(defmacro expand-locals
  "Establish  bindings (as returned by `get-binding-fn`) as local symbols via `let`.
  `get-binding-fn` must return  map of symbols to their values.
  This dynamic lookup mechanism make it possible to defer the symbol resolution until runtime;
  while compile-time resolution would be mostly fine (I use this macro manually via `exl`),
  it would be come a problem when the values were not serializable in the bytecode,
  such as HickariCP connection pool object stored in ring's request map."
  [get-binding-fn & body]
  (let [binding-syms (keys (get-binding-fn))
        ;; For details about this implementation see:
        ;; 
        binding (gensym "binding-")]
    `(let [~binding (~get-binding-fn)
           ~@(mapcat (fn [sym] [sym (list 'get binding (list 'quote sym))])
                     binding-syms)]
       ~@body))).

(defmacro exl
  "Like `expand-local` but assumes that the var storing the bindings is called `my-locals` (see the `locals` macro)."
  [& body]
  (let [get-binding (fn [] (peek @my-locals))]
    `(expand-locals ~get-binding ~@body)))

(exl [x y z a b c])
;; => [5 6 7 210 211 211/10]

(defmacro exl1
  "Like `exl` but accepts an index to select proper binding from all the bindings
  saved in the `my-locals` atom."
  [index & body]
  (let [get-binding #(nth @my-locals index)]
    `(expand-locals ~get-binding ~@body)))

(exl1 0 [x y z a b c])
;;=>
1. Caused by java.lang.IllegalArgumentException
   No matching ctor found for class clojure_experiments.debugging$exl1$get_binding__32203

            Reflector.java:  288  clojure.lang.Reflector/invokeConstructor
           LispReader.java: 1317  clojure.lang.LispReader$EvalReader/invoke
           LispReader.java:  853  clojure.lang.LispReader$DispatchReader/invoke
           LispReader.java:  285  clojure.lang.LispReader/read
           LispReader.java:  216  clojure.lang.LispReader/read
           LispReader.java:  205  clojure.lang.LispReader/read
                   RT.java: 1876  clojure.lang.RT/readString

(macroexpand-1 '(expand-locals #(peek @my-locals)
                [x y z a b c]))
;;=>
1. Caused by java.lang.ClassCastException
   class clojure.lang.PersistentList cannot be cast to class clojure.lang.IFn
   (clojure.lang.PersistentList and clojure.lang.IFn are in unnamed module of loader 'app')

                      REPL:  359  clojure-experiments.debugging/expand-locals

Sorry, gotta run now - will take a look in a few hours, unless someone does it sooner.

No problem, thanks again!

that would require expanding the macro at runtime ? i wouldn't recommend it but you can try using eval

When you pass #() to a macro, it effectively gets rewritten as (fn []). When (fn []) gets passed to that macro, it's not a function - it's a list of two items, fn and [] (with other elements if there's a body, of course). In order to call it, you have to make it into a function with, as rolt said, eval. Alternatively, you can pass a symbol that specifies a function and resolve that symbol within a macro.

i was talking about this line: let [binding-syms (keys (get-binding-fn)) the list of symbols need to be known at compile time (unless you call expand-locals within eval) you can pass a function to retrieve the value from a symbol though, and make the list of symbols a constant

Yes, I know the symbols must be known at compile time - we talked about that above. That is expected and fine 🙂 But I wanted to know if there's a good way to pass a function to the macro. I was sort of able to do that (although not inline anonymous function - I might not need that at all) but I had problems with unexpected behavior when using slightly more complex function (the function closing over an argument index inside the exl1 macro). See https://clojurians.slack.com/archives/C03S1KBA2/p1659609223735649?thread_ts=1659601131.210959&cid=C03S1KBA2

what's the point of passing a function then ? if you need to call it at read time anyway ? why not directly pass the result of the function

That's a good point- I must try that

https://clojuredocs.org/clojure.core/counted_q

nice! thanks

But I would question why you want to know whether a collection is lazy. If you want to be strict about input then you can also assert that something is vector? or set? for example

I'm doing some generic collection handling for a lazy value inspector (not interested in the specific coll implementations) but need to be careful on infinite ones

Ah in that case counted? will return false for any lazy seq, because it can't be counted in constant time

bounded-count is probably closer to what you want

I think counted? is enough for my purpose, but nice to know bounded-count exist, keep discovering useful clojure.core fns, thanks!!

one reason could be that *data-readers* was only added in clojure 1.4, and clj-http wanted to support older clojure version ? In this case it would throw anyway when trying to deref nil but this may be an obsolete code

Oh, that would make sense, yeah.

You are very close. Java (the jvm) is weird and uses L<typename>; to mean array of typename

user=> (type (Object.))
java.lang.Object
user=> (type (object-array []))
"[Ljava.lang.Object;"

often times this can come about due to varags in java. the signUrl signature might be signUrl (info, time, unit, …options). And in java you pass the invidiual options but the java compiler clumps them all together into an array of options. Calling from Clojure doesn’t get this benefit so you need to make the array yourself

https://javadoc.io/doc/com.google.cloud/google-cloud-storage/1.113.14-sp.1/com/google/cloud/storage/Blob.html#signUrl-long-java.util.concurrent.TimeUnit-com.google.cloud.storage.Storage.SignUrlOption...-

and indeed that is the case from that link: `

Storage.SignUrlOption... options

oh, right thanks a lot!

and reference from http://clojure.org: https://clojure.org/reference/java_interop#_vararg_methods

and it worked! thank you very much!

fyi L<typename>; is not an array

it's the [ prefix that makes it an array

ah. what does the L prefix mean then? is that just a separator between the typename and the [?

I suspect the original post accidentally missed a [ on the error line (there's one [ in the parentheses, but there should be one before the parens too)

(and thank you for the correction. love hearing your answers especially around jvm stuff that i’m a bit handwavy on)

Lx.y.z; is the "descriptor" of the class

https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Class.html#descriptorString()

names, simple names, binary names, internal names, descriptors

that’s some neat arcane technical bits that i didn’t know. thanks!

Good to know, thanks

https://github.com/aysylu/loom comes to mind

for graph algo stuff

Thanks!

Seems ok to me for global singletons

cool. made me twitch a bit but for namespace objects specifically seems maybe ok

thanks for the gut check

it is only evaluated once

the reader producers unevaluated forms

just like the result of reading (+ 1 2) is the list (+ 1 2) not the number 3

I guess I'm confused why syntaxQuote doesn’t produce the same. I assume that when the Compiler sees (+ 1 2 3), it outputs a function call to plus. How does it know that (seq (concat (list + 1 2 3))) should also output a function call to plus?

why would it?

user=> `(+ 1 2)
(clojure.core/+ 1 2)
user=>

it evalutes to a list, the first element is the symbol clojure.core/+ (due to how syntax quote namespace qualifies symbols)

user=> '(+ 1 2)
(+ 1 2)
user=>

just like regular quote (regular quote doesn't namespace qualify symbols)

you are conflating that macros are often written using syntax quote with syntax quote itself

macro expansion does a sort of staged evaluation thing, where there is macro call, the macro is expanded, which means whatever the macro returns replaces the call to the macro and that is evaluated instead

user=> (defmacro m [] '(+ 1 2))
#'user/m
user=> (m)
3
user=>

user=> (defmacro m [] (list (symbol "+") 1 2))
#'user/m
user=> (m)
3
user=>

Hmmm interesting

I’m specifically thinking about nested syntax quote situations, so maybe that's where I’m running into issues in my mental model

usually you don't actually nest syntax quote, usually you have syntax quote, then inside an unquote you have another syntax quote

not `` but `~` or `~@

Right, but in the cases you do nest them (building complex forms), the output from there inner syntax quote is not the simple list but the (seq (concat …)) form

even in the complex form case

a double quote is not the original thing, but the quoted thing

user=> (+ 1 2)
3
user=> '(+ 1 2)
(+ 1 2)
user=> ''(+ 1 2)
(quote (+ 1 2))
user=> `(+ 1 2)
(clojure.core/+ 1 2)
user=> ``(+ 1 2)
(clojure.core/seq (clojure.core/concat (clojure.core/list (quote clojure.core/+)) (clojure.core/list 1) (clojure.core/list 2)))
user=>

layering quotes just doesn't work that way (syntax quote or otherwise)

mixing and matching for fun

user=> `'(+ 1 2)
(quote (clojure.core/+ 1 2))
user=> '`(+ 1 2)
(clojure.core/seq (clojure.core/concat (clojure.core/list (quote clojure.core/+)) (clojure.core/list 1) (clojure.core/list 2)))
user=>

Is the output form necessary for the compiler? Could LispReader.syntaxQuote output a “simple” list and everything would still work?

no

forget syntax quote, just think about normal quote, it is the same thing, and it won't trip you up on syntax quote's bells and whistles

reader doesn't return the list (+ 1 2) when it reads '(+ 1 2) it returns the list (quote (+ 1 2))

Thanks for explaining all of this to me. I really appreciate it