This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-01-05
Channels
- # aleph (1)
- # announcements (18)
- # babashka (145)
- # beginners (70)
- # calva (34)
- # cider (3)
- # clj-kondo (98)
- # cljdoc (5)
- # cljs-dev (13)
- # clojure (134)
- # clojure-europe (57)
- # clojure-nl (4)
- # clojure-uk (4)
- # clojurescript (40)
- # code-reviews (3)
- # conjure (1)
- # core-async (5)
- # data-science (3)
- # datomic (8)
- # fulcro (9)
- # google-cloud (2)
- # inf-clojure (9)
- # jobs (1)
- # lsp (9)
- # malli (25)
- # polylith (4)
- # reitit (4)
- # releases (2)
- # remote-jobs (3)
- # rewrite-clj (8)
- # shadow-cljs (34)
- # tools-build (1)
- # tools-deps (67)
Maybe a silly question, but why does this api use File instead of Pathhttps://clojure.github.io/tools.build/clojure.tools.build.api.html#var-uber
it doesn't - all paths are strings there
or do you mean in the conflict handler?
if so, because a) a File is what is in hand and, the things useful to do are things like slurp, which takes a File (and not currently a Path), and c) it really is a File on disk, not just a path
@U3JH98J4R Maybe an example conflict handler is illustrative? https://github.com/seancorfield/build-uber-log4j2-handler/blob/main/src/org/corfield/log4j2_conflict_handler.clj
Er, that is a conflict handler...
OK, gotcha. Hence my response 🙂
Yeah, it's the only thing that takes File
and it's just the natural API at that point in the (uber) process flow.
and yeah - i can see it being more useful with the apis that clojure has. The “File” class though is just a representation of a path, it doesn’t imply one way or the other that a file is actually there - hence there being a toPath
Oh, is this for your idiomatic Java wrapper project?
the most complex part of the api (from a translation perspective) is uber
on account of conflict handlers being able to carry through arbitrary state
AFAIK, my log4j2 plugins merger is to only conflict handler out there -- tools.build
's built-in stuff handles (almost) everything you would need I think.
(and my handler is just a translation of the Maven plugin that does this)
log4j 3.x should do away with that stuff, apparently.
also looking at your code there it seems like you are using a regex as a key
(def log4j2-conflict-handler
{#"^META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat$"
log4j2-plugin-merger})
I don’t think that is guaranteed to work
The conflict handlers are a map of string regex pattern
to:
a keyword (to use a built-in handler) or
a symbol (to resolve and invoke) or
a function instance
Default conflict handlers map:
{"^data_readers.clj[cs]?$" :data-readers
"^META-INF/services/.*" :append
"(?i)^(META-INF/)?(COPYRIGHT|NOTICE|LICENSE)(\\.(txt|md))?$" :append-dedupe
:default :ignore}
The state is global. The expectations is each handlers uses a unique key to hold its data across the process.
Well, tools.build
itself builds a hash map with java.util.regex.Pattern
keys.
Whilst I should probably have a String
there, the keys in the map end up being either :default
or the result of re-pattern
: https://github.com/clojure/tools.build/blob/master/src/main/clojure/clojure/tools/build/tasks/uber.clj#L237
And re-pattern
is a no-op when passed java.util.regex.Pattern
So the docs are correct. My code is "wrong" but works.
(well, I guess, strictly, you could argue the docs should say "a map of string regex pattern or actual regex pattern to...")
But I'd be happy to fix my handler if tools.build
validated that it has a map of string to ...
The intent was string
Isn't there also issues with the Pattern equality semantics for use as map keys?
user=> (= #"blah" #"blah")
false
They use object identity and not pattern equality, which means it's difficult at best to retrieve items put into a map with a regex key unless it's only used for iteration.
which I believe it is in this particular case in the bowels of the uber task
but that's my dirty laundry, not public api here (which says strings)
I've updated my conflict handler to use the string instead of the Pattern
and will cut a new release either if log4j gets updated or tools.build starts to validate that 🙂
It feels strange to check a linear sequence of regexes that come from an unordered container, but considering how rare this must be to want to do and the fact that maps start as array maps, I doubt the project that would run into unexpected behavior on account of that has yet been born
(and sorting the regex strings internally would make it deterministic, just not user defined which would probably be enough)
it's probably a bad choice in that code, for sure
but I think it's unlikely that more than one regex will apply so unlikely to be a problem
What is the main usage difference between with-redefs
with-redefs-fn
with-bindings
with-bindings*
and with-local-vars
? I was trying to provide some logging meta-information, such that I can log anything with a prefixed id without parameter-chaining. But I'm struggling to figure out which would be the most suitable approach to that. I think ideally something like
(def ^:dynamic *id* nil)
(with-redefs {*id* 123}
(do-stuff))
And does it make a difference if I pmap something in do-stuff
? It says thread-local, so do I lose *id*
?
Yes, it will revert to the top level binding (if that’s what you mean by lose i.e. here it will be
nil
, but if you’d done (def ^:dynamic *id* 456)
it would revert to 456
.If you don’t want that, use bound-fn
to capture the bindings into your pmap
threads.
I think you really want bindings
not any of those, and pmap uses futures I believe which should transfer bindings
Yes I can confirm they do copy the bindings via binding-conveyor-fn
Ok so in this case I would use
(def ^:dynamic *id*)
(binding [*id* 123]
(do-stuff))
Thanks 🙂
But what actually are those other fns for? And why wouldn't I use with-bindings
for this?well with-bindings*
and with-redefs-fn
are functions, each with a macro providing a sprinkling of sugar
bindings provide thread local values for dynamic vars. with-redefs provides temporary root values for vars (useful in some testing scenarios but also quite dangerous for multithreaded). with-local-vars I have never used, deeply weird.
bindings is a let style macro, with-bindings provides the binding set as a map (if you're calling it with data)
bindings is far and away the most commonly used of all these
ah! great! so I guess I would use with-bindings
preferably in a macro where I already do a ns-quote? I imagine with-redefs
would in turn be useful when mocking data, replacing otherwise stateful functions and such.
nb about with-redefs
- it is not fully thread safe and if used concurrently can cause the original definition to be permanently lost due to a data race
I'm writing a edn
into a file
I want to set some custom print-methods, but just locally
Like, (pr-str v {:print-methods {java.time.Duration (fn [v] (str v))}})
But I don't want to set a global print-method
to Duration
How I do this?
someone already developed a library to write edn
in a more flexible way?!
like data.json
but for edn
I think that if you want to serialize something differently you should somehow make a custom writer that writes your data with #tag
tags. And then during your reading you should specify these tags to be read with your deserializer. I don't know how to exactly do it but it should be this way. #inst
works this way.
They are called reader tags I think.
I am not aware of a complete solution, clojure.pprint lets you set a "dispatch" function which can sort of do that
greetings! is there any way to get output of (1) in the (2)?
;; (1)
`[println]
=> [clojure.core/println]
;; (2) I want here the same output as in (1), somehow. (it will not be let, but will be an arg to a macro)
(let [x '[println]]
`~x) <--- change this to something, which will return [clojure.core/println]
=> [println]
in general the symbol println
is not at all the same as the symbol clojure.core/println
, there is no connection from them that you can just traverse
if you want to "ns-resolve" the symbol println relative to some namespace, you will get a var, and then you can get the name of the var which will be a namespaced symbol
what syntax quote is doing to the symbol println
when reading is resolving it against the current value (at read time) of *ns*
(I think) I need just fully qualified names of everything in the form I am getting as a n arg to macro
the symbol clojure.core/println
is a completely different thing from the symbol println
i need to walk the form (inside my macro), and dispatch on symbol to rearrange things, so dispatching on, say, let
would work, unless it is used as clojure.core/let
or alias/let
in the form.
so my kneejerk reaction was to just qualify all the things beforehand to avoid issues with aliases, etc.
so what you need is a map somewhere from the symbol println
to the symbol clojure.core/println
which doesn't exist, but a given namespace might have a map from the symbol println
to the var #'clojure.core/println
and you can turn the var into a fully qualified symbol
if you are doing code transformation you should also keep in mind that clojure special forms are not namespaced
yes, but backtick "handles" that for me, I think:
`let
=> clojure.core/let
`let*
=> let*
which is to say, whatever value *ns*
has when the compiler is called to compile the code will be the value it has when macroexpansion happens
depends, clojure.walk will also descend into quoted forms and other kinds of literals
yes, terms and conditions apply™ but I expected there is already a function for this :( like https://github.com/clojure/tools.reader/blob/6918abc8b3c009228790cf091097eb3037858ad6/src/main/clojure/clojure/tools/reader.clj#L694
Another, sort of related, question:
are there any tricks to turn closed predicate based cond
into open defmethod
?
Unless you can add some kind type kind of tag to the inputs, your "open" defmethod wont actually be open
oh, right, cond is sequential, and defmethod is not, and there would be N matching branches
If you trying to convert a tree walker, it is common to turn a single cond in to two or more defmethods
The first defmethod dispatching on the type: list, symbol, whatever, and those methods call a second defmethod that does type specific dispatch
yes, treewalker.
actual use case is: I need to collect form's dependencies on locals.
for example I need to figure out that nobody here needs x
, because x in (fn [x] ...) is different from x
in x 1
(let [x 1
y (fn [x] (inc x))]
(y 1))
It is the same kind of pattern the clojure compiler uses to analyze clojure code. The compiler doesn't use multimethods, but there is a Java method analyze that is passed a form, and it looks at it, and if it is a seq it calls analyzeSeq
I started the way you suggest, but then destructuring [a b & body] gives me body of the ChunkedSeq, which is an implementation detail, and I would not want to depend on it
(let [[a b & body] [1 2 3]]
(type body))
=> clojure.lang.PersistentVector$ChunkedSeq
(let [[a b & body] (range 10)]
(type body))
=> clojure.lang.LongRange
+ there is a cljs to worry about too...Yeah, you won't be able to share type names between clj and cljs, they are different
I am pretty sure if you look at the cljs compiler it does the same double multimethods thing for analysis, but does actually use multimethods for it
still, (instance? clojure.lang.ISeq body)
– is a predicate. looking how to turn it into dispatch value
user=> (defmulti f type)
#'user/f
user=> (defmethod f java.lang.Number [x] (* x x))
#object[clojure.lang.MultiFn 0x48bfb884 "clojure.lang.MultiFn@48bfb884"]
user=> (defmethod f clojure.lang.ISeq [x] (for [i x ii x] (list i ii)))
#object[clojure.lang.MultiFn 0x48bfb884 "clojure.lang.MultiFn@48bfb884"]
user=> (f 1)
1
user=> (f 2)
4
user=> (f '(2 2))
((2 2) (2 2) (2 2) (2 2))
user=> (f '(1 2))
((1 1) (1 2) (2 1) (2 2))
user=>
Is there an established standard for code generation in Clojure?
I would like to create an auto-generated library wrapper for OpenGL in Clojure, but one of my preferences about it is that I do not provide only a macro which does all the expansion at compile time but rather that code generation is an actual step in the build process to allow jump to definition as well as allowing a published artifact to be source-based but not incur the cost of generating the code at startup.
I've looked at and used rewrite-clj a little bit for working with clj-kondo hooks, but it adds a lot of complexity over what I think could reasonably just be an equivalent of clojure.pprint/pprint
that's slightly smarter about eliding namespaces in the generated data structures to make it more reader-friendly.
> Is there an established standard for code generation in Clojure? Doesn't clojure core already offer enough?
I usually don't do the code generation as part of a build step, I just generate it and check it in, maybe with the code to generate it in a comment at the bottom
there’s a really cool project called Archimedes that uses core.logic and the vendored asm lib to write a class file 🙂
my most recent generated code, it looks like I didn't bother eliding namespaces at all
Yeah, pprint is probably fine
I'm not 100% either way on if I want to check in the generated code. On the one hand it'd be real easy for the version checked in to get out of date with the generator as it's updated, and on the other hand being able to view it in github is useful.
@suskeyhose fwiw, I generate some code for rewrite-clj. A step in my build/ci workflow fails if the generated code is out of date.
That makes sense, but I haven't touched any of github actions' stuff and I'm not really planning on starting now for just an opengl wrapper
For me it is just part of a babashka task. Happens to be also invoked by CI, but also invoked by me.
Right, I'm going for a deps/prep step for git dependencies and hand-invoked by me whenever I want to publish a clojars version
Would generated sources be deterministic? I think it would require some tinkering.
Just to prevent version mismatch between the data spec (which will be in VC) and the generated code.
You need to set a logging implementation and ensure that it has stdout (or whatever your repl is) as one of the appenders.
So STDOUT must not be the repl's out, but I'm not sure how to figure out what that is
if you started your project and hosted like a repl port with nrepl and then did a cider connect or something then the stdout of your program is the stdout of the original process, not the cider connection
This project doesn't have a built-in nrepl server, I've just used cider's jack-in to connect
I recently released cascade, a lib that provides a bunch of CPS versions of Clojure's seq operations. here's a cool example of converting a non-tail recursive algorithm into a tail-recursive trampolined version https://github.com/lilactown/pyramid/commit/21a5829dcb9d912c06bc7aa34decf478c87ed9a3?diff=split it felt quite easy all in all 😄
I’m pretty inexperienced with app deployment and have never done it in Clojure before. I started working on an idea for a language-learning related web app that requires a database (but not user-editable; it’s essentially a dictionary). What is the recommended tech stack & PaaS for something like this, backend-wise?
Heroku seems to offer Clojure support out of the box, but prefers/requires(?) lein
and MongoDB Atlas. For this project, I’m trying the deps.edn
approach for the first time so I kinda groan at the thought of restarting the project with lein
. Also, I found XTDB, which seems to be a lot simpler to deal with than MongoDB, but I’m not sure how it would work exactly. Would I generate the data store locally, bundle it with the .jar
and deploy that?
You can get it done with deps.edn
(you still need a token project.clj
file) but it is a bit difficult figuring it all out. Check out my repo here to see how I eventually got it done: https://github.com/Chase-Lambert/clojure-ai
Thanks! I’ll have a look at it later. Good to know it’s possible (whether it’s worth it is another question, I suppose)