Fork me on GitHub
#clojure
<
2022-01-05
>
emccue01:01:04

Maybe a silly question, but why does this api use File instead of Pathhttps://clojure.github.io/tools.build/clojure.tools.build.api.html#var-uber

Alex Miller (Clojure team)02:01:42

it doesn't - all paths are strings there

Alex Miller (Clojure team)02:01:13

or do you mean in the conflict handler?

Alex Miller (Clojure team)02:01:16

if so, because a) a File is what is in hand and, the things useful to do are things like slurp, which takes a File (and not currently a Path), and c) it really is a File on disk, not just a path

emccue02:01:08

i meant the conflict handler

seancorfield02:01:10

Er, that is a conflict handler...

emccue02:01:21

no i’m responding to alex

seancorfield02:01:33

OK, gotcha. Hence my response 🙂

seancorfield02:01:10

Yeah, it's the only thing that takes File and it's just the natural API at that point in the (uber) process flow.

emccue02:01:11

and yeah - i can see it being more useful with the apis that clojure has. The “File” class though is just a representation of a path, it doesn’t imply one way or the other that a file is actually there - hence there being a toPath

seancorfield02:01:49

Oh, is this for your idiomatic Java wrapper project?

emccue02:01:17

the most complex part of the api (from a translation perspective) is uber on account of conflict handlers being able to carry through arbitrary state

seancorfield02:01:46

AFAIK, my log4j2 plugins merger is to only conflict handler out there -- tools.build's built-in stuff handles (almost) everything you would need I think.

emccue02:01:00

so the example conflict handler helps greatly^

seancorfield02:01:15

(and my handler is just a translation of the Maven plugin that does this)

seancorfield02:01:47

log4j 3.x should do away with that stuff, apparently.

emccue02:01:42

actually - do you know if the :state map is shared between conflict handlers?

emccue02:01:02

like a conflict handler could potentially see the state of a previous one?

emccue02:01:49

also looking at your code there it seems like you are using a regex as a key

(def log4j2-conflict-handler
  {#"^META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat$"
   log4j2-plugin-merger})
I don’t think that is guaranteed to work
The conflict handlers are a map of string regex pattern
to:
  a keyword (to use a built-in handler) or
  a symbol (to resolve and invoke) or
  a function instance

Default conflict handlers map:
  {"^data_readers.clj[cs]?$" :data-readers
   "^META-INF/services/.*" :append
   "(?i)^(META-INF/)?(COPYRIGHT|NOTICE|LICENSE)(\\.(txt|md))?$" :append-dedupe
   :default :ignore}

seancorfield02:01:54

The state is global. The expectations is each handlers uses a unique key to hold its data across the process.

seancorfield03:01:47

Well, tools.build itself builds a hash map with java.util.regex.Pattern keys.

emccue03:01:21

ah, then that should probably be a docs update

seancorfield03:01:04

Whilst I should probably have a String there, the keys in the map end up being either :default or the result of re-pattern: https://github.com/clojure/tools.build/blob/master/src/main/clojure/clojure/tools/build/tasks/uber.clj#L237

seancorfield03:01:24

And re-pattern is a no-op when passed java.util.regex.Pattern

seancorfield03:01:53

So the docs are correct. My code is "wrong" but works.

seancorfield03:01:52

(well, I guess, strictly, you could argue the docs should say "a map of string regex pattern or actual regex pattern to...")

seancorfield03:01:36

But I'd be happy to fix my handler if tools.build validated that it has a map of string to ...

Joshua Suskalo16:01:27

Isn't there also issues with the Pattern equality semantics for use as map keys?

Joshua Suskalo16:01:26

user=> (= #"blah" #"blah")
false

Joshua Suskalo16:01:13

They use object identity and not pattern equality, which means it's difficult at best to retrieve items put into a map with a regex key unless it's only used for iteration.

Alex Miller (Clojure team)16:01:56

which I believe it is in this particular case in the bowels of the uber task

Alex Miller (Clojure team)16:01:19

but that's my dirty laundry, not public api here (which says strings)

seancorfield17:01:10

I've updated my conflict handler to use the string instead of the Pattern and will cut a new release either if log4j gets updated or tools.build starts to validate that 🙂

emccue17:01:59

It feels strange to check a linear sequence of regexes that come from an unordered container, but considering how rare this must be to want to do and the fact that maps start as array maps, I doubt the project that would run into unexpected behavior on account of that has yet been born

emccue17:01:15

(and sorting the regex strings internally would make it deterministic, just not user defined which would probably be enough)

Alex Miller (Clojure team)17:01:25

it's probably a bad choice in that code, for sure

Alex Miller (Clojure team)17:01:25

but I think it's unlikely that more than one regex will apply so unlikely to be a problem

Al Z. Heymer13:01:38

What is the main usage difference between with-redefs with-redefs-fn with-bindings with-bindings* and with-local-vars ? I was trying to provide some logging meta-information, such that I can log anything with a prefixed id without parameter-chaining. But I'm struggling to figure out which would be the most suitable approach to that. I think ideally something like

(def ^:dynamic *id* nil)
(with-redefs {*id* 123}
  (do-stuff))

Al Z. Heymer13:01:50

And does it make a difference if I pmap something in do-stuff? It says thread-local, so do I lose *id*?

rickmoynihan13:01:53

Yes, it will revert to the top level binding (if that’s what you mean by lose i.e. here it will be nil, but if you’d done (def ^:dynamic *id* 456) it would revert to 456. If you don’t want that, use bound-fn to capture the bindings into your pmap threads.

Alex Miller (Clojure team)13:01:10

I think you really want bindings not any of those, and pmap uses futures I believe which should transfer bindings

1
rickmoynihan13:01:24

Yes I can confirm they do copy the bindings via binding-conveyor-fn

Al Z. Heymer13:01:29

Ok so in this case I would use

(def ^:dynamic *id*)

(binding [*id* 123]
  (do-stuff))
Thanks 🙂 But what actually are those other fns for? And why wouldn't I use with-bindings for this?

rickmoynihan13:01:28

well with-bindings* and with-redefs-fn are functions, each with a macro providing a sprinkling of sugar

Alex Miller (Clojure team)14:01:35

bindings provide thread local values for dynamic vars. with-redefs provides temporary root values for vars (useful in some testing scenarios but also quite dangerous for multithreaded). with-local-vars I have never used, deeply weird.

Alex Miller (Clojure team)14:01:29

bindings is a let style macro, with-bindings provides the binding set as a map (if you're calling it with data)

Alex Miller (Clojure team)14:01:56

bindings is far and away the most commonly used of all these

Al Z. Heymer14:01:08

ah! great! so I guess I would use with-bindings preferably in a macro where I already do a ns-quote? I imagine with-redefs would in turn be useful when mocking data, replacing otherwise stateful functions and such.

noisesmith17:01:19

nb about with-redefs - it is not fully thread safe and if used concurrently can cause the original definition to be permanently lost due to a data race

souenzzo19:01:19

I'm writing a edn into a file I want to set some custom print-methods, but just locally Like, (pr-str v {:print-methods {java.time.Duration (fn [v] (str v))}}) But I don't want to set a global print-method to Duration How I do this?

hiredman19:01:46

you cannot 😞

souenzzo19:01:08

someone already developed a library to write edn in a more flexible way?! like data.json but for edn

Martynas M19:01:29

I think that if you want to serialize something differently you should somehow make a custom writer that writes your data with #tag tags. And then during your reading you should specify these tags to be read with your deserializer. I don't know how to exactly do it but it should be this way. #inst works this way.

Martynas M19:01:12

They are called reader tags I think.

hiredman19:01:49

I am not aware of a complete solution, clojure.pprint lets you set a "dispatch" function which can sort of do that

misha19:01:17

greetings! is there any way to get output of (1) in the (2)?

;; (1)
`[println]
=> [clojure.core/println]

;; (2) I want here the same output as in (1), somehow. (it will not be let, but will be an arg to a macro)
(let [x '[println]]
  `~x)   <--- change this to something, which will return [clojure.core/println]
=> [println] 

hiredman19:01:40

depends what you mean

hiredman19:01:32

in general the symbol println is not at all the same as the symbol clojure.core/println , there is no connection from them that you can just traverse

hiredman19:01:49

if you want to "ns-resolve" the symbol println relative to some namespace, you will get a var, and then you can get the name of the var which will be a namespaced symbol

misha19:01:30

(I think let does not really illustrate, ... or maybe it does)

hiredman19:01:21

what syntax quote is doing to the symbol println when reading is resolving it against the current value (at read time) of *ns*

misha19:01:32

(I think) I need just fully qualified names of everything in the form I am getting as a n arg to macro

hiredman19:01:40

which is how it ends up with clojure.core/println

hiredman19:01:22

the symbol clojure.core/println is a completely different thing from the symbol println

misha19:01:20

i need to walk the form (inside my macro), and dispatch on symbol to rearrange things, so dispatching on, say, let would work, unless it is used as clojure.core/let or alias/let in the form. so my kneejerk reaction was to just qualify all the things beforehand to avoid issues with aliases, etc.

lilactown19:01:38

what are you trying to do?

hiredman20:01:55

so what you need is a map somewhere from the symbol println to the symbol clojure.core/println which doesn't exist, but a given namespace might have a map from the symbol println to the var #'clojure.core/println and you can turn the var into a fully qualified symbol

hiredman20:01:58

if you are doing code transformation you should also keep in mind that clojure special forms are not namespaced

misha20:01:29

but within macro, *ns* would be bound to the macro's ns, right? not the call site ns?

misha20:01:17

yes, but backtick "handles" that for me, I think:

`let
=> clojure.core/let
`let*
=> let*

hiredman20:01:29

it will be the ns of the expansion site

hiredman20:01:03

which is to say, whatever value *ns* has when the compiler is called to compile the code will be the value it has when macroexpansion happens

misha20:01:06

so do I just clojure/walk with a manual lookup in *ns* aliases,etc?

hiredman20:01:41

depends, clojure.walk will also descend into quoted forms and other kinds of literals

misha20:01:40

Another, sort of related, question: are there any tricks to turn closed predicate based cond into open defmethod?

hiredman20:01:04

Unless you can add some kind type kind of tag to the inputs, your "open" defmethod wont actually be open

misha20:01:09

oh, right, cond is sequential, and defmethod is not, and there would be N matching branches

hiredman20:01:35

If you trying to convert a tree walker, it is common to turn a single cond in to two or more defmethods

hiredman20:01:43

The first defmethod dispatching on the type: list, symbol, whatever, and those methods call a second defmethod that does type specific dispatch

hiredman20:01:05

E g dispatching on the first element of a list

misha20:01:13

yes, treewalker. actual use case is: I need to collect form's dependencies on locals. for example I need to figure out that nobody here needs x , because x in (fn [x] ...) is different from x in x 1

(let [x 1
      y (fn [x] (inc x))]
  (y 1)) 

hiredman20:01:44

Yeah, a double defmethod will do that

hiredman20:01:11

It is the same kind of pattern the clojure compiler uses to analyze clojure code. The compiler doesn't use multimethods, but there is a Java method analyze that is passed a form, and it looks at it, and if it is a seq it calls analyzeSeq

misha20:01:29

I started the way you suggest, but then destructuring [a b & body] gives me body of the ChunkedSeq, which is an implementation detail, and I would not want to depend on it

(let [[a b & body] [1 2 3]]
  (type body))
=> clojure.lang.PersistentVector$ChunkedSeq

(let [[a b & body] (range 10)]
  (type body))
=> clojure.lang.LongRange
+ there is a cljs to worry about too...

hiredman20:01:40

It is a seq

hiredman20:01:51

clojure.lang.ISeq

hiredman20:01:09

Yeah, you won't be able to share type names between clj and cljs, they are different

hiredman20:01:49

I am pretty sure if you look at the cljs compiler it does the same double multimethods thing for analysis, but does actually use multimethods for it

misha20:01:48

still, (instance? clojure.lang.ISeq body) – is a predicate. looking how to turn it into dispatch value

hiredman21:01:20

user=> (defmulti f type)
#'user/f
user=> (defmethod f java.lang.Number [x] (* x x))
#object[clojure.lang.MultiFn 0x48bfb884 "[email protected]"]
user=> (defmethod f clojure.lang.ISeq [x] (for [i x ii x] (list i ii)))
#object[clojure.lang.MultiFn 0x48bfb884 "[email protected]"]
user=> (f 1)
1
user=> (f 2)
4
user=> (f '(2 2))
((2 2) (2 2) (2 2) (2 2))
user=> (f '(1 2))
((1 1) (1 2) (2 1) (2 2))
user=>

misha21:01:05

does it work because of global type hierarchy?

hiredman21:01:32

it works because multimethods dispatch via isa? not =

misha22:01:04

thank you!

hiredman21:01:27

you use type as the dispatch function

Joshua Suskalo21:01:07

Is there an established standard for code generation in Clojure? I would like to create an auto-generated library wrapper for OpenGL in Clojure, but one of my preferences about it is that I do not provide only a macro which does all the expansion at compile time but rather that code generation is an actual step in the build process to allow jump to definition as well as allowing a published artifact to be source-based but not incur the cost of generating the code at startup. I've looked at and used rewrite-clj a little bit for working with clj-kondo hooks, but it adds a lot of complexity over what I think could reasonably just be an equivalent of clojure.pprint/pprint that's slightly smarter about eliding namespaces in the generated data structures to make it more reader-friendly.

borkdude21:01:11

> Is there an established standard for code generation in Clojure? Doesn't clojure core already offer enough?

hiredman21:01:47

I usually just use pprint

hiredman21:01:51

I usually don't do the code generation as part of a build step, I just generate it and check it in, maybe with the code to generate it in a comment at the bottom

dpsutton21:01:25

there’s a really cool project called Archimedes that uses core.logic and the vendored asm lib to write a class file 🙂

hiredman21:01:44

my most recent generated code, it looks like I didn't bother eliding namespaces at all

Joshua Suskalo21:01:09

Yeah, pprint is probably fine

Joshua Suskalo21:01:29

I'm not 100% either way on if I want to check in the generated code. On the one hand it'd be real easy for the version checked in to get out of date with the generator as it's updated, and on the other hand being able to view it in github is useful.

lread21:01:22

@suskeyhose fwiw, I generate some code for rewrite-clj. A step in my build/ci workflow fails if the generated code is out of date.

Joshua Suskalo21:01:56

That makes sense, but I haven't touched any of github actions' stuff and I'm not really planning on starting now for just an opengl wrapper

lread21:01:16

For me it is just part of a babashka task. Happens to be also invoked by CI, but also invoked by me.

Joshua Suskalo21:01:48

Right, I'm going for a deps/prep step for git dependencies and hand-invoked by me whenever I want to publish a clojars version

borkdude21:01:23

why not check the gen-ed code into source control?

Martynas M22:01:40

Would generated sources be deterministic? I think it would require some tinkering.

borkdude21:01:56

seems easier on the tooling as well (cursive, clj-kondo)

Joshua Suskalo21:01:33

Just to prevent version mismatch between the data spec (which will be in VC) and the generated code.

winsome21:01:54

I'm using clojure.tools.logging, how do I get logs to show up in my repl?

winsome21:01:29

I've tried using log-capture!, but that didn't work.

Joshua Suskalo21:01:11

You need to set a logging implementation and ensure that it has stdout (or whatever your repl is) as one of the appenders.

hiredman21:01:44

stdout may not be the same as the repl's out

☝️ 1
winsome22:01:33

<root level="INFO">
    <appender-ref ref="STDOUT" />
  </root>

winsome22:01:13

I think I'm using logback classic

winsome22:01:14

So STDOUT must not be the repl's out, but I'm not sure how to figure out what that is

winsome22:01:50

*out* looks like some sort of PrintWriter

Joshua Suskalo22:01:51

if you started your project and hosted like a repl port with nrepl and then did a cider connect or something then the stdout of your program is the stdout of the original process, not the cider connection

winsome22:01:43

This project doesn't have a built-in nrepl server, I've just used cider's jack-in to connect

lilactown23:01:11

I recently released cascade, a lib that provides a bunch of CPS versions of Clojure's seq operations. here's a cool example of converting a non-tail recursive algorithm into a tail-recursive trampolined version https://github.com/lilactown/pyramid/commit/21a5829dcb9d912c06bc7aa34decf478c87ed9a3?diff=split it felt quite easy all in all 😄

tabidots23:01:27

I’m pretty inexperienced with app deployment and have never done it in Clojure before. I started working on an idea for a language-learning related web app that requires a database (but not user-editable; it’s essentially a dictionary). What is the recommended tech stack & PaaS for something like this, backend-wise? Heroku seems to offer Clojure support out of the box, but prefers/requires(?) lein and MongoDB Atlas. For this project, I’m trying the deps.edn approach for the first time so I kinda groan at the thought of restarting the project with lein. Also, I found XTDB, which seems to be a lot simpler to deal with than MongoDB, but I’m not sure how it would work exactly. Would I generate the data store locally, bundle it with the .jar and deploy that?

Chase23:01:41

You can get it done with deps.edn (you still need a token project.clj file) but it is a bit difficult figuring it all out. Check out my repo here to see how I eventually got it done: https://github.com/Chase-Lambert/clojure-ai

1
Chase23:01:25

You need that bin/build file, the Procfile , etc.

tabidots00:01:39

Thanks! I’ll have a look at it later. Good to know it’s possible (whether it’s worth it is another question, I suppose)

Chase00:01:47

You need these buildpacks (I think in this order):

λ ~/projects/clojure/ai : heroku buildpacks
=== ai-marketing-copy Buildpack URLs
1. heroku/nodejs
2. heroku/clojure

Chase00:01:29

Search heroku and you will see some recent convos I've had about it all in September if you get stuck.