This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-01-19
Channels
- # announcements (14)
- # babashka (16)
- # beginners (81)
- # calva (18)
- # chlorine-clover (8)
- # cider (4)
- # clj-kondo (11)
- # cljdoc (4)
- # cljsrn (5)
- # clojure (163)
- # clojure-europe (30)
- # clojure-nl (3)
- # clojure-spec (7)
- # clojure-uk (4)
- # cursive (3)
- # emacs (12)
- # events (1)
- # figwheel-main (3)
- # fulcro (60)
- # introduce-yourself (3)
- # jackdaw (12)
- # juxt (3)
- # lsp (77)
- # malli (23)
- # mid-cities-meetup (9)
- # music (2)
- # off-topic (12)
- # portal (6)
- # releases (2)
- # shadow-cljs (3)
- # spacemacs (7)
- # sql (1)
- # tools-deps (2)
- # vim (3)
I have a project with a package (p) i've written in Clojure that has several modules (a.clj , b.clj, etc) and want to call it in Java. First confusion is that I have to instantiate a new Java object of the function (p.a/foo) and obj.invoke it. That works but fails when the first function tries to call a second function (p.b/bar) with the following exception.
Caused by: java.lang.IllegalStateException: Attempting to call unbound fn: #'p.b/bar
Is there some initialization of clojure and the package that needs to happen before I call a function? Why do I have to get an object and invoke to call it?Have you seen https://clojure.github.io/clojure/javadoc/? It has some examples using the API that clojure exposes for java
The way to do it is to use Clojure.var to get clojure.core/require, use require to load whatever namespace you want, then use Clojure.var to get the functions you want from that namespace
Thanks for the quick response. Your references are helpful. I've looked at other similar examples from my google searching. My message was not clear in expressing my issue. I am doing what is shown in the clojure/javadoc.
IFn plus = Clojure.var("clojure.core", "+");
plus.invoke(1, 2);
The difference is that the function I invoke calls a function in another namespace which fails with the unbound
error. The call from Java to Clojure works but the function calls a function in another file which fails. Is there some kind of Clojure runtime that is able to perform the equivalent of the Clojure.var
.invoke()?All of the examples I've seen show a function that does not reference code in another file. My simple code example is below. Java file contains:
IFn foo = Clojure.var("mod.a", "foo");
foo.invoke(1, 2);
file mod/a.clj contains:
(ns mod.a
(:require [mod.b])
(gen-class)
)
(defn foo [a b]
(mod.b/bar a b)
)
file mod/b.clj contains:
(ns mod.b
(gen-class)
)
(defn bar [a b]
nil
)
Failure occurs when foo
tries to call bar
http://Insideclojure.org might have a more indepth example (I've seen one somewhere, but don't entirely recall where)
Thanks for the reference to the gist. I will study it. I guess I expected something simpler. Calling Clojurescript from Javascript required only include extern
in the defn. I assumed it would be just as easy for Java.
Some important info on periods in keywords here. It seems to be allowed. The docs need updating.
^ perhaps something that encodes normal keywords into morse code keywords, :sos
-> :...---...
?
Making Clojure resemble some esoteric languages (like brainf*) 😂
Can you add an issue on clojure-site?
Awesome sleuthing @U04V15CAJ!
there's some web related lib (I forget which) that uses keywords like :
in tests
data.xml does some interesting stuff like that :)
but I cannot deny that I also encountered this in a project which keywordized RDF URIs 😂
the issue we ran into with clj-kondo analysis was that we stored info about keywords, like this, given keyword :1.9
, {:name 1.9 ;;<- symbol ...}
so when you would read back in that EDN, then the name value would not be a symbol. We should have stored this as a string anyway, but that's when an "invalid" keyword tripped us up.
Amazing. I just had to try this with babashka.
$ bb -cp src -e "((requiring-resolve 'kmorse.core/keyword->morse) :foobar)"
:..-._---_---_-..._.-_.-.
Were it my call, I'd probably name the repo remorse
. :D Only because those keywords look terrifying.
@U051N6TTC that reminds me of my encoder that disguises a plaintext to look like homerow keyboard mashing https://gist.github.com/noisesmith/561c215ffa39f71645b5
user> (base-_SEMI_lkjfdsa-encode "hello")
";ddks;jjfddas;;;"
Hey everyone! Is it possible to write a "higher-order" transducer which takes another transducer and transform a specific key in a map with it. For example, I want to calculate a moving average in a series of maps like this:
(def values
[{:ix 0, :val 0.1009376937865758}
{:ix 1, :val 0.4809050647549886}
{:ix 2, :val 0.9892799406144329}
{:ix 3, :val 0.017281836069744982}
{:ix 4, :val 0.8858032296740233}
{:ix 5, :val 0.9732218596402712}
{:ix 6, :val 0.4625438903362259}
{:ix 7, :val 0.9466645843710969}
{:ix 8, :val 0.5040162806448868}
{:ix 9, :val 0.7839764149216033}])
(require '[net.cgrand.xforms :as x])
;; transducer to calculate ma
(defn ma [n]
(x/partition n 1 x/avg))
;; get raw ma
(sequence (comp
(map :val)
(ma 5)) values)
;; => [0.49484155297995314 0.6692983861506921 0.6656261512669397 0.6571030800182724 0.7544499689333009 0.7340846059828168]
;; what I want is
[{:ix 4 :val 0.49484155297995314}
{:ix 5 :val 0.6692983861506921}
{:ix 6 :val 0.6656261512669397}
{:ix 7 :val 0.6571030800182724}
{:ix 8 :val 0.7544499689333009}
{:ix 9 :val 0.7340846059828168}]
Something like:
(defn higher-order-update
[key xfrom]
(fn [rf]
(fn
([] (rf))
([result] (rf result))
([result input]
(rf result (assoc input key
(get input key) ;; <-- stick xform somwhere here???
))))))
Can't wrap my head around this...(defn map-key
[k xform]
(let [xf (xform (completing #(assoc %1 k %2))]
(fn [rf]
(fn
([] (rf))
([result] (rf result))
([result input]
(rf result (xf input (get input k))))))))
I haven't tested this at a repl, but this is approximately correct I think. It should be checking for reduced
and similar though
plus it should call the completing part of xf too
checking for reduced
like this?
(defn- multiplexable
"Returns a multiplexable reducing function (doesn't init or complete the uderlying rf, wraps reduced -- like preserving-reduced)"
[rf]
(let [rf (ensure-kvrf rf)]
(kvrf
([])
([acc] acc) ; no init no complete rf
([acc x]
(let [acc (rf acc x)]
(if (reduced? acc)
(reduced acc)
acc)))
([acc k v]
(let [acc (rf acc k v)]
(if (reduced? acc)
(reduced acc)
acc))))))
yeah. If you give me a couple minutes I'll have access to a computer and can give a more complete and correct example
Thank you very much! I will try it out tonight !
Okay @UTTHNCF5L I've got a couple minutes to work on this and I have a question now: how should this act when the transducer produces multiple values, or none?
Or is this a very limited usecase transducer that requires that xform
be a stack that produces one value for each result with an optional reduced?
result at the end?
I think my tranducers are expected to produce exactly one value per map in the series. But in more general case I think we should just duplicate items from the series or omit them.
For example if xform is going to be (filter #(> % 0.5)
than just omit all the maps where the :val
is less than 0.5
So is there a particular reason you're looking for a transducer that operates on maps rather than doing something like this? Just to make it a little shorter?
(filter (comp #(> % 0.5) :the-key))
(defn key-xf
[k xform]
(fn [rf]
(let [xf (xform conj)
last-map (volatile! nil)]
(fn
([] (rf))
([result]
(try
(rf (if (nil? @last-map)
result
(reduce #(unreduced (rf %1 (assoc @last-map k %2)))
result
(xf []))))
(finally
(vreset! last-map nil))))
([result input]
(vreset! last-map input)
(let [res (xf [] (get input k))]
(cond-> (reduce #(rf %1 (assoc input k %2))
result
(unreduced res))
(reduced? res) ensure-reduced)))))))
There's a more complete version of ithandles reduced, and when the xf produces one value or more than one. It handles producing more than one value by adding the same map multiple times with different values for the key
Works like a charm! Thank you very much once again!
> So is there a particular reason you're looking for a transducer that operates on maps rather than doing something like this? Just to make it a little shorter? There might be two reasons. First is to decouple the logic of filtering values from the specific position inside the map. Second, most of the time transducers need to map something (and often do aggregations like with moving average example)
Sure. Just be careful with aggregations because as I said, only the map paired with the last item before it emits an item will be assoc'ed into. Which won't be a problem if you're just keeping an aggregate and emitting on each input, but with e.g. partition-all it acts a little strangely.
glad I could be of help though
so wonder if someone can help, I have logback installed with tools.logging but it seems like tools.logging is not picking up logback if i run (log-impl/find-factory) I can see it lists org.slf4j.helpers.NOPLoggerFactory which seems wrong to me, but I don't get how I change them / it decides to use that (I am not familiar with java logging). so perhaps I am missing something I have logback.xml in my resources folder with a basic config so whats my next step ?
Are you passing in -Dlogback.configurationFile=resources/logback.xml
as a JVM option when you run your app?
In my projects, I definitely don't specify it explicitly anywhere - logback.xml
is just picked up from the classpath.
yeah I just read that is the case and the other option is if you want to override for say prod, not sure how i can confirm if its even picking up the file
I do get this when I launch SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". which I have read should go when you import logback-classic into your dependency list
yes I do get a file back, that is now going at the start of my app with a prn so I know which file its loading 🙂
I also notice that when launching my app I get this,
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See for further details.
"file:/var/www/beanbag/workspace/projects/office-api/resources/logback.xml"
MLog initialization issue: slf4j found no binding or threatened to use its (dangerously silent) NOPLogger. We consider the slf4j library not found.
Jan 19, 2022 2:23:00 PM com.mchange.v2.log.MLog
INFO: MLog clients using java 1.4+ standard logging.
Jan 19, 2022 2:23:00 PM com.mchange.v2.c3p0.C3P0Registry
INFO: Initializing c3p0-0.9.5.4 [built 23-March-2019 23:00:48 -0700; debug? true; trace: 10]
after that last info all logging stopsI import tools.logging into my app should i also be importing logback even if I am not using it directly ?
You mean like via a (:import ...)
clause in a namespace declaration? I don’t think so.
I wonder if there’s a typo somewhere in your logback.xml that is blowing up the xml parser and resulting in the “Failed to load class” error?
yeah, I guess the bit that bothers me is the magic part where its supposed to just use logback some how seems the magic is failing somewhere 😕
What's your dependency list? Usually logback-classic + c.t.logging + logback.xml in the classpath does the job. In some cases you need to add org.slf4j/slf4j-api
Maybe you could use clojure.data.xml to slurp and parse the logback.xml file, just to make sure it’s not blowing up during the parsing.
(related article giving an overview just in case you haven't seen it: https://lambdaisland.com/blog/2020-06-12-logging-in-clojure-making-sense-of-the-mess)
Yeah, Java logging is kind of a mess, and lucky us, we get to inherit the mess. 😕
ch.qos.logback/logback-classic {:mvn/version "1.2.10"}
org.slf4j/slf4j-api {:mvn/version "1.7.33"}
org.slf4j/jul-to-slf4j {:mvn/version "1.7.33"}
org.slf4j/jcl-over-slf4j {:mvn/version "1.7.33"}
org.slf4j/log4j-over-slf4j {:mvn/version "1.7.33"}
org.slf4j/osgi-over-slf4j {:mvn/version "1.7.33"}
I did not have osgi so added thatI will try reading the xml file, I will be annoyed if it just ignores an invalid xml file with out some kind of warning though
Your logging output above shows that some thing called MLog
is being used. I don't know for sure but my hypothesis is that it's being picked up instead of Logback. I'd track that dependency and try to exclude it.
Are you sure that when you're getting that log output with MLog
you are not using some aliases that might affect the classpath?
The fully qualified class name that's being mentioned is com.mchange.v2.log.MLog
.
So, try using (io/resource "com/mchange/v2/log/MLog.class")
.
Perhaps it’s bundled into c3p0? :thinking_face:
yeah https://stackoverflow.com/questions/5686619/turning-off-logging-for-hibernate-c3p0/5698667 it mentions logback in the answer but does not have a full example
I get this back when dumping the mlog resource
file:/home/oly/.m2/repository/com/mchange/mchange-commons-java/0.2.15/mchange-commons-java-0.2.15.jar!/com/mchange/v2/log/MLog.class
at least the documentation for c3p0 claims compatibility with slf4j & logback (https://www.mchange.com/projects/c3p0/#configuring_logging). I'd suggest looking closely at the dependency tree and verifying you see the correct version of logback-classic
there
com.mchange.v2.log.MLogInitializationException: slf4j found no binding or threatened to use its (dangerously silent) NOPLogger. We consider the slf4j library not found
yeah, as long as the versions of slf4j and logback are compatible
> Note that slf4j-api versions 2.0.x and later use the https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html mechanism. Backends such as logback 1.3 and later which target slf4j-api 2.x, do not ship with `org.slf4j.impl.StaticLoggerBinder`.
e.g. ^^ that might be a problem if your project somehow loads a different newer version of logback rather than the one you specified above. But this is just a hypothetical guess
I was not getting these before INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to ERROR
anyway thanks for all the help guys I think its solved now, the app even launches with these messages at the start now
15:14:23,706 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
15:14:23,706 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/var/www/beanbag/workspace/projects/office-api/resources/logback.xml]
Has anyone here ever seen an actual bug that would have been prevented (or was fixed) by declaring a function with defn-
instead of defn
?
My theory is that defn-
is a holdover from OOP, and adds no practical value to the art of functional programming.
I’m mostly using it to mark things in my ns which should not be relied upon, mainly “don’t use that, it might change” (e.g. not a part of the API)
OOP generally emphasizes private stuff - i will disagree though. It still makes sense to have a public and private api for things
(defn- length-helper [collection n]
(if (empty? collection)
n
(recur (rest collection) (inc n))))
(defn length [collection]
(length-helper collection 0))
basic example - but its pretty easy to have functions in a namespace that dont make sense to expose and are really implementation details
Ok, I’ll buy the use case where you’re publishing an API and you want to distinguish between what’s “official API” and what’s “internal implementation detail subject to change.” My beef is with seeing it used in standalone proprietary app code where it complicates things without adding any value.
Our standalone proprietary app code also has some kind of internal API. I liked @U3JH98J4Rs example. If we have a ns with a lot of such functions which are only used internally to clean things up, the devs might be happy to not have them provided by the code completion tools. But I get that this is more opinion based
just checked, and there are 114 private vars in clojure.core
. Thought it was an interesting datapoint
clojure.core
is the exception that proves the rule -- that there's no point in defn-
for 99% of cases. Instead use namespaces to separate API from implementation and make everything public
clojure.set
has a single private function. Would seem strange for there to be a clojure.set.impl
with a single function and only one namespace that would call it. That seems far worse than a little metadata marking something private
I first learned about namespaces (packages) in CL. There is no access control enforcement mechanism there, so I don't consider them as being OOP related at all.
I find defn-
useful. Like @U0N9SJHCH mentioned, it helps to distinguish internal from external.
For maintenance, it also helps me to more easily find and obliterate unused code, although clojure-lsp now helps with finding unused public things too.
There is also the ^:no-doc
convention to mark non :private
vars and nses as internal. I use this too.
I like to signal intent with
(alter-meta! #'your-var assoc :private false)
I hope you documented why that function was private@U025L93BC1M Smalltalk uses the same convention - you can categorize methods as private purely to the inform the user of the class, no enforcement
@U0CKDHF4L ha, bold!
@U11BV7MTK clojure.core
does many things which mortal programmers don't need to do, either for technical or historical reasons
I think calling defn-
a holdover from oop is like calling spec a holdover from static typing.
Its usecase is to allow you to implement your api, the namespace's public functions, in a way that holds a stable interface even as you improve the internals both in style, readability, performance, and other ways.
Otherwise I think yes using defn
over defn-
does cause "bugs" in a sense, where a library dev either has to keep old helper functions around etc for fear of breaking callers, or they do break some small subset of callers by updating their internals. Either way is not ideal when the callers don't have a clear signal that it's internals subject to change.
You could put internals in an impl namespace, but I think that only makes sense when you have a lot of stuff to put in there or for tests and similar. A lot of smaller helpers get tested implicitly by testing the public api though, so in many cases it's just nicer to stick the helper next to where it's used, and in those cases defn-
is appropriate.
another way is to make everything implicitly (public) impl and include an .api
namespace for the api
I use it to signal intent 🙂
I like ^:private instead, it signals intent with a bullhorn
If I don't want consumers to use it, it goes in an impl namespace, otherwise I'm just making work for myself if I want to test it, too
Luckily, it’s not THAT much work.
=> (clojure.core/spread [:a :b [:c]])
Syntax error (IllegalStateException) compiling clojure.core/spread at (REPL:1:1).
var: #'clojure.core/spread is not public
=> (#'clojure.core/spread [:a :b [:c]])
(:a :b :c)
Why does this work? In my mind it goes symbol -> var -> value. This passes var instead of value, but that has not be looked up as some point, right? Or is #'foo a different var than the var which foo resolves to?
after thinking, it’s probably because the meta check happens when resolving the symbol -> var, right? when you access the var directly this is obviously skipped.
I think Alex will have a better answer, but I’ll present what I know.
clojure.core/spread
is a symbol. When eval
-ed then it is looked for in the namespace. This lookup will find a Var
object, but at this point it will see that the object referenced by the Var
is marked as private, and an exception will be thrown. This happened in the .resolve
part of the evaluation, and before it looks at the Var
. If it weren’t private, then the var returned would contain the IFn
and the eval
usually continues by doing that dereference.
#'clojure.core/spread
is a reader macro that is equivalent to (var clojure.core/spread)
. Evaluating this creates the var object, and looks it up in the current namespace, but importantly, it uses an internal method called .lookupVar
and does not use the .resolve
method. Both of these methods lookup the value of the var from the namespace, but only .resolve
checks the privacy flag.
The resolveIn
code is at: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L7395
While lookupVar
is at: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L7475
thanks, think that sort of matches where I ended up when thinking it through. always nice 🙂
also interesting to look at the compiler code. never dived in before but this seems fairly approachable.
seems there’s also a path to accessing a private var by calling (clojure.lang.Compiler/resolveIn ns sym true)
. Neat!
Another way to really get inaccessible helper functions (well, you get still get to them if you're a hacker):
(let [private-fn (fn ...)]
(defn public-fn-1 [] (private-fn ...))
(defn public-fn-2 [] (private-fn ...)))
but why....? Just don't ;).I am used to updating functions live when the app is running and having them change behavior. For some reason, updating functions f
added to tap with (add-tap f)
does not seem to work. I need to restart the app.
Is there a workaround?
Will try 😄
It's described here: https://clojure.org/guides/repl/enhancing_your_repl_workflow#writing-repl-friendly-programs
I assumed it had something to do with pointers. Will read it :thumbsup:
As a short description, when you write (add-tap f)
what's happening is it's looking up the var f
and fetching its current value, which is a function, and passing that value to add-tap
.
Then when you update the function what's happening is you're making a brand new function and assigning that value back to the var f
, but this doesn't update the tap because it doesn't look up the var again, it just uses the returned value.
When you pass #'f
then you're passing the var itself to add-tap
, which only works because vars implement the IFn
interface by looking up the value in the var and calling it as a function. If that weren't a part of vars, this wouldn't work.
Fun question perhaps, even if I think the answer will be no: Can I add many jars to the classpath as one entry, without making an uberjar?
you could add a single jar that has nothing but a meta-inf that includes a bunch of other relative paths to other jars to add
what's the gist of this solution? I took a look at https://s3.amazonaws.com/athena-downloads/drivers/JDBC/SimbaAthenaJDBC-2.0.27.1000/AthenaJDBC41_2.0.27.1000.jar but found various stuff, not sure of what is the one using the technique you refer to
I'd observed it on the redis jar @U45T93RA6
I know this is how e.g. the amazon redis jdbc driver jars declare a dependency on log4j
you can use a Class-Path
manifest attribute (was designed for applets)
this is sufficiently obscure that your colleagues may hate you for it though :)
![clojure-spin](https://emoji.slack-edge.com/T03RZGPFR/clojure-spin/aea1be92af1f8856.gif)
I have a pattern where I'm building up a map by chaining some initial input though several handler functions. I have multiple function chains I need to execute on vaguely similar input data with some common steps between them. Any one of these functions can result in some error-like/termination condition, at which point I want the chain to halt and log the built up map at this point. The map being built up is pretty huge and not really ideal to have printed (but small enough where I want to log it entirely).
In this case, what do you prefer (or what factors would lead you to prefer one of these):
1. throwing exceptions in the handler functions with ex-info
and include as the ex-data the built up map at that point. Then wrap the handler chain in a try/catch, and use the ex-data to log the data. This seems the most normal and obvious, I only pause because there end up being lots of throw
statements large data structures included in the ex-data. Edit: and I'm not sure how I feel about depending on the the existence of a key in the ex-data in the the thing that invokes the function chain. Is that normal?
2. Use something like https://github.com/cognitect-labs/anomalies or really just a keyword indicating an error that any of the functions can return in the map. If you use this, how do you deal with early termination in a large chain of functions? I have tried out a custom threading macro for this which checks for the error key at each step but this isn't particularly satisfying.
3. Use some library which implements the interceptor pattern? There are some lightweight ones, but this feels like overkill since the only feature i need is early termination (and not the whole enter/exit lifecycle)
4. anything else?
I've built a few data processing pipelines that sound like that, and usually end up with what is basically an either monad
where the value is either some bit of data that the pipeline operating on, or a value representing an error
and then each step of the pipeline operates on the data or passes through errors unchanged
I've never used a library or anything for that, usually just define a few helper functions
(defn just [x] {:data x})
(defn error [x] {:error x})
(defn app [f x] (if (:error x) x (f (:data x))))
@jjttjj I wrote up a little thing recently that might work for you, it decomplects mapping thrown exceptions to return values from early return points: https://clojurians.slack.com/archives/C03RZGPG3/p1642025029110800