This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-06-04
Channels
- # announcements (13)
- # asami (68)
- # babashka (19)
- # beginners (51)
- # calva (4)
- # cider (3)
- # clj-kondo (13)
- # clojure (161)
- # clojure-australia (3)
- # clojure-europe (21)
- # clojure-nl (4)
- # clojure-spec (9)
- # clojure-sweden (13)
- # clojure-uk (9)
- # clojured (14)
- # clojurescript (7)
- # conjure (4)
- # cursive (16)
- # data-science (2)
- # datalog (1)
- # figwheel-main (7)
- # graalvm (18)
- # helix (14)
- # introduce-yourself (1)
- # jackdaw (9)
- # jobs (1)
- # kaocha (1)
- # malli (8)
- # meander (8)
- # off-topic (7)
- # pathom (11)
- # proletarian (3)
- # shadow-cljs (29)
- # tools-deps (106)
- # xtdb (8)
Hi, does anyone know if and how I can extract a list of all required namespaces from a given namespace programmatically in Clojure?
@U0220KPQ5CH a namespace is either already loaded or it is not, but it doesn't matter which namespace did this, so this information isn't saved anywhere I believe. you can inspect the aliases that are valid within a namespace
This can give you some info:
user=> (binding [clojure.core/*loading-verbosely* true] (require '[clojure.set] :reload))
(clojure.core/load "/clojure/set")
nil
Thanks for the help.
I'm trying to solve a problem where I want to test if the given namespace loads the proper list, so I can have lein test
signal if something weird is going on. We're using a bit of dynamic namespace parsing, so it really matters for us which namespace loads what
user=> (with-redefs [require (fn [& args] (prn :req args))] (require 'clojure.set))
:req (clojure.set)
nil
@U05224H0W that's not sufficient here:
user=> (require '[clojure.set])
nil
user=> (ns-aliases *ns*)
{}
true, it is not 100%. most of the time things will have :as
aliases though so should be good enough
What you can also do as a test is, require a namespace and test if after doing that, the other namespaces have been loaded?
hmm yeah then it gets tricky, especially if there are some namespaces that are AOT compiled since those won't show up in regular require
logs
Hi, following my previous post. I re-did evaluations with criterium. 16ns for the java method, vs 64ns for the native clojure one. Is there any reason why clojure does not implement rand-int
that way ?
https://gist.github.com/a30660391c68255e65ff60970e706c22
Try to benchmark java version when you create seed each time. Notice that clojure version probably allocates it since it is not an input argument
Seed is expected to be done once ! If I re-init seed each time, I'll have always the same result
I donβt know why. Why does it matter?
Sorry @U064X3EF3 I don't understand. Why the performance is important? the seed?
What's the reason behind the whole thread?
1. I feel like a newbie so I may have done something fundamentally wrong 2. If that implementation is quicker and equivalent (what I think), it may be an improvement, something like :
(def ^java.util.Random seed (java.util.Random.))
(defn rand-int-alternative
"Returns a random integer between 0 (inclusive) and n (exclusive)."
{:added "1.0"
:static true}
[n] (.nextInt seed n))
If you need more context I am trying to port an operations research tool I built years ago in C++. That kind of algorithms are combinatorial optimization, with a huge number of iterations on int-array
. I start with some benchmarking between java and clojure and find out the global execution time gap was due to that rand-int
function. That kind of use case are demanding from a time execution prospective (billion of iterations are expected).
thanks, that was the context I was looking for. I don't think anything is preventing you from using your alternative method. I don't know why it's implemented the way it is. I'll make a ticket out of it when I get a chance.
yes, I agree this is not a critical issue. Thanks for your confirmation to use the 'alternative method'. It's all what I need.
I think this would be a good question for http://ask.clojure.org as well
And I followed the question to http://ask.clojure.org following your recommandation. Thx.
Procrastination alert! I can implement this operation
[{:key :foo :value 1} {:key :bar :value 2}] => {:foo 1, :bar 2}
like this
(reduce #(assoc %1 (:key %2) (:value %2)) {} coll)
or maybe
(into {} (map (juxt :key :value) coll))
but not really happy with either. Any ideas?last one is good except it could be in transducer form:
(into {} (map (juxt :key :value)) coll)
what do you think is wrong with it?
not sure. i think it has something todo with the fact this the expression is saying βadd this value under this keyβ, but in my head iβm thinking βadd as this keyβ. i suppose (juxt :key identity)
would express that. thanks!
(juxt :key idenity)
creates a different output though
whenever I find myself writing (juxt ... identity)
I reach for Medley's index-by
I have a data reader #object
defined. This works:
(def x #object[org.joda.time.DateTime 0x3ff80a66 "2021-07-04T09:54:54.937Z"])
but this doesnβt
(identity #object[org.joda.time.DateTime 0x3ff80a66 "2021-07-04T09:54:54.937Z"])
Why is that?What does #object read into ?
(defn joda-time-data-reader [[class-symbol _objid date-time-string :as value]]
(if (= "org.joda.time.DateTime" (name class-symbol))
(tf/parse date-time-string)
(throw (ex-info "Unsupported #object instance" {:value value}))))
So this should be a jodatime instanceorg.joda.time.Datetime
in what way does it "not work" ?
When I put
(def x #object[org.joda.time.DateTime 0x3ff80a66 "2021-07-04T09:54:54.937Z"])
in namespace code and load the namespace in a REPL, the value of X is as expected. But if I do something like:
(defn y [] #object[org.joda.time.DateTime 0x3ff80a66 "2021-07-04T09:54:54.937Z"])
and load that namespace I get:
Syntax error compiling fn* at (src/.../plans/presentation.clj:324:1).
Can't embed object in code, maybe print-dup not defined: 2021-07-04T09:54:54.937Z
Similarly if I just put like (identity #objectβ¦.)
in namespace top-level and try to load it, same error
seems odd that def
parses differently than other expressions
right, so the data reader is reading into an object. that object can't be represented in bytecode as a value, so you can't compile the function like that. def is different case
you can probably add a printer for this type of object though and get it to work
you do that by extending the print-method and print-dup multimethods
you then have roundtripping of read/print which are complementary operations
I see, thank you
Also if you define a print operation for these values, then it would be preferable for the reader to not read #object, but rather read something like #joda/datetime
This will prevent issues with other objects being read as date times.
Iβve just type hinted the return value of a protocol function with a class.
Itβs a common pattern in our code to call this function inside a with-open
.
However it seems this is strictly speaking a breaking change, because consuming namespaces now need to import the class that the type hint relates to, is that right?!
I know get compile errors at call sites that call (with-open [(myprotocol/open-the-thing ,,,)] ,,,)
I know about this article that advocates for passing system as a first argument for a protocol impl: https://puredanger.github.io/tech.puredanger.com/2014/01/03/clojure-dependency-injection/
as the author, I think "advocates" is a strong description - this is an idea, it may be useful, or it may not :)
There's some other article that says this for functions that does IO?
To make your IO execution to not depend on global variables and to pass the dependency in a explicit way (a.k.a. as a first argument of a function)
I've been programming a lot in this way since I began on Clojure and "it just felt right", how can I explain that it gets your code better more than simply saying that It's easier to debug / It's easier to mock?
To your question about how to explain the value proposition of passing arguments vs. depending on globals: It sounds like you're trying to convince colleagues about the benefits of pure functions. Unfortunately, you'll probably have to tailor your arguments to your audience so your pitch will resonate with their specific experiences. Some arguments you could you could be: parallelization of tests - without global state you can safely parallelize your tests. transparency - global state is an additional, silent, side-channel input. It doesn't come through the front door like other function arguments. So the developer who picks up the test/code needs to know all of the relevant context to figure out how the function will behave. If you use pure functions and you know that the behavior only depends on the arguments supplied to the function, I've found it's much easier to understand and easier for other developers to understand.

Yeah, some form as component system, can be a map of system stuff (as right now I'm using https://github.com/piotr-yuxuan/closeable-map to make the system component)
Note that component itself discourages it: https://github.com/stuartsierra/component#do-not-pass-the-system-around
the idea is that instead of my-insertion-db
receiving system
it receives the datasource
itself?
Just the public APIs generally needs the system-map, other private functions downstream generaly will get only what it needs
yeah, I generally agree on pass only what the function needs, but destructuring helps a lot with this on most part of the time π
(although I have to admit that this is quite a common pattern in the codebase I'm working on π )
yeah, but right now I'm not using component itself, if I have to make IO to a database I can do something like
(defn my-insertion-db! [{::config/keys [datasource]} data]
(log "some data!" (pr-str data))
(->> data
some-transformation
(jdbc/execute! datasource))
and for testing I can do something like
(with-open [system (-> config make-dev-system!)]
(my-insertion-db! system data))
I think is more simple / clean to have just a map π
No function should take the entire system as an argument => I know that sometimes you would prefer to pass part of the system (the part that your IO needs to use to handle it) as the argument of your function
but destructuring seems to fit very well for these stuff
but the thing I don't like is to have the system as a global variable (people are arguing that we need to make functions that has global vars embedded and returns a function that does the IO, because is simple to mock using with-redefs
)
but there's an article talking about this on clojure?
I'm struggling to find this on clojure π, it has for Elixir though https://dashbit.co/blog/mocks-and-explicit-contracts
because polutes functions to pass around system for every stuff (I've said not every stuff, just IO stuff haha)
but there's an article talking about this on clojure?
I'm trying to use Apache Tika to detect a file type. I'm getting some reflection errors I can't seem to figure out, and I haven't done much Java interop prior to this. Here's what I have:
(with-open [xin ( (get-in params [:uploaded_file :bytes]))
tika (Tika.)
ftype (org.apache.tika.Tika/.detect tika)]
(prn "tika ---->" ftype))
So the first complaint is 'reference to field close can't be resolved' and then 'reference to field detect on org.apache.tika.Tika can't be resolved'
I assume this is either an import issue or a namespace issue
That was my first attempt, and I got: 'No matching field found: close for class org.apache.tika.Tika'
I'm conflating 2 issues, probably because I tried a plain 'let' in place of 'with-open' to see if I could get past the 'close' error
Oh, silly me. A separate 'let'
And now it cannot resolve the 'detect'...
Which is why I tried the full namespace... let's try that again...
Hmm. So the 'close' is resolved, but not 'detect'
Got it!
(.detect tika xin)
Does anyone know a file-system backed TTL cache? I'm communicating with an API with limits with a process that will stop/start on my machine. I've not found a java thing surprisingly!
@dominicm Do you mean for http caching, etags etc?
@rickmoynihan I can hit an api 20 times a day, and I only want to do that every hour to make sure I don't deplete that. I want to make sure while I'm developing I don't deplete it either by restarting my jvm a few times.
something like https://github.com/oliyh/martian/tree/master/vcr ?
There are some other similar projects βvcrβ will probably help you mind them.
@rickmoynihan vcr is more about testing. I am looking for something that expires after 1 hour, and to fetch a new one when that happens.
could you use varnish as a proxy?
@borkdude I mean, if I'm gonna roll my own I'll probably just use duragent and the local-file thing π
I am not sure I understand why it works: ns A has a macro that outputs a call to another macro from ns B ; ns C calls that macro but does not require ns B ; all is cool
(defmacro macro-a [x] `(B/macro-b ~x))
when macro-a expands it will make the symbol B/macro-b fully qualified
and if the namespace with that symbol is loaded by any other namespace it will be resolved correctly
Right, I might have been confused on that last point in my CLJC project. Indeed, ns B was being loaded in Clojure but not in CLJS, so it was misbehaving in CLJS. Make sense!
Just being lazy instead of translating this but anyone know of a clojure bigdec natural log implementation in the wild, like this java one https://stackoverflow.com/questions/739532/logarithm-of-a-bigdecimal
this relation seems really handy: log(AB) = log(A) + log(B)
. This might get into things like numerical stability and errors related to that
Is there a way to have the std{out,err}
from shell/sh
sent somewhere in real-time? Now I only get :out
and :err
info after the job is done π
I believe Java ProcessBuilder class can enable you to do that. I don't think that the built-in-to-Clojure shell/sh can do that.
yeah I have a snippet doing that with ProcessBuilder:
(let [pb (ProcessBuilder. ["lua"])
p (.start pb)
out (.getOutputStream p)
in (.getInputStream p)]
(spit out "print \"hello from lua\"\n")
(slurp in))
very straightforward, compared to most java apisof course for real interactivity you have to use something other than slurp / spit (and that means more complexity) - I'm sure there's some good libs that simplify that streaming IO stuff out there
sounds like you want the stdout / stderr for monitoring? you should be able to .read
from the output stream and error stream in a loop and trigger a callback for each line / chunk / whatever
you can also use processbuilder to stream to a file or pipe instead of a stream, and to join the streams if that simplifies your use case as well
Thanks to both of you π
I am using my software to run multiple (shell) jobs concurrently and want output from them ASAP and continually so I can see if anything goes wrong. So yes, I guess monitoring is the word.
@endrebak Example with babashka.process (which you can use as a regular Clojure library):
(require '[babashka.process :refer [process]])
(process ["foo"] {:inherit true})
(process ["bar"] {:inherit true})
.. wait .. e.g. @(promise)
they mentioned they don't want the output to go to stdio of the repl
> and want output from them ASAP and continually so I can see if anything goes wrong oh, I thought he/she/they did say that here
But if multiple jobs send to stdout concurrently they might overlap. Also, I want to send some to the front-end and so on so it would be best to send it to a tap or something π
just make a loop that reads the output line by line, prints it and does whatever else with it
This is all new to me. Sounds like it is not that hard then. Thanks!
I have a similar example to that here: https://book.babashka.org/#child_processes
Here is a simple example that reads the output from yes
:
(require '[babashka.process :as p :refer [process]]
'[ :as io])
(def yes (process ["yes"] {:err :inherit
:shutdown p/destroy}))
(with-open [rdr (io/reader (:out yes))]
(binding [*in* rdr]
(loop []
(let [line (read-line)]
(println :line line))
(recur))))
@borkdude I guess there should be a (when (not (nil? line)) (recur))
above?
I tried it and it just keeps on printing :line nil
forever π
@UT770EY2K you are correct, but the output from yes
never ends :)
@endrebak With babashka.process (https://github.com/babashka/process) this is very easy:
@(babashka.process/process ["ls"] {:inherit true})
and vanilla clojure / interop version of the same:
(doto (ProcessBuilder. ["ls"]) (.inheritIO) (.start))
Cool! I'll have to look more into babashka
. But let's say I have multiple processes running at the same time in futures. Can each of them send data to a tap or something with inherit? So I get messages like:
jobid1 :stdout "message from job1"
jobid3 :stderr "this job failed"
...
I will have to play around with :inherit
to understand how it works.inherit just sends to *out*
and *err*
- see my reply in the other thread for using the streams with a callback
oh, good catch
why does evaluating an empty anonymous function using the #()
syntax yield an empty list? I expected... maybe nil?
user=> (#())
()
user=> ((fn []))
nil