Fork me on GitHub
#clojure
<
2020-03-05
>
dpsutton00:03:17

> Default reader tags are defined in > clojure.core/default-data-readers but may be overridden in > data_readers.clj, data_readers.cljc, or by rebinding this Var.

(binding [*data-readers* {'foo/bar `int}]
  #foo/bar 3)
Syntax error reading source at (REPL:74:13).
No reader function for tag foo/bar

dpsutton00:03:30

from the docstring of data-readers i expect i can dynamically add a namespaced data reader and use it in this fashion but i seem to be missing something

hiredman00:03:29

you are adding it too late

hiredman00:03:53

the whole form has to be read, then evaled before your binding is in effect

dpsutton00:03:05

like so?

(binding [*data-readers* {'foo/bar `int}]
  (clojure.edn/read-string "#foo/bar 3"))

hiredman00:03:44

if you like, that will error or return nil

dpsutton00:03:58

it throws an error for me still

hiredman00:03:11

which error

dpsutton00:03:25

No reader function for tag foo/bar

dpsutton00:03:38

and using read:

(binding [*data-readers* {'foo/bar `int}]
  (read-string "#foo/bar 3"))
Execution error at find.main/eval8981 (REPL:104).
No dispatch macro for: f

hiredman00:03:10

the clojure.edn reading functions take those things as explicit passed in maps

hiredman00:03:37

the var for rebinding only effects the actual reader

dpsutton00:03:11

the read-string version here, right?

dpsutton00:03:22

i don't understand why that complained about a dispatch macro for f

dpsutton00:03:56

ah, i don't need the quote on this version. int not `int

hiredman00:03:53

it is a map from symbols to functions

hiredman00:03:32

so a quoted symbol is pretty much never what you want there

dpsutton00:03:50

Map from reader tag symbols to data reader Vars.
 When Clojure starts, it searches for files named 'data_readers.clj'
and 'data_readers.cljc' at the root of the classpath. Each such file
must contain a literal map of symbols, like this:
     {foo/bar my.project.foo/bar
     foo/baz my.project/baz}

dpsutton00:03:16

That’s where I got confused. Map of symbols so I assumed it needed namespaces qualified symbols

dpsutton00:03:21

Thanks for your help

hiredman00:03:17

yeah, symbols are only if you are creating a data_readers.clj file

hiredman00:03:47

they are resolved before being installed in *data-readers* then

👍 4
markx06:03:38

Hi all! A question about https://github.com/l3nz/cli-matic: Do I have to use a sub-command? Does it support no command style? Something like: my_app --some_param=1 some_argument

Matthew Davidson (kingmob)17:03:31

I had the exact same question today. The docs on cli-matic are a bit vague on this point, and I couldn’t find the author on Clojurians. But a little bit of local testing suggests that, yes, cli-matic requires subcommands.

orestis06:03:53

Was surprised to find that my ETL pipeline spends a considerable amount in SSL processing (JDK 8, for now). I don’t control the code that handles the connections (MongoDB and JDBC Postgres) - is there a way to speed things up other than throwing hardware at the problem?

jumar08:03:24

Would be good to mention that the follow up is in #java

orestis09:03:22

Yes, sorry :)

orestis10:03:07

With ScheduledThreadPoolExecutor available, is there any Clojure job scheduler (at-at, overtone, tea-time) that gives any added value?

Ivar Refsdal11:03:25

How easy is it to test code using ScheduledThreadPoolExecutor? tea-time has a virtual clock that makes it very easy to test schedules. https://github.com/aphyr/tea-time#testing-with-virtual-time tea-time also has defer! task, which I don't think ScheduledThreadPoolExecutor has directly. This feature "feels" very natural and common sense to me.

Ivar Refsdal11:03:51

fwiw I've written* a library recurring-cup that let's you work with and schedule (lazy) sequences of java.time.ZonedDateTime. This library makes it easy to run a function at (for example) a fixed time of the day. It's built on top of tea-time. * Not released, not well documented. https://github.com/ivarref/recurring-cup

noisesmith16:03:31

ScheduledThreadPoolExecutor has always worked for me when using a single vm, I have never run into complications or extra problems from not using a wrapper over it

noisesmith16:03:47

it can use clojure functions directly since they are Runnable / Callable

noisesmith16:03:37

(on the other hand, if your scheduling has to work across redeploys or across multiple instances, you need bigger guns and the clojure wrappers really help a lot)

wotbrew10:03:00

I like ScheduledThreadPoolExecutor but it does not make many decisions for you and by default you may do the wrong thing (e.g long running jobs blocking the pool)

wotbrew10:03:00

I am not too familiar with clojure schedulers, as I am pretty comfortable ScheduledThreadPoolExecutor but perhaps they provide an easier api with sane defaults

seegy13:03:20

Hi there! 👋 I try to solve a special problem and I am looking for a function/lib, that could help me. Let’s say, I got a function and a list of items. Now I want to call the function on parallel with each item of the list and just want the result of the fastest thread (and dump all other threads without waiting for them). Something like wanted-fun in here:

(def l [100 200 300])

(defn fun [x] (Thread/sleep x) x)

(wanted-fun fun l) ; >> 100 (after ~100 miliseconds)
Does this already exists or do I have to build it by my own?

borkdude13:03:34

@soeren.gutzeit isn't fun already that function? or do you mean something like (defn wanted-fun [f x] (f x))?

seegy13:03:50

fun is an example what wanted-fun should could handle

borkdude13:03:58

@soeren.gutzeit :

(def l [100 200 300])
(defn fun [x] (Thread/sleep x) x)
(some fun l) ;;=> 100 

borkdude14:03:14

if you want the first succeeding result from multiple threads, you can use something like core.async or promises:

(def prom (promise))

(future (Thread/sleep 100) (deliver prom 1))
(future (deliver prom 2))

@prom ;;=> 2

👍 4
manutter5114:03:27

I’m not sure I’ve really got the hang of core.async yet, but here’s what I came up with:

(require '[clojure.core.async :refer [go chan >!!  <!! alts!!]])
=> nil
(def l [100 200 300])
=> #'user/l
(defn fun [x] (Thread/sleep x) x)
=> #'user/fun
(defn wanted-fun [f coll]
  (let [ports (map (fn [l]
                     (let [ch (chan 1)]
                       (>!! ch (f l))
                       ch))
                   coll)]
    (first (alts!! ports))))
=> #'user/wanted-fun
(wanted-fun fun l)
=> 100

seegy14:03:55

wow, thanks!

borkdude14:03:39

@manutter51 slight variation, where the sleeping happens not on the main thread:

(require '[clojure.core.async :refer [go alts!!]])

(def l [100 200 300])

(defn fun [x] (Thread/sleep x) x)

(defn wanted-fun [f coll]
  (let [ports (map (fn [l]
                     (go (f l)))
                   coll)]
    (first (alts!! ports))))

(prn (wanted-fun fun l))

manutter5114:03:55

Nice, the go automatically creates the chans for you? I never realized that but then again I did know that go returns a chan so I guess I shoulda known.

borkdude14:03:39

go and async/thread both return channels that contain the result of that async computation. in this case async/thread might be a better choice than go since sleeping in a go block might not make sense.

manutter5114:03:35

Would that work with alts!! though? If you just want to get whichever one comes back first, and ignore the others, I’m not sure how to do that with async/thread (due to my lack of experience with the async stuff)

jumar14:03:54

Rather frequently, I have some data that contains object literals referencing some java class but if I try to copy-paste this into repl it fails:

{:id 308, :time-now "2020-03-04T16:48:39.000Z" :created-at #object[org.joda.time.DateTime 0x799dfbe "2020-03-04T16:48:39.000Z"], :updated-at #object[org.joda.time.DateTime 0x9cae937 "2020-03-04T16:48:42.000Z"], :status :failure}
...
No reader function for tag object
Is there an easy way how to make it work?

borkdude14:03:02

@jumar you can transform this stuff using edn/read-string with :reader or :default and a good default could be tagged-literal

borkdude14:03:00

@jumar

user=> (edn/read-string {:default tagged-literal} "{:id 308, :time-now \"2020-03-04T16:48:39.000Z\" :created-at #object[org.joda.time.DateTime 0x799dfbe \"2020-03-04T16:48:39.000Z\"]}")
{:id 308, :time-now "2020-03-04T16:48:39.000Z", :created-at #object [org.joda.time.DateTime 127524798 "2020-03-04T16:48:39.000Z"]}

jumar15:03:35

Thanks; is there any way how to make this a default behavior so I don't need to wrap it every time I need this?

borkdude15:03:49

don't know, maybe by setting it in data_readers.clj but I'm not sure if that's a good idea

4
denik16:03:05

Say one wants to pass a server port as an arg to main. Is there a way to pass opts or an opts map to be read/coerced to edn? currently everything is interpreted as a string, so port becomes "port" , :port ":port" and 8080 "8080" while the goal is {:port 8080} clj -m my.server port 8080

dominicm16:03:16

Nothing built in. Boot defined some stuff.

denik16:03:06

perfect thank you @alexmiller

jeff tang17:03:56

hi all! today im taking my first steps to learning Clojure. i was mainly convinced by the positive response to this tweet 🙂 https://twitter.com/tangjeff0/status/1234688533367050244?s=20 hope to keep in touch with everyone for the next 10+ years! haha

🎉 20
👍 12
👋 8
Aleed22:03:32

I started learning Clojure a few months ago 🙌 I definitely recommend reading “Clojure for the Brave and True” and doing Clojure Koans and 4Clojure exercises Have fun!

👍 4
jeff tang01:03:52

haha im part-way through ch 3, i absolutely love Clojure for the Brave so far. will check out Koans 4Clojure too. thanks!

Alex Miller (Clojure team)17:03:17

welcome! #beginners is a great place to ask questions...

Alex Miller (Clojure team)17:03:57

or https://ask.clojure.org for searching or creating a record of good q&a

😃 4
robertfw18:03:46

I'm in need of a bit of code to take an accept-lang header and turn it into an iso 639-1 language code. I'm poking around to see if I can find something pre-existing to do that - does anyone know of anything in clojure-land (or optionally java land) to do that?

robertfw18:03:22

that looks quite promising, thanks!

bfabry18:03:39

user=> (.getISO3Language (java.util.Locale/forLanguageTag "en-US"))
"eng"

robertfw18:03:22

it's close - I need to end up with the 2 letter code, but I suspect I can find what I need here

Jon Walch19:03:48

What do you all use for moving the current time forward in tests?

ghadi20:03:18

I don't -- I make the time a parameter

8
dominicm20:03:57

Java has a clock object for that.

👌 4
vlaaad21:03:21

(defn inject-$val [val form]
  `(let [~'$val ~val]
     ~form))

(eval (inject-$val (StringBuilder.) '(pr-str $val)))
=> 
Syntax error compiling fn*.
Can't embed object in code, maybe print-dup not defined: 
How can I inject an arbitrary value into local scope for eval?

vlaaad21:03:46

Figured it out: transform form to a function that is then called with arbitrary value:

(defn ->$val-fn [form]
  `(fn [~'$val]
     ~form))

((eval (->$val-fn '(pr-str $val))) (StringBuilder.)) 
=> "#object[java.lang.StringBuilder 0x3909a854 \"\"]"

lilactown22:03:32

yeah, either that or quote the form returning the val you want to replace with

lilactown22:03:05

(eval (inject-$val '(StringBuilder.) '(pr-str $val)))
=> "#object[java.lang.StringBuilder 0x6380a835 \"\"]"

vlaaad22:03:30

I can't quote it: it's created dynamically

Ben Grabow22:03:45

I'm processing messages coming over a core.async channel, and I'd like to pipe them to another channel with a weird caveat. The messages that represent success I want to pipe unconditionally and immediately, but the messages that represent failure I want to delay sending them for some grace period, and forgo sending them if a success message arrives before the grace period elapses. My first thought is to use some kind of stateful transducer but I'm having a hard time finding examples of transducers that do delayed delivery of the result, or that make behavior for the nth item conditional based on the n+1th item. Am I trying to use transducers for something they weren't designed for?

bfabry22:03:08

imo doesn't sound very transducery. sounds like a go-loop with some state

Ben Grabow22:03:17

There are plenty of options for doing it in a go-loop. I am hoping to find something that is a bit more constrained and concise than that.

ghadi22:03:33

seems arbitrary

ghadi22:03:34

channel transducers run within the context of the channel mutex; interactions with other channels are verboten

hiredman22:03:01

channel transducers also run in band, they can only pass or not pass through (possibly transformed) data, they can't sit on data and wait

hiredman22:03:21

the reason you can't find examples of transducers that do delayed delivery is it isn't something they can do

Ben Grabow22:03:24

Yeah I think the delayed aspect is what's going to stop me here. I can imagine storing some internal state in a transducer that only gets returned when some other new input arrives, but that would have to happen synchronously with the new input arriving, and can't happen at an arbitrary time between inputs.

ghadi22:03:33

when you have a hammer transducer, everything looks like a nail

ghadi22:03:50

the tricky requirement is:

ghadi22:03:52

> forgo sending them if a success message arrives before the grace period elapses

ghadi22:03:38

which means you have to listen to both a timeout and some success/cancel signal

Ben Grabow22:03:32

This is essentially a low-pass filter, so the idea is conceptually somewhat simple and I'm struggling to find the correspondingly simple implementation.

hiredman22:03:15

it is pretty straight forward with alts!

hiredman22:03:31

(fn [in out] (async/go-loop [t nil] (let [[a-val a-ch] (async/alts! (some->> t (conj [in]))] (cond (and (= ch in) (success? val)) (do (async/>! out a-val) (recur t)) (= ch in) (recur (asnc/go (async/<! (async/timeout 1000)) a-val)) (= ch t) (do (async/>! out a-val) (recur nil))))))

🙏 4
hiredman22:03:38

(dictated but not read, etc)

Ben Grabow23:03:59

Putting the pending failures each in their own delayed channel helps a lot

Ben Grabow23:03:11

There's another wrinkle here where there may be several failure messages that are pending before the timeout elapses or a success arrives, but it's relatively simple to extend this to a queue of pending failures

hiredman23:03:00

the thing to watch out for if you are managing multiple timeout channels is avoid using them directly as keys for maps or members of sets

hiredman23:03:30

because timeout can return the same channel more than once, so not a good key or set member, but if you create a go block that takes from the timeout and returns something useful that is less likely to bite you

Ben Grabow23:03:31

Are you talking about the equality gotcha related to optimizing multiple timeouts that happen close to each other?

Ben Grabow23:03:59

When creating a function that is an intermediate processing step in a core.async process, is it more idiomatic to have the function take the in-chan as an argument and return an out-chan that was created internally, or take the out-chan as an argument and return an in-chan that was created internally, or take both as arguments, or take neither?

hiredman23:03:52

I like to take both an in and out, and then the function returns the channel from the go block it starts, which another process can wait on if they want to wait for that process to stop

Matthew Davidson (kingmob)17:03:31

I had the exact same question today. The docs on cli-matic are a bit vague on this point, and I couldn’t find the author on Clojurians. But a little bit of local testing suggests that, yes, cli-matic requires subcommands.