This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-07-11
Channels
- # babashka (13)
- # beginners (17)
- # bristol-clojurians (1)
- # calva (23)
- # cider (5)
- # clj-kondo (32)
- # cljsrn (2)
- # clojure (167)
- # clojure-dev (23)
- # clojure-europe (6)
- # clojure-uk (8)
- # clojurescript (5)
- # cursive (7)
- # datomic (12)
- # emacs (4)
- # events (6)
- # fulcro (2)
- # graalvm (1)
- # interop (10)
- # leiningen (8)
- # local-first-clojure (1)
- # luminus (2)
- # meander (36)
- # planck (3)
- # re-frame (3)
- # reagent (3)
- # sci (2)
- # spacemacs (4)
- # sql (1)
- # test-check (6)
- # tools-deps (22)
- # xtdb (75)
is there a lib out there that takes clojure data structures (maps) and spits out reasonably good looking hiccup/html data?
if you're just looking for pretty print, you can do:
[:pre
(with-out-str (clojure.pprint/pprint
obj))]
if you're on the jvm, there are tools for generic data exploration like rebl and reveal
that's what i thought, still hoped for something more involved that i could just import. does the job though
whatâs possibly wrong with this:user=> (def instant #inst â2018-03-28T10:48:00.000â) Syntax error reading source at (REPL:1:45). No reader function for tag inst
@i What does *clojure-version*
return?
1.10.0. used to be working fine. havenât done clojure in months and then today encountered this ;-(
Something must be broken with your setup. lein
? clojure
/ clj
?
I tried it on Clojure versions 1.10 back down to 1.2 and it works all the way back to 1.4, and fails on 1.3 & earlier.
(but with a different error -- so the version is not the problem)
Is this lein repl
? clj
?
Or some other REPL setup?
Do you have the Clojure CLI installed? If so, could you just try that and check it works with that def
?
Do you have a project.clj
file in the current folder? Do you have stuff in your ~/.lein/profiles.clj
that might be breaking this?
(it seems common that folks who take a break from Clojure, then come back and find weird stuff broken, have something in their profiles that causes weird behavior)
@i Did you empty/remove your ~/.lein/profiles.clj
file?
Just reinstalling Leiningen won't replace that file and it's the cause of so many people's problems...
Oh, wow, so I just tried that in a plain lein repl
and it gave the same error!
It works perfectly in clj
so I never thought to actually try lein repl
...
seanc@DESKTOP-QU2UJ1N:~/clojure$ lein repl
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
nREPL server started on port 56280 on host 127.0.0.1 -
REPL-y 0.4.4, nREPL 0.6.0
Clojure 1.10.0
OpenJDK 64-Bit Server VM 13.0.2+6-MTS
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Exit: Control+D or (exit) or (quit)
Results: Stored in vars *1, *2, *3, an exception in *e
user=> (def instant #inst "2018-03-28T10:48:00.000")
Syntax error reading source at (REPL:1:45).
No reader function for tag inst
user=> Bye for now!
seanc@DESKTOP-QU2UJ1N:~/clojure$ clj
Clojure 1.10.1
user=> (def instant #inst "2018-03-28T10:48:00.000")
#'user/instant
user=> instant
#inst "2018-03-28T10:48:00.000-00:00"
user=>
That makes no sense to me...
I haven't used Leiningen for years, except to test things that folks have problems with here on Slack, but I'm stunned something so basic doesn't work.
OK, I switched to my main computer so I can try different versions of Leiningen... It works fine in 2.8.1:
(! 832)-> lein repl
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
nREPL server started on port 53601 on host 127.0.0.1 -
REPL-y 0.3.7, nREPL 0.2.12
Clojure 1.8.0
OpenJDK 64-Bit Server VM 14+36-1461
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Exit: Control+D or (exit) or (quit)
Results: Stored in vars *1, *2, *3, an exception in *e
user=> (def instant #inst "2018-03-28T10:48:00.000")
#'user/instant
user=> instant
#inst "2018-03-28T10:48:00.000-00:00"
user=>
(and, note, that's on Clojure 1.8 but I also tried a 1.10 project and it worked fine)
Tried 2.8.2 -- that won't even start a REPL, it just crashes -- 2.8.3 -- #inst
works fine -- 2.9.4 -- No reader function for tag inst.
2.9.0 to 2.9.4 -- none of those work.
So you could do lein upgrade 2.8.3
and work with that version @i -- or you could switch to the Clojure CLI instead đ
You probably want to follow up in #leiningen at this point ...
Yup. Tried it above. Doesn't work. It has nREPL 0.7.0.
Leiningen has seemed really flaky through the last set of releases. I've seen no end of problems reported with the 2.9.x releases that are nearly always solved by "upgrading" to 2.8.3 @dpsutton
I think that was a giant mistake, frankly.
but its been a bit too rocky agreed. difficult to fix something so widespread and core
All because Bug didn't want to work on nREPL inside the Contrib ecosystem.
but these upgrades haven't been pain free. and the pain would have been present in contrib or nrepl. its a fork he's taken and improved so i'm all for it
The pain would not have been present if everything had continued to depend on org.clojure/tools.nrepl and he'd just upgraded that -- most of the breakages I saw people run into was due to the namespace changes and the REPLy 0.4.3 / 0.4.4 cutover (I think that's the versions that caused the biggest pain, due to switching from tools.nrepl in Contrib to nrepl outside it). That could have been handled much more smoothly by making the code work with both sets of namespaces and just give warnings about the old names.
(REPLy 0.4.3 depended on tools.nrepl; REPLy 0.4.4 on nrepl I think?)
Not that I really care much: I stopped using Leiningen in 2015. I just felt bad for all the newcomers who kept bumping into those sharp edges đ
(but, yeah, there does seem to have been a number of "quality control" issues with Leiningen releases lately and I can't blame the nREPL/CIDER folks for that đ )
@dpsutton Do you have any insight into the #inst
issue reported above? Is it something you've heard folks mentioned in that tool chain?
Given the number of moving parts, I'm not sure where to go digging for a workaround/fix...
I repro'd on Windows/WSL first, then macOS
(def ^{:added "1.4"} default-data-readers
"Default map of data reader functions provided by Clojure. May be
overridden by binding *data-readers*."
(merge
{'uuid #'clojure.uuid/default-uuid-reader}
(when-class "java.sql.Timestamp"
{'inst #'clojure.instant/read-instant-date})))
i wonder if lein starts up without java.sql.Timestamp
? first time actually looking at this definitionI haven't tested on Powershell/CMD. I'm not sure I even have lein
on those setups on my Windows laptop.
Hmm, would that be a java.base
issue, related to Leiningen's bootclasspath stuff and how that changed between 2.8.x and 2.9.x?
Java versions were 13 on Windows/WSL and 14 on macOS, if that helps.
(! 852)-> JAVA_HOME=$OPENJDK8_HOME lein repl
...
REPL-y 0.4.4, nREPL 0.7.0
Clojure 1.10.1
...
user=> (def instant #inst "2018-03-28T10:48:00.000")
#'user/instant
So it's Java 9+That was lein
2.9.4.
lein
2.8.3 works on Java 8 / 13 / 14, 2.9.4 works on 8, but not on 13 / 14 so that's why I said "9+".
works = #inst
is understood.
(weird stuff like this makes me so glad I switched to a simpler toolchain with the Clojure CLI!)
yeah i agree with that. unfortunate the tooling support hasn't carried along. plugins and middleware abound
Scissors are good tools đ
I'll be interested to see how that issue plays out (since you tagged me in it) đ
OK, I'm out. I think I'll be in the kitchen most of tomorrow -- the wife has a major baklava project planned, with three different recipes to try... her form of A/B testing. Probably be online in the evening/Sunday.
that sounds awesome. enjoy it. my friend in SF loves going to this place to get backlava ice cream. (souvla maybe?)
Hey All! Iâve been exploring reloadable REPL workflow + web server. In the past, I have used ring.reload/wrap-reload
and clojure.tools.namespace
for reloading, but iâve seen others design their code to be reloadable using Var
e.g. https://github.com/seancorfield/usermanager-example/blob/develop/src/usermanager/main.clj - I have tried to setup a simple version for my own side project: https://gist.github.com/athomasoriginal/15ab9f5e01832fda677f80dd635aff46.
With the above, after changes, I have to evaluate health-check-route
and then app
. What I would like is to only have to modify + re-evaluate health-check-route
and see the changes picked up without needing to re-evaluate app
. What would I need to modify in the design of this code to make that possible? :thinking_face:
I think if you change your router definition to :
(defn router []
(reitit/router
[(#'health-check-route)]))
that should workI added #'
before health-check-route
you may also have to change your app definition to:
(def app
(reitit/ring-handler (#'router) (constantly {:status 404})))
I've never used reitit, so I'm not sure if the second change is necessary
You don't need #'
in a function call -- only a direct var reference (i.e., function passed as a value/argument)
But with a def
the problem is that's going to be evaluated once, at namespace load time, so it will have been evaluated and captured the current values of things at load time.
If you change app
to a function (of request), it would likely be easier to work with in a REPL
(defn app [req]
((reitit/ring-handler (router) (constantly {:status 404})) req))
I think that should work (but, like @U7RJTCH6J I don't use Reitit).some of the routing libraries also use macros, which can make a difference too.
Which also means that the change to [(#'health-check-route)]
is probably not needed either -- it already refers to a function in a call.
@U7RJTCH6J Ah, good point...
Thanks for the suggestions above đ !! It seems that Seanâs second one worked. I updated the gist with the working version (also kept a reference to the one that did not workâŚ.could be helpful for others). @U7RJTCH6J good point on the macros! I will have to keep that in mind as iâm going.
Iâm writing a library that wraps an underlying java library, and I want two thingsâŚ
Firstly I want to wrap one of those java objects (which is at the root of a tree of other object instances), and present a clojure map like interface to the wrapped object.
I could obviously do:
(defrecord [internal-java-object other-opts ,,,])
However Iâd also like to hide internal-java-object
from API users (for various reasons).
I could obviously reify
and implement all the appropriate parts of a clojure map; however Iâll need to repeat this for quite a large number of objects in that subtree.
Potemkin can do this with reify-map-type
, but in the past Iâve heard people complain about potemkin causing lots of problemsâŚ
Whenever Iâve used potemkin in the past itâs been fine, and caused me no problems.
What do people think, should I do?
I knew folk would ask đ
đ I think it is a core constraint here so itâs worth asking. My knee jerk reaction, not knowing anything about your use case, is twisting yourself into knots purposefully obscuring things is a code smell
Iâm not convinced Iâll hide it yet, but perhaps the main reason is that we have code that already uses this library, and there are numerous anti patterns in its use that I want to stamp out⌠i.e. the new library aims to fix these problems. Iâm considering exposing the valid stuff as data etc, but avoid people accessing the internal stuff.
Essentially Iâd like the option to grow the API⌠if people really DO need access to the internals if I were to use reify-map-type
I could grow the API by adding the key for it.
@rickmoynihan Do you need to maintain the Java object directly, or could you just turn it all into data instead? Or have the option to turn it all back into a Java object again at will?
(I'm thinking clojure.java.data
)
my initial thought was to use datafy
(similar in idea to clojure.java.data
I guess), but that would also mean exposing the underlying object potentially
@seancorfield Yeah I need to hold a reference to the internal library as a reference in either a closure or a key in the map
And why do you need to prevent users from accessing it? Could you just document that folks should not touch the object?
Itâs ok Iâve decided to make it a record. Essentially I was wanting to avoid thinking about subtle breakage that might occur if people mixed decorated behaviour with undecorated behaviour when using new and old code together that both provide wrappers on that old library. However I donât think the problems actually exist; but itâs quite hard to think about â I was thinking there may be ways people could combine things that would for example break equality of some operations.
Obviously Iâm well aware of the speculation stuff; and we do that⌠but there are always a few more subtleties in the real world.
@rickmoynihan Do you own all the protocols? If so you could use :extend-via-metadata
on the protocols and just use maps with metadata https://clojure.org/reference/protocols#_extend_via_metadata
Yeah I considered that option too, but decided to just go with a record
I was going to set that option on some of my protocols, but was meaning to ask about the perf cost of that option. At least one of the protocols may be dispatched 100 million times or more
Most will be called infrequently though, so Iâll set it on those
i'm trying to translate the Concurrent crawler with channels from here https://pdos.csail.mit.edu/6.824/notes/crawler.go. One area of confusion is how to mimic how the master is iterating over the channel. I assume this function keeps blocking forever, though the docs here don't make that clear https://gobyexample.com/range-over-channels Ideally the solution would have a timeout. But i don't know how to recreate the effect of "keep taking" from the channel. I tried a loop, but its likely failing for a number of reasons, including the chance that the loops take (which is already awkward) completes before the go block puts anything on the channel.
(let [url->urls {:a [:b]
:b [:d]}
ch (chan 10)
results (atom [])]
(>!! ch (keys url->urls ))
(loop [urls (<!! ch)
seen? #{}]
(let [url (first urls)]
(cond
(nil? url) (println "done")
(seen? url) (recur (<!! ch) seen?)
:else
(do
(go (>! ch
(do (swap! results conj url)
(get url->urls url []))))
(recur (<!! ch) (conj seen? url))))))
@results)
with just an initial look, the atom seems out of place. I would probably do something like:
(let [url->urls {:a [:b]
:b [:d]}
ch (chan 10)]
(async/onto-chan ch (keys url->urls) false)
(loop [url (<!! ch)
seen? #{}]
(prn "looking at " url)
(cond
(nil? url) (do (println "done") seen?)
(seen? url) (recur (<!! ch) seen?)
:else
(do
(go
(async/onto-chan ch (get url->urls url) false))
(recur (<!! ch) (conj seen? url))))))
seen?
captures all the urls that have been visited. results
seems unnecessary and mixing atoms with async blocks is usually not the right thing
if you want to capture other data about the process. depending on the context, I would do one of the following:
1. have the accumulated data as a value in the loop
if you want to accumulate the result of the process and return it when the result is finished
2. have an output channel that this process puts values onto if you'd like to have the results as they are available
Yeah only glanced at this but the loop probably wants to be in a go block itself, and pass the result back to the main thread via a chan.
To implement timeouts you want to use the timeout
function with an alts!(!)
over it
atoms can block when contended, which could stall all of the go blocks
probably not really an issue here; but it rings alarm bells.
the main issue with the atom isn't whether it's blocking or not (although that might be a practical issue), but that logically, core.async in built on CSP. in this paradigm, you want to communicate communicate your data via channels, as data within recursively built state with loop variables, or return values and function parameters
one other note is that if (get url->urls url)
is replaced by an actual http request, you probably don't want to do that in the go
block thread pool. you either want to spawn a new thread or provide some other thread pool for IO or long running computations
:thumbsup: Yeah definitely agree that shared mutable state isnât part of CSP, but clojure doesnât care and allows you to mix paradigms. Rich is famous for offering things a la carte. Practically though you will care if you mix blocking ops with asyncâs coroutines and you stall every coroutine in your program. Philosophical issues are one thing but you really donât want to be debugging these interactions in a production system. Essentially spinning on an atom wonât yield the go block, and eventually under sufficient load may starve all of the go blocks. This is more commonly seen when people combine go blocks with blocking i/o, which suffers the same issue.
while it is possible to mix different paradigms for reasoning about concurrency/parallelism, I wouldn't recommend it.
@rickmoynihan @U7RJTCH6J Thanks a lot. the results atom was added at the last second to try to debug things. i should have removed it before posting. But the discussion was still very enlightening to me. > one other note is that if `(get url->urls url)` is replaced by an actual http request, you probably don't want to do that in the `go` block thread pool. you either want to spawn a new thread or provide some other thread pool for IO or long running computations This would be because an http call will require an actual jvm/os thread and tie up the whole green thread/core async.go.thread pool anyway?
I wasn't aware of onto-chan at all!
I suppose mostly i'm weired out by the recur with a channel take as the arg. Is that ideal?
yes, there's only a limited number of threads in the green thread/core async.go.thread pool so anything that blocks one of them is undesirable
recurring with a channel take is pretty common and normal
at least, that's the style I usually use, you could also use this style:
(let [url->urls {:a [:b]
:b [:d]}
ch (chan 10)]
(async/onto-chan ch (keys url->urls) false)
(loop [seen? #{}]
(prn "looking at " url)
(let [url (<!! ch)]
(cond
(nil? url) (do (println "done") seen?)
(seen? url) (recur seen?)
:else
(do
(go
(async/onto-chan ch (get url->urls url) false))
(recur (conj seen? url)))))))
i'm not terminating the loop correctly it seems. When you try to call onto-chan with a channel and nil (when no urls are left) it returns a channel? i guess i need to check for that and ask if its closed?
or close it
or maybe not call onto chan in that case.
or no. i should timeout
umm, onto-chan seems to return nil when nothing is left, possible bc its closed. i guess i need to close it.
it depends. if this is an ongoing process that can continue to receive new urls to explore [from another process putting values onto ch
, then this process should stop as written when ch
is closed.
if this is a process that should stop when all the urls initially provided and the urls they can reach have been explored , then I probably wouldn't read new urls from ch
and I would set it up slightly differently
It was intended to be the later.
In the model when where you can get new urls then only a timeout makes sense.
or have the channel be explicitly closed by the producer
Any idea why this taking from the channel will sometimes return a multichannel? It happens when their are no urls to follow. (url->urls url) returns nil. What i expect is that it will either take a url from a channel that i put on or it will find nothing and return nil.
(let [url->urls {:a [:b]
:c [:d]}
ch (chan 10)]
(onto-chan ch (keys url->urls) false)
(loop [url (<!! ch)
seen? #{}]
(cond
(nil? url) (println "finished: " seen?)
(seen? url) (recur (<!! ch) seen?)
:else
(do
(when-let [urls (url->urls url)]
(>!! ch (onto-chan ch urls false)))
(recur (<!! ch) (conj seen? url))))))
you're putting the result of onto-chan
on the ch
channel
onto-chan
returns a channel itself that only signals that all of the values have been put
so i am. i should take a break it seems. thanks!
I was thinking something like:
(let [url->urls {:a [:b]
:b [:d]
:d [:a]}
to-download (chan 10)
downloaded (chan 10)]
(go
(loop []
(when-let [url (<! to-download)]
(>! downloaded [url
(<! (thread (get url->urls url [])))])
(recur))))
(go
(loop [to-visit (set (keys url->urls))
pending? #{}
seen? #{}]
(prn to-visit pending? seen?)
(if (and (empty? pending?)
(empty? to-visit))
seen?
(let [next-url (first to-visit)
ports (if next-url
[[to-download next-url]
downloaded]
[downloaded])
[val port] (alts! ports)]
(cond
(= port to-download)
(recur (disj to-visit next-url)
(conj pending? next-url)
seen?)
(= port downloaded)
(let [[from-url to-urls] val]
(recur (into to-visit (remove #(or (pending? %)
(seen? %)))
to-urls)
(disj pending? from-url)
(conj seen? from-url)))))))))
it might be overcomplicating it, but I think this should work. I think explicitly keeping track of the in-flight requests (in this case pending?
) is a key idea
otherwise, you don't know when to stop.
@U7RJTCH6J what's putting anything on the to-download channel in that?
the alts!
if there are any urls in to-visit
, then the alts! try to read from either to-download
or downloaded
.
otherwise, it will just try to read from downloaded and park until a result is available
i see. i didn't understand how alts worked it seemed.
@U7RJTCH6J this has been very helpful. Would it be ok to show your solution alongside mine for the purely fun not for credit class this is part of? I have a group of people working through a distributed systems class and the some of the lab's were translated from Go to clojure. Now i feel responsible for rapidly learning how to translate the rest.
:thumbsup:
Is there any way to call a default method of an interface in reify when implementing that method?
reify doesn't have access to parent class concrete implementation, proxy does
see proxy-super
, but NB proxy-super is not thread safe
(.forEach (proxy [Iterable] []
(iterator []
(.iterator (doto (ArrayList.)
(.add 3)))))
(reify Consumer
(accept [_ x]
(println x))))
Execution error (UnsupportedOperationException) at dev.mccue.class_system.proxy$java.lang.Object$Iterable$e10044ca/forEach (REPL:-1).
forEach
(.forEach (doto (ArrayList.)
(.add 3))
(reify Consumer
(accept [_ x]
(println x))) )
3
=> nil
(.forEach (proxy [Iterable] []
(iterator []
(.iterator (doto (ArrayList.)
(.add 3))))
(forEach [c]
(proxy-super forEach c)))
(reify Consumer
(accept [_ x]
(println x))))
Execution error (UnsupportedOperationException) at dev.mccue.class_system.proxy$java.lang.Object$Iterable$e10044ca/forEach (REPL:-1).
forEach