Fork me on GitHub

Any crux users here?


i'm a crux noob


Me too, I'm confused as to why a tx isn't working.


join the #crux channel


post the errors there


is there a lib out there that takes clojure data structures (maps) and spits out reasonably good looking hiccup/html data?


i'm trying to use my browser as a pretty printer


if you're just looking for pretty print, you can do:

 (with-out-str (clojure.pprint/pprint

💯 3

if you're on the jvm, there are tools for generic data exploration like rebl and reveal


that's what i thought, still hoped for something more involved that i could just import. does the job though

✔️ 3

what’s possibly wrong with this:user=> (def instant #inst “2018-03-28T10:48:00.000”) Syntax error reading source at (REPL:1:45). No reader function for tag inst


@i What does *clojure-version* return?


1.10.0. used to be working fine. haven’t done clojure in months and then today encountered this ;-(


Something must be broken with your setup. lein? clojure / clj?


I tried it on Clojure versions 1.10 back down to 1.2 and it works all the way back to 1.4, and fails on 1.3 & earlier.


(but with a different error -- so the version is not the problem)


yes. i guess, probably.


Is this lein repl? clj?


Or some other REPL setup?


Do you have the Clojure CLI installed? If so, could you just try that and check it works with that def?


Do you have a project.clj file in the current folder? Do you have stuff in your ~/.lein/profiles.clj that might be breaking this?


(it seems common that folks who take a break from Clojure, then come back and find weird stuff broken, have something in their profiles that causes weird behavior)


things become a little bit hairy. reinstalled lein and java, same issue.


@i Did you empty/remove your ~/.lein/profiles.clj file?


Just reinstalling Leiningen won't replace that file and it's the cause of so many people's problems...


i removed that file


Oh, wow, so I just tried that in a plain lein repl and it gave the same error!


It works perfectly in clj so I never thought to actually try lein repl ...


seanc@DESKTOP-QU2UJ1N:~/clojure$ lein repl
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
nREPL server started on port 56280 on host - 
REPL-y 0.4.4, nREPL 0.6.0
Clojure 1.10.0
OpenJDK 64-Bit Server VM 13.0.2+6-MTS
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

user=> (def instant #inst "2018-03-28T10:48:00.000")
Syntax error reading source at (REPL:1:45).
No reader function for tag inst

user=> Bye for now!
seanc@DESKTOP-QU2UJ1N:~/clojure$ clj
Clojure 1.10.1
user=> (def instant #inst "2018-03-28T10:48:00.000")
user=> instant
#inst "2018-03-28T10:48:00.000-00:00"

😲 3

That makes no sense to me...


I haven't used Leiningen for years, except to test things that folks have problems with here on Slack, but I'm stunned something so basic doesn't work.


OK, I switched to my main computer so I can try different versions of Leiningen... It works fine in 2.8.1:

(! 832)-> lein repl
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
nREPL server started on port 53601 on host - 
REPL-y 0.3.7, nREPL 0.2.12
Clojure 1.8.0
OpenJDK 64-Bit Server VM 14+36-1461
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

user=> (def instant #inst "2018-03-28T10:48:00.000")
user=> instant
#inst "2018-03-28T10:48:00.000-00:00"


(and, note, that's on Clojure 1.8 but I also tried a 1.10 project and it worked fine)


Tried 2.8.2 -- that won't even start a REPL, it just crashes -- 2.8.3 -- #inst works fine -- 2.9.4 -- No reader function for tag inst.


2.9.0 to 2.9.4 -- none of those work.


So you could do lein upgrade 2.8.3 and work with that version @i -- or you could switch to the Clojure CLI instead 🙂


You probably want to follow up in #leiningen at this point ...


I think 2.9.4 just came out with a newer nrepl too


Yup. Tried it above. Doesn't work. It has nREPL 0.7.0.


Leiningen has seemed really flaky through the last set of releases. I've seen no end of problems reported with the 2.9.x releases that are nearly always solved by "upgrading" to 2.8.3 @dpsutton

🙃 3

the move from contrib nrepl 0.2.13 to nrepl 0.6 was painful but necessary (imo)


I think that was a giant mistake, frankly.


but its been a bit too rocky agreed. difficult to fix something so widespread and core


All because Bug didn't want to work on nREPL inside the Contrib ecosystem.


i don't begrudge that decision at all


but these upgrades haven't been pain free. and the pain would have been present in contrib or nrepl. its a fork he's taken and improved so i'm all for it


The pain would not have been present if everything had continued to depend on org.clojure/tools.nrepl and he'd just upgraded that -- most of the breakages I saw people run into was due to the namespace changes and the REPLy 0.4.3 / 0.4.4 cutover (I think that's the versions that caused the biggest pain, due to switching from tools.nrepl in Contrib to nrepl outside it). That could have been handled much more smoothly by making the code work with both sets of namespaces and just give warnings about the old names.


(REPLy 0.4.3 depended on tools.nrepl; REPLy 0.4.4 on nrepl I think?)


Not that I really care much: I stopped using Leiningen in 2015. I just felt bad for all the newcomers who kept bumping into those sharp edges 😞


(but, yeah, there does seem to have been a number of "quality control" issues with Leiningen releases lately and I can't blame the nREPL/CIDER folks for that 🙂 )


@dpsutton Do you have any insight into the #inst issue reported above? Is it something you've heard folks mentioned in that tool chain?


Given the number of moving parts, I'm not sure where to go digging for a workaround/fix...


i'm creating an issue for it


yeah. i'm not sure where to start. lots of bits involved


are you recreating on windows? i'm on mac and want to list all the OSes involved


I repro'd on Windows/WSL first, then macOS


(def ^{:added "1.4"} default-data-readers
  "Default map of data reader functions provided by Clojure. May be
  overridden by binding *data-readers*."
    {'uuid #'clojure.uuid/default-uuid-reader}
    (when-class "java.sql.Timestamp"
      {'inst #'clojure.instant/read-instant-date})))
i wonder if lein starts up without java.sql.Timestamp? first time actually looking at this definition


I haven't tested on Powershell/CMD. I'm not sure I even have lein on those setups on my Windows laptop.


Hmm, would that be a java.base issue, related to Leiningen's bootclasspath stuff and how that changed between 2.8.x and 2.9.x?


it recognizes the #uuid tag


Java versions were 13 on Windows/WSL and 14 on macOS, if that helps.


yeah i was 13


i think i can change it easily with sdkman


(! 852)-> JAVA_HOME=$OPENJDK8_HOME lein repl
REPL-y 0.4.4, nREPL 0.7.0
Clojure 1.10.1
user=> (def instant #inst "2018-03-28T10:48:00.000")
So it's Java 9+


That was lein 2.9.4.


the plot thickens


lein 2.8.3 works on Java 8 / 13 / 14, 2.9.4 works on 8, but not on 13 / 14 so that's why I said "9+".


works = #inst is understood.


(weird stuff like this makes me so glad I switched to a simpler toolchain with the Clojure CLI!)


yeah i agree with that. unfortunate the tooling support hasn't carried along. plugins and middleware abound


love stu's debugging with a sub-repl with an error handler that just prints "BOOM"


Scissors are good tools 🙂


I'll be interested to see how that issue plays out (since you tagged me in it) 🙂


wanted to attribute accolades for the help 🙂 not to pester


OK, I'm out. I think I'll be in the kitchen most of tomorrow -- the wife has a major baklava project planned, with three different recipes to try... her form of A/B testing. Probably be online in the evening/Sunday.

💯 3

that sounds awesome. enjoy it. my friend in SF loves going to this place to get backlava ice cream. (souvla maybe?)


Souvla is a meat dish, at least here in Cyprus.


Oh neat. It’s the name of the restaurant there.


Ah, I thought you were talking about the name for "baklava ice cream". :)


yeah. its greek yogurt with baklava and honey


How prudent of me to have eaten before making that comment. :D


and i bet homemade is even better


Hey All! I’ve been exploring reloadable REPL workflow + web server. In the past, I have used ring.reload/wrap-reload and for reloading, but i’ve seen others design their code to be reloadable using Var e.g. - I have tried to setup a simple version for my own side project: With the above, after changes, I have to evaluate health-check-route and then app . What I would like is to only have to modify + re-evaluate health-check-route and see the changes picked up without needing to re-evaluate app . What would I need to modify in the design of this code to make that possible? :thinking_face:


I think if you change your router definition to :

(defn router []
that should work


I added #' before health-check-route


you may also have to change your app definition to:

(def app
    (reitit/ring-handler (#'router) (constantly {:status 404})))


I've never used reitit, so I'm not sure if the second change is necessary


You don't need #' in a function call -- only a direct var reference (i.e., function passed as a value/argument)


But with a def the problem is that's going to be evaluated once, at namespace load time, so it will have been evaluated and captured the current values of things at load time.


If you change app to a function (of request), it would likely be easier to work with in a REPL

(defn app [req]
  ((reitit/ring-handler (router) (constantly {:status 404})) req))
I think that should work (but, like @U7RJTCH6J I don't use Reitit).


some of the routing libraries also use macros, which can make a difference too.


Which also means that the change to [(#'health-check-route)] is probably not needed either -- it already refers to a function in a call.


Thanks for the suggestions above 🙏 !! It seems that Sean’s second one worked. I updated the gist with the working version (also kept a reference to the one that did not work….could be helpful for others). @U7RJTCH6J good point on the macros! I will have to keep that in mind as i’m going.


I’m writing a library that wraps an underlying java library, and I want two things… Firstly I want to wrap one of those java objects (which is at the root of a tree of other object instances), and present a clojure map like interface to the wrapped object. I could obviously do: (defrecord [internal-java-object other-opts ,,,]) However I’d also like to hide internal-java-object from API users (for various reasons). I could obviously reify and implement all the appropriate parts of a clojure map; however I’ll need to repeat this for quite a large number of objects in that subtree. Potemkin can do this with reify-map-type, but in the past I’ve heard people complain about potemkin causing lots of problems… Whenever I’ve used potemkin in the past it’s been fine, and caused me no problems. What do people think, should I do?


what are the various reasons you want to hide the internal java object?


I knew folk would ask 🙂


😄 I think it is a core constraint here so it’s worth asking. My knee jerk reaction, not knowing anything about your use case, is twisting yourself into knots purposefully obscuring things is a code smell


I’m not convinced I’ll hide it yet, but perhaps the main reason is that we have code that already uses this library, and there are numerous anti patterns in its use that I want to stamp out… i.e. the new library aims to fix these problems. I’m considering exposing the valid stuff as data etc, but avoid people accessing the internal stuff.


Essentially I’d like the option to grow the API… if people really DO need access to the internals if I were to use reify-map-type I could grow the API by adding the key for it.


@rickmoynihan Do you need to maintain the Java object directly, or could you just turn it all into data instead? Or have the option to turn it all back into a Java object again at will?

☝️ 3

(I'm thinking


my initial thought was to use datafy (similar in idea to I guess), but that would also mean exposing the underlying object potentially


@seancorfield Yeah I need to hold a reference to the internal library as a reference in either a closure or a key in the map


And why do you need to prevent users from accessing it? Could you just document that folks should not touch the object?


It’s ok I’ve decided to make it a record. Essentially I was wanting to avoid thinking about subtle breakage that might occur if people mixed decorated behaviour with undecorated behaviour when using new and old code together that both provide wrappers on that old library. However I don’t think the problems actually exist; but it’s quite hard to think about — I was thinking there may be ways people could combine things that would for example break equality of some operations.


Obviously I’m well aware of the speculation stuff; and we do that… but there are always a few more subtleties in the real world.


@rickmoynihan Do you own all the protocols? If so you could use :extend-via-metadata on the protocols and just use maps with metadata


Yeah I considered that option too, but decided to just go with a record


I was going to set that option on some of my protocols, but was meaning to ask about the perf cost of that option. At least one of the protocols may be dispatched 100 million times or more


Most will be called infrequently though, so I’ll set it on those

Drew Verlee19:07:40

i'm trying to translate the Concurrent crawler with channels from here One area of confusion is how to mimic how the master is iterating over the channel. I assume this function keeps blocking forever, though the docs here don't make that clear Ideally the solution would have a timeout. But i don't know how to recreate the effect of "keep taking" from the channel. I tried a loop, but its likely failing for a number of reasons, including the chance that the loops take (which is already awkward) completes before the go block puts anything on the channel.

(let [url->urls {:a [:b]
                 :b [:d]}
      ch        (chan 10)
      results   (atom [])]

  (>!! ch (keys url->urls ))

  (loop [urls  (<!! ch)
         seen? #{}]
    (let [url (first urls)]
        (nil? url)  (println "done")
        (seen? url) (recur (<!! ch) seen?)
          (go (>! ch
                (do (swap! results conj url)
                    (get url->urls url  []))))
          (recur (<!! ch) (conj seen? url))))))


with just an initial look, the atom seems out of place. I would probably do something like:

(let [url->urls {:a [:b]
                 :b [:d]}
      ch        (chan 10)]
  (async/onto-chan ch (keys url->urls) false)
  (loop [url (<!! ch)
         seen? #{}]
    (prn "looking at " url)
      (nil? url)  (do (println "done") seen?)
      (seen? url) (recur (<!! ch) seen?)
          (async/onto-chan ch (get url->urls url) false))
        (recur (<!! ch) (conj seen? url))))))


seen? captures all the urls that have been visited. results seems unnecessary and mixing atoms with async blocks is usually not the right thing


if you want to capture other data about the process. depending on the context, I would do one of the following: 1. have the accumulated data as a value in the loop if you want to accumulate the result of the process and return it when the result is finished 2. have an output channel that this process puts values onto if you'd like to have the results as they are available


Yeah only glanced at this but the loop probably wants to be in a go block itself, and pass the result back to the main thread via a chan. To implement timeouts you want to use the timeout function with an alts!(!) over it


atoms can block when contended, which could stall all of the go blocks


probably not really an issue here; but it rings alarm bells.


the main issue with the atom isn't whether it's blocking or not (although that might be a practical issue), but that logically, core.async in built on CSP. in this paradigm, you want to communicate communicate your data via channels, as data within recursively built state with loop variables, or return values and function parameters


one other note is that if (get url->urls url) is replaced by an actual http request, you probably don't want to do that in the go block thread pool. you either want to spawn a new thread or provide some other thread pool for IO or long running computations


:thumbsup: Yeah definitely agree that shared mutable state isn’t part of CSP, but clojure doesn’t care and allows you to mix paradigms. Rich is famous for offering things a la carte. Practically though you will care if you mix blocking ops with async’s coroutines and you stall every coroutine in your program. Philosophical issues are one thing but you really don’t want to be debugging these interactions in a production system. Essentially spinning on an atom won’t yield the go block, and eventually under sufficient load may starve all of the go blocks. This is more commonly seen when people combine go blocks with blocking i/o, which suffers the same issue.


while it is possible to mix different paradigms for reasoning about concurrency/parallelism, I wouldn't recommend it.

👍 3
Drew Verlee21:07:56

@rickmoynihan @U7RJTCH6J Thanks a lot. the results atom was added at the last second to try to debug things. i should have removed it before posting. But the discussion was still very enlightening to me. > one other note is that if `(get url->urls url)`  is replaced by an actual http request, you probably don't want to do that in the `go` block thread pool. you either want to spawn a new thread or provide some other thread pool for IO or long running computations This would be because an http call will require an actual jvm/os thread and tie up the whole green thread/core async.go.thread pool anyway?

Drew Verlee21:07:23

I wasn't aware of onto-chan at all!

Drew Verlee21:07:52

I suppose mostly i'm weired out by the recur with a channel take as the arg. Is that ideal?


yes, there's only a limited number of threads in the green thread/core async.go.thread pool so anything that blocks one of them is undesirable


recurring with a channel take is pretty common and normal


at least, that's the style I usually use, you could also use this style:

(let [url->urls {:a [:b]
                 :b [:d]}
      ch        (chan 10)]
  (async/onto-chan ch (keys url->urls) false)
  (loop [seen? #{}]
    (prn "looking at " url)
    (let [url (<!! ch)]
       (nil? url)  (do (println "done") seen?)
       (seen? url) (recur seen?)
           (async/onto-chan ch (get url->urls url) false))
         (recur (conj seen? url)))))))

Drew Verlee21:07:04

i'm not terminating the loop correctly it seems. When you try to call onto-chan with a channel and nil (when no urls are left) it returns a channel? i guess i need to check for that and ask if its closed?

Drew Verlee21:07:22

or maybe not call onto chan in that case.

Drew Verlee22:07:30

or no. i should timeout

Drew Verlee22:07:56

umm, onto-chan seems to return nil when nothing is left, possible bc its closed. i guess i need to close it.


it depends. if this is an ongoing process that can continue to receive new urls to explore [from another process putting values onto ch, then this process should stop as written when ch is closed. if this is a process that should stop when all the urls initially provided and the urls they can reach have been explored , then I probably wouldn't read new urls from ch and I would set it up slightly differently

Drew Verlee22:07:34

It was intended to be the later.

Drew Verlee22:07:27

In the model when where you can get new urls then only a timeout makes sense.


or have the channel be explicitly closed by the producer

Drew Verlee22:07:30

Any idea why this taking from the channel will sometimes return a multichannel? It happens when their are no urls to follow. (url->urls url) returns nil. What i expect is that it will either take a url from a channel that i put on or it will find nothing and return nil.

(let [url->urls {:a [:b]
                 :c [:d]}
      ch        (chan 10)]

  (onto-chan ch (keys url->urls) false)

  (loop [url   (<!! ch)
         seen? #{}]
      (nil? url)  (println "finished: " seen?)
      (seen? url) (recur (<!! ch) seen?)
        (when-let [urls (url->urls url)]
          (>!! ch (onto-chan ch urls false)))
        (recur (<!! ch) (conj seen? url))))))


you're putting the result of onto-chan on the ch channel


onto-chan returns a channel itself that only signals that all of the values have been put

Drew Verlee22:07:16

so i am. i should take a break it seems. thanks!


I was thinking something like:

(let [url->urls {:a [:b]
                 :b [:d]
                 :d [:a]}
        to-download (chan 10)
        downloaded (chan 10)]
    (loop []
      (when-let [url (<! to-download)]
        (>! downloaded [url
                        (<! (thread (get url->urls url [])))])
    (loop [to-visit (set (keys url->urls))
           pending? #{}
           seen? #{}]
      (prn to-visit pending? seen?)
      (if (and (empty? pending?)
               (empty? to-visit))

        (let [next-url (first to-visit)
              ports (if next-url
                      [[to-download next-url]
              [val port] (alts! ports)]
            (= port to-download)
            (recur (disj to-visit next-url)
                   (conj pending? next-url)

            (= port downloaded)
            (let [[from-url to-urls] val]
              (recur (into to-visit (remove #(or (pending? %)
                                                 (seen? %)))
                     (disj pending? from-url)
                     (conj seen? from-url)))))))))


it might be overcomplicating it, but I think this should work. I think explicitly keeping track of the in-flight requests (in this case pending?) is a key idea


otherwise, you don't know when to stop.

Drew Verlee00:07:35

@U7RJTCH6J what's putting anything on the to-download channel in that?


if there are any urls in to-visit, then the alts! try to read from either to-download or downloaded . otherwise, it will just try to read from downloaded and park until a result is available

Drew Verlee00:07:28

i see. i didn't understand how alts worked it seemed.

Drew Verlee00:07:49

@U7RJTCH6J this has been very helpful. Would it be ok to show your solution alongside mine for the purely fun not for credit class this is part of? I have a group of people working through a distributed systems class and the some of the lab's were translated from Go to clojure. Now i feel responsible for rapidly learning how to translate the rest.


Is there any way to call a default method of an interface in reify when implementing that method?


trivial example


public interface Thing {
   default int up(int x) { return x + 1; }


(reify Thing
  (up [_ x] (+ 2 (somehow-call-super-up x))))


fine with a solution that requires reflection, since i am doing some evil with this


reify doesn't have access to parent class concrete implementation, proxy does


see proxy-super, but NB proxy-super is not thread safe


so that is somewhat working


(.forEach (proxy [Iterable] []
            (iterator []
              (.iterator (doto (ArrayList.)
                           (.add 3)))))
          (reify Consumer
            (accept [_ x]
              (println x))))          
Execution error (UnsupportedOperationException) at dev.mccue.class_system.proxy$java.lang.Object$Iterable$e10044ca/forEach (REPL:-1).
(.forEach (doto (ArrayList.)
            (.add 3))
          (reify Consumer
            (accept [_ x]
              (println x))) )         
=> nil


but even before proxy super comes into play I get some nonsense with proxy


(.forEach (proxy [Iterable] []
            (iterator []
              (.iterator (doto (ArrayList.)
                           (.add 3))))
            (forEach [c]
              (proxy-super forEach c)))
          (reify Consumer
            (accept [_ x]
              (println x))))          
Execution error (UnsupportedOperationException) at dev.mccue.class_system.proxy$java.lang.Object$Iterable$e10044ca/forEach (REPL:-1).


i remember that proxy dispatches on name only, but that shouldn't be the issue here