Fork me on GitHub

Is there already a project that combines Clojure (ring/compojure) and mitm proxies like Betwixt or Openresty?


I guess littleproxy and this is what I need


newbie question.. But Im looking for a nice way to cope with dos2unix in a file i am slurp-ing.


Have a file thats windows based and has a carriage ruturn and a new line. \r\n . Am running on a mac, and when I slurp and then split the file, the final string has the “0\r"


(clojure.string/replace "_queue-1\\r" #"\r" "") => "_queue-1\\r"


(its basically ^M im tring to cope with)


Don’t you mean \r\n?


Hmm, correct, but Im using split to parse through the file


(clojure.string/split my-string #"\n")


and therefore its not included. Logically now Im trying to change that to #”\r\n” but having some interesting


This could be a time where you might want to reach for and line-seq.


Check out the first example.


Thanks @akiva I’ll give it a go


looks like the wayto go




i think i’m missing something here


sorry, would this question have been better in beginners?


krchia: you did the same thing twice and got a different result?


i wanted something like {:left 1 :left 2}


then i just realized now what a dumb thing that is


i’m just trying to associate a bunch of values with the direction left


actually, i could’ve just done {:left i-already-had-this-list} 😞


sorry to waste your time


Can anyone recommend some reads/intros on protocol based design, I rarely use them and I feel like I'm missing out 😄


it's mostly useful when designing libraries and allowing your users to extend your library. so if you're not into that a lot, you're probably not missing out a lot either 🙂


martinklepsch: not sure I could recommend anything in particular... What are you struggling with? As lmergen says it's really about where you want extensibility to other things... I use protocols quite a lot though... but not only in libraries for end users to extend. Places to look to apply them are basically anywhere where you're doing a conditional on type, where the condition is doing the same kind of operation... i.e. anywhere you want polymorphism on the first argument. If you're doing java interop they can also be handy


For example here: I use a protocol to define two way type coercions between types in a library and in clojure/java


There's nothing in particular I'm struggling with I guess, I just feel like I don't have the intuition on when protocols might be a better solution. Also I remember that protocols can make testing easier which is something that sounds generally good 🙂


the benefit isn't just that end users can extend it, it also expresses the fact that it's polymorphic


I think polymorphism is the main thing you want to think about... they can to a lesser extent model domain entities/interface expectations... but I find that very much plays second fiddle...


though if you do start using protocols you'll probably end up with records modelling some domain entities e.g. User etc...


cyanite is a good example


it has multiple implementations of its building blocks for different contexts/loads


Hi, Is there a way to write clojure .clj file to a .edn file (I would like to convert functions , maps,etc)


I found com.theoryinpractise to generate edn file automatically when a maven build is done.


The pom.xml has the following plugin: <plugin> <groupId>com.theoryinpractise</groupId> <artifactId>clojure-maven-plugin</artifactId> </plugin>


@martinklepsch: I find protocols are best when you have more than one polymorphic function being a part of a whole thing. If you have just one it's OK to use multimethods. However, if the functionality is part of a library I find that protocols are always better as they are more "open".


multimethods are also more powerful as you can dispatch on an arbitrary function, but I find that useful in very rare cases


performance-wise protocols are always better


dm3: Interesting, I have a slightly different "style rule" on choosing between protocols/multimethods... I'll use a protocol even for a single function on the whole... And would always prefer protocols over multimethods for all cases where we're dispatching on class. One difficulty with multimethods is that their dispatch rules can be arbitrarily complex, so by saying it's a protocol you immediately communicate precisely how dispatch works.


I actually use defmulti quite a bit. The moment a bunch of if statements become cond statements and complexity ensues, I find defmulti really steps in, helps with extensibility, etc. I do keep the dispatch-fn very simple, though.


akiva: multimethods are super cool - and the extensibility (as with protocols) is great - I'm just saying that if you can do it with a protocol, use a protocol - otherwise use a multimethod.


For me protocols are specifically for types. Sometimes I want to dispatch based on a key in a map.


e.g. anytime you see (defmulti foo class) you should replace with a protocol


That I agree with.


akiva: agreed - and that's what I'm saying


you can't use a protocol to dispatch on a map key... so you need a multimethod


likewise if you want to dispatch on (comp :some-keyword meta) or a set/subset etc...


Yep. But I rarely use records. I tend to stick with maps unless there’s a good reason to enforce formality.


akiva: records are maps - you can assoc new keys on them if you want... so the reason I'd use a record over a map is if I ever wanted to create a map that had a :type key - that I would want to dispatch on.


and if I wanted type based polymorphic dispatch on a map - I'd prefer to make those maps records/protocols than use a multi method


That’s one of the good things about Clojure and Lisps in general: there are multiple ways of getting things done and not one of those ways is necessarily the right way.


sure... I'm not saying my preference is the right way... I'm just justifying my reasoning


Same here.


this discussion is really about crude heuristics for when to prefer one to another... there are tradeoffs behind every decision and specific circumstances lead you down different paths.


Indeed. I guess my guiding principle is to shy away from formal structures such as records unless you absolutely need it. I’ve messed around with it both ways (loose maps vs. ’strict' records) and, as you point out, there are pros and cons to both.


I'm trying to build some core.async consumers/subscribers, and I'm a little confused on how to approach the problem. I want to make sure I get this right first time. Is there any information on how to set up something like this? My only conclusion is some kind of async/thread (avoiding go?) loop in a component. Is there any other approaches I could/should take?

Alex Miller (Clojure team)15:07:33

I don’t think there’s enough there to answer that question. why avoid go? (btw, there is #C05423W6H too)

Alex Miller (Clojure team)15:07:03

@rickmoynihan: I capture my thoughts on this subject in Clojure Applied if you’re curious. too much to repeat here.

Alex Miller (Clojure team)15:07:51

@dominicm: I also cover what you’re asking about at a broad level in Clojure Applied (using async, and async + components), but again too much to repeat here


@alexmiller: Have a copy at work, I'll make sure to check it out. I thought go was a bad idea due to the limited thread-pool and that go was more appropriate for async pushing onto channels instead of processing them. I'm not too well versed in async understanding.

Alex Miller (Clojure team)15:07:48

it’s something to be aware of, but using go loops is fine for processing async data. it’s really a question of whether you need a dedicated thread or are able to multiple over a pool.

Alex Miller (Clojure team)15:07:04

also, as of the last release, you can configure the size of that pool

Alex Miller (Clojure team)15:07:44

from a design point of view, I think it’s more important that the parts you expose are the channel ends


thanks alexmiller I'll take a look


Question: I'm able to call Clojure functions starting from strings by using an approach like (ns-resolve *ns* (symbol "+")). Trying the same approach with Java instance methods fails (`(ns-resolve ns (symbol "System/getenv")) => nil`), I assume because the namespace does not resolve the .instanceMember macro or the dot operator. Is there another approach that would allow me to call Java methods starting from a string describing the method?

Alex Miller (Clojure team)15:07:31

@dominicm: and it’s a bit more flexible if a component does not have to create the channels but is instead given them - that allows a higher-level thing to create the channels, with appropriate buffer semantics, and fit those pipes into the components


@alexmiller: That's the exact design I was leaning towards. The only things that's internal is a "kill-chan" which serves to let the thread know when to shut down.

Alex Miller (Clojure team)15:07:53

@daveliepmann: starting from strings, you are basically in the realm of Java reflection

Alex Miller (Clojure team)15:07:04

you can find a class with (Class/forName “java.lang.System”)and then use (.getMethods class …) etc to find a method, which can then be invoked with an array of args.

Alex Miller (Clojure team)15:07:32

you could also read-string and eval


@alexmiller: TBH I'm more interested in when to choose between transducers and lazy sequences - than records/maps which I think I have a good grasp on... Regarding transducers/lazy-sequences which method should be the default for sequence-like manipulation... how to consider the tradeoffs etc... I understand the costs of laziness - but for where the performance doesn't matter so much what is your default option these days?


I still reach for lazy-seqs - but it's probably because I'm a lot more familiar with them than transducers

Alex Miller (Clojure team)15:07:18

@daveliepmann: (eval (read-string "(System/getenv)”))


I hadn't seen it no... thanks... the guide looks good too... I think I understand many of the pros/cons of each.... I think I'm more looking for an "if in doubt - use XXXX` kinda rule... I suspect lazy seqs are still the default default... but I definitely need to write more transducers

Alex Miller (Clojure team)16:07:33

if your collection is small it doesn’t matter

Alex Miller (Clojure team)16:07:48

if your number of transformations is 1 or 2, it probably doesn’t matter

Alex Miller (Clojure team)16:07:01

so use whatever’s comfortable there (prob most situations)

Alex Miller (Clojure team)16:07:28

most people have found that into is their favorite place to use transducers

Alex Miller (Clojure team)16:07:42

as it allows you to transform into exactly the output collection you want


yeah... re into that's definitely the impression I'm getting

Alex Miller (Clojure team)16:07:03

think harder about transducers when dealing with: lots of data, lots of composed transforms, places where you want to reuse transforms in multiple contexts (async + collections), places where your source is an external resource and you want to know when you’re done (eagerly)


There are lots of times where transducers are simply easy quality of life improvements, like mapping over a channel


My feeling is transducers require more thinking but provide more opportunities for efficiency etc... as is frequently the case when decomplecting


dg: I think if I used core.async - I'd reach for them more often... but I've not needed to do much event/stream stuff over the past few years.... I have a number of times when I've needed to reached for juc.BlockingQueue's etc to marshal things across threads... but it's usually been pretty simple stuff... I don't think core.async would've helped massively (and I suspect it won't be quite as fast).

Alex Miller (Clojure team)16:07:08

one benefit of async go over BlockingQueues is you don’t have to tie up a thread on the consumer side - if nothing is in the queue, the block is parked and awakes only when there is work to do

Alex Miller (Clojure team)16:07:25

really a benefit over all the Java queues


I didn't mean to say core.async is the solution, it's just an example of a place where transducers fit so well that they may be easier than other solutions


They don't always need to be a heavily considered "is this the right time?" option. If the solution is expressed very simply with the built-in transducers, I'd default to just using that.


yeah that is definitely a benefit... but we're probably only running half a dozen of these extra threads in the worst cases.


dg: agree on just using it - if its not that important... that's why I've been reluctant to change the BlockingQueue's for async


I think the other thing that's yet to become clear to me are all the tradeoffs around dealing with I/O... I'm currently wrapping an API that uses a thread-based I/O model - and this code basically does it the old-school clojure way of hiding the I/O behind a lazy-seq abstraction.... Obviously it can and will leak resources in some circumstances; but, with some careful coding in a few places in 3+ years of production usage its not been a big source of problems. Regardless I do consider the potential for resource leaks a problem; and would like to move to a better alternative... but I now basically have 3 alternatives to look at... 1) Use core.async channels 2) Use reducers 3) Use transducers There are some additional issues in moving away from the lazy-seq model in that some libraries such as also manage I/O in this way. Curious if I'm missing something and what peoples thoughts are on this.


btw alexmiller thanks for the tips it's useful


@rickmoynihan: adding to your pile, i've been using for async i/o for a while - the Deferred and Stream impls can convert to/from other abstractions (e.g. core.async chan to Stream) which has made integration with existing libs a snap


mccraigmccraig: thanks...


Anybody knows a good way to turn Schema's errors into a user-facing error messages? 🙂


Is there any existing macro for defining my own syntax, eg a new form that uses banana brackets: (| ... |)


or do i have to roll that myself?


I believe what you’re asking for is reader macros, which are not open for extension in Clojure


@semperos: ah right, reader macros, haven't used them much. So I can't extend my own reader macros? So my use case is more or less not possible?


@josh.freckleton: correct, as a design decision, Clojure does not allow end-users to add further reader macros; what use-case were you envisioning for your new syntax?


@semperos: I'm playing with some ideas, and trying to learn what's possible/not. Right now, I'm wondering how I could go about creating custom collection types, defined by syntax. For example, this is similar to maps, but I'm thinking a type like data RoseTree a = RoseTree [RoseTree a], or my choice of clojure forms: (| <node value> <branch 1> <branch 2> <etc.> |)


then I could define utility functions like tree?


I’d put out two rules of thumb on that front


first as a general rule of thumb, you want to stick with using Clojure core collections and its sequence api as much as possible


second, before moving onto syntax, I’d play with defining custom deftypes/defrecords if you need to experiment with how your tree needs to behave


I’m having some trouble understanding how keywords work with respect to clojure.spec… Why is it that (s/def :spec-test-ns/name string?) works, but (s/def (keyword “spec-test-ns” “name”) string?) fails with CompilerException java.lang.AssertionError: Assert failed: k must be namespaced keyword or resolvable symbol? The type of both is clojure.lang.Keyword.


might get more focused discussion @joshmiller if you ask in #C1B1BB2Q3


@josh.freckleton: often one finds that a combination of well-designed and well-named functions using regular Clojure maps, for example


gets you where you want to be + the added benefit of being a data structure with lots of built-in support and idioms


@semperos: > first ... sequence api ... agreed, so I was thinking of extending that. > second ... good advice. I get carreid away sometimes > well-desigined ... functions ... I think you're right. It'd still be cool to have reader macro-access!


I’ve done personal projects on that front @josh.freckleton where I play with the LispReader and Compiler classes; I think you can learn a lot about how Clojure on the JVM is put together by those experiments, but yeah, in Clojure-as-released you don’t have the flexibility


EDN, however, does have a story around data literals which is related


and probably worth learning about if you haven’t already


> EDN has a story around data literals what do you mean, I thought I was familiar with EDN but are you saying it's extensible?


it’s not a syntactic free-for-all, but it is extensible


ah, that's what the "E" stands for! haha


oh that's right! I vaguely recall elarning this, so with the # symbol, I can define arbitrary read-time macros? (i'm still reading throughyour link, forgive if this is obvious)...


it provides you a hook into the reader, so that what follows your #-prefixed tag is passed to a function you write


you’ll need to read into how one tells the reader where to find those functions (differs for core Clojure and something like tools.reader) and restrictions, but that link is a starting point for EDN’s functionality there


that and the next section


but again, I stress the need to focus on the API and its behaviors/boundaries before getting too caught up on the syntactic front


> before getting too caught up on the syntactic front yes, I agree, thanks for pointing me to these references!


I'll put them away to mull over at a future time when/if they'll help after I've focused on behaviors/boundaries 🙂


@sekao: Most exciting mailing list announcement ever. 👏


Is there an easy way to turn a tree of nested maps into a "flattened" map of the leaf nodes, keyed by vector paths to each leaf?


I'm working through walking the tree correctly, but it seems like the kind of thing that might turn out to be two core functions plugged together if you know what you're doing. 🙂


From there you could map the result across a get-in call if performance isn't super important


is there an idiomatic way to force a (conj nil foo #{})? (force conjs’ struct type as opposed to automatically being a list?)


i could do some (if (nil? thing) #{foo} (conj thing foo)) but that feels .. verbose


Are you looking for (fnil conj #{})?


that’s pretty sweet


@peeja: I did that exact thing the other day, using the same stackoverflow answer and this was my solution, using keys-in from that SO answer. It's terribly inefficient I think, though.

(defn key-paths [m]
  (into {}
        (map (fn [path] [path (get-in m path)]))
        (keys-in m)))


I'm currently running with

(defn leaf-seq
  [branch? children root]
  (let [walk (fn walk [node]
                (if (branch? node)
                  (mapcat walk (children node))
    (walk root)))

(defn flatten-with-paths [tree]
  (fn [[path node]]
    (associative? node))
  (fn [[path node]]
    (reduce-kv #(conj %1 [(conj path %2) %3]) [] node))
  [[] tree]))


where leaf-seq is a modified tree-seq that doesn't include branches


Is there a way to run a function after all tests have been run with lein test?


I saw that, seems to be per namespace. I'm trying to figure out the best way to run something after all have been run.


Add a shutdown hook in your code, perhaps?


(the JVM lets you register a function to run as the JVM is shutting down)


I'll take a look at it, thanks!


So, wrt I have the following in my ~/.lein/profiles. :injections [(require '[ :as repl])] However, it seems that whenever I call (repl/refresh), it gets rid of the injection, and the next time I want to call (repl/refresh) I have spell out the whole module name. Any tips?


Another random question. Is there a difference between some? and boolean?


Oh I see. (some? false) #=> true, but (boolean false) #=> false. It would appear this is the only difference.


(some? x) is literally (not (nil? x)), and (boolean x) is a coercion to a type


Even if they're usually the same result, I'd differentiate based on which kind of thing I'm trying to do