This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-07-13
Channels
- # admin-announcements (4)
- # beginners (14)
- # boot (41)
- # capetown (1)
- # carry (10)
- # clojure (168)
- # clojure-czech (11)
- # clojure-mexico (3)
- # clojure-quebec (1)
- # clojure-russia (63)
- # clojure-spec (108)
- # clojure-uk (44)
- # clojurescript (37)
- # component (5)
- # data-science (2)
- # datascript (5)
- # datomic (3)
- # defnpodcast (9)
- # dirac (14)
- # emacs (18)
- # events (1)
- # funcool (2)
- # garden (2)
- # hoplon (48)
- # leiningen (6)
- # numerical-computing (1)
- # off-topic (8)
- # om (61)
- # onyx (22)
- # proton (14)
- # re-frame (50)
- # reagent (2)
- # uncomplicate (1)
- # untangled (41)
- # vim (5)
- # yada (5)
Is there already a project that combines Clojure (ring/compojure) and mitm proxies like Betwixt or Openresty?
I guess littleproxy and this is what I need https://github.com/ganskef/LittleProxy-mitm
newbie question.. But Im looking for a nice way to cope with dos2unix in a file i am slurp-ing.
Have a file thats windows based and has a carriage ruturn and a new line. \r\n . Am running on a mac, and when I slurp and then split the file, the final string has the “0\r"
and therefore its not included. Logically now Im trying to change that to #”\r\n” but having some interesting
This could be a time where you might want to reach for
and line-seq
. http://clojuredocs.org/clojure.core/line-seq
krchia: you did the same thing twice and got a different result?
krchia: how about this? https://www.refheap.com/121472
Can anyone recommend some reads/intros on protocol based design, I rarely use them and I feel like I'm missing out 😄
it's mostly useful when designing libraries and allowing your users to extend your library. so if you're not into that a lot, you're probably not missing out a lot either 🙂
martinklepsch: not sure I could recommend anything in particular... What are you struggling with? As lmergen says it's really about where you want extensibility to other things...
I use protocols quite a lot though... but not only in libraries for end users to extend. Places to look to apply them are basically anywhere where you're doing a cond
itional on type, where the condition is doing the same kind of operation... i.e. anywhere you want polymorphism on the first argument.
If you're doing java interop they can also be handy
For example here: https://github.com/Swirrl/grafter/blob/master/src/rdf-common/grafter/rdf/io.clj#L207 I use a protocol to define two way type coercions between types in a library and in clojure/java
There's nothing in particular I'm struggling with I guess, I just feel like I don't have the intuition on when protocols might be a better solution. Also I remember that protocols can make testing easier which is something that sounds generally good 🙂
the benefit isn't just that end users can extend it, it also expresses the fact that it's polymorphic
I think polymorphism is the main thing you want to think about... they can to a lesser extent model domain entities/interface expectations... but I find that very much plays second fiddle...
though if you do start using protocols you'll probably end up with records modelling some domain entities e.g. User
etc...
Hi, Is there a way to write clojure .clj file to a .edn file (I would like to convert functions , maps,etc)
I found com.theoryinpractise to generate edn file automatically when a maven build is done.
The pom.xml has the following plugin: <plugin> <groupId>com.theoryinpractise</groupId> <artifactId>clojure-maven-plugin</artifactId> </plugin>
@martinklepsch: I find protocols are best when you have more than one polymorphic function being a part of a whole thing. If you have just one it's OK to use multimethods. However, if the functionality is part of a library I find that protocols are always better as they are more "open".
multimethods are also more powerful as you can dispatch on an arbitrary function, but I find that useful in very rare cases
dm3: Interesting, I have a slightly different "style rule" on choosing between protocols/multimethods... I'll use a protocol even for a single function on the whole... And would always prefer protocols over multimethods for all cases where we're dispatching on class. One difficulty with multimethods is that their dispatch rules can be arbitrarily complex, so by saying it's a protocol you immediately communicate precisely how dispatch works.
I actually use defmulti
quite a bit. The moment a bunch of if
statements become cond
statements and complexity ensues, I find defmulti
really steps in, helps with extensibility, etc. I do keep the dispatch-fn very simple, though.
akiva: multimethods are super cool - and the extensibility (as with protocols) is great - I'm just saying that if you can do it with a protocol, use a protocol - otherwise use a multimethod.
For me protocols are specifically for types. Sometimes I want to dispatch based on a key in a map.
e.g. anytime you see (defmulti foo class)
you should replace with a protocol
akiva: agreed - and that's what I'm saying
you can't use a protocol to dispatch on a map key... so you need a multimethod
likewise if you want to dispatch on (comp :some-keyword meta)
or a set/subset etc...
Yep. But I rarely use records. I tend to stick with maps unless there’s a good reason to enforce formality.
akiva: records are maps - you can assoc new keys on them if you want... so the reason I'd use a record over a map is if I ever wanted to create a map that had a :type
key - that I would want to dispatch on.
and if I wanted type based polymorphic dispatch on a map - I'd prefer to make those maps records/protocols than use a multi method
That’s one of the good things about Clojure and Lisps in general: there are multiple ways of getting things done and not one of those ways is necessarily the right way.
sure... I'm not saying my preference is the right way... I'm just justifying my reasoning
this discussion is really about crude heuristics for when to prefer one to another... there are tradeoffs behind every decision and specific circumstances lead you down different paths.
Indeed. I guess my guiding principle is to shy away from formal structures such as records unless you absolutely need it. I’ve messed around with it both ways (loose maps vs. ’strict' records) and, as you point out, there are pros and cons to both.
I'm trying to build some core.async consumers/subscribers, and I'm a little confused on how to approach the problem. I want to make sure I get this right first time. Is there any information on how to set up something like this? My only conclusion is some kind of async/thread
(avoiding go
?) loop in a component. Is there any other approaches I could/should take?
I don’t think there’s enough there to answer that question. why avoid go
? (btw, there is #C05423W6H too)
@rickmoynihan: I capture my thoughts on this subject in Clojure Applied if you’re curious. too much to repeat here.
@dominicm: I also cover what you’re asking about at a broad level in Clojure Applied (using async, and async + components), but again too much to repeat here
@alexmiller: Have a copy at work, I'll make sure to check it out.
I thought go
was a bad idea due to the limited thread-pool and that go was more appropriate for async pushing onto channels instead of processing them. I'm not too well versed in async understanding.
it’s something to be aware of, but using go loops is fine for processing async data. it’s really a question of whether you need a dedicated thread or are able to multiple over a pool.
also, as of the last release, you can configure the size of that pool
from a design point of view, I think it’s more important that the parts you expose are the channel ends
thanks alexmiller I'll take a look
Question: I'm able to call Clojure functions starting from strings by using an approach like (ns-resolve *ns* (symbol "+"))
. Trying the same approach with Java instance methods fails (`(ns-resolve ns (symbol "System/getenv")) => nil`), I assume because the namespace does not resolve the .instanceMember macro or the dot operator. Is there another approach that would allow me to call Java methods starting from a string describing the method?
@dominicm: and it’s a bit more flexible if a component does not have to create the channels but is instead given them - that allows a higher-level thing to create the channels, with appropriate buffer semantics, and fit those pipes into the components
@alexmiller: That's the exact design I was leaning towards. The only things that's internal is a "kill-chan" which serves to let the thread know when to shut down.
@daveliepmann: starting from strings, you are basically in the realm of Java reflection
@alexmiller: ah, I see
you can find a class with (Class/forName “java.lang.System”)
and then use (.getMethods class …)
etc to find a method, which can then be invoked with an array of args.
you could also read-string
and eval
@alexmiller: TBH I'm more interested in when to choose between transducers and lazy sequences - than records/maps which I think I have a good grasp on... Regarding transducers/lazy-sequences which method should be the default for sequence-like manipulation... how to consider the tradeoffs etc... I understand the costs of laziness - but for where the performance doesn't matter so much what is your default option these days?
I still reach for lazy-seqs - but it's probably because I'm a lot more familiar with them than transducers
@rickmoynihan: did you see my response here: https://www.reddit.com/r/Clojure/comments/4sgjhz/new_clojurians_ask_anything/d5a3fax ?
@daveliepmann: (eval (read-string "(System/getenv)”))
@alexmiller: thank you! 🙂
I hadn't seen it no... thanks... the http://clojure.org guide looks good too... I think I understand many of the pros/cons of each.... I think I'm more looking for an "if in doubt - use XXXX` kinda rule... I suspect lazy seqs are still the default default... but I definitely need to write more transducers
if your collection is small it doesn’t matter
if your number of transformations is 1 or 2, it probably doesn’t matter
so use whatever’s comfortable there (prob most situations)
most people have found that into
is their favorite place to use transducers
as it allows you to transform into exactly the output collection you want
yeah... re into
that's definitely the impression I'm getting
think harder about transducers when dealing with: lots of data, lots of composed transforms, places where you want to reuse transforms in multiple contexts (async + collections), places where your source is an external resource and you want to know when you’re done (eagerly)
There are lots of times where transducers are simply easy quality of life improvements, like mapping over a channel
My feeling is transducers require more thinking but provide more opportunities for efficiency etc... as is frequently the case when decomplecting
dg: I think if I used core.async - I'd reach for them more often... but I've not needed to do much event/stream stuff over the past few years.... I have a number of times when I've needed to reached for juc.BlockingQueue
's etc to marshal things across threads... but it's usually been pretty simple stuff... I don't think core.async would've helped massively (and I suspect it won't be quite as fast).
one benefit of async go over BlockingQueues is you don’t have to tie up a thread on the consumer side - if nothing is in the queue, the block is parked and awakes only when there is work to do
really a benefit over all the Java queues
I didn't mean to say core.async is the solution, it's just an example of a place where transducers fit so well that they may be easier than other solutions
They don't always need to be a heavily considered "is this the right time?" option. If the solution is expressed very simply with the built-in transducers, I'd default to just using that.
yeah that is definitely a benefit... but we're probably only running half a dozen of these extra threads in the worst cases.
dg: agree on just using it - if its not that important... that's why I've been reluctant to change the BlockingQueue's for async
I think the other thing that's yet to become clear to me are all the tradeoffs around dealing with I/O... I'm currently wrapping an API that uses a thread-based I/O model - and this code basically does it the old-school clojure way of hiding the I/O behind a lazy-seq abstraction.... Obviously it can and will leak resources in some circumstances; but, with some careful coding in a few places in 3+ years of production usage its not been a big source of problems. Regardless I do consider the potential for resource leaks a problem; and would like to move to a better alternative... but I now basically have 3 alternatives to look at... 1) Use core.async channels 2) Use reducers 3) Use transducers There are some additional issues in moving away from the lazy-seq model in that some libraries such as clojure.data.csv also manage I/O in this way. Curious if I'm missing something and what peoples thoughts are on this.
btw alexmiller thanks for the tips it's useful
@rickmoynihan: adding to your pile, i've been using https://github.com/ztellman/manifold for async i/o for a while - the Deferred and Stream impls can convert to/from other abstractions (e.g. core.async chan to Stream) which has made integration with existing libs a snap
mccraigmccraig: thanks...
Anybody knows a good way to turn Schema's errors into a user-facing error messages? 🙂
Is there any existing macro for defining my own syntax, eg a new form that uses banana brackets: (| ... |)
or do i have to roll that myself?
incidentally: https://github.com/clojure/clojure/blob/6d48ae372a540903173be2974b66b8911371e05d/src/jvm/clojure/lang/LispReader.java#L103
I believe what you’re asking for is reader macros, which are not open for extension in Clojure
@semperos: ah right, reader macros, haven't used them much. So I can't extend my own reader macros? So my use case is more or less not possible?
@josh.freckleton: correct, as a design decision, Clojure does not allow end-users to add further reader macros; what use-case were you envisioning for your new syntax?
@semperos: I'm playing with some ideas, and trying to learn what's possible/not. Right now, I'm wondering how I could go about creating custom collection types, defined by syntax. For example, this is similar to map
s, but I'm thinking a type like data RoseTree a = RoseTree [RoseTree a]
, or my choice of clojure forms: (| <node value> <branch 1> <branch 2> <etc.> |)
then I could define utility functions like tree?
first as a general rule of thumb, you want to stick with using Clojure core collections and its sequence api as much as possible
second, before moving onto syntax, I’d play with defining custom deftypes/defrecords if you need to experiment with how your tree needs to behave
I’m having some trouble understanding how keywords work with respect to clojure.spec
… Why is it that (s/def :spec-test-ns/name string?)
works, but (s/def (keyword “spec-test-ns” “name”) string?)
fails with CompilerException java.lang.AssertionError: Assert failed: k must be namespaced keyword or resolvable symbol
? The type
of both is clojure.lang.Keyword
.
might get more focused discussion @joshmiller if you ask in #C1B1BB2Q3
@josh.freckleton: often one finds that a combination of well-designed and well-named functions using regular Clojure maps, for example
gets you where you want to be + the added benefit of being a data structure with lots of built-in support and idioms
@semperos: > first ... sequence api ... agreed, so I was thinking of extending that. > second ... good advice. I get carreid away sometimes > well-desigined ... functions ... I think you're right. It'd still be cool to have reader macro-access!
I’ve done personal projects on that front @josh.freckleton where I play with the LispReader and Compiler classes; I think you can learn a lot about how Clojure on the JVM is put together by those experiments, but yeah, in Clojure-as-released you don’t have the flexibility
> EDN has a story around data literals what do you mean, I thought I was familiar with EDN but are you saying it's extensible?
ah, that's what the "E" stands for! haha
oh that's right! I vaguely recall elarning this, so with the #
symbol, I can define arbitrary read-time macros? (i'm still reading throughyour link, forgive if this is obvious)...
it provides you a hook into the reader, so that what follows your #
-prefixed tag is passed to a function you write
you’ll need to read into how one tells the reader where to find those functions (differs for core Clojure and something like tools.reader) and restrictions, but that link is a starting point for EDN’s functionality there
there’s a section on http://clojure.org as well: http://clojure.org/reference/reader#_extensible_data_notation_edn
but again, I stress the need to focus on the API and its behaviors/boundaries before getting too caught up on the syntactic front
> before getting too caught up on the syntactic front yes, I agree, thanks for pointing me to these references!
I'll put them away to mull over at a future time when/if they'll help after I've focused on behaviors/boundaries 🙂
yeah, that was fun :)
Is there an easy way to turn a tree of nested maps into a "flattened" map of the leaf nodes, keyed by vector paths to each leaf?
I'm working through walking the tree correctly, but it seems like the kind of thing that might turn out to be two core functions plugged together if you know what you're doing. 🙂
This might be a starting point: http://stackoverflow.com/questions/21768802/how-can-i-get-the-nested-keys-of-a-map-in-clojure
From there you could map the result across a get-in call if performance isn't super important
is there an idiomatic way to force a (conj nil foo #{})
? (force conjs’ struct type as opposed to automatically being a list?)
Are you looking for (fnil conj #{})
?
@peeja: I did that exact thing the other day, using the same stackoverflow answer and this was my solution, using keys-in from that SO answer. It's terribly inefficient I think, though.
(defn key-paths [m]
(into {}
(map (fn [path] [path (get-in m path)]))
(keys-in m)))
I'm currently running with
(defn leaf-seq
[branch? children root]
(let [walk (fn walk [node]
(lazy-seq
(if (branch? node)
(mapcat walk (children node))
[node])))]
(walk root)))
(defn flatten-with-paths [tree]
(leaf-seq
(fn [[path node]]
(associative? node))
(fn [[path node]]
(reduce-kv #(conj %1 [(conj path %2) %3]) [] node))
[[] tree]))
This helped me come to that solution: http://stackoverflow.com/questions/27208128/clojure-recursively-flatten-nested-maps
@tom: You might be able to use a :once
fixture: https://clojuredocs.org/clojure.test/use-fixtures
I saw that, seems to be per namespace. I'm trying to figure out the best way to run something after all have been run.
Add a shutdown hook in your code, perhaps?
(the JVM lets you register a function to run as the JVM is shutting down)
@tom: ah, maybe you could wrap lein test
as described here: https://github.com/technomancy/leiningen/blob/master/doc/PLUGINS.md#hooks
So, wrt clojure.tools.namespace.repl/refresh
. I have the following in my ~/.lein/profiles
.
:injections [(require '[clojure.tools.namespace.repl :as repl])]
However, it seems that whenever I call (repl/refresh)
, it gets rid of the injection, and the next time I want to call (repl/refresh)
I have spell out the whole module name. Any tips?