This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-04-25
Channels
- # announcements (2)
- # architecture (7)
- # aws (1)
- # babashka (105)
- # beginners (88)
- # braveandtrue (2)
- # calva (9)
- # cider (18)
- # cljs-dev (265)
- # cljsrn (22)
- # clojure (138)
- # clojure-argentina (3)
- # clojure-austin (1)
- # clojure-france (14)
- # clojure-italy (6)
- # clojure-uk (8)
- # clojurescript (283)
- # community-development (4)
- # conjure (11)
- # datomic (43)
- # docker (12)
- # duct (16)
- # emacs (1)
- # figwheel (1)
- # figwheel-main (27)
- # fulcro (10)
- # graalvm (6)
- # kaocha (4)
- # malli (9)
- # off-topic (13)
- # rdf (2)
- # reagent (12)
- # shadow-cljs (86)
- # spacemacs (1)
- # vrac (1)
@noisesmith How is it different than swapping into the atom in your code directly, (instead of calling tap>
)?
@potetm The tap>
can remain as-is, even in production. You can add (and remove) as many "taps" (watchers) as you want and they can each do different things.
It allows you to have tap>
in a running process, attach to it via a REPL, add some tap instrumentation, do some debugging, remove the tap instrumentation and then move on...
right, no weird defs / printlns sneaking into the codebase
I do wish tap> had a predicate / filter mechanism, but I'm also glad it's simple
:thinking_face: I’m gonna have to consider that more. I usually refrain from connecting to a live process, so logging seems to be a better fit for my usage.
Pretty rare that the problem is on a single box and not “the environment” (i.e. db+queues+network).
I don't use tap> outside my dev repl, but it's still more ergonomic to control compared to printlns or defs
Vs println, benefits are obvious. It’s hard for me to see how it’s better than def in a dev env.
with the swap! into atom, I get data from multiple sources in the order the tap> calls were hit
@seancorfield Do you often use it like that? Connect to a prod repl?
@noisesmith Sure. Swap into a def’d atom?
I use tap>
a lot of debugging -- because I use REBL and it can automatically accumulate results, and then let you browse them in tabular and graphical form.
right, my tap function does a swap, maybe also attaching a time stamp and call stack
sure - but this way I only edit in one place to get the data (call tap> on each target), then separately refine the data gathering as I figure out what I need
and yes I'm doing exactly what you said - attaching data with the fn I provide to swap - with no risk of errors caused by stray debug code sneaking into the file
then I can group-by, filter... use clojure generally
And, yes, I do connect to REPLs in production and will add taps for debugging. In those cases, I'll usually redefine a few functions with tap>
in them, then I'll call add-tap
to get hold of the data to process/store it somewhere for analysis, then remove the taps (but leave the tap>
calls in place).
That sounds roughly like adding printlns and re-defing, no? (albeit it has all the benefits of data instead of bytes)
One of the nice things about tap>
is that you can put it into a very high traffic function and it simply drops data when there's too much being processed, so you won't block your production app 🙂
Yeah I saw that in the impl. Still, not cost free to leave it there (though probably cost negligible).
Logging isn't cost free either, yet we use that for debugging a lot 🙂 and even putting a def
into a function to capture values isn't free (and only captures one values at a time).
Thing is: Logging can be configured ad nauseam, even at runtime. And a lot of effort has gone into optimizing them. I’ve seen it fail, but I certainly trust it a lot more than whatever fn I cook up to slide into a hot path.
and yes I'm doing exactly what you said - attaching data with the fn I provide to swap - with no risk of errors caused by stray debug code sneaking into the file
“add it to something that’s called 1000x a second” -> end up with a bunch of data, but not a stalled server
tap> is internally rate limited and drops data if your debug fn is too slow. and you remove the tapping fn and now it's a noop
acts like a dropping buffer on a chan
oh I misread what you said here
Yeah, I originally misunderstood Sean’s point. I thought he was talking about the impl, but I think it was more to do with the user experience.
I would never have thought of tap>
-- but it has been amazingly useful since Rich thought it up and added it. The debugging function I never knew I needed! 🤯
Hmm… I’ll think on it more. I’m still skeptical that it’s worth the indirection except in the case of tooling.
I appreciate ya’lls thoughts and sharing ya’lls experiences @seancorfield and @noisesmith
(defn median [ns]
(let [ns (sort ns)
cnt (count ns)
mid (bit-shift-right cnt 1)]
(if (odd? cnt)
(nth ns mid)
(/ (+ (nth ns mid) (nth ns (dec mid))) 2))))
In the above function (from rosetta stone) what is bit-shift-right
achieving?in base 10 it "pushes" the number and drops off the last one. so 143 -> 14 and is effectively dividing by 10. in binary the base is 2 not ten so when pushing over its not dividing by 10 but by 2
I would like to propose the macro implies
to be added to clojure.core
(defmacro implies [condition implication]
`(if ~condition
~implication
true))
is this the right place to talk about it?This macro is useful when used in a conditional expression. It’s an alternative to the less readable (or (not condition) implication)
exemple of usage
1. If you want to discuss language features, #clojure-dev or https://clojure.atlassian.net/projects/CLJ/issues is the place 2. I do not expect such a proposal to get very far
Actually, the preferred place to make such a request is https://ask.clojure.org with a request
tag
https://clojure.org/community/contributing#_reporting_problems_and_requesting_enhancements
Thanks @U064X3EF3 I won’t request it as an enhancement because now I think that adding one more function in the core library would only amplify its learning curve unnecessarily.
The use case is already covered. What you’re asking for is purely syntactic and, as you’ve shown, easy to achieve with a macro.
Aside: As frustrating as that might sound, it’s an important design goal that Clojure give users the power of the language maintainers.
that’s not frustrating, don’t worry
maybe you are right, this should not be in clojure.core
Java interop question, given a ring path like "/foo/bar", how do I convert that into a (presumably) windows path, but preferably something where I don't care and could use nio.Path or similar.
I suppose something like File/separator is what I want 🙂 Replace / with File/separator
(last time I had to do something like this, a REPL running in a VM was really, really helpful)
Yes, forward slashes work via Java on both platforms (good) but you have to be careful about any code that either accepts paths as input from any config or user data, or constructs filesystem paths from filenames (or portable /
-separated paths). Writing correct cross-platform code can be tricky, even with Java's help 🙂
The other thing now with Windows 10 and WSL, you can also map c:\path\to\file.ext
on the Windows side to /mnt/s/path/to/file.ext
in WSL (Chlorine for Atom does this so it can support connecting from Atom on Windows to a REPL running on WSL).
Yes, I remember when I caught a bug because Java returned /C:/Some Folder/Some File
as a valid windows path...
I'm still surprised by that one, since it indicated different behavior on different JDKs, right?
I don't think so... I remember that I tested on Hotspot OpenJDK, OpenJ9, and Graal and they all gave the same results
I think it's more of a case to be aware that Java can return a path that will not work with some libraries' code.
(that, or Microsoft decides to accept /
as path separators, but I would not bet on that one 😄)
I was just reading this Reddit post: https://www.reddit.com/r/Clojure/comments/g7qyoy/why_does_orm_have_omg_complexity/ Anyone know what Rich meant when he said, “What’s the dual of value? Is it covalue? What’s a covalue? It’s an inconsistent thing.”
A value is a fixed unchanging thing, in category theory the prefix "co" means the reverse or inverse, so a covalue is like something always changing
He was a big fp/category theory guy. I don't really recall his talk, but the covalue stuff was I am sure added to Rich's talk in response
@hiredman any idea the particular talk? Or even what conference Rich’s talk was at? Any idea how/whether it relates to ORMs?
https://www.thestrangeloop.com/2011/category-theory-monads-and-duality-in-big-data.html
I remember Alex Miller saying that Eric Meijers requested the recording not be published.
Hi, I have a function like this:
(defn wrap-api [rout dev?]
(-> (if dev? (#'rout) (rout))
do-something))
Now if I try to compile that, the compiler complains it cannot resolve the var rout
I assume #'
does not work for function parameters? Is there a different way to do that?@U0677JTQX this does not appear correct
is it possible to read a def'ed expression as a string?
(defn some-function [] (+ 1 2 3))
(magic-reader some-function)
=> "(defn some-function [] (+ 1 2 3))"
i wasn't able to get clojure.repl/source
to work with non-core functions:
(defn some-function [] (+ 1 2 3))
=> #'myns/some-function
(clojure.repl/source some-function)
Source not found
=> nil
"Prints the source code for the given symbol, if it can find it.
This requires that the symbol resolve to a Var defined in a
namespace for which the .clj is in the classpath.
Example: (source filter)"
hmm. so if i defn my function in a namespace, load the namespace into a repl and call source on the symbol, it should work? silly example:
(ns mynamespace)
(defn some-example []
(+ 1 2 3))
(defn provide-source []
(clojure.repl/source some-example))
also clojure.repl
is injected into clojure by clojure.main when running a repl, but isn't guaranteed to be present in your app - require it if you use it
Is partition-all
supposed to work with async/pipeline
? I have a transducer and it looks like everything is just getting split into batches of size 1
and passed on from partition-all
even though there’s far more items than that on the channel.
do you have some sample code?
Yeah, my transducer looks something like this:
(comp (map transform-rows)
(partition-all 100)
(map some-batched-fn))
If I run that with transduce
, my data comes through in batches of 100. If I ask async/pipeline
to process the data, then everything comes through in batches of 1.I’m guessing it has something to do with this from the pipeline
docs:
Because
it is parallel, the transducer will be applied independently to each
element, not across elements
What should I use if I want to process a sequence of data in parallel with my aforementioned transducer?seems like it should be possible as long as you’re partitioning before you pipeline. let me give it a shot. one sec.
Yeah, I’m probably going to move away from pipeline
and just replace my map
calls with pmap
for simplicity. Thanks for the help!
sounds good, but for future reference, it seems like this is possible as long as you partition before hand. you can do this easily with clojure.core.async/pipe
:
(defn some-batched-fn [xs]
(map inc xs))
(def to-chan (async/chan 30))
(def from-chan (async/to-chan (range 20)))
(async/pipeline 3 to-chan (map some-batched-fn)
(async/pipe from-chan (async/chan 1 (partition-all 3))))
(loop [x (<!! to-chan)]
(when x
(prn x)
(recur (<!! to-chan))))
@hiredman I don’t know if you can see my comments from the thread above. What’s my best alternative then if I want to parallelize my transducer?
Hard to say, to some degree it depends on how you expect partitioning to work in parallel
pipeline preserves order when executing in parallel, but if you wanted partitioning to be have exactly like partitioning does on a sequence that eliminates parallelism
So maybe you can split your transducer around partition, a pipeline before, partition on the output channel, then feed the output into another pipeline for the bit after
Yeah, that could work. I think my transducer is basic enough that I’ll probably ditch async
in this particular case and just use pmap
for the parts before and after the partitioning. Thanks @hiredman
Reducers can only process vectors or maps in parallel because they use the tree shape structure to divide and conquer. The linear structure of a sequence isn't amenable
https://github.com/aphyr/tesser/blob/master/README.markdown might be worth a look