This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-01-25
Channels
- # arachne (1)
- # beginners (22)
- # boot (21)
- # cider (23)
- # cljs-dev (16)
- # cljsrn (9)
- # clojure (118)
- # clojure-dev (11)
- # clojure-greece (16)
- # clojure-italy (10)
- # clojure-losangeles (4)
- # clojure-russia (14)
- # clojure-serbia (4)
- # clojure-spec (58)
- # clojure-uk (33)
- # clojurescript (30)
- # cursive (17)
- # datomic (48)
- # docs (22)
- # events (1)
- # fulcro (24)
- # hoplon (3)
- # jobs (6)
- # jobs-discuss (4)
- # keechma (4)
- # leiningen (11)
- # luminus (4)
- # midje (1)
- # off-topic (107)
- # onyx (30)
- # other-languages (12)
- # pedestal (4)
- # re-frame (72)
- # reagent (6)
- # remote-jobs (1)
- # shadow-cljs (16)
- # spacemacs (3)
- # specter (9)
- # uncomplicate (4)
- # unrepl (40)
@aria2 - a few people who hang out in #data-science have some new things involving just that use case (Jupyter)
@aria42 I think Stu has been using https://github.com/stuarthalloway/clj-xchart ...
I want to update values in a map of any depth. Specifically any value that contains the key :db/ident I want to replace with the value of :db/ident
{:a {:b {:db/ident :enumerated/value}}}
-> {:a {:b :enumerated/value}}
(clojure.walk/postwalk #(if (and (map? %) (:db/ident %)) (:db/ident %) %) {:c {:b {:db/ident :relation/spouse}}})
Still need to clean it up a bit but that’s basically it
@caleb.macdonaldblack with specter it's:
(def ALL-MAPS (recursive-path [] p (stay-then-continue map? MAP-VALS p)))
(transform [ALL-MAPS #(contains? % :db/ident)] :db/ident data)
a lot faster than doing a postwalk
you can also re-use ALL-MAPS
for doing queries or other transformations on that kind of data structure
@nathanmarz Sorry I didn’t mention this in my problem. My data structure contains a map in a vector.
The postwalk works with vector or map and takes about a millisecond.
I couldn’t get spector to work with a vector as well but it took about 5 milliseconds even when it didn’t work
@caleb.macdonaldblack running just once in time
is not a good way to benchmark
nothing has a chance to optimize the code (both specter and the jvm)
this is better:
user=> (time (dotimes [_ 1000000] (clojure.walk/postwalk #(if (and (map? %) (:db/ident %)) (:db/ident %) %) data)))
"Elapsed time: 5055.333616 msecs"
user=> (time (dotimes [_ 1000000] (transform [ALL-MAPS #(contains? % :db/ident)] :db/ident data)))
"Elapsed time: 522.451801 msecs"
that shows 10x performance difference
it's easy to make modify ALL-MAPS
to handle whatever structure your data has
I also find it's better to be exact about what you're targeting to manipulate – postwalk
can lead to surprising bugs like descending into records, keys, etc.
Hey guys, what you think about DDD in clojure?
Does anyone know whether Clojure ended up using invokedynamic
? All I can find from googling and searching the mailing list is discussion about it from years ago, during which Rich seemed lukewarm.
Not it hasn’t and that’s due to a few things
Invokedynamic had some nasty bugs in older jvms
And today this would require adding runtime switches to enable or disable invokedynamic depending on the jvm. Support for compiler switches is fairly new.
But in the end, Clojure isn’t as dependent on invokedynamic due to it being mostly statically typed. That is to say protocols already use interfaces so they’re already quite fast
Not that invokedynamic would be useless, it not super critical
Thanks @U07TDTQNL. I suspected as much. Tbh, my understanding of how invokedynamic
works is very shaky. I get the sense from talking to people across JVM languages that it maybe didn't quite pan out as expected.
The main benefit I was imagining for Clojure would be closing the performance gap between protocols and multimethods, even if that is of relatively little importance to me personally.
There are several places where invokedynamic could be very useful and I think in late java 8 it’s finally reliable enough to use (without catastrophic edge cases). I have no doubt we will eventually explore it for things like multimethods, lazy var initialization, etc.
Thanks @U064X3EF3. I never thought about it being useful for lazy-seqs (I'm imagining those generated through recursive calls using lazy-seq
directly, which I do use often enough), but that makes complete sense.
Can you think of anything I can read on invokedynamic more recent than the speculative blog posts from around when it was introduced? As mentioned, I hardly understand how it actually works and would like to.
I don't think he meant lazy-seqs, but lazily loading vars, maybe for improving startup time
That doesn't make sense to me. I assumed Alex meant actual lazy vars, i.e. the thunks that make up lazy-seqs.
@U4R5K5M0A the more I think about it you may be right, although I doubt it would improve startup time.
@U2TCUSM2R here’s documentation that lazy var loading is a startup time optimization https://dev.clojure.org/display/design/Lazy+var+loading
Lazy vars do improve startup time noticeably if you’re loading AOT code (esp in combination with direct linking). Some data is at the end of https://dev.clojure.org/display/design/Improving+Clojure+Start+Time (patch is an attachment on that page too) - improvements are 25-30%. The downside is that you also make all subsequent var accesses slower (as you have to go through a guard). invokedynamic has the ability to actually remove the guard once you load, which mitigates that issue completely.
Woah, thanks guys. I see you what you meant now @U064X3EF3.
@nathanmarz Thanks, I wasn’t aware of benchmarking that way. I can see the specter transform is clearly faster than postwalk now. I’ll read a little more into specter and try to understand how it works.
I struggle to pass a pattern for a hash-map to clojure.core.match/match
. This examples shows would I would like to do (pass pattern
as a variable):
(let [pattern {:a 3 :b 4}
x {:a 3 :b 4 :c 5}]
(clojure.core.match/match [[x]]
[pattern] :foo))
;; => IllegalArgumentException No matching clause: [{:a 3, :b 4, :c 5}] server.repl/eval69751 (form-init673047409690541468.clj:3)
How can this be done? I tried a couple of variants, especially around using more/less square brackets, but nothing worked so far.
Yeah that is the exact problem. However, I still want to pass it in. Is there a way to achieve that?
you’ll have to quote it and expand, something like (untested):
(let [pattern {:a 3 :b 4}
x {:a 3 :b 4 :c 5}]
`(clojure.core.match/match [~x]
[~pattern] :foo))
I have a simple program:
(ns myprogram.core
(:gen-class))
(defn -main []
(pmap identity [1 2 3]))
that hangs with lein run
.
If I exchange pmap
with map
, it exists correctly. Should I somehow clean up after pmap
?I think pmap
uses the agents threadpool, so you need to call shutdown-agents
on exit: http://clojuredocs.org/clojure.core/shutdown-agents
How come reify
is so much faster than wrapping in a fn here:
(let [f0 (reify IFn
(invoke [_ x] (Integer/parseInt x)))
f1 (fn [x] (Integer/parseInt x))]
(doseq [f [f0 f1]]
(time (dotimes [_ 1000000000] (f "1")))))
Shouldn't they be pretty much identical?try checking it with https://github.com/gtrak/no.disassemble
f0 compiles into something like public final class ricochet_robots.core$reify__12494 implements clojure.lang.IFn, clojure.lang.IObj
while f1 is public final class ricochet_robots.core$f1 extends clojure.lang.AFunction
@schmee Yeah I did that. But there isn't a whole lot of difference. They both just implement a class that defines the .invoke(Object x)
in the same manner. Reify is straight class implementing IFn
and the function will exten AFunction
But still, a huge 5x difference just because of extending from a function? I'm surprised...
I guess that the JVM optimizes them differently because one is interface dispatch and one is not
My hunch is the JVM inlines the one call but not the other... But then, checking that would require some JVM-fu.
the problem is in the bench. try changing the order, the fastest is always the first to run
ehh, when I do this, the results are reversed…
(def f0
(reify clojure.lang.IFn
(invoke [_ x] (Integer/parseInt x))))
(defn f1 [x] (Integer/parseInt x))
(defn asdf []
(time (dotimes [_ 1000000000] (f0 "1")))
(time (dotimes [_ 1000000000] (f1 "1"))))
@schmee Looks like the JVM first optimizes the call site for exactly one class and can dispatch right to the implementation (or inlines it). Once seen a second class it deoptimizes.
Doing (doseq [f [f0 f0 f0]])
(good), vs (doseq [f [f0 f1 f0 f1]])
(bad), (doseq [f [f0 f0 f0 f1]])
(bad after 3rd)
I thought the JVM can easily handle 2-3 different classes and only becomes megamorphic after that... But maybe that was JS and not the JVM.
Howdy folks, I'm struggling with a very strange exception when writing midje tests: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.IFn$OLOO
. I'm deeply confused, does anyone know what it means?
Yes, I have a few provided
functions, that's probably where the issue reside since the code works in normal conditions (not under midje)
@rborer IIRC you should redefine them with the same primitive type hints. In your instance the second arg (the L
in OLOO
) is a primitive long (the other 1st/3rd are objects, return is object)
How would I do this in Clojure?
new AttributeProviderFactory() {
public AttributeProvider create(AttributeProviderContext context) {
return new ImageAttributeProvider();
}
})
@ghadi thanks, how do I tell if something is concrete? (source of that class is here: https://github.com/atlassian/commonmark-java/blob/fa299a6d565f3a7e66132410a7f7cf5752d1c700/commonmark/src/main/java/org/commonmark/renderer/html/AttributeProviderFactory.java)
Nice, I got it to work:
(def testfac
(proxy [AttributeProviderFactory] []
(create [context]
(reify AttributeProvider
(setAttributes [_ node tag-name attrs]
(prn node))))))
thanks @ghadi! Also Clojure is so concise, this is the Java version of what I’m doing: https://github.com/atlassian/commonmark-java#add-or-change-attributes-of-html-elements ❤️@martinklepsch Just in case you've missed it, have you seen https://github.com/yogthos/markdown-clj ?
until you need to do things with numbers maybe
what’s the best channel to discuss cljfmt related issues?
haha - trying to be more careful about using the right channels, anyway turns out there’s a known bug in cljfmt for indenting letfn and a workaround is documented in the github issue https://github.com/weavejester/cljfmt/issues/73
noisesmith: I’ve been working on a cljfmt fork that does a lot more than the original version. Still a few outstanding bugs to clean up, but may solve your problem: https://github.com/greglook/cljfmt
@U8XJ15DTK nice, will we get parallel fix and check there? cljfmt's maintainer ignored my PR for that 😞 https://github.com/weavejester/cljfmt/pull/104
@U5LPUJ7AP parallel fix sounds good to me, but parallel check seems like it would interleave diff output
I could imagine a nice compromise where the actual checking is parallel, and the printing of the check output is owned by a single thread
well, maybe my PR does that, but it's not difficult to avoid that, just put the diffs on a channel and print them in a go-loop
so basically what noisesmith said 😛
I am open to said PR. 🙂 At this point, my fork is ~3x the size of the original cljfmt, so I’m not sure if merging back upstream will be viable. We’re using the current snapshot release at work though, so it’s seen some battle testing. You should be able to ignore any forms that it misbehaves on, anyway. (another new feature!)
will take a look when I have the time
I suggest #cljfmt I’d be very interested in following and probably participating in an active cljfmt fork. It looks like James doesn’t have a whole lot of time for the project at present.
I love this commit 🙂 https://github.com/greglook/cljfmt/commit/567a02dab63ec375bd0f13797d30689cfd58f155
noisesmith: I’ve been working on a cljfmt fork that does a lot more than the original version. Still a few outstanding bugs to clean up, but may solve your problem: https://github.com/greglook/cljfmt
I’ve made a ring handler that reloads all my test files and shows the results in a div in the frontend
It seems like this would be a cool way of deploying little executable system checks to all your environments (whitelisting on production only "production safe" tests). All those ad-hoc scripts you write and paste into your ssh session to inspect the health of the system when something has gone wrong can be encoded as clojure tests and run at the click of a button. Does anyone do this?
I would differentiate health checks from unit tests, both are useful
I haven’t considered using clojure.test to implement health checks though
or, put differently, I don’t run ad-hoc scripts to inspect the health of the system, I have health check endpoints on the http server that allow that (it is accessed by an automated system that sends us alerts if the checks don’t respond, or if they fail, etc.)
a transducer question. I have a long chain of comp
ed transducers doing various map and filter operations where the data once it gets applied will be a sequence of maps. Now I would like for one of the comped transducer steps to assoc a “ranking” value to the maps where the ranking is that map’s index if sorting the full sequence of maps based on the value of one of the keys. I get the feeling this makes the ranking operation unsuitable for my transducer chain since it’s no longer “step based” but rather operates on the whole sequence of maps, but I’m still noob with transducers. Am I correct in assuming it would be a bad fit to try to do this kind of ranking using a transducer in the existing chain?
I guess I could do a custom stateful transducer with a completion that does the ranking…
That seems not great - what you’re describing requires fully realizing the collection first, so it’s not really a (per-element) transformation
sounds more like a transduce that happens to use the transducer you’ve defined
that has all the pieces you need - building up a result step by step, final transformation before returning the value, not lazy