This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-01-25
Channels
- # aws (2)
- # bangalore-clj (2)
- # beginners (90)
- # boot (89)
- # bristol-clojurians (1)
- # cider (23)
- # cljs-dev (48)
- # cljsjs (2)
- # cljsrn (3)
- # clojure (118)
- # clojure-argentina (3)
- # clojure-austin (8)
- # clojure-czech (1)
- # clojure-dev (18)
- # clojure-ireland (1)
- # clojure-italy (4)
- # clojure-russia (6)
- # clojure-spec (75)
- # clojure-uk (224)
- # clojurescript (103)
- # core-async (28)
- # cursive (3)
- # datascript (7)
- # datomic (15)
- # dirac (30)
- # emacs (14)
- # events (3)
- # figwheel (1)
- # hispano (1)
- # hoplon (176)
- # lambdaisland (1)
- # lein-figwheel (6)
- # off-topic (21)
- # om (7)
- # onyx (2)
- # pedestal (6)
- # re-frame (4)
- # reagent (15)
- # spacemacs (67)
- # specter (13)
- # testing (9)
- # untangled (65)
- # vim (6)
- # yada (1)
Saw the log while running the latest version of https://github.com/mfikes/core.rrb-vector
http://insideclojure.org/2014/12/15/warn-on-boxed/ is something I wrote about this warning that might help.
Taking an example: Boxed math warning, clojure/core/rrb_vector/nodes.clj:88:11 - call: public static boolean
this is saying that at line 88 of nodes.clj there is a call to <
where the first argument is a long (primitive) argument but the second argument has no type information and will thus be a boxed number object (like Long).
Boxed math is much slower (~100x) so this can be a performance issue.
here, the Clojure compiler can infer the type of step and thus the type of r
but it has no info about cnt
https://github.com/clojure/core.rrb-vector/blob/master/src/main/clojure/clojure/core/rrb_vector/nodes.clj#L88 so rrb-vector is not optimized and can be slow?
one way to change that would be to type-hint the argument like:
(defn regular-ranges [shift ^long cnt] …
it’s hard to say whether the boxed warnings you’re seeing are things that actually affect the performance in practice of an actual program.
from a glance, my guess is that it could be faster with more attention paid to leveraging primitive numerics
whether that is 10% or 100% or 1000% faster, I have a hard time predicting
the whole point of this hint is to help you more easily find places that can be optimized
(as opposed to reading the generated bytecode, which is what I usually have to do)
sounds like a fun project then :)
the only primitive type hints that matter are ^double (prob not relevant here) and ^long - those are generally the only ones that actually affect the generated bytecode
the exception being within Java interop or loop/recur where occasionally int casts are useful when interacting with int indexes
whatever you’re doing, make benchmarks that can tell you whether you’re doing something useful and to double-check that it’s having the effect you expect
i have a question about loop
/ recur
. specifically, i'm getting an error that the recur argument count is mismatched, but the loop has an even number of two bindings and my recur
has two parameters:
(defn get-constraints [m]
(loop [[k {:keys [children constraints]}] m
all-constraints []]
(let [new-constraints (reduce conj all-constraints constraints)]
(if children
(map (fn [child] (recur child new-constraints)) children)
(println all-constraints)))))
recur goes to the nearest loop or function enclosing it, which here if (fn [child] … )
you can’t recur to the outer loop from within the inner function
it’s unclear to me what you are actually intending here so hard to suggest a rewrite
ah, thanks, that makes total sense. my brain was ignoring the anonymous function. while i'm here, maybe there's a much better way to accomplish my goal. i'm trying to recursively step through nested maps and retrieve all values of a specific key. in my example i'm looking to collect all :constraints
values:
{"Gene" {:children {"symbol" {:visible true
:constraints [{:op "="
:value "zen"}]}
"organism" {:children {"name" {:visible true
:constraints [{:op "="
:value "Homo spaiens"}]}}}
"alleles" {:children {"name" {:visible true}
"id" {:visible true}}}}}}
and the result would be [{:op "=" :value "zen"}] {:op "=" :value "Homo spaiens"}
but i need to keep a trail of where i am in the tree and concat the keys together, so it would really produce something like this: [{:path "Gene.symbol" :op "=" :value "zen"} {:path "Gene.organism.name" :op "=" :value "Homo spaiens"}]
if I had a dollar for every time I’d said that
Many folk like specter
for the kinds of operations you describe
i'm trying to learn it 'the hard' way. i also found that specter adds 300kb to my clojurescript projects
okay, so in lieu of using a zipper (which solves my problem but doesn't teach me anything), is there an efficient way to pull out all :constraints
values while concat'ing the :`path` values in this slightly more sane structure?
(def query {:path "Gene"
:visible true
:children [{:path "symbol"
:visible true
:constraints [{:op "="
:value "zen"}]}
{:path "organism"
:visible true
:children [{:path "name"
:visible true
:constraints [{:op "="
:value "Homo sapiens"}]}]}]})
[
{:path "Gene.symbol" :op "=" :value "zen"}
{:path "Gene.organism.name" :op "=" :value "Homo sapiens"}
]
tree-seq
gets me close but i lose my location in the map
(->> query (tree-seq map? :children) (mapcat :constraints))
http://stackoverflow.com/a/35032544/508458 might be helpful
Anyone here particularly adept at figwheel? Built a Reagent app from the default Reagent lein template and deployed to heroku (http://www.archetyp.io) but even in prod there’s a brief “ClojureScript has not been compiled! please run lein figwheel” which displays and then disappears after compile. We don’t use :requires outside of a namespace declaration (as per one post on SO) and I think our :builds :externs are all correct. Any ideas or pointers in troubleshooting? Our project.clj is here https://gist.github.com/ekhall/d852a29bd4acce0256975ca6af910bdd
@seancorfield thanks for the reply, now that I think about it, it might be best to create a proper namespace for this kind of configuration. it's basically for ETL where each data source gives you the same data in different ways and so the configuration was storing the composition of functions that each source needs to normalize the data
I could probably accomplish the same thing using spec with conform functions and then I just have to store the spec for each data source
Does anyone have good resources for learning how to work with dependencies, in particular conflicts and how to deal with them (either specific to Leiningen or in general)? I've just scratched the surface (with :exclusions
and lein deps tree
) but it's got me interested in figuring out more on how to troubleshoot and actually understand how people normally solve dependency issues in Clojure.
have you used lein ancient
?
the readme https://github.com/xsc/lein-ancient might be useful reading
the canonical maven resource is https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html but you may find it leaves you with more questions than answers :)
@alexmiller: I use lein ancient
but I don't really understand anything beside the basics - so those are probably good resources to start with.
The actual problem I'm looking into may be totally unrelated, maybe I'm starting in the wrong place.
I tried adding onyx-local-rt
to a project using clojure 1.9.0-alpha14
, but onyx-local-rt
depends on clojure 1.8.0
and clojure-future-spec
When I start up the project I get some errors that go away if I exclude clojure-future-spec
, but then onyx-local-rt
doesn't work.
It's not remotely critical but it made me curious about how to deal with it.
dependencies form a tree
building a classpath (the thing Java actually uses) is a linearization of that tree such that things earlier in the classpath shadow things later in the classpath
exclusions are a tool to filter that tree before linearization
(as are scopes and other features of maven dependency resolution)
the canonical “dependency hell” problem is when you have two leaves of the tree that are different versions of the same library AND where the branches are incompatible
also refer to Rich’s keynote at the conj :)
the linearization of the tree in this case is going to include both (presumably incompatible) versions of the same lib but only one is going to take effect, thus making one branch unhappy
Been burned in a Java project recently by later versions removing an internal class
this exact problem is the motivation behind the new brand new module system in Java 9, which theoretically allows you to go back to a tree like runtime model (supported by classloaders) where both incompatible versions can be in use simultaneously
I am skeptical that this is going to improve the state of the world :)
Is that Jigsaw @alexmiller ?
yes, although it’s become a labyrinth of names and jsrs
Usual thing in the Java world...I struggle to keep up with the JSR spiders web
Are there good ways to view the difference between the dependency tree and the classpath that is the result of linearizing that tree? i.e. lein deps tree
turns into ?
lein classpath
Well that makes me feel like I should have been able to figure that out on my own 🙂
lein help
might have led you there
Here's the error message I'm getting, dependencies are [org.clojure/clojure "1.9.0-alpha14"] [org.onyxplatform/onyx-local-rt "0.9.15.5"]
only.
I know the error goes away if I exclude clojure-future-spec
, and I suspect the root of the error message is in https://github.com/tonsky/clojure-future-spec/blob/master/src/clojure/core_instant18.clj
Is there a nice way to troubleshoot or somehow fix it, or is this the dependency hell error message?
that’s likely tricky
that ns is conditionally loaded in Clojure itself
as it requires classes that don’t exist in all Java versions supported by Clojure
are you perhaps using java 1.7?
and if so, can you use 1.8?
java -version tells me 1.8, so that's what I should be on unless there's something else weird going on
so if you’re using Clojure 1.9 AND future-spec, then it seems like some of these things are going to conflict - like that ns will exist in both Clojure and in future-spec
it seems like you’ll want to exclude future-spec
so the question then is what breaks with onyx
Ok, at least I was on the right track there.
If I add the exclusion, when I require onyx-local-rt.api in the repl I get the error message
CompilerException java.io.FileNotFoundException: Could not locate clojure/future__init.class or clojure/future.clj on classpath., compiling:(onyx/spec.cljc:1:1)
Which seems to make sense, but I'm not sure what the normal next step would be - does it require a change to onyx or can it be dealt with from within my project?
seems like onyx to me
or you could just fall back to clojure 1.8
Ok that's good to know - I wasn't sure if there was some other Clojure thing that I just didn't know about. Thanks for the help!
@shaun-mahood @alexmiller We’ve been playing a tricky game keeping Onyx core on 1.8 until 1.9 is out officially, but we needed onyx-local-rt to have access to Clojure spec so it could do its thing in the browser. We’ve mostly been working around this by either: (1) Using 1.9 everywhere, and putting stub files into the project for clojure.future which calls back to Clojure core. (2) Using 1.8 everywhere. Everything should “just work” since Clojure.future would be in place.
I might have linked this to you before, but I can’t remember. This how you can handle (1) https://github.com/onyx-platform/onyx-local-rt/issues/5
@michaeldrogalis: Oh that's great, thanks!
Agh, I didn’t send you that link the first time? My apologies, I thought I did.
No problem, I should have seen it in the issues anyway. Glad I went on the journey I did, this is the first time I've looked into most of this part of Clojure in any depth at all so it's been a very good learning experience.
Awesome, glad to hear it. 🙂
I want to map over a collection of collections of booleans and output if the nth elements are all true? Put another way, i have some column oriented data, and i want to ask a row question x | c1 | c2 r1|t | f r2|t| t i want to be able to ask if all of r1 is true? This sounds like a map function
(map ?? [t f] [t t])
but the question i want to ask is if their all true? Ok so if i just had two booleans i can use and. So should be able to just apply map with that?
clojure
(map and [false true] [true false])
… Can't take value of a macro: #'clojure.core/and
so i can’t map over and, because its a macro? Thats disappointing, and at his point somewhat confusing. (Why is and a marco but + isn’t, whats the advantage?). Ok, whats the alternative?
clojure
(map #(every? true? [%1 %2]) [false true] [true true])
isn’t sufficient because it requires i know the number of collectionsmaybe i should re-arrange the data?
and is a macro because it returns the first truthy value, it doesn't evaluate remaining arguments after that.
@drewverlee something like:
(map #(every? true? %) [[false true] [true false] [true true]])
should work
@chronno Thanks! This works just like i would expect. In order to solve my problem i just transposed the data first then did as you suggested. @dominicm Gotcha. I understand that in order for it to not eval the second value it needs to be a macro. But i still dont grok why macros can’t be passed around like functions can be or used as map. I recall reading it had to do with steps necessary at compile time, but i didn’t understand what that was all about.