Fork me on GitHub
#beginners
<
2017-01-25
>
Jon03:01:21

What does it mean in Java?

Jon03:01:51

Saw the log while running the latest version of https://github.com/mfikes/core.rrb-vector

Jon03:01:12

which is based on a snapshot version of rrb-vector

Alex Miller (Clojure team)04:01:56

http://insideclojure.org/2014/12/15/warn-on-boxed/ is something I wrote about this warning that might help.

Alex Miller (Clojure team)04:01:17

Taking an example: Boxed math warning, clojure/core/rrb_vector/nodes.clj:88:11 - call: public static boolean (long,java.lang.Object).

Alex Miller (Clojure team)04:01:44

this is saying that at line 88 of nodes.clj there is a call to < where the first argument is a long (primitive) argument but the second argument has no type information and will thus be a boxed number object (like Long).

Alex Miller (Clojure team)04:01:03

Boxed math is much slower (~100x) so this can be a performance issue.

Alex Miller (Clojure team)04:01:21

here, the Clojure compiler can infer the type of step and thus the type of r

Alex Miller (Clojure team)04:01:26

but it has no info about cnt

Alex Miller (Clojure team)04:01:06

one way to change that would be to type-hint the argument like: (defn regular-ranges [shift ^long cnt] …

Alex Miller (Clojure team)04:01:57

it’s hard to say whether the boxed warnings you’re seeing are things that actually affect the performance in practice of an actual program.

Alex Miller (Clojure team)04:01:27

from a glance, my guess is that it could be faster with more attention paid to leveraging primitive numerics

Alex Miller (Clojure team)04:01:09

whether that is 10% or 100% or 1000% faster, I have a hard time predicting

Alex Miller (Clojure team)04:01:06

the whole point of this hint is to help you more easily find places that can be optimized

Alex Miller (Clojure team)04:01:25

(as opposed to reading the generated bytecode, which is what I usually have to do)

Jon04:01:30

@bbloom 's fipp is relying on it, I'm using fipp, so it could be faster if optimized..

Alex Miller (Clojure team)04:01:46

sounds like a fun project then :)

Alex Miller (Clojure team)04:01:01

the only primitive type hints that matter are ^double (prob not relevant here) and ^long - those are generally the only ones that actually affect the generated bytecode

Alex Miller (Clojure team)04:01:35

the exception being within Java interop or loop/recur where occasionally int casts are useful when interacting with int indexes

Alex Miller (Clojure team)04:01:21

whatever you’re doing, make benchmarks that can tell you whether you’re doing something useful and to double-check that it’s having the effect you expect

Jon04:01:39

Didn't look into rrb-vector... I almost spent all my time in JavaScript

joshkh14:01:20

i have a question about loop / recur. specifically, i'm getting an error that the recur argument count is mismatched, but the loop has an even number of two bindings and my recur has two parameters:

(defn get-constraints [m]
  (loop [[k {:keys [children constraints]}] m
         all-constraints []]
    (let [new-constraints (reduce conj all-constraints constraints)]
      (if children
        (map (fn [child] (recur child new-constraints)) children)
        (println all-constraints)))))

Alex Miller (Clojure team)15:01:53

recur goes to the nearest loop or function enclosing it, which here if (fn [child] … )

Alex Miller (Clojure team)15:01:32

you can’t recur to the outer loop from within the inner function

Alex Miller (Clojure team)15:01:54

it’s unclear to me what you are actually intending here so hard to suggest a rewrite

joshkh15:01:23

ah, thanks, that makes total sense. my brain was ignoring the anonymous function. while i'm here, maybe there's a much better way to accomplish my goal. i'm trying to recursively step through nested maps and retrieve all values of a specific key. in my example i'm looking to collect all :constraints values:

{"Gene" {:children {"symbol"   {:visible     true
                                :constraints [{:op    "="
                                               :value "zen"}]}
                    "organism" {:children {"name" {:visible     true
                                                   :constraints [{:op    "="
                                                                  :value "Homo spaiens"}]}}}
                    "alleles"  {:children {"name" {:visible true}
                                           "id"   {:visible true}}}}}}
and the result would be
[{:op "=" :value "zen"}] {:op "=" :value "Homo spaiens"}

joshkh15:01:27

but i need to keep a trail of where i am in the tree and concat the keys together, so it would really produce something like this: [{:path "Gene.symbol" :op "=" :value "zen"} {:path "Gene.organism.name" :op "=" :value "Homo spaiens"}]

joshkh15:01:05

actually, in hindsight that's a terrible data structure. 😉

Alex Miller (Clojure team)15:01:07

if I had a dollar for every time I’d said that

donaldball15:01:15

Many folk like specter for the kinds of operations you describe

joshkh15:01:28

yes, and specter is what spoiled me 😉

joshkh15:01:56

i'm trying to learn it 'the hard' way. i also found that specter adds 300kb to my clojurescript projects

joshkh15:01:29

but yes you're absolutely right, specter is great for accomplishing just that!

joshkh15:01:38

okay, so in lieu of using a zipper (which solves my problem but doesn't teach me anything), is there an efficient way to pull out all :constraints values while concat'ing the :`path` values in this slightly more sane structure?

(def query {:path     "Gene"
            :visible  true
            :children [{:path        "symbol"
                        :visible     true
                        :constraints [{:op    "="
                                       :value "zen"}]}
                       {:path     "organism"
                        :visible  true
                        :children [{:path        "name"
                                    :visible     true
                                    :constraints [{:op    "="
                                                   :value "Homo sapiens"}]}]}]})
[
{:path "Gene.symbol" :op "=" :value "zen"}
{:path "Gene.organism.name" :op "=" :value "Homo sapiens"}
]
tree-seq gets me close but i lose my location in the map
(->> query (tree-seq map? :children) (mapcat :constraints))

joshkh16:01:04

neat, i'll have a look. thanks.

eugekev16:01:02

Anyone here particularly adept at figwheel? Built a Reagent app from the default Reagent lein template and deployed to heroku (http://www.archetyp.io) but even in prod there’s a brief “ClojureScript has not been compiled! please run lein figwheel” which displays and then disappears after compile. We don’t use :requires outside of a namespace declaration (as per one post on SO) and I think our :builds :externs are all correct. Any ideas or pointers in troubleshooting? Our project.clj is here https://gist.github.com/ekhall/d852a29bd4acce0256975ca6af910bdd

gdeer8116:01:12

@seancorfield thanks for the reply, now that I think about it, it might be best to create a proper namespace for this kind of configuration. it's basically for ETL where each data source gives you the same data in different ways and so the configuration was storing the composition of functions that each source needs to normalize the data

gdeer8116:01:02

I could probably accomplish the same thing using spec with conform functions and then I just have to store the spec for each data source

shaun-mahood17:01:40

Does anyone have good resources for learning how to work with dependencies, in particular conflicts and how to deal with them (either specific to Leiningen or in general)? I've just scratched the surface (with :exclusions and lein deps tree) but it's got me interested in figuring out more on how to troubleshoot and actually understand how people normally solve dependency issues in Clojure.

Alex Miller (Clojure team)17:01:28

have you used lein ancient ?

Alex Miller (Clojure team)17:01:07

the canonical maven resource is https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html but you may find it leaves you with more questions than answers :)

shaun-mahood17:01:13

@alexmiller: I use lein ancient but I don't really understand anything beside the basics - so those are probably good resources to start with. The actual problem I'm looking into may be totally unrelated, maybe I'm starting in the wrong place. I tried adding onyx-local-rt to a project using clojure 1.9.0-alpha14, but onyx-local-rt depends on clojure 1.8.0 and clojure-future-spec When I start up the project I get some errors that go away if I exclude clojure-future-spec, but then onyx-local-rt doesn't work. It's not remotely critical but it made me curious about how to deal with it.

Alex Miller (Clojure team)17:01:14

dependencies form a tree

Alex Miller (Clojure team)17:01:39

building a classpath (the thing Java actually uses) is a linearization of that tree such that things earlier in the classpath shadow things later in the classpath

Alex Miller (Clojure team)17:01:25

exclusions are a tool to filter that tree before linearization

Alex Miller (Clojure team)17:01:45

(as are scopes and other features of maven dependency resolution)

Alex Miller (Clojure team)17:01:49

the canonical “dependency hell” problem is when you have two leaves of the tree that are different versions of the same library AND where the branches are incompatible

Alex Miller (Clojure team)17:01:14

also refer to Rich’s keynote at the conj :)

Alex Miller (Clojure team)17:01:10

the linearization of the tree in this case is going to include both (presumably incompatible) versions of the same lib but only one is going to take effect, thus making one branch unhappy

agile_geek17:01:26

Been burned in a Java project recently by later versions removing an internal class

Alex Miller (Clojure team)17:01:40

this exact problem is the motivation behind the new brand new module system in Java 9, which theoretically allows you to go back to a tree like runtime model (supported by classloaders) where both incompatible versions can be in use simultaneously

Alex Miller (Clojure team)17:01:11

I am skeptical that this is going to improve the state of the world :)

Alex Miller (Clojure team)17:01:59

yes, although it’s become a labyrinth of names and jsrs

jeffh-fp17:01:50

I think npm (Node package manager) tried to do a similar thing with v2->v3

agile_geek17:01:55

Usual thing in the Java world...I struggle to keep up with the JSR spiders web

shaun-mahood18:01:19

Are there good ways to view the difference between the dependency tree and the classpath that is the result of linearizing that tree? i.e. lein deps tree turns into ?

shaun-mahood18:01:57

Well that makes me feel like I should have been able to figure that out on my own 🙂

Alex Miller (Clojure team)18:01:43

lein help might have led you there

shaun-mahood18:01:50

Here's the error message I'm getting, dependencies are [org.clojure/clojure "1.9.0-alpha14"] [org.onyxplatform/onyx-local-rt "0.9.15.5"] only. I know the error goes away if I exclude clojure-future-spec, and I suspect the root of the error message is in https://github.com/tonsky/clojure-future-spec/blob/master/src/clojure/core_instant18.clj Is there a nice way to troubleshoot or somehow fix it, or is this the dependency hell error message?

Alex Miller (Clojure team)18:01:47

that ns is conditionally loaded in Clojure itself

Alex Miller (Clojure team)18:01:23

as it requires classes that don’t exist in all Java versions supported by Clojure

Alex Miller (Clojure team)18:01:58

are you perhaps using java 1.7?

Alex Miller (Clojure team)18:01:12

and if so, can you use 1.8?

shaun-mahood18:01:21

java -version tells me 1.8, so that's what I should be on unless there's something else weird going on

Alex Miller (Clojure team)18:01:53

so if you’re using Clojure 1.9 AND future-spec, then it seems like some of these things are going to conflict - like that ns will exist in both Clojure and in future-spec

Alex Miller (Clojure team)18:01:54

it seems like you’ll want to exclude future-spec

Alex Miller (Clojure team)18:01:12

so the question then is what breaks with onyx

shaun-mahood18:01:07

Ok, at least I was on the right track there. If I add the exclusion, when I require onyx-local-rt.api in the repl I get the error message CompilerException java.io.FileNotFoundException: Could not locate clojure/future__init.class or clojure/future.clj on classpath., compiling:(onyx/spec.cljc:1:1) Which seems to make sense, but I'm not sure what the normal next step would be - does it require a change to onyx or can it be dealt with from within my project?

Alex Miller (Clojure team)18:01:19

or you could just fall back to clojure 1.8

shaun-mahood18:01:18

Ok that's good to know - I wasn't sure if there was some other Clojure thing that I just didn't know about. Thanks for the help!

michaeldrogalis19:01:27

@shaun-mahood @alexmiller We’ve been playing a tricky game keeping Onyx core on 1.8 until 1.9 is out officially, but we needed onyx-local-rt to have access to Clojure spec so it could do its thing in the browser. We’ve mostly been working around this by either: (1) Using 1.9 everywhere, and putting stub files into the project for clojure.future which calls back to Clojure core. (2) Using 1.8 everywhere. Everything should “just work” since Clojure.future would be in place.

michaeldrogalis19:01:15

I might have linked this to you before, but I can’t remember. This how you can handle (1) https://github.com/onyx-platform/onyx-local-rt/issues/5

shaun-mahood19:01:30

@michaeldrogalis: Oh that's great, thanks!

michaeldrogalis19:01:13

Agh, I didn’t send you that link the first time? My apologies, I thought I did.

shaun-mahood19:01:50

No problem, I should have seen it in the issues anyway. Glad I went on the journey I did, this is the first time I've looked into most of this part of Clojure in any depth at all so it's been a very good learning experience.

michaeldrogalis19:01:33

Awesome, glad to hear it. 🙂

Drew Verlee21:01:30

I want to map over a collection of collections of booleans and output if the nth elements are all true? Put another way, i have some column oriented data, and i want to ask a row question x | c1 | c2 r1|t | f r2|t| t i want to be able to ask if all of r1 is true? This sounds like a map function

(map ?? [t f] [t t])
but the question i want to ask is if their all true? Ok so if i just had two booleans i can use and. So should be able to just apply map with that?
clojure
(map and [false true] [true false])

… Can't take value of a macro: #'clojure.core/and
so i can’t map over and, because its a macro? Thats disappointing, and at his point somewhat confusing. (Why is and a marco but + isn’t, whats the advantage?). Ok, whats the alternative?
clojure 
(map #(every? true? [%1 %2]) [false true] [true true])
isn’t sufficient because it requires i know the number of collections

Drew Verlee21:01:09

maybe i should re-arrange the data?

dominicm21:01:30

and is a macro because it returns the first truthy value, it doesn't evaluate remaining arguments after that.

chronno22:01:28

@drewverlee something like: (map #(every? true? %) [[false true] [true false] [true true]]) should work

chronno22:01:22

Notice that map is being called with only one collection.

Drew Verlee22:01:00

@chronno Thanks! This works just like i would expect. In order to solve my problem i just transposed the data first then did as you suggested. @dominicm Gotcha. I understand that in order for it to not eval the second value it needs to be a macro. But i still dont grok why macros can’t be passed around like functions can be or used as map. I recall reading it had to do with steps necessary at compile time, but i didn’t understand what that was all about.