Fork me on GitHub
#beginners
<
2017-01-25
>
jiyinyiyong03:01:21

What does it mean in Java?

jiyinyiyong03:01:51

Saw the log while running the latest version of https://github.com/mfikes/core.rrb-vector

jiyinyiyong03:01:12

which is based on a snapshot version of rrb-vector

alexmiller04:01:56

http://insideclojure.org/2014/12/15/warn-on-boxed/ is something I wrote about this warning that might help.

alexmiller04:01:17

Taking an example: Boxed math warning, clojure/core/rrb_vector/nodes.clj:88:11 - call: public static boolean (long,java.lang.Object).

alexmiller04:01:44

this is saying that at line 88 of nodes.clj there is a call to < where the first argument is a long (primitive) argument but the second argument has no type information and will thus be a boxed number object (like Long).

alexmiller04:01:03

Boxed math is much slower (~100x) so this can be a performance issue.

alexmiller04:01:21

here, the Clojure compiler can infer the type of step and thus the type of r

alexmiller04:01:26

but it has no info about cnt

alexmiller04:01:06

one way to change that would be to type-hint the argument like: (defn regular-ranges [shift ^long cnt] …

alexmiller04:01:57

it’s hard to say whether the boxed warnings you’re seeing are things that actually affect the performance in practice of an actual program.

alexmiller04:01:27

from a glance, my guess is that it could be faster with more attention paid to leveraging primitive numerics

alexmiller04:01:09

whether that is 10% or 100% or 1000% faster, I have a hard time predicting

alexmiller04:01:06

the whole point of this hint is to help you more easily find places that can be optimized

alexmiller04:01:25

(as opposed to reading the generated bytecode, which is what I usually have to do)

jiyinyiyong04:01:30

@bbloom 's fipp is relying on it, I'm using fipp, so it could be faster if optimized..

alexmiller04:01:46

sounds like a fun project then :)

alexmiller04:01:01

the only primitive type hints that matter are ^double (prob not relevant here) and ^long - those are generally the only ones that actually affect the generated bytecode

alexmiller04:01:35

the exception being within Java interop or loop/recur where occasionally int casts are useful when interacting with int indexes

alexmiller04:01:21

whatever you’re doing, make benchmarks that can tell you whether you’re doing something useful and to double-check that it’s having the effect you expect

jiyinyiyong04:01:39

Didn't look into rrb-vector... I almost spent all my time in JavaScript

joshkh14:01:20

i have a question about loop / recur. specifically, i'm getting an error that the recur argument count is mismatched, but the loop has an even number of two bindings and my recur has two parameters:

(defn get-constraints [m]
  (loop [[k {:keys [children constraints]}] m
         all-constraints []]
    (let [new-constraints (reduce conj all-constraints constraints)]
      (if children
        (map (fn [child] (recur child new-constraints)) children)
        (println all-constraints)))))

alexmiller15:01:53

recur goes to the nearest loop or function enclosing it, which here if (fn [child] … )

alexmiller15:01:32

you can’t recur to the outer loop from within the inner function

alexmiller15:01:54

it’s unclear to me what you are actually intending here so hard to suggest a rewrite

joshkh15:01:23

ah, thanks, that makes total sense. my brain was ignoring the anonymous function. while i'm here, maybe there's a much better way to accomplish my goal. i'm trying to recursively step through nested maps and retrieve all values of a specific key. in my example i'm looking to collect all :constraints values:

{"Gene" {:children {"symbol"   {:visible     true
                                :constraints [{:op    "="
                                               :value "zen"}]}
                    "organism" {:children {"name" {:visible     true
                                                   :constraints [{:op    "="
                                                                  :value "Homo spaiens"}]}}}
                    "alleles"  {:children {"name" {:visible true}
                                           "id"   {:visible true}}}}}}
and the result would be
[{:op "=" :value "zen"}] {:op "=" :value "Homo spaiens"}

joshkh15:01:27

but i need to keep a trail of where i am in the tree and concat the keys together, so it would really produce something like this: [{:path "Gene.symbol" :op "=" :value "zen"} {:path "Gene.organism.name" :op "=" :value "Homo spaiens"}]

joshkh15:01:05

actually, in hindsight that's a terrible data structure. 😉

alexmiller15:01:07

if I had a dollar for every time I’d said that

donaldball15:01:15

Many folk like specter for the kinds of operations you describe

joshkh15:01:28

yes, and specter is what spoiled me 😉

joshkh15:01:56

i'm trying to learn it 'the hard' way. i also found that specter adds 300kb to my clojurescript projects

joshkh15:01:29

but yes you're absolutely right, specter is great for accomplishing just that!

joshkh15:01:38

okay, so in lieu of using a zipper (which solves my problem but doesn't teach me anything), is there an efficient way to pull out all :constraints values while concat'ing the :`path` values in this slightly more sane structure?

(def query {:path     "Gene"
            :visible  true
            :children [{:path        "symbol"
                        :visible     true
                        :constraints [{:op    "="
                                       :value "zen"}]}
                       {:path     "organism"
                        :visible  true
                        :children [{:path        "name"
                                    :visible     true
                                    :constraints [{:op    "="
                                                   :value "Homo sapiens"}]}]}]})
[
{:path "Gene.symbol" :op "=" :value "zen"}
{:path "Gene.organism.name" :op "=" :value "Homo sapiens"}
]
tree-seq gets me close but i lose my location in the map
(->> query (tree-seq map? :children) (mapcat :constraints))

joshkh16:01:04

neat, i'll have a look. thanks.

eugekev16:01:02

Anyone here particularly adept at figwheel? Built a Reagent app from the default Reagent lein template and deployed to heroku (http://www.archetyp.io) but even in prod there’s a brief “ClojureScript has not been compiled! please run lein figwheel” which displays and then disappears after compile. We don’t use :requires outside of a namespace declaration (as per one post on SO) and I think our :builds :externs are all correct. Any ideas or pointers in troubleshooting? Our project.clj is here https://gist.github.com/ekhall/d852a29bd4acce0256975ca6af910bdd

gdeer8116:01:12

@seancorfield thanks for the reply, now that I think about it, it might be best to create a proper namespace for this kind of configuration. it's basically for ETL where each data source gives you the same data in different ways and so the configuration was storing the composition of functions that each source needs to normalize the data

gdeer8116:01:02

I could probably accomplish the same thing using spec with conform functions and then I just have to store the spec for each data source

shaun-mahood17:01:40

Does anyone have good resources for learning how to work with dependencies, in particular conflicts and how to deal with them (either specific to Leiningen or in general)? I've just scratched the surface (with :exclusions and lein deps tree) but it's got me interested in figuring out more on how to troubleshoot and actually understand how people normally solve dependency issues in Clojure.

alexmiller17:01:28

have you used lein ancient ?

alexmiller17:01:07

the canonical maven resource is https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html but you may find it leaves you with more questions than answers :)

shaun-mahood17:01:13

@alexmiller: I use lein ancient but I don't really understand anything beside the basics - so those are probably good resources to start with. The actual problem I'm looking into may be totally unrelated, maybe I'm starting in the wrong place. I tried adding onyx-local-rt to a project using clojure 1.9.0-alpha14, but onyx-local-rt depends on clojure 1.8.0 and clojure-future-spec When I start up the project I get some errors that go away if I exclude clojure-future-spec, but then onyx-local-rt doesn't work. It's not remotely critical but it made me curious about how to deal with it.

alexmiller17:01:14

dependencies form a tree

alexmiller17:01:39

building a classpath (the thing Java actually uses) is a linearization of that tree such that things earlier in the classpath shadow things later in the classpath

alexmiller17:01:25

exclusions are a tool to filter that tree before linearization

alexmiller17:01:45

(as are scopes and other features of maven dependency resolution)

alexmiller17:01:49

the canonical “dependency hell” problem is when you have two leaves of the tree that are different versions of the same library AND where the branches are incompatible

alexmiller17:01:14

also refer to Rich’s keynote at the conj :)

alexmiller17:01:10

the linearization of the tree in this case is going to include both (presumably incompatible) versions of the same lib but only one is going to take effect, thus making one branch unhappy

agile_geek17:01:26

Been burned in a Java project recently by later versions removing an internal class

alexmiller17:01:40

this exact problem is the motivation behind the new brand new module system in Java 9, which theoretically allows you to go back to a tree like runtime model (supported by classloaders) where both incompatible versions can be in use simultaneously

alexmiller17:01:11

I am skeptical that this is going to improve the state of the world :)

agile_geek17:01:30

Is that Jigsaw @alexmiller ?

alexmiller17:01:59

yes, although it’s become a labyrinth of names and jsrs

jeffh-fp17:01:50

I think npm (Node package manager) tried to do a similar thing with v2->v3

agile_geek17:01:55

Usual thing in the Java world...I struggle to keep up with the JSR spiders web

shaun-mahood18:01:19

Are there good ways to view the difference between the dependency tree and the classpath that is the result of linearizing that tree? i.e. lein deps tree turns into ?

alexmiller18:01:04

lein classpath

shaun-mahood18:01:57

Well that makes me feel like I should have been able to figure that out on my own 🙂

alexmiller18:01:43

lein help might have led you there

shaun-mahood18:01:50

Here's the error message I'm getting, dependencies are [org.clojure/clojure "1.9.0-alpha14"] [org.onyxplatform/onyx-local-rt "0.9.15.5"] only. I know the error goes away if I exclude clojure-future-spec, and I suspect the root of the error message is in https://github.com/tonsky/clojure-future-spec/blob/master/src/clojure/core_instant18.clj Is there a nice way to troubleshoot or somehow fix it, or is this the dependency hell error message?

alexmiller18:01:35

that’s likely tricky

alexmiller18:01:47

that ns is conditionally loaded in Clojure itself

alexmiller18:01:23

as it requires classes that don’t exist in all Java versions supported by Clojure

alexmiller18:01:58

are you perhaps using java 1.7?

alexmiller18:01:12

and if so, can you use 1.8?

shaun-mahood18:01:21

java -version tells me 1.8, so that's what I should be on unless there's something else weird going on

alexmiller18:01:53

so if you’re using Clojure 1.9 AND future-spec, then it seems like some of these things are going to conflict - like that ns will exist in both Clojure and in future-spec

alexmiller18:01:54

it seems like you’ll want to exclude future-spec

alexmiller18:01:12

so the question then is what breaks with onyx

shaun-mahood18:01:07

Ok, at least I was on the right track there. If I add the exclusion, when I require onyx-local-rt.api in the repl I get the error message CompilerException java.io.FileNotFoundException: Could not locate clojure/future__init.class or clojure/future.clj on classpath., compiling:(onyx/spec.cljc:1:1) Which seems to make sense, but I'm not sure what the normal next step would be - does it require a change to onyx or can it be dealt with from within my project?

alexmiller18:01:12

seems like onyx to me

alexmiller18:01:19

or you could just fall back to clojure 1.8

shaun-mahood18:01:18

Ok that's good to know - I wasn't sure if there was some other Clojure thing that I just didn't know about. Thanks for the help!

michaeldrogalis19:01:27

@shaun-mahood @alexmiller We’ve been playing a tricky game keeping Onyx core on 1.8 until 1.9 is out officially, but we needed onyx-local-rt to have access to Clojure spec so it could do its thing in the browser. We’ve mostly been working around this by either: (1) Using 1.9 everywhere, and putting stub files into the project for clojure.future which calls back to Clojure core. (2) Using 1.8 everywhere. Everything should “just work” since Clojure.future would be in place.

michaeldrogalis19:01:15

I might have linked this to you before, but I can’t remember. This how you can handle (1) https://github.com/onyx-platform/onyx-local-rt/issues/5

shaun-mahood19:01:30

@michaeldrogalis: Oh that's great, thanks!

michaeldrogalis19:01:13

Agh, I didn’t send you that link the first time? My apologies, I thought I did.

shaun-mahood19:01:50

No problem, I should have seen it in the issues anyway. Glad I went on the journey I did, this is the first time I've looked into most of this part of Clojure in any depth at all so it's been a very good learning experience.

michaeldrogalis19:01:33

Awesome, glad to hear it. 🙂

drewverlee21:01:30

I want to map over a collection of collections of booleans and output if the nth elements are all true? Put another way, i have some column oriented data, and i want to ask a row question x | c1 | c2 r1|t | f r2|t| t i want to be able to ask if all of r1 is true? This sounds like a map function

(map ?? [t f] [t t])
but the question i want to ask is if their all true? Ok so if i just had two booleans i can use and. So should be able to just apply map with that?
clojure
(map and [false true] [true false])

… Can't take value of a macro: #'clojure.core/and
so i can’t map over and, because its a macro? Thats disappointing, and at his point somewhat confusing. (Why is and a marco but + isn’t, whats the advantage?). Ok, whats the alternative?
clojure 
(map #(every? true? [%1 %2]) [false true] [true true])
isn’t sufficient because it requires i know the number of collections

drewverlee21:01:09

maybe i should re-arrange the data?

dominicm21:01:30

and is a macro because it returns the first truthy value, it doesn't evaluate remaining arguments after that.

chronno22:01:28

@drewverlee something like: (map #(every? true? %) [[false true] [true false] [true true]]) should work

chronno22:01:22

Notice that map is being called with only one collection.

drewverlee22:01:00

@chronno Thanks! This works just like i would expect. In order to solve my problem i just transposed the data first then did as you suggested. @dominicm Gotcha. I understand that in order for it to not eval the second value it needs to be a macro. But i still dont grok why macros can’t be passed around like functions can be or used as map. I recall reading it had to do with steps necessary at compile time, but i didn’t understand what that was all about.