Fork me on GitHub
#beginners
<
2017-12-26
>
Naylyn04:12:28

Hi all, I'm trying to populate a java array with values from a clojure seq and it seems super slow. e.g. (time (def my-seq (take (Math/pow 10 6) (repeatedly #(rand))))) runs in under 1 ms, while (time (def my-arr (double-array my-seq))) takes more than 150 ms What am I doing wrong?

noisesmith04:12:39

the first one, you aren't measuring anything meaningful

noisesmith04:12:04

take and repeatedly are both lazy, and you don't force the value inside the timing

noisesmith04:12:36

so the second thing you time is really the time usage of both together

noisesmith04:12:54

try putting doall around take - also repeatedly takes an optional number of repeats argument and #(rand) is identical to rand

noisesmith04:12:33

so you can replace the first one with (time (def my-seq (doall (repeatedly (Math/pow 10 6) rand))))

Naylyn05:12:16

Ah, great. Thank you!

porkostomus10:12:13

For one of my first projects, I made a command-line runner for the 4clojure problems: https://github.com/porkostomus/ctrain

jedahan17:12:57

Is there a way to overload a function by type, even if its the same arity?

jedahan17:12:18

I want someone to be able to pass a string or a BufferedImage object and have this function do the right thing

noisesmith17:12:06

@slack1478 yes, multimethods and protocols are the two easiest ways to do this in clojure

noisesmith17:12:33

multimethods are better in your case unless you are doing this for performance

jedahan17:12:01

not for performance, all for ergonomics

jedahan17:12:07

I already have a major performance flaw

jedahan17:12:10

yeah that page was not helpful for me, read it twice but maybe was too slow

jedahan17:12:18

thanks for the pointer

jedahan17:12:08

ohh thanks

jedahan17:12:17

hmm, javaclassnotfound....but i am using it

jedahan17:12:39

(defmulti bake (fn [image-or-file & params] (class image-or-file)))

 (defmethod bake IIOImage [image filename metadata]
    (write-image filename
      (reduce set-metadata image metadata)))

 (defmethod bake String [input-filename filename metadata]
    (let [image (load-image input-filename)
          metadata (conj (metadata input-filename))]
     (bake image filename metadata)))

jedahan17:12:21

(load-image returns an IIOImage)

noisesmith17:12:36

you either need to use import in that ns to use it without qualification, or use it by its fully package qualified name

jedahan17:12:48

I have it imported above

jedahan17:12:11

javax.imageio.IIOImage also gives me that error

noisesmith17:12:34

are you sure you have the package right?

jedahan17:12:37

problem is the ClassNotFoundError doesn't point me towards any helpful line, just URLClassLoader

jedahan17:12:45

which means it might not be in the bake method

jedahan17:12:53

time to sprinkle printlns everywhere 😞

noisesmith17:12:12

this is a class loading problem, printlns won’t help

noisesmith17:12:20

the most likely thing is subtly getting the package name wrong

jedahan17:12:43

IIOImage works in other functions

jedahan17:12:30

maybe my deps.edn needs it?

noisesmith17:12:54

no, it comes with your jvm

noisesmith17:12:29

I don’t know how you’d get class not found for that unless there was a typo somewhere honestly (or a headless vm but you use the class elsewhere in the code without problem so…)

noisesmith18:12:37

@slack1478 I just checkout out your repo, looks fine?

[email protected]: ~/metapng$ git checkout multimethod
Branch multimethod set up to track remote branch multimethod from origin.
Switched to a new branch 'multimethod'
[email protected]: ~/metapng$ clj
Clojure 1.9.0
(ins)user=> (doto 'metapng.core require in-ns)
metapng.core
metapng.core=>

jedahan18:12:05

I've been using (use 'metapng.core)

noisesmith18:12:44

I’d expect that to act the same

noisesmith18:12:03

same result - just leaves me in the user ns

jedahan18:12:37

aaand now it works

jedahan18:12:45

is this...common

noisesmith18:12:55

did you reload the code without restarting your repl?

jedahan18:12:05

i restarted

jedahan18:12:15

i closed my terminal, reopened and tried again

noisesmith18:12:16

might be a caching bug with the clj tool

jedahan18:12:17

then it worked

jedahan18:12:25

but caching should be easy!

jedahan18:12:38

welp, im glad im not crazy, thanks for double checking

jedahan18:12:58

now, there is a separate issue, that i'm not actually writing the metadata

jedahan18:12:03

but thats squarely my problem 🙂

noisesmith18:12:20

it would be cool to replicate the caching issue and submit a bug report to the clj team- but since this is #beginners just knowing we sorted it might be enough

jedahan18:12:39

and it is most definitely a caching issue because its printing out lines where I have 0 printlns in my code

noisesmith18:12:50

not only is clj replacing a subset of leiningen’s funcitonality, it’s recreating a subset of past leiningen bugs :P

jedahan18:12:39

I should: write more tests, move all the png handling to pngj and get off of javax

jedahan18:12:58

but this is nice for now

noisesmith18:12:30

since caching issues already came up, another thing to pay attention to when using multimethods is that when you reload your code the multimethod isn’t redefined - if you need to happen you need to explicitly remove it ((def mymulti nil) is the easiest way to do that)

jedahan18:12:30

does it make sense to restat clj every time i get an error with my source file in (use 'calendar) or something like that

noisesmith18:12:04

you can use load-file or add the optional :reload arg to require

noisesmith18:12:56

I find that use is useful for my custom user namespace definition, and clojure.repl and clojure.pprint and otherwise (and especially for code that I am actively developing) require with an alias and/or switching to that namespace in the repl is more productive

noisesmith18:12:37

but if you are in ns foo.bar you can still run (require 'foo.bar :reload) and it will work (with caveats about how some constructs reload of course - but generally it works)

noisesmith18:12:33

when you use the ns, you might need to run use again after loading more definitions though - the mapping is done to your current ns at the time use is called, so it won’t see future definitions and map them

jedahan20:12:35

can i use override-deps to point to a local/root even if my regular deps shows a mvn/version?

jedahan20:12:41

because that doesn't seem to work too well...

noisesmith20:12:07

load-file works regardless of where you initially got the dep

noisesmith20:12:22

but the rest only really work with the current project, or if you are using lein checkouts

noisesmith20:12:43

(of course you need a file on disk to use load-file but I assume that’s the case if you are editing it)

jedahan20:12:22

not using load file, using deps.edn and clj -R:dev with :aliases {:dev {:override-deps {metapng {:local-root "/my/dev/dir/metapng"}}}}

noisesmith20:12:26

usually the way people do this is they have a keystroke in their editor. the editor is connected to the repl and calls load-file behind the scenes

noisesmith20:12:54

what I’m saying is load-file will work for loading file changes

noisesmith20:12:28

and I don’t know enough about clj to tell you if it can make require work - but the easier path is to use load-file which is guaranteed to work

noisesmith20:12:19

I wonder if there’s a #clj channel yet for that tool (looks like no)

jedahan20:12:46

so I tried (load-file "/path/to/metapng/core.clj") then (require 'calendar :reload)` but it seems to be using the installed maven version, not my local one TT

noisesmith20:12:54

the :reload caused require to undo the changes load-file did - anyway it’s one or the other, there’s never a reason to use both in a row for the same ns

noisesmith20:12:48

also - what is this “calendar” - is it a single segment namespace? surely it’s not a namespace defined in metapng/core.clj - what are you actually doing here, where does this “calendar” come from?

jedahan20:12:44

yay it works now

jedahan20:12:59

calendar is a single segment namespace in the project that i'm in

jedahan20:12:08

its just a sketch I made with clojure2d

jedahan20:12:39

I really am not a fan of the million.namespace.deep.folders.in.a.src.directory

jedahan20:12:47

at least for sketches

jedahan20:12:52

if i start making libraries, sure

jedahan20:12:06

metapng should be is.jonathan.metapng or something like that

noisesmith20:12:11

there’s a few problems with single segment namespaces - but if it’s just something you only use from a source file in the current project that should be fine

jedahan20:12:30

yeah, single segment is only for 'leaf' projects, in my head

jedahan20:12:13

i also havent decoupled paths from namespaces as well as I should yet

noisesmith20:12:17

also .core is based on leiningen not knowing enough about your project to come up with a better name and isn’t needed - you can give your namespace any name you like

jedahan20:12:26

well, metapng really only has 2 functions it should expose, and honestly I would like people to add it via something like [jedahan/metapng :as metapng]

jedahan20:12:42

or maybe jedahan.metapng :as metapng

jedahan20:12:47

not sure the difference there

noisesmith20:12:58

the / version isn’t valid

jedahan20:12:17

well I see that version in deps, which is what confuses me

noisesmith20:12:28

and imho jedahan.metapng is a much better ns than metapng.core

noisesmith20:12:05

yeah, deps are for artifacts, require is for things provided by those artifacts - it’s too bad they look so similar because they are very different things

noisesmith20:12:08

given current conventions, if anyone else decided “metapng” would be a good name for a thing, the likelyhood that metapng.core would clash is high

jedahan20:12:10

so like, deps for pngj is ar.com.hjg/pngj, but I import it via ar.com.hjg.pngj, I don't understand

jedahan20:12:28

well, i got it on clojars, but yeah

noisesmith20:12:10

one is a maven group and artifact name, separated by a slash, the other is a package which has some classes in it which could be in any artifact

jedahan20:12:11

so deps of jedahan/metapng with import jedahan.metapng :as metapng I think reads nicely

noisesmith20:12:34

yes, that is intuitive, and avoids conflicts in a way that metapng.core doesn’t

jedahan20:12:34

or require, if its native clojure i guess

noisesmith20:12:57

import never allows :as so I assumed you meant require :)

jedahan20:12:09

so then, can i have jedahan.metapng.tests and is there a way to make jedahan.metapng functions private (with defn-) but visible/testable to the tests?

noisesmith20:12:32

you can use @#’foo to call a private var called foo

jedahan20:12:35

or is that kinda a paradox 'test the public interface' vs the implementation

jedahan20:12:55

i guess the ugliness is good

jedahan20:12:05

makes you not wanna mess with private vars

seancorfield20:12:09

"private" in Clojure is much like "private" in Groovy -- somewhat advisory 🙂

jedahan20:12:21

yeah, i mean, seems best of both worlds I guess

noisesmith20:12:38

breaking it down - #’foo returns the var for foo, and @ extracts the contents of that var (deref)

noisesmith20:12:06

so it’s a low level workaround skipping the check for privateness that the clojure compiler would do if you used the name directly

seancorfield20:12:48

(and you only need @ for Vars -- with def ^:private -- for functions you can omit @ and Clojure will automatically dereference the Var for you)

noisesmith20:12:27

that’s true - I had forgotten whether #' quote of vars alone was enough to avoid the privacy check and included @ to be safe - turns out it’s not needed

seancorfield20:12:05

My recommendation is to think very carefully about whether you really want a function to be private. If it has testable behavior, it probably has useful behavior and might as well be public and part of your API.

seancorfield20:12:34

An alternative is to put "implementation details" in a .impl namespace (and leave them public).

seancorfield20:12:59

That makes the implementation easy to test and clearly separates implementation functions from the public API as well.

noisesmith20:12:53

bonus points if you give the namespace a cumbersome name and / or use some trick like implementing everything in terms of protocols so that people have to use reify or deftype or defrecord to use any of it, sure to keep the riffraff out (no, don’t actually do this, this is just a joke)

Sabbatical201720:12:54

What's the idiomatic way to return true if any of a sequence of bools is true? reduce with or does not work since or is a macro (which makes sense given the desirability of short-circuiting its evaluation).

Sabbatical201720:12:54

I ran in to a similar problem yesterday where I could not use if directly in a datascript query, for the same reason.

Sabbatical201720:12:56

Perhaps there are non-short circuited versions of the logical operations somewhere?

Sabbatical201720:12:16

It would seem to make more sense though to have a short-circuting reduce though

jgh20:12:27

bit-or maybe?

Sabbatical201720:12:33

I am assuming there is something more idiomatic out there than (defn aor [x y] (or x y))...

donaldball20:12:52

(partial some? true?)

donaldball20:12:00

depends on your taste for partial

donaldball20:12:12

and of course I meant (partial some true?) 😛

noisesmith20:12:30

the way I would do it with reduce: (reduce #(when (true? %2) (reduced true)) false coll)

noisesmith20:12:36

but yeah, use some for that

Sabbatical201720:12:49

Ah, some is what I was looking for! (some identity x) does exactly what I wanted. Thank you, @jgh @donaldball @noisesmith!

noisesmith20:12:17

also you mention short circuiting above - to be clear, reduced causes reduce to short-circuit with the value provided