Fork me on GitHub
#beginners
<
2017-12-26
>
naylyn.gaffney04:12:28

Hi all, I'm trying to populate a java array with values from a clojure seq and it seems super slow. e.g. (time (def my-seq (take (Math/pow 10 6) (repeatedly #(rand))))) runs in under 1 ms, while (time (def my-arr (double-array my-seq))) takes more than 150 ms What am I doing wrong?

noisesmith04:12:39

the first one, you aren't measuring anything meaningful

noisesmith04:12:04

take and repeatedly are both lazy, and you don't force the value inside the timing

noisesmith04:12:36

so the second thing you time is really the time usage of both together

noisesmith04:12:54

try putting doall around take - also repeatedly takes an optional number of repeats argument and #(rand) is identical to rand

noisesmith04:12:33

so you can replace the first one with (time (def my-seq (doall (repeatedly (Math/pow 10 6) rand))))

naylyn.gaffney05:12:16

Ah, great. Thank you!

porkostomus10:12:13

For one of my first projects, I made a command-line runner for the 4clojure problems: https://github.com/porkostomus/ctrain

slack147817:12:57

Is there a way to overload a function by type, even if its the same arity?

slack147817:12:18

I want someone to be able to pass a string or a BufferedImage object and have this function do the right thing

noisesmith17:12:06

@slack1478 yes, multimethods and protocols are the two easiest ways to do this in clojure

noisesmith17:12:33

multimethods are better in your case unless you are doing this for performance

slack147817:12:01

not for performance, all for ergonomics

slack147817:12:07

I already have a major performance flaw

slack147817:12:10

yeah that page was not helpful for me, read it twice but maybe was too slow

slack147817:12:18

thanks for the pointer

slack147817:12:17

hmm, javaclassnotfound....but i am using it

slack147817:12:39

(defmulti bake (fn [image-or-file & params] (class image-or-file)))

 (defmethod bake IIOImage [image filename metadata]
    (write-image filename
      (reduce set-metadata image metadata)))

 (defmethod bake String [input-filename filename metadata]
    (let [image (load-image input-filename)
          metadata (conj (metadata input-filename))]
     (bake image filename metadata)))

slack147817:12:21

(load-image returns an IIOImage)

noisesmith17:12:36

you either need to use import in that ns to use it without qualification, or use it by its fully package qualified name

slack147817:12:48

I have it imported above

slack147817:12:11

javax.imageio.IIOImage also gives me that error

noisesmith17:12:34

are you sure you have the package right?

slack147817:12:37

problem is the ClassNotFoundError doesn't point me towards any helpful line, just URLClassLoader

slack147817:12:45

which means it might not be in the bake method

slack147817:12:53

time to sprinkle printlns everywhere 😞

noisesmith17:12:12

this is a class loading problem, printlns won’t help

noisesmith17:12:20

the most likely thing is subtly getting the package name wrong

slack147817:12:43

IIOImage works in other functions

slack147817:12:30

maybe my deps.edn needs it?

noisesmith17:12:54

no, it comes with your jvm

noisesmith17:12:29

I don’t know how you’d get class not found for that unless there was a typo somewhere honestly (or a headless vm but you use the class elsewhere in the code without problem so…)

noisesmith18:12:37

@slack1478 I just checkout out your repo, looks fine?

[email protected]: ~/metapng$ git checkout multimethod
Branch multimethod set up to track remote branch multimethod from origin.
Switched to a new branch 'multimethod'
[email protected]: ~/metapng$ clj
Clojure 1.9.0
(ins)user=> (doto 'metapng.core require in-ns)
metapng.core
metapng.core=>

slack147818:12:05

I've been using (use 'metapng.core)

noisesmith18:12:44

I’d expect that to act the same

noisesmith18:12:03

same result - just leaves me in the user ns

slack147818:12:37

aaand now it works

slack147818:12:45

is this...common

noisesmith18:12:55

did you reload the code without restarting your repl?

slack147818:12:15

i closed my terminal, reopened and tried again

noisesmith18:12:16

might be a caching bug with the clj tool

slack147818:12:17

then it worked

slack147818:12:25

but caching should be easy!

slack147818:12:38

welp, im glad im not crazy, thanks for double checking

slack147818:12:58

now, there is a separate issue, that i'm not actually writing the metadata

slack147818:12:03

but thats squarely my problem 🙂

noisesmith18:12:20

it would be cool to replicate the caching issue and submit a bug report to the clj team- but since this is #beginners just knowing we sorted it might be enough

slack147818:12:39

and it is most definitely a caching issue because its printing out lines where I have 0 printlns in my code

noisesmith18:12:50

not only is clj replacing a subset of leiningen’s funcitonality, it’s recreating a subset of past leiningen bugs :P

slack147818:12:39

I should: write more tests, move all the png handling to pngj and get off of javax

slack147818:12:58

but this is nice for now

noisesmith18:12:30

since caching issues already came up, another thing to pay attention to when using multimethods is that when you reload your code the multimethod isn’t redefined - if you need to happen you need to explicitly remove it ((def mymulti nil) is the easiest way to do that)

slack147818:12:30

does it make sense to restat clj every time i get an error with my source file in (use 'calendar) or something like that

noisesmith18:12:04

you can use load-file or add the optional :reload arg to require

noisesmith18:12:56

I find that use is useful for my custom user namespace definition, and clojure.repl and clojure.pprint and otherwise (and especially for code that I am actively developing) require with an alias and/or switching to that namespace in the repl is more productive

noisesmith18:12:37

but if you are in ns foo.bar you can still run (require 'foo.bar :reload) and it will work (with caveats about how some constructs reload of course - but generally it works)

noisesmith18:12:33

when you use the ns, you might need to run use again after loading more definitions though - the mapping is done to your current ns at the time use is called, so it won’t see future definitions and map them

slack147820:12:35

can i use override-deps to point to a local/root even if my regular deps shows a mvn/version?

slack147820:12:41

because that doesn't seem to work too well...

noisesmith20:12:07

load-file works regardless of where you initially got the dep

noisesmith20:12:22

but the rest only really work with the current project, or if you are using lein checkouts

noisesmith20:12:43

(of course you need a file on disk to use load-file but I assume that’s the case if you are editing it)

slack147820:12:22

not using load file, using deps.edn and clj -R:dev with :aliases {:dev {:override-deps {metapng {:local-root "/my/dev/dir/metapng"}}}}

noisesmith20:12:26

usually the way people do this is they have a keystroke in their editor. the editor is connected to the repl and calls load-file behind the scenes

noisesmith20:12:54

what I’m saying is load-file will work for loading file changes

noisesmith20:12:28

and I don’t know enough about clj to tell you if it can make require work - but the easier path is to use load-file which is guaranteed to work

noisesmith20:12:19

I wonder if there’s a #clj channel yet for that tool (looks like no)

slack147820:12:46

so I tried (load-file "/path/to/metapng/core.clj") then (require 'calendar :reload)` but it seems to be using the installed maven version, not my local one TT

noisesmith20:12:54

the :reload caused require to undo the changes load-file did - anyway it’s one or the other, there’s never a reason to use both in a row for the same ns

noisesmith20:12:48

also - what is this “calendar” - is it a single segment namespace? surely it’s not a namespace defined in metapng/core.clj - what are you actually doing here, where does this “calendar” come from?

slack147820:12:44

yay it works now

slack147820:12:59

calendar is a single segment namespace in the project that i'm in

slack147820:12:08

its just a sketch I made with clojure2d

slack147820:12:39

I really am not a fan of the million.namespace.deep.folders.in.a.src.directory

slack147820:12:47

at least for sketches

slack147820:12:52

if i start making libraries, sure

slack147820:12:06

metapng should be is.jonathan.metapng or something like that

noisesmith20:12:11

there’s a few problems with single segment namespaces - but if it’s just something you only use from a source file in the current project that should be fine

slack147820:12:30

yeah, single segment is only for 'leaf' projects, in my head

slack147820:12:13

i also havent decoupled paths from namespaces as well as I should yet

noisesmith20:12:17

also .core is based on leiningen not knowing enough about your project to come up with a better name and isn’t needed - you can give your namespace any name you like

slack147820:12:26

well, metapng really only has 2 functions it should expose, and honestly I would like people to add it via something like [jedahan/metapng :as metapng]

slack147820:12:42

or maybe jedahan.metapng :as metapng

slack147820:12:47

not sure the difference there

noisesmith20:12:58

the / version isn’t valid

slack147820:12:17

well I see that version in deps, which is what confuses me

noisesmith20:12:28

and imho jedahan.metapng is a much better ns than metapng.core

noisesmith20:12:05

yeah, deps are for artifacts, require is for things provided by those artifacts - it’s too bad they look so similar because they are very different things

noisesmith20:12:08

given current conventions, if anyone else decided “metapng” would be a good name for a thing, the likelyhood that metapng.core would clash is high

slack147820:12:10

so like, deps for pngj is ar.com.hjg/pngj, but I import it via ar.com.hjg.pngj, I don't understand

slack147820:12:28

well, i got it on clojars, but yeah

noisesmith20:12:10

one is a maven group and artifact name, separated by a slash, the other is a package which has some classes in it which could be in any artifact

slack147820:12:11

so deps of jedahan/metapng with import jedahan.metapng :as metapng I think reads nicely

noisesmith20:12:34

yes, that is intuitive, and avoids conflicts in a way that metapng.core doesn’t

slack147820:12:34

or require, if its native clojure i guess

noisesmith20:12:57

import never allows :as so I assumed you meant require :)

slack147820:12:09

so then, can i have jedahan.metapng.tests and is there a way to make jedahan.metapng functions private (with defn-) but visible/testable to the tests?

noisesmith20:12:32

you can use @#’foo to call a private var called foo

slack147820:12:35

or is that kinda a paradox 'test the public interface' vs the implementation

slack147820:12:55

i guess the ugliness is good

slack147820:12:05

makes you not wanna mess with private vars

seancorfield20:12:09

"private" in Clojure is much like "private" in Groovy -- somewhat advisory 🙂

slack147820:12:21

yeah, i mean, seems best of both worlds I guess

noisesmith20:12:38

breaking it down - #’foo returns the var for foo, and @ extracts the contents of that var (deref)

noisesmith20:12:06

so it’s a low level workaround skipping the check for privateness that the clojure compiler would do if you used the name directly

seancorfield20:12:48

(and you only need @ for Vars -- with def ^:private -- for functions you can omit @ and Clojure will automatically dereference the Var for you)

noisesmith20:12:27

that’s true - I had forgotten whether #' quote of vars alone was enough to avoid the privacy check and included @ to be safe - turns out it’s not needed

seancorfield20:12:05

My recommendation is to think very carefully about whether you really want a function to be private. If it has testable behavior, it probably has useful behavior and might as well be public and part of your API.

seancorfield20:12:34

An alternative is to put "implementation details" in a .impl namespace (and leave them public).

seancorfield20:12:59

That makes the implementation easy to test and clearly separates implementation functions from the public API as well.

noisesmith20:12:53

bonus points if you give the namespace a cumbersome name and / or use some trick like implementing everything in terms of protocols so that people have to use reify or deftype or defrecord to use any of it, sure to keep the riffraff out (no, don’t actually do this, this is just a joke)

sabbatical201720:12:54

What's the idiomatic way to return true if any of a sequence of bools is true? reduce with or does not work since or is a macro (which makes sense given the desirability of short-circuiting its evaluation).

sabbatical201720:12:54

I ran in to a similar problem yesterday where I could not use if directly in a datascript query, for the same reason.

sabbatical201720:12:56

Perhaps there are non-short circuited versions of the logical operations somewhere?

sabbatical201720:12:16

It would seem to make more sense though to have a short-circuting reduce though

jgh20:12:27

bit-or maybe?

sabbatical201720:12:33

I am assuming there is something more idiomatic out there than (defn aor [x y] (or x y))...

donaldball20:12:52

(partial some? true?)

donaldball20:12:00

depends on your taste for partial

donaldball20:12:12

and of course I meant (partial some true?) 😛

noisesmith20:12:30

the way I would do it with reduce: (reduce #(when (true? %2) (reduced true)) false coll)

noisesmith20:12:36

but yeah, use some for that

sabbatical201720:12:49

Ah, some is what I was looking for! (some identity x) does exactly what I wanted. Thank you, @jgh @donaldball @noisesmith!

noisesmith20:12:17

also you mention short circuiting above - to be clear, reduced causes reduce to short-circuit with the value provided