Fork me on GitHub
#clojure
<
2017-11-06
>
gonewest81800:11:46

On the other hand, there’s the philosophy that you shouldn’t need to “unit test” with an external dependency like a database

zignd00:11:54

@gonewest818 I don't think it would work for the Client API, it only connects to a peer server. And even though the Client API has similar methods for creating, deleting and listing databases (http://docs.datomic.com/clojure-client/index.html#datomic.client.admin), I'm not being able to make them work in memory databases. I guess I will have to make use of this dirty hack or I could use the older library, but wouldn't be this entertaining xD

qqq00:11:17

When clojure tells me:

java.lang.IllegalArgumentException: More than one matching method found: add
clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: More than one matching method found: add, compiling:(
is there a way to respond: great, please list all the matching methods for me?

zignd00:11:44

In Cursive you can use Ctrl+B and check the method overloads. But it seems that you will have to use type hints

qqq00:11:41

yeah, so I try type hints, and it tells me "you ca't type hint this local"

qqq00:11:47

maybe I should produce a minimal failure case

qqq00:11:25

(ns snip.fail
  (:require [primitive-math :as p]))


(let [a (float-array 20) 
      y (aget ^floats a 0)]
  (loop [x (float 0.0)] 
    (p/+ x y)))



^--- please help me typehint this (yes, the loop is important, you can't change the loop to a ledt)

zignd01:11:10

(let [a (float-array 20)
      y (aget a 0)]
  (loop [x 0]
    (p/+ (float x) (float y))))

zignd01:11:29

^ It seems that you can do it like this @qqq

qqq01:11:43

yes, indeed, how expensive is the (float ...) given the arg is already a float?

qqq01:11:01

I genuinely don't know, and am trying to avoid as much boxing/unboxinga s possible

zignd01:11:37

public static float floatCast(Object x) {
        if (x instanceof Float) {
            return ((Float)x).floatValue();
        } else {
            double n = ((Number)x).doubleValue();
            if (n >= -3.4028234663852886E38D && n <= 3.4028234663852886E38D) {
                return (float)n;
            } else {
                throw new IllegalArgumentException("Value out of range for float: " + x);
            }
        }
    }

zignd01:11:02

This is what happens every time you call float. Lots of unboxing

qqq01:11:25

this also confuses me:

;; fails 
(let [a (float-array 20) 
      y (aget ^floats a 0)] 
  (loop [x (float 0.0)] 
    (p/+ x y)))


;; works
(let [a (float-array 20) 
      y (aget ^floats a 0)
      x (float 0.0)] 
  (p/+ x y))

qqq01:11:39

so for whatever reason, let = works, loop = fails, (unfortunately, I need to loop)

zignd01:11:47

(let [a [0.0 0.0 0.0 0.0 0.0]
      y (a 0)]
  (loop [x 0.0]
    (p/+ x y)))

zignd01:11:13

It works this way too. Do you actually need to use float-array?

qqq01:11:26

I think these are all doubles, not floats right ?

qqq01:11:37

(I do have to use float-array, as I'm aset -ing alot)

qqq01:11:08

(let [a [(float 0.0)]
      y (a 0)]
  (loop [x (float 0.0)]
    (p/+ x y)))


this also fails

qqq01:11:26

question is: why is clojure making this a float vs double issue

zignd01:11:06

Oh, you also have the option do use unchecked-float, it's a bit less expensive:

public static float uncheckedFloatCast(Object x) {
        return ((Number)x).floatValue();
    }

qqq01:11:37

;; works 
(let [a (double-array 20) 
      y (aget ^double a 0)] 
  (loop [x (double 0.0)] 
    (p/+ x y)))



;; fails 
(let [a (float-array 20) 
      y (aget ^float a 0)] 
  (loop [x (float 0.0)] 
    (p/+ x y)))


qqq01:11:42

seems like double/float are treated differently

qqq01:11:51

@ztellman: any chance you can help? 🙂

qqq01:11:38

alright, I have an even simple minimal failure case:

;; works 
(let [y (double 0.0)] 
  (loop [x (double 0.0)] 
    (p/+ x y)))

;; fails 
(let [y (float 0.0)] 
  (loop [x (float 0.0)] 
    (p/+ x y)))

zignd01:11:20

That's interesting, I wonder why loop works like that

(loop [x (float 0.0)]
  (type x))
=> java.lang.Double

zignd01:11:56

Btw, @qqq that's what it's causing the impossibility to find the correct overload

(let [y (float 0.0)]
  (loop [x (float 0.0)]
    (println (type x) (type y))))
=> java.lang.Double java.lang.Float

qqq01:11:10

that clarifies so much

qqq01:11:37

I don't know how to file clojure bug reports, but I think your example above has to be a bug

zignd01:11:05

I don't think it's a bug, it's seems to be related to the quotes and unquotes being performed inside the loop macro. Check this out:

qqq01:11:30

Regardless of the macro internals,

(loop [x (float 0.0)]
  (type x))
=> java.lang.Double
should return Float instead of Double, IMHO

zignd01:11:50

I guess you can try other alternatives to the loop macro, or the unchecked-float that seems to be a bit less expensive

qqq01:11:18

@tbaldridge @alexmiller: When you get a chance, can you please let us know what Clojure's semi-official stance on

(loop [x (float 0.0)]
  (type x))
=> java.lang.Double
is intended behaviour -- or if it should return java.lang.Float, and thus a bug ?

qqq01:11:29

@zignd: I'm going to go study loop* 🙂

zignd01:11:00

You will be taken to the depths of the compiler internals, be careful mate. src/jvm/clojure/lang/Compiler.java#L6318 more precisely

qqq01:11:46

yeah, it's kinda a bad sign when there are no docs, i.e.

boot.user> (clojure.repl/doc loop*)
nil
boot.user> (clojure.repl/doc loop)
-------------------------
clojure.core/loop
  (loop [bindings*] exprs*)

qqq01:11:04

correct me if I"m wrong, the problem here is the loop macro is smart, so at compile time, it's like "oh, I see (float 0.0)" let me evaluate this at macro expansion time "oh, we got 0.0" [next stage:] "oh, 0.0 is a double"

qqq01:11:16

actually, my theory is wrong:

(macroexpand 
 '(loop [x (float 0.0)]
    x))
;; ==> (loop* [x (float 0.0)] x)

zignd01:11:39

I don't think you will be able to expand everything inside this macro, loop* is a symbol defined in the compiler's code. I don't even think there's documentation for it. It's something we should never have to look at

qqq01:11:27

(macroexpand 
 '(loop [x (float 0.0)]
    (type x)))
;; => (loop* [x (float 0.0)] (type x))

(loop [x (float 0.0)]
    (type x))
;; => java.lang.Double

(loop* [x (float 0.0)] (type x))
;; => java.lang.Double
looking at that, it looks like everything is passed to loop* correctly (still as a float), but loop* [which is implemented in Java?] is doing the weird float -> double translation

zignd01:11:04

Yup, the macro takes your bindings and body and send them to loop* so that it can perform the magic

Alex Miller (Clojure team)01:11:49

Clojure only supports primitive long and double

Alex Miller (Clojure team)01:11:27

Loops support a wider set of loop local primitive types

Alex Miller (Clojure team)01:11:10

But once you leave the loop you’re back to long and double

qqq01:11:37

@alexmiller: what does 'support' mean in 'Clojure only supports primitive long and double.' We have float-array , aget, float -- this seems to contradict only supporting double.

Alex Miller (Clojure team)01:11:27

Some of that is vestigial from an earlier time when a broader set were supported. Some is for limited interop

Alex Miller (Clojure team)01:11:05

Only float and double are supported as primitive args/returns in functions

Alex Miller (Clojure team)01:11:53

Sorry, only long and double

qqq01:11:13

Only float and double are supported as primitive args/returns in functions <-- Can you please explain what happens when I call a java function of type sig float java_add(float x, float y) and call it from clojure as (java_add (float 2.0) (float 3.0)) in particular, what is returned, a float or a double ?

Alex Miller (Clojure team)01:11:45

Sorry, I’m signing off for the night

Alex Miller (Clojure team)01:11:53

Too long, catch you tomorrow

qqq01:11:56

thanks for your help; take care

zignd01:11:45

Considering what alex said, you should try to use doubles instead of floats, they will fit your float values anyway. At least for these operations involving the loop macro.

qqq02:11:38

I'm interfacing with other libraries that want floats.

qqq02:11:03

I'm going to write this performance critical code in java 🙂.

qqq02:11:18

@zignd: thanks for all your help, on a sunday evening, with debugging this.

zignd02:11:33

No problem :)

zignd02:11:27

Regarding Datomic, I've been struggling with this for an hour or two, please help me out. I'm trying to run a parameterized query, but apparently I'm doing something wrong, does anyone knows why is the first query not returning the entity? I tried to follow the documentation, but it doesn't seems to be working... ;-; I'm also using the Datomic's Client API (http://docs.datomic.com/clojure-client/index.html)

csm03:11:06

Just pass the value to bind to ?p-username in :args, e.g. just pass [db "zignd"] as :args

zignd10:11:35

Thank you! It worked!

csm03:11:17

(There is also #datomic)

zignd10:11:00

I will post there next time

qqq04:11:48

I have tried .size, .length, .-length how do I get, in O(1) time, the length of a (float-array n) ?

tbaldridge05:11:40

@qqq acount I think?

qqq05:11:41

alength, thanks!

tbaldridge05:11:57

arrays are really strange on the JVM, they don't have methods, or fields

tbaldridge05:11:22

So in Java you have to call something to Array/length, and even that I think might be some sort of strange compiler intrinsic

souenzzo14:11:39

Can I "append" a pre/post function to a function?

zignd14:11:24

(defn constrained-fn [f x]
   {:pre [(pos? x)]
   :post [(= % (* 2 x))]}
   (f x))
:pre and :post assertions

dpsutton14:11:02

you mean after the fact?

taylor14:11:06

depending on what you mean by append, this might be helpful too https://github.com/technomancy/robert-hooke

qqq15:11:40

(defmacro foo [x y]
  `(let [^floats ~x ~y]
     ~x))

(macroexpand-1
 '(foo x (float-array 20)))
;; ==> (clojure.core/let [x (float-array 20)] x)
It appears my type hint of ^floats is being stripped away. Is there a way for macros to generate / keep type hints ?

qqq15:11:55

[ I need to keep type hints as I end up using primitive math. ]

borkdude16:11:47

Probably not, but is there something in key destructuring that lets you rename the keys in one go as you destructure them?

borkdude16:11:55

@U09LZR36F I mean something like this:

(let [{:keys [x y]} {:x 1 :y 2}
      original-x x
      original-y y
      {:keys [x y]} {:x 2 :y 3}
      new-x x
      new-y y]
  {:x (+ original-x new-x)
   :y (+ original-y new-y)})
where I want to assign the destructured x directly to the intended name

borkdude16:11:05

Ah, that is what {new-name :old-name} can do, I see

borkdude16:11:37

(let [{original-x :x original-y :y} {:x 1 :y 2}
      {new-x :x new-y :y} {:x 2 :y 3}]
  {:x (+ original-x new-x)
   :y (+ original-y new-y)})

borkdude16:11:09

Actually the first time I needed this renaming 🙂

dominicm16:11:00

{new-name :old-name}

dyba16:11:07

Is anyone here using the s3-private-wagon plugin with Leiningen 2.7.1 on Java 1.8? I’m getting authentication errors when using the plugin and running lein deps; however, when I run the aws cli tool, I’m able to retrieve objects from the private bucket.

dyba16:11:50

It seems s3-private-wagon isn’t retrieving the EC2 instance profile credentials

dyba16:11:06

[email protected]:~/workspace/movr3-api-dev$ lein deps
Could not transfer artifact wardrobe:wardrobe:pom:0.1.3 from/to releases (): Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: AA5EF73EDF617F8E)
This could be due to a typo in :dependencies or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
[email protected]:~/workspace/movr3-api-dev$ cd ../../.m2/repository/wardrobe/wardrobe/0.1.3/
[email protected]:~/.m2/repository/wardrobe/wardrobe/0.1.3$ aws s3 sync  .
download:  to ./wardrobe-0.1.3.jar.sha1
download:  to ./wardrobe-0.1.3.pom
download:  to ./wardrobe-0.1.3.jar
download:  to ./wardrobe-0.1.3.pom.sha1
download:  to ./wardrobe-0.1.3.jar.md5
download:  to ./wardrobe-0.1.3.pom.md5
[email protected]:~/.m2/repository/wardrobe/wardrobe/0.1.3$

dyba16:11:08

By the way, I’m using version 1.3.0 of s3-private-wagon

dyba16:11:53

https://github.com/s3-wagon-private/s3-wagon-private/issues/18 It seems this has already been solved as of version 1.3.0

ghadi16:11:01

You may want to try leiningen 2.8.1 @daniel.dyba, I know some attention went into updating s3-private-wagon

ghadi16:11:37

Not sure it handles your issue, but it does appear that related issues have been closed upstream

dyba16:11:10

@ghadi Thanks for the tip! I’ll give that a try

dyba16:11:30

Could not find artifact wardrobe:wardrobe:jar:0.1.3 in central ()
Could not find artifact wardrobe:wardrobe:jar:0.1.3 in clojars ()
Could not transfer artifact wardrobe:wardrobe:jar:0.1.3 from/to releases (): Access key cannot be null.
Could not transfer artifact wardrobe:wardrobe:jar:0.1.3 from/to snapshots (): Access key cannot be null.
Could not transfer artifact wardrobe:wardrobe:pom:0.1.3 from/to releases (): Access key cannot be null.
This could be due to a typo in :dependencies, file system permissions, or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.

dyba16:11:58

I’m getting a similar error with the upgrade to Lein 2.8.1. I’m going to try Lein 2.8.0 next

ghadi16:11:48

you're getting a much better error message now

ghadi16:11:59

Access key cannot be null

ghadi16:11:31

Might want to ask in #aws @daniel.dyba -- usually for IAM instance profiles you don't have to do anything

qqq17:11:44

(defmacro ifor [[i start end] & body]
  `(let [start# (long ~start)
         end#   (long ~end)] 
     (loop [~i start#]
       (when (p/< ~i end#)
         [email protected]
         (recur (unchecked-inc ~i))))))

(defmacro iproc [expr [data i v] & body]
  `(let [~data ~expr]
     (ifor [~i 0 (alength ~data)]
           (let [~v (aget ^floats ~data ~i)] 
             [email protected]))))

(def x (float-array (* 1000 100)))

(cc/quick-bench 
 (ifor [i 0 (alength x)]
       (aset ^floats x i (float i))))
Evaluation count : 3276 in 6 samples of 546 calls.
             Execution time mean : 183.588777 µs
    Execution time std-deviation : 923.219982 ns
   Execution time lower quantile : 182.996929 µs ( 2.5%)
   Execution time upper quantile : 185.164436 µs (97.5%)
                   Overhead used : 1.962587 ns

(cc/quick-bench 
 (let [^long end (alength x)]
   (loop [i 0]
     (when (p/< i end)
       (aset ^floats x i (float i))
       (recur (unchecked-inc i))))))
Evaluation count : 2412 in 6 samples of 402 calls.
             Execution time mean : 248.133557 µs
    Execution time std-deviation : 926.589005 ns
   Execution time lower quantile : 247.272398 µs ( 2.5%)
   Execution time upper quantile : 249.071233 µs (97.5%)
                   Overhead used : 1.962587 ns

(cc/quick-bench 
 (iproc x [x i v]
        (aset ^floats x i (float i))))
Evaluation count : 6 in 6 samples of 1 calls.
             Execution time mean : 1.121665 sec
    Execution time std-deviation : 70.439931 ms
   Execution time lower quantile : 1.067037 sec ( 2.5%)
   Execution time upper quantile : 1.206947 sec (97.5%)
                   Overhead used : 1.962587 ns

qqq17:11:04

any idea why the running time increases ? I would expect the three to be about the same

bronsa17:11:20

your type hint in iproc isn't doing what you think it's doing

qqq17:11:48

should I be doing

(aget ~(vary-meta data assoc :tag 'floats) ~i) 

bronsa17:11:48

use ~(with-meta data {:tag 'floats}) instead

alexstokes17:11:20

i have a question about idiomatic application architecture — i’m writing a server application with a series of modules acting as various services; my thinking now is that i can wrap the important state to each module in an atom and then connect each module w/ core.async channels to pass messages around as various events come into the top-level port

alexstokes17:11:17

does anyone have any feedback on this plan? or a better suggestion for structuring the software? i’m used to the situation in Go where i can just connect all the modules w/ channels to pass messages around; what i’m used to doing in clojure for concurrency is using atoms and i’m trying to resolve the two approaches

bfabry17:11:53

@alexstokes I would caution against that architecture unless you specifically know you need concurrency already

qqq17:11:22

is that basically the erlang / actor model ?

qqq17:11:59

@bronsa :

(defmacro ifor [[i start end] & body]
  `(let [start# (long ~start)
         end#   (long ~end)] 
     (loop [~i start#]
       (when (p/< ~i end#)
         [email protected]
         (recur (unchecked-inc ~i))))))

(defmacro iproc [expr [data i v] & body]
  `(let [~(vary-meta data assoc :tag 'floats) ~expr]
     (ifor [~i 0 (alength ~data)]
           (let [~v (aget ~(vary-meta data assoc :tag 'floats) ~i)] 
             [email protected]))))


(do

  (def x (float-array (* 1000 1000)))

  (println "\n\n\n\n\n")
  (println "=====")

  (cc/quick-bench 
   (ifor [i 0 (alength x)]
         (aset ^floats x i (float i))))

  (cc/quick-bench 
   (iproc x [x i v]
          (aset ^floats x i (float i)))))


=====
Evaluation count : 360 in 6 samples of 60 calls.
             Execution time mean : 1.680640 ms
    Execution time std-deviation : 3.250002 µs
   Execution time lower quantile : 1.677290 ms ( 2.5%)
   Execution time upper quantile : 1.685090 ms (97.5%)
                   Overhead used : 1.962587 ns
Evaluation count : 456 in 6 samples of 76 calls.
             Execution time mean : 1.324811 ms
    Execution time std-deviation : 255.773342 ns
   Execution time lower quantile : 1.324424 ms ( 2.5%)
   Execution time upper quantile : 1.324963 ms (97.5%)
                   Overhead used : 1.962587 ns


not complaining at all -- iproc is now faster than ifor, weird

alexstokes17:11:09

ok great @bfabry i’m working on a bitcoin node — messages from peers will end up going to one module and then that will have downstream effects to some other modules

alexstokes17:11:32

i’m wondering what the best way to do this in clojure is

bfabry17:11:10

I don't know enough about erland/actor model to comment on that. you absolutely can set up an app where you have components that own little worker threads and the components talk to each other via channels. I've written apps like that. almost all of them i either have or wish I had gone back and rewritten them in a more functional way

qqq17:11:10

to reinforce @bfabry’s point: I've found core.async hard to debug (both stack traces and reasoning about it) -- and found it's much simpler to take the FRP / reagent model /reactive atom route

bfabry17:11:31

turns out it's hard to predict which parts of your app will benefit from asynchronous programming 🙂

bfabry17:11:29

now when I use core.async I prefer to keep it in one namespace that wires everything together. gives you one place to read/understand your asynchronous algorithm and one place to vary the number of threads for the different parts etc

jeff.terrell17:11:34

@alexstokes - That sounds to me like a use case that Storm is designed to solve. However, I'm speaking from relative ignorance here on both the application architecture and Storm itself, so take that with a grain of salt. http://www.storm-project.net/

alexstokes17:11:05

thanks! seems a little like overkill

alexstokes17:11:12

I may try putting all the state in a single atom

alexstokes17:11:34

There will just be a little overhead for each module to only get the pieces it wants

eriktjacobsen17:11:46

It might also be overkill, but you could take a look at Onyx. It's written in clojure and will handle things like state for your modules and has a clear model for describing how the data flows between your tasks.

alexstokes17:11:10

Anyone use anything like lenses?

eriktjacobsen17:11:38

look into Specter

alexstokes17:11:33

I think that will help if I go with one big ball of state approach

alexstokes17:11:34

Each module can define a lens over the substate it cares about and a single atom can handle concurrent writes

alexstokes18:11:21

Does anyone have any suggestions on writing maintainable accessors with specter?

alexstokes18:11:55

Can I avoid having to re-write all my accessors if the shape of the state changes?

bfabry18:11:59

onyx/storm are great, but I don't think they're really related to the problem here

alexstokes18:11:21

This is p simple and I just want well factored code

eriktjacobsen18:11:42

If you just go with a single key per module, you probably won't need lenses. Although that's assuming each module has its own subsection rather than different modules needing state from different pieces.

bfabry18:11:52

@alexstokes specter is probably helpful here. but if you have lots of different functions mutating things then you're not writing very clojurey code

bfabry18:11:27

humble suggestion: write functions that take data and return data 🙂

alexstokes18:11:27

@bfabry can you point to an example that shows the difference?

alexstokes18:11:08

Yeah I guess the issue is going from that to “here is a stream of events that impact various modules”

bfabry18:11:20

if at all possible pull state tracking/munging up to the very top level

alexstokes18:11:42

Yeah I think that is the single top-level atom idea I like

alexstokes18:11:15

Then depending on what happens each module will need to read/write the state

bfabry18:11:23

right, but if you have a single top level atom I would manipulate it only at that top level, ie deref the atom and pass the contents down, not the atom itself. then receive the changes back up the top and modify the atom

alexstokes18:11:52

Ok that’s a good suggestion

alexstokes18:11:57

I’ll keep that in mind

alexstokes18:11:29

Which invariably has me asking — what’s a good way to diff data structures ?

alexstokes18:11:52

Is this map different from that map, etc

sundarj18:11:44

@alexstokes

(= {:a 1} {:a 1})
true

(= {:a 1} {:a 2})
false

bfabry18:11:55

so long as you're staying in the good world of immutable clojure data structures then comparing is just = ya

plins18:11:43

hello every one, is there a better way to achieve this? (apply str (map str (interpose “, ” [1 2 3 4 5])))

eriktjacobsen18:11:43

Unless diffing maps is specific to the task... I'd say it's probably bad practice in general. Think more about how the data flows through the system, the output of a pure function should depend on the input, generally not the input + diff against another piece of changing state.

sundarj18:11:44

(clojure.string/join ", " [1 2 3 4 5]) @plins

jeff.terrell18:11:44

@alexstokes - Agreed with the other comments about YAGNI, but in case you do need it: https://clojuredocs.org/clojure.data/diff

josh.freckleton19:11:55

I need to parse simple text like: "Important Field: {{important-field}}" and replace {{important-field}} with a value from a map should I use something as heavy as instaparse, or what would be an idiomatic way in clojure?

aengelberg19:11:11

Instaparse isn't well suited for "find/replace" use cases

aengelberg19:11:12

It's more designed to parse a whole block of text data and interpret it as data

noisesmith19:11:24

there are templating libraries that use that syntax already

noisesmith19:11:51

and selmer, and antlers

seancorfield19:11:26

Selmer would be my first choice (we use it very heavily at World Singles), but clostache is probably simpler/more lightweight for this use case(?).

aengelberg19:11:07

or (clojure.string/replace my-str #"\{\{(.*?)\}\}" (fn ...))

josh.freckleton19:11:02

thanks for the options, I'll weigh em out for my use case!

qqq19:11:06

x = float-array of size 1000 y = float-array of size 2000 is there a builtin to say y[1000:1999] = x ? (I'm hoping for a java builtin faster than 1000 aset / agets)

ghadi19:11:18

System/arraycopy (no slices on JVM)

ghadi19:11:26

yup (but the long arity version), that one

qqq19:11:30

(def n (* 1000 1000 1000)) 

(def x (float-array n)) 

(def y (float-array (+ 1000 n)))

(cc/quick-bench 
 (System/arraycopy x 0 y 999 n))
gives:
Evaluation count : 6 in 6 samples of 1 calls.
             Execution time mean : 1.047620 sec
    Execution time std-deviation : 98.981008 ms
   Execution time lower quantile : 954.274457 ms ( 2.5%)
   Execution time upper quantile : 1.137644 sec (97.5%)
                   Overhead used : 1.962587 ns
that is copying 4GB in 1.13 seconds -- that's insane

noisesmith19:11:42

I bet that’s leaning heavily on OS level optimizations - might be interesting to benchmark on various platforms

qqq20:11:08

agreed, short of a memcpy, I'm not sure how it can be that fast

mping21:11:38

hi there, I’m having an issue with futures

mping21:11:02

I have a fn with (try ... (catch ) ... (finally)) that works ok and returns as expected

mping21:11:11

but when I run it through an executor it returns nil

mping21:11:20

do you guys have any pointers?

mping21:11:31

(defn submit-job [fn] (.submit threadpool fn))

@(submit-job (constantly 1))
nil

hiredman21:11:34

submit can take a Runnable or a Callable, and fn's implement both, so the reflector has to pick one

hiredman21:11:41

^Callable fn

hiredman21:11:17

submit returns a future that returns a value when you deref it

hiredman21:11:30

if you submit a Callable you get the return value of the callable

seancorfield21:11:34

Runnable doesn't return a value, Callable does.

mping21:11:02

(deref (submit-job (cast java.util.concurrent.Callable (fn [] 1)))) still nil

seancorfield21:11:02

(well, in an imprecise way 🙂 )

seancorfield21:11:16

Use a type hint.

mping21:11:16

I feel stupid 🙂 gonna learn something today I guess

hiredman21:11:24

you can't use cast for that

seancorfield21:11:36

(defn submit-job [^Callable fn] (.submit threadpool fn)) I think?

hiredman21:11:37

and you can't type hint a (fn ..) form like that either

mping21:11:46

cast is runtime, I want to tell the compiler how to dispatch right?

mping21:11:58

nope 😄 @(let [^java.util.concurrent.Callable f (fn [] 1)] (.submit threadpool f))

mping21:11:10

still nil

hiredman21:11:55

you also need to type hint threadpool

hiredman21:11:28

if you (set! *warn-on-reflection* true) you will see that the method call is still reflective

seancorfield21:11:53

boot.user=> (defn submit-job ^java.util.concurrent.Future [^java.util.concurrent.Callable f] (.submit ^java.util.concurrent.ForkJoinPool fjpool f))
#'boot.user/submit-job
boot.user=> (deref (submit-job (constantly 1)))
1

seancorfield21:11:34

(I'm using a ForkJoinPool for ease of creation but should work with a general executor service)

seancorfield21:11:16

(switching the type hint to ^java.util.concurrent.ExecutorService works -- I just tried it)

hiredman21:11:21

the reflected method chosen may actually vary by clojure release, with the beta of 1.9 even with reflection it chooses the callable version of submit

hiredman21:11:34

user=> (set! *warn-on-reflection* true)
true
user=> (def pool1 (java.util.concurrent.Executors/newFixedThreadPool 2))
#'user/pool1
@(let [^Runnable f (fn [] 1)]
   (.submit ^java.util.concurrent.ExecutorService pool1 f))
nil
@(let [^Callable f (fn [] 1)]
   (.submit ^java.util.concurrent.ExecutorService pool1 f))
1
user=> 

hiredman21:11:12

(if I explicitly type hint I can get the Runnable version)

mping21:11:43

thanks all, it seems type hints fixed it

qqq22:11:10

when we base64 encode a string, how much does the size increase by ? does it mean each byte is now only storing 6 bits instead of 8, so a 3MB file -> 4 MB ?

ghadi22:11:07

yes, but probably a topic for #off-topic

bcbradley22:11:32

does anyone know of a good resources to learn about clojure zippers, preferably excersizes I can try to solve?

bcbradley22:11:53

I've read through the docs, and read an overview and a few tutorials

bcbradley22:11:11

but i really want to try to solve some problems with them, but its hard to create your own problems

johanatan23:11:41

does anyone know if :ret and :fn aspects of fdef on macros are no longer being checked? The docs sort of discourage checking :ret (although they do not explicitly forbid it) and I'm seeing in practice that they are not being checked and just wondered if the "discouragement" became a "mandate enforced by code itself"

johanatan23:11:27

apart from the docs being updated to reflect such

Alex Miller (Clojure team)23:11:06

They are only checked during stest/check, not during instrument

johanatan23:11:29

@alexmiller so what aspects are checked during instrument? just :args?

Alex Miller (Clojure team)23:11:54

Yes - the purpose of instrument is to check for correct invocation

johanatan23:11:04

got it. thanks

Alex Miller (Clojure team)23:11:21

Check is for checking whether a function does what it says