This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-11-06
Channels
- # bangalore-clj (16)
- # beginners (120)
- # boot (21)
- # cider (24)
- # clara (9)
- # cljs-experience (1)
- # cljsrn (1)
- # clojure (218)
- # clojure-dev (3)
- # clojure-italy (12)
- # clojure-losangeles (4)
- # clojure-norway (2)
- # clojure-russia (2)
- # clojure-spec (19)
- # clojure-uk (178)
- # clojurescript (52)
- # cursive (7)
- # data-science (55)
- # datomic (25)
- # defnpodcast (11)
- # emacs (5)
- # fulcro (27)
- # hoplon (2)
- # leiningen (14)
- # midje (9)
- # off-topic (132)
- # onyx (19)
- # other-languages (23)
- # portkey (2)
- # re-frame (31)
- # reagent (1)
- # ring-swagger (15)
- # shadow-cljs (58)
- # slack-help (13)
- # spacemacs (22)
- # sql (7)
- # test-check (13)
On the other hand, there’s the philosophy that you shouldn’t need to “unit test” with an external dependency like a database
@gonewest818 I don't think it would work for the Client API, it only connects to a peer server. And even though the Client API has similar methods for creating, deleting and listing databases (http://docs.datomic.com/clojure-client/index.html#datomic.client.admin), I'm not being able to make them work in memory databases. I guess I will have to make use of this dirty hack or I could use the older library, but wouldn't be this entertaining xD
When clojure tells me:
java.lang.IllegalArgumentException: More than one matching method found: add
clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: More than one matching method found: add, compiling:(
is there a way to respond: great, please list all the matching methods for me?In Cursive you can use Ctrl+B and check the method overloads. But it seems that you will have to use type hints
(ns snip.fail
(:require [primitive-math :as p]))
(let [a (float-array 20)
y (aget ^floats a 0)]
(loop [x (float 0.0)]
(p/+ x y)))
^--- please help me typehint this
(yes, the loop is important, you can't change the loop to a ledt)public static float floatCast(Object x) {
if (x instanceof Float) {
return ((Float)x).floatValue();
} else {
double n = ((Number)x).doubleValue();
if (n >= -3.4028234663852886E38D && n <= 3.4028234663852886E38D) {
return (float)n;
} else {
throw new IllegalArgumentException("Value out of range for float: " + x);
}
}
}
this also confuses me:
;; fails
(let [a (float-array 20)
y (aget ^floats a 0)]
(loop [x (float 0.0)]
(p/+ x y)))
;; works
(let [a (float-array 20)
y (aget ^floats a 0)
x (float 0.0)]
(p/+ x y))
Oh, you also have the option do use unchecked-float
, it's a bit less expensive:
public static float uncheckedFloatCast(Object x) {
return ((Number)x).floatValue();
}
;; works
(let [a (double-array 20)
y (aget ^double a 0)]
(loop [x (double 0.0)]
(p/+ x y)))
;; fails
(let [a (float-array 20)
y (aget ^float a 0)]
(loop [x (float 0.0)]
(p/+ x y)))
alright, I have an even simple minimal failure case:
;; works
(let [y (double 0.0)]
(loop [x (double 0.0)]
(p/+ x y)))
;; fails
(let [y (float 0.0)]
(loop [x (float 0.0)]
(p/+ x y)))
That's interesting, I wonder why loop
works like that
(loop [x (float 0.0)]
(type x))
=> java.lang.Double
Btw, @qqq that's what it's causing the impossibility to find the correct overload
(let [y (float 0.0)]
(loop [x (float 0.0)]
(println (type x) (type y))))
=> java.lang.Double java.lang.Float
I don't know how to file clojure bug reports, but I think your example above has to be a bug
I don't think it's a bug, it's seems to be related to the quotes and unquotes being performed inside the loop
macro. Check this out:
Regardless of the macro internals,
(loop [x (float 0.0)]
(type x))
=> java.lang.Double
should return Float instead of Double, IMHOI guess you can try other alternatives to the loop macro, or the unchecked-float
that seems to be a bit less expensive
@tbaldridge @alexmiller: When you get a chance, can you please let us know what Clojure's semi-official stance on
(loop [x (float 0.0)]
(type x))
=> java.lang.Double
is intended behaviour -- or if it should return java.lang.Float, and thus a bug ?You will be taken to the depths of the compiler internals, be careful mate. src/jvm/clojure/lang/Compiler.java#L6318
more precisely
yeah, it's kinda a bad sign when there are no docs, i.e.
boot.user> (clojure.repl/doc loop*)
nil
boot.user> (clojure.repl/doc loop)
-------------------------
clojure.core/loop
(loop [bindings*] exprs*)
correct me if I"m wrong, the problem here is the loop macro is smart, so at compile time, it's like "oh, I see (float 0.0)" let me evaluate this at macro expansion time "oh, we got 0.0" [next stage:] "oh, 0.0 is a double"
actually, my theory is wrong:
(macroexpand
'(loop [x (float 0.0)]
x))
;; ==> (loop* [x (float 0.0)] x)
I don't think you will be able to expand everything inside this macro, loop*
is a symbol defined in the compiler's code. I don't even think there's documentation for it. It's something we should never have to look at
(macroexpand
'(loop [x (float 0.0)]
(type x)))
;; => (loop* [x (float 0.0)] (type x))
(loop [x (float 0.0)]
(type x))
;; => java.lang.Double
(loop* [x (float 0.0)] (type x))
;; => java.lang.Double
looking at that, it looks like everything is passed to loop* correctly (still as a float), but loop* [which is implemented in Java?] is doing the weird float -> double translationYup, the macro takes your bindings and body and send them to loop* so that it can perform the magic
Clojure only supports primitive long and double
Loops support a wider set of loop local primitive types
But once you leave the loop you’re back to long and double
@alexmiller: what does 'support' mean in 'Clojure only supports primitive long and double.'
We have float-array
, aget
, float
-- this seems to contradict only supporting double.
Some of that is vestigial from an earlier time when a broader set were supported. Some is for limited interop
Only float and double are supported as primitive args/returns in functions
Sorry, only long and double
Only float and double are supported as primitive args/returns in functions
<-- Can you please explain what happens when I call a java function of type sig
float java_add(float x, float y)
and call it from clojure as (java_add (float 2.0) (float 3.0))
in particular, what is returned, a float or a double ?
Sorry, I’m signing off for the night
Too long, catch you tomorrow
Considering what alex said, you should try to use doubles instead of floats, they will fit your float values anyway. At least for these operations involving the loop macro.
Regarding Datomic, I've been struggling with this for an hour or two, please help me out. I'm trying to run a parameterized query, but apparently I'm doing something wrong, does anyone knows why is the first query not returning the entity? I tried to follow the documentation, but it doesn't seems to be working... ;-; I'm also using the Datomic's Client API (http://docs.datomic.com/clojure-client/index.html)
Just pass the value to bind to ?p-username
in :args
, e.g. just pass [db "zignd"]
as :args
I have tried .size, .length, .-length how do I get, in O(1) time, the length of a (float-array n) ?
@qqq acount I think?
https://clojuredocs.org/search?q=acount%20 doesn'ts eem ot exist
arrays are really strange on the JVM, they don't have methods, or fields
So in Java you have to call something to Array/length, and even that I think might be some sort of strange compiler intrinsic
https://stackoverflow.com/questions/8755812/array-length-in-java was throwing me off
(defn constrained-fn [f x]
{:pre [(pos? x)]
:post [(= % (* 2 x))]}
(f x))
:pre and :post assertionsdepending on what you mean by append, this might be helpful too https://github.com/technomancy/robert-hooke
(defmacro foo [x y]
`(let [^floats ~x ~y]
~x))
(macroexpand-1
'(foo x (float-array 20)))
;; ==> (clojure.core/let [x (float-array 20)] x)
It appears my type hint of ^floats is being stripped away.
Is there a way for macros to generate / keep type hints ?found answer: https://stackoverflow.com/questions/11919602/generating-clojure-code-with-type-hints
Is there a more updated view of definline ? http://bytopia.org/2014/07/07/inline-functions-in-clojure/
Probably not, but is there something in key destructuring that lets you rename the keys in one go as you destructure them?
@U09LZR36F I mean something like this:
(let [{:keys [x y]} {:x 1 :y 2}
original-x x
original-y y
{:keys [x y]} {:x 2 :y 3}
new-x x
new-y y]
{:x (+ original-x new-x)
:y (+ original-y new-y)})
where I want to assign the destructured x
directly to the intended name(let [{original-x :x original-y :y} {:x 1 :y 2}
{new-x :x new-y :y} {:x 2 :y 3}]
{:x (+ original-x new-x)
:y (+ original-y new-y)})
Is anyone here using the s3-private-wagon plugin with Leiningen 2.7.1 on Java 1.8? I’m getting authentication errors when using the plugin and running lein deps
; however, when I run the aws
cli tool, I’m able to retrieve objects from the private bucket.
jenkins@ip-10-110-61-60:~/workspace/movr3-api-dev$ lein deps
Could not transfer artifact wardrobe:wardrobe:pom:0.1.3 from/to releases ( ): Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: AA5EF73EDF617F8E)
This could be due to a typo in :dependencies or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
jenkins@ip-10-110-61-60:~/workspace/movr3-api-dev$ cd ../../.m2/repository/wardrobe/wardrobe/0.1.3/
jenkins@ip-10-110-61-60:~/.m2/repository/wardrobe/wardrobe/0.1.3$ aws s3 sync .
download: to ./wardrobe-0.1.3.jar.sha1
download: to ./wardrobe-0.1.3.pom
download: to ./wardrobe-0.1.3.jar
download: to ./wardrobe-0.1.3.pom.sha1
download: to ./wardrobe-0.1.3.jar.md5
download: to ./wardrobe-0.1.3.pom.md5
jenkins@ip-10-110-61-60:~/.m2/repository/wardrobe/wardrobe/0.1.3$
https://github.com/s3-wagon-private/s3-wagon-private/issues/18 It seems this has already been solved as of version 1.3.0
You may want to try leiningen 2.8.1 @daniel.dyba, I know some attention went into updating s3-private-wagon
Not sure it handles your issue, but it does appear that related issues have been closed upstream
Could not find artifact wardrobe:wardrobe:jar:0.1.3 in central ( )
Could not find artifact wardrobe:wardrobe:jar:0.1.3 in clojars ( )
Could not transfer artifact wardrobe:wardrobe:jar:0.1.3 from/to releases ( ): Access key cannot be null.
Could not transfer artifact wardrobe:wardrobe:jar:0.1.3 from/to snapshots ( ): Access key cannot be null.
Could not transfer artifact wardrobe:wardrobe:pom:0.1.3 from/to releases ( ): Access key cannot be null.
This could be due to a typo in :dependencies, file system permissions, or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
I’m getting a similar error with the upgrade to Lein 2.8.1. I’m going to try Lein 2.8.0 next
Might want to ask in #aws @daniel.dyba -- usually for IAM instance profiles you don't have to do anything
(defmacro ifor [[i start end] & body]
`(let [start# (long ~start)
end# (long ~end)]
(loop [~i start#]
(when (p/< ~i end#)
~@body
(recur (unchecked-inc ~i))))))
(defmacro iproc [expr [data i v] & body]
`(let [~data ~expr]
(ifor [~i 0 (alength ~data)]
(let [~v (aget ^floats ~data ~i)]
~@body))))
(def x (float-array (* 1000 100)))
(cc/quick-bench
(ifor [i 0 (alength x)]
(aset ^floats x i (float i))))
Evaluation count : 3276 in 6 samples of 546 calls.
Execution time mean : 183.588777 µs
Execution time std-deviation : 923.219982 ns
Execution time lower quantile : 182.996929 µs ( 2.5%)
Execution time upper quantile : 185.164436 µs (97.5%)
Overhead used : 1.962587 ns
(cc/quick-bench
(let [^long end (alength x)]
(loop [i 0]
(when (p/< i end)
(aset ^floats x i (float i))
(recur (unchecked-inc i))))))
Evaluation count : 2412 in 6 samples of 402 calls.
Execution time mean : 248.133557 µs
Execution time std-deviation : 926.589005 ns
Execution time lower quantile : 247.272398 µs ( 2.5%)
Execution time upper quantile : 249.071233 µs (97.5%)
Overhead used : 1.962587 ns
(cc/quick-bench
(iproc x [x i v]
(aset ^floats x i (float i))))
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 1.121665 sec
Execution time std-deviation : 70.439931 ms
Execution time lower quantile : 1.067037 sec ( 2.5%)
Execution time upper quantile : 1.206947 sec (97.5%)
Overhead used : 1.962587 ns
i have a question about idiomatic application architecture — i’m writing a server application with a series of modules acting as various services; my thinking now is that i can wrap the important state to each module in an atom and then connect each module w/ core.async
channels to pass messages around as various events come into the top-level port
does anyone have any feedback on this plan? or a better suggestion for structuring the software? i’m used to the situation in Go where i can just connect all the modules w/ channels to pass messages around; what i’m used to doing in clojure for concurrency is using atoms and i’m trying to resolve the two approaches
@alexstokes I would caution against that architecture unless you specifically know you need concurrency already
@bronsa :
(defmacro ifor [[i start end] & body]
`(let [start# (long ~start)
end# (long ~end)]
(loop [~i start#]
(when (p/< ~i end#)
~@body
(recur (unchecked-inc ~i))))))
(defmacro iproc [expr [data i v] & body]
`(let [~(vary-meta data assoc :tag 'floats) ~expr]
(ifor [~i 0 (alength ~data)]
(let [~v (aget ~(vary-meta data assoc :tag 'floats) ~i)]
~@body))))
(do
(def x (float-array (* 1000 1000)))
(println "\n\n\n\n\n")
(println "=====")
(cc/quick-bench
(ifor [i 0 (alength x)]
(aset ^floats x i (float i))))
(cc/quick-bench
(iproc x [x i v]
(aset ^floats x i (float i)))))
=====
Evaluation count : 360 in 6 samples of 60 calls.
Execution time mean : 1.680640 ms
Execution time std-deviation : 3.250002 µs
Execution time lower quantile : 1.677290 ms ( 2.5%)
Execution time upper quantile : 1.685090 ms (97.5%)
Overhead used : 1.962587 ns
Evaluation count : 456 in 6 samples of 76 calls.
Execution time mean : 1.324811 ms
Execution time std-deviation : 255.773342 ns
Execution time lower quantile : 1.324424 ms ( 2.5%)
Execution time upper quantile : 1.324963 ms (97.5%)
Overhead used : 1.962587 ns
not complaining at all -- iproc is now faster than ifor, weirdok great @bfabry i’m working on a bitcoin node — messages from peers will end up going to one module and then that will have downstream effects to some other modules
i’m wondering what the best way to do this in clojure is
I don't know enough about erland/actor model to comment on that. you absolutely can set up an app where you have components that own little worker threads and the components talk to each other via channels. I've written apps like that. almost all of them i either have or wish I had gone back and rewritten them in a more functional way
to reinforce @bfabry’s point: I've found core.async hard to debug (both stack traces and reasoning about it) -- and found it's much simpler to take the FRP / reagent model /reactive atom route
turns out it's hard to predict which parts of your app will benefit from asynchronous programming 🙂
now when I use core.async I prefer to keep it in one namespace that wires everything together. gives you one place to read/understand your asynchronous algorithm and one place to vary the number of threads for the different parts etc
@alexstokes - That sounds to me like a use case that Storm is designed to solve. However, I'm speaking from relative ignorance here on both the application architecture and Storm itself, so take that with a grain of salt. http://www.storm-project.net/
thanks! seems a little like overkill
for me now
I may try putting all the state in a single atom
There will just be a little overhead for each module to only get the pieces it wants
It might also be overkill, but you could take a look at Onyx. It's written in clojure and will handle things like state for your modules and has a clear model for describing how the data flows between your tasks.
Thanks
Anyone use anything like lenses?
Zippers etc
look into Specter
Ah right
I think that will help if I go with one big ball of state approach
Each module can define a lens over the substate it cares about and a single atom can handle concurrent writes
Does anyone have any suggestions on writing maintainable accessors with specter?
Can I avoid having to re-write all my accessors if the shape of the state changes?
Agreed
This is p simple and I just want well factored code
If you just go with a single key per module, you probably won't need lenses. Although that's assuming each module has its own subsection rather than different modules needing state from different pieces.
@alexstokes specter is probably helpful here. but if you have lots of different functions mutating things then you're not writing very clojurey code
@bfabry can you point to an example that shows the difference?
Yeah I guess the issue is going from that to “here is a stream of events that impact various modules”
Yeah I think that is the single top-level atom idea I like
Then depending on what happens each module will need to read/write the state
right, but if you have a single top level atom I would manipulate it only at that top level, ie deref the atom and pass the contents down, not the atom itself. then receive the changes back up the top and modify the atom
Ok that’s a good suggestion
I’ll keep that in mind
Which invariably has me asking — what’s a good way to diff data structures ?
Is this map different from that map, etc
Great
so long as you're staying in the good world of immutable clojure data structures then comparing is just =
ya
hello every one, is there a better way to achieve this?
(apply str (map str (interpose “, ” [1 2 3 4 5])))
Unless diffing maps is specific to the task... I'd say it's probably bad practice in general. Think more about how the data flows through the system, the output of a pure function should depend on the input, generally not the input + diff against another piece of changing state.
@alexstokes - Agreed with the other comments about YAGNI, but in case you do need it: https://clojuredocs.org/clojure.data/diff
thanks!
I need to parse simple text like: "Important Field: {{important-field}}"
and replace {{important-field}}
with a value from a map
should I use something as heavy as instaparse, or what would be an idiomatic way in clojure?
Instaparse isn't well suited for "find/replace" use cases
It's more designed to parse a whole block of text data and interpret it as data
there are templating libraries that use that syntax already
and selmer, and antlers
Selmer would be my first choice (we use it very heavily at World Singles), but clostache is probably simpler/more lightweight for this use case(?).
or (clojure.string/replace my-str #"\{\{(.*?)\}\}" (fn ...))
thanks for the options, I'll weigh em out for my use case!
x = float-array of size 1000 y = float-array of size 2000 is there a builtin to say y[1000:1999] = x ? (I'm hoping for a java builtin faster than 1000 aset / agets)
@ghadi: https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#arraycopy(java.lang.Object,%20int,%20java.lang.Object,%20int,%20int) ?
(def n (* 1000 1000 1000))
(def x (float-array n))
(def y (float-array (+ 1000 n)))
(cc/quick-bench
(System/arraycopy x 0 y 999 n))
gives:
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 1.047620 sec
Execution time std-deviation : 98.981008 ms
Execution time lower quantile : 954.274457 ms ( 2.5%)
Execution time upper quantile : 1.137644 sec (97.5%)
Overhead used : 1.962587 ns
that is copying 4GB in 1.13 seconds -- that's insaneI bet that’s leaning heavily on OS level optimizations - might be interesting to benchmark on various platforms
I have a fn with (try ... (catch ) ... (finally))
that works ok and returns as expected
submit can take a Runnable or a Callable, and fn's implement both, so the reflector has to pick one
Runnable
doesn't return a value, Callable
does.
(well, in an imprecise way 🙂 )
Use a type hint.
(defn submit-job [^Callable fn] (.submit threadpool fn))
I think?
correct, as @seancorfield has it
if you (set! *warn-on-reflection* true)
you will see that the method call is still reflective
boot.user=> (defn submit-job ^java.util.concurrent.Future [^java.util.concurrent.Callable f] (.submit ^java.util.concurrent.ForkJoinPool fjpool f))
#'boot.user/submit-job
boot.user=> (deref (submit-job (constantly 1)))
1
(I'm using a ForkJoinPool for ease of creation but should work with a general executor service)
(switching the type hint to ^java.util.concurrent.ExecutorService
works -- I just tried it)
the reflected method chosen may actually vary by clojure release, with the beta of 1.9 even with reflection it chooses the callable version of submit
user=> (set! *warn-on-reflection* true)
true
user=> (def pool1 (java.util.concurrent.Executors/newFixedThreadPool 2))
#'user/pool1
@(let [^Runnable f (fn [] 1)]
(.submit ^java.util.concurrent.ExecutorService pool1 f))
nil
@(let [^Callable f (fn [] 1)]
(.submit ^java.util.concurrent.ExecutorService pool1 f))
1
user=>
when we base64 encode a string, how much does the size increase by ? does it mean each byte is now only storing 6 bits instead of 8, so a 3MB file -> 4 MB ?
does anyone know of a good resources to learn about clojure zippers, preferably excersizes I can try to solve?
but i really want to try to solve some problems with them, but its hard to create your own problems
does anyone know if :ret
and :fn
aspects of fdef
on macros are no longer being checked? The docs sort of discourage checking :ret
(although they do not explicitly forbid it) and I'm seeing in practice that they are not being checked and just wondered if the "discouragement" became a "mandate enforced by code itself"
They are only checked during stest/check, not during instrument
@alexmiller so what aspects are checked during instrument? just :args
?
Yes - the purpose of instrument is to check for correct invocation
Check is for checking whether a function does what it says