Fork me on GitHub

@zignd I haven't worked with Datomic, so I can't speak to that. But, in the past when I've encountered ClassNotFoundException, it's because I had my classpath set up incorrectly one way or another.


@zignd What build tool are you using? Have you declared all the appropriate dependencies as required by the documentation that you're following?


@mingp I think it's somehow related to my dependencies, some sort of conflict between versions, because I just created a new leiningen project and it worked there. I have also found some Stack Overflow questions and old conversations from the #datomic channel where another person were facing the same problem


It seems to be related to ring as I'm also using it in my project and the fact that it uses Jetty on a certain version...


@zignd I see. In that case, you may need to bump up some versions. Although, of course, that has other effects on the project as a whole.


@mingp I don't think I can, my dependencies are at latest versions


@zignd That is a bit unexpected, then. Sorry, but it doesn't match what I've seen, so I don't think I know how to help you, unfortunately.


@mingp I have an idea, I'm inspecting the dependency tree of a new leiningen project from which the connection works and then I will compare it to the dependency tree of my project with lein deps :tree


And, if it turns out a transitive dependency is resolving to a different version than what works, you can always explicitly pin it.


@mingp Oh, by the way no problem, I appreciate your attention :)


The problem in, for example, NodeJS land is that, sometimes, there does not exist a set of mutually compatible versions for some sets of dependencies. For what it's worth, Java land I feel like has been better about this on average.


I still have to learn how that thing you mentioned works


If there is a transitive dependency (not directly declared in your dependencies, but pulled in as a result of what was directly declared) that needs to be at a specific version, you can pin it (declare it explicitly, at the version needed, in your dependencies).


@mingp Oh, now it makes sense. Merging what you said with what I'm seeing in the comparison of both dependency trees. It seems that the ring/ring-jetty-adapter dependency in my project contains a transitive dependency as you said to org.eclipse.jetty/jetty-io which is also used by com.datomic/clj-client, but because it was already available for ring it was not downloaded for datomic.


Hopefully, you can find a version that makes everything happy.


Yup, I had to add a :exclusions modifier for org.eclipse.jetty/jetty-io and it worked fine. Thanks for the help @mingp!


You're welcome. Actually, thank you for explaining, because I learned something new there, also.


@qqq instead of aset in a doseq, use amap or areduce for performance


@noisesmith: in the full problem. I am modifying an existing float-array -- amap is out of the question; also, areduce only returns one item right?


an array is one item, but areduce will walk an array input faster than doseq will


have you tried using your same array as the ret in amap?


it should work


generalized seq-oriented code is not going to be as fast as array specific code for the kind of thing you are doing


@noisesmith: this is my fault, my general problem is: I have matrices A, B, C I have a function f I want each eleme of A[i, j] to be repalced with f(A[i-1, j], A[i, j-1], B[i, j], C[i, j]) I don't think areduce / amap is right for this I posted a much simpler questino (for which areduce/amap works fine) mainly to learn how to do 'numeric unboxing stuff' in clojure


honestly beyond some very simple levels of complexity, just write some java


if unboxing matters that much, it's easier


yeah; I'm seriously considering doing part clojure + part scala (this problem arises from dealing with a scala library in particular)


also a multi-dimensional array is an array of arrays right? so you just need nested array ops


the library I'm using has blas / vectorized tensor ops


OK - what's the actual type?


unfortunately, I'm doing something dynamic-programmikng ish, where A[i,j] dedpends on A[i-1, j] A[i, j-1]


"actual type?' <-- I don't get what you are asking


like what does (type A) return. I would assume [[F


so that you would use (aget A 0 0) to get the first item, etc.


Hey folks, I'm trying to understand how to wrap ServerSocketChannel with core.async... can someone point me towards the right direction?


for side-effecty looping, does clojure have something faster than

(doseq [i (range n)]
or is that the standard, optimal way to od it?


I’d expect loop/`recur` to be the fastest way available, as it should be closer to the metal than other options built on reduce etc… If performance matters so much measure it, it might be negligible in practice, and reduce/run! etc will probably be nicer.


I can't believe!is a real function -- somehow I've managed to never hear of it all these years


any compojure api users here? what's the url supposed to be if you want to send a query param set? specifying {id :- [Long] []} for "query?id=0&id=1=3" works fine but expecting a set, i.e. {id :- #{Long} #{}} gives me "{:id (not (set? ["0" "1" "3"]))}


the documentation on that is ridiculously thin or I just can't find the right place


if I'm doing tensor math, should I just accept that any pure clj+java solution is going to be about 10x slower than C ?


 (let [n (* 10 1000 1000) 
       z (float-array n)]
   (doseq [^long i (range 1 n)]
     (aset-float z i
                 (float i)))))

results in:
Evaluation count : 6 in 6 samples of 1 calls.
             Execution time mean : 514.302217 ms
    Execution time std-deviation : 1.078130 ms
   Execution time lower quantile : 512.321577 ms ( 2.5%)
   Execution time upper quantile : 515.160140 ms (97.5%)
                   Overhead used : 2.944295 ns

does this seem right 10M aset-float takes 500ms ==> each aset-float takes 50ns ? this seems a bit high, as I expect, on a GHz CPU, for each aset-float to take about 1ns


@qqq remember what I said about amap? on my machine that benches 405ms, and this benches 26 ms

(let [n (* 10 1000 1000)                                                        
        z (float-array n)                                                       
        indexes (range 1 n)]                                                    
     (amap z i r                                                                
       (float i))))


@qqq it does the same thing, 20 times faster, you rejected what I was saying without even trying it


notice that I don't need to use r or z - for your math where you need to look at other indexes of the array, r and z are there (plus i to do math on for index math) - it's much more general than you think


@noisesmith: I tried it just now; $&@($#* amap is fast


maybe you'll listen next time


not sure if it matters, but the two different snippets aren’t measuring the same thing


one is creating the float array in the benchmark, while the other is not


@smith.adriane it's a silly refactor I can undo 😄


yea, i don’t think it would change the results too much


I also benchmarked that refactor on the original, it makes it slower


or more likely ,the difference was less than epsilon haha


yeah, my quick bench says pulling the quick-bench to the outside again adds .2 ms to the run time


great oracle of @noisesmith, can you show me how to make this code faster too? (note that computing a[i] depends on a[i-1] which makes me unsure how to use amap)

 (let [n   (* 100 1000 1000)
       dst (float-array n)
       out (float-array n)]
   (aset out 0 (aget ^floats dst 0))
   (aset out 1 (aget ^floats dst 1))
   (aset out 2 (aget ^floats dst 2))
   (doseq [^long i (range 3 n)]
     (aset out i  

            (+ (aget ^floats dst (- i 0))
               (aget ^floats out (- i 2)))

            (+ (aget ^floats dst (- i 1))
               (aget ^floats out (- i 1)))

            (+ (aget ^floats dst (- i 2))
               (aget ^floats out (- i 0))))))))
my current bench output is:
Evaluation count : 6 in 6 samples of 1 calls.
             Execution time mean : 1.578544 sec
    Execution time std-deviation : 21.633538 ms
   Execution time lower quantile : 1.548456 sec ( 2.5%)
   Execution time upper quantile : 1.599705 sec (97.5%)
                   Overhead used : 2.944295 ns


@qqq like I said, the bindings i, r, z allow you to access index, original, output respectively - you can do math on i and do an array lookup on r or z


the weird part is starting on item index 3, you'll need a conditional clearly (or maybe just copy the input back to the first 4 indexes of the output, to skip the conditional on each iteration)


@noisesmith: amap just sped up my code by a factor of 10. I'm not sure what you could have done to get me to take amap seriously earlier, but I'm glad you kept repeating it.


i thought part of the problem was that it doesn’t modify the array in place


you can do that inside amap - if you want to


maybe areduce is better if that is what you are trying to do actually


nobody says the accumulated value maintained by areduce can't be the input


alternatively amap is only like 10 lines of code


(defmacro amap
  "Maps an expression across an array a, using an index named idx, and
  return value named ret, initialized to a clone of a, then setting 
  each element of ret to the evaluation of expr, returning the new 
  array ret."
  {:added "1.0"}
  [a idx ret expr]
  `(let [a# ~a
         ~ret (aclone a#)]
     (loop  [~idx 0]
       (if (< ~idx  (alength a#))
           (aset ~ret ~idx ~expr)
           (recur (unchecked-inc ~idx)))


it’d be pretty easy to make an amap-inplace that just doesn’t create a clone


@smith.adriane: why is it faster than doseq, is it the loop+unchecked-inv vs range ?


the main reason is aset-float


aset-float is slower than a regular aset ?


I think i mislead you there


I thought giving it more info (hey, this is a float) would make it faster


if you do (aset z i ^float i)


it runs faster


but not as fast as amap


(let [n (* 10 1000 1000) 
        z (float-array n)]
    (doseq [^long i (range 1 n)]
      (aset z i ^float i)))


> (macroexpand-1 '(doseq [^long i (range 1 n)]
                                       (aset ^floats z ^long i ^float i)))
 [seq_75583 (clojure.core/seq (range 1 n)) chunk_75584 nil count_75585 0 i_75586 0]
  (clojure.core/< i_75586 count_75585)
   [i (.nth chunk_75584 i_75586)]
   (do (aset z i i))
   (recur seq_75583 chunk_75584 count_75585 (clojure.core/unchecked-inc i_75586)))
   [seq_75583 (clojure.core/seq seq_75583)]
    (clojure.core/chunked-seq? seq_75583)
     [c__4685__auto__ (clojure.core/chunk-first seq_75583)]
      (clojure.core/chunk-rest seq_75583)
      (clojure.core/int (clojure.core/count c__4685__auto__))
      (clojure.core/int 0)))
     [i (clojure.core/first seq_75583)]
     (do (aset z i i))
     (recur (clojure.core/next seq_75583) nil 0 0))))))


interesting, aset-float -> aset changed runtime from "500ms" to "90ms"


but the doseq version is also doing a bunch of other stuff that makes it slower than the amap version as seen in the macroexpand


it does seem crazy that aset-float is slower than aset


but those are similar results to what i saw on my computer as well


@smith.adriane this is so insightful: aset-float -> aset = 500ms -> 90 ms doseq -> loop + unchecked-inc = 90ms -> 20ms which basically matches amap


now I'm happy


I feel like I've made every $&@(#$ possible numeri-clojure mistake there is to make :-)


i’m learning a lot from this too


@qqq nah there's lots of numeri-clojure mistakes to make, you're still making the one where you write clojure instead of java when it has to be fast for numerics for example


@noisesmith: now you're ignoring what I'm saying ... see "numeri-clojure" 🙂


@smith.adriane: showing the source of the amap macro was especially useful; as I actually need a 2d version


why wouldn't nested amap work for 2d?


because it returns an array of array, instead of a flat float-array


and what I meant is that if you care about your time, writing it java in the first place is more expedient, no matter how much we all prefer writing clojure


actually, most of the code is running the GPU using tensor libraries


why would anything give you a flat float array out of a 2d array? in java a 2d array is an array of arrays


there just happens to be some $@#($ ETL type work I need to do beforehand


i have jump to source setup and it’s really useful for clojure. either looking at source in clojure core or in libraries I consume


@noisesmith BufferedImage -> getRaster -> getData is an flat array


sounds like you could skip a bunch of trouble and use the flat array and do some simple indexing math instead of pretending it's 2d


and I hear you saying that the GPU is doing most of the work, but if you'd written this in java in the first place you would have saved a lot of time, and that's 100% of my experience when numeric performance is an issue in clojure


my workflow is write in clojure, if it's slow figure out the bottleneck, if the bottleneck is math related write a small java class for it


is float-array guarnateed to be all 0? does not mention it, but I ran (seq (float-array 20)) a bunch of times, and it's all 0 all the time


@noisesmith: I see, so float-array just creates a java float array, but then according to that defaults to 0


Regarding Datomic, I'd like to have my unit tests running in isolated environments just like what was demonstrated in this video, [Test-driven Development with Datomic](, but using Datomic's Client API (`datomic.client`). The author demonstrates a technique in which he creates an in memory database for every unit test in his suite, but it only seems to be possible to do using the Datomic API (`datomic.api`) because it has function create-database ( that provides some facilities for creating in memory databases whenever you want to. I've been thinking about a work around for his technique using the Client API and the only thing that comes to my mind is executing from within my test code a peer server and then stopping it after the test finishes its execution. Something like: bin/run -m datomic.peer-server -h localhost -p 8998 -a myaccesskey,mysecret -d hello,datomic: It sounds like an overkill solution for me and I believe it's a pretty common use case to execute your unit tests against a fresh database instance. Does anyone have an easier solution that I could use?


FWIW, this is pretty much exactly what we do for tests in our services that use the client API. It’s a little heavyweight, but not overly so. Running tests works fine with this procedure.


@U1WMPA45U Good to know I'm not entirely mad in doing so xD Thanks for the letting me know!


On the one hand, you can create a fixture that deletes the database and reloads the necessary test data between every test. I found this example on SO: