Fork me on GitHub
#clojure
<
2017-08-25
>
didibus03:08:22

Question: Do clojure.test tests run in arbitrary order and in parallel?

noisesmith03:08:34

the latter could easily be yes if people stopped using with-redefs actually (and it isn't hard to stop using with-redefs and make a new runner that goes in parallel)

noisesmith03:08:46

I guess the output as you go would be tricky maybe...

didibus04:08:38

I thought with-redef was thread safe? Or is it just that it restores the original def when leaving its scope?

didibus04:08:06

Also, how would you avoid with-redef ?

noisesmith04:08:40

with-redef effects all threads, and you can avoid it by making code that isolates pushes side effects to the edges and takes other parts of the app as an explicit argument so that "mocking" simply consists of passing a different function in

noisesmith04:08:56

eg. component makes it very easy to write your system this way

qqq05:08:13

I like lisp () for everything except for math notation. Is there a nice way to embed a math DSL in lisp notation, or are the two mutually exclusive ?

seancorfield06:08:57

@qqq I've seen several examples of infix macros that accept "normal" math DSLs and convert them to prefix notation

seancorfield06:08:16

I thought there was one in Joy of Clojure but I just searched it and didn't find it.

seancorfield06:08:39

Clojure in Action 2nd Ed has one...

seancorfield06:08:02

...but it's overly simplistic.

danm08:08:26

Did you mean the one in Clojure for the Brave and True? https://www.braveclojure.com/writing-macros/

danm08:08:32

That has a simple infix example

mikeb08:08:02

@qqq it might be an interesting project to see if you could create a math dsl using tagged literals. https://stackoverflow.com/questions/20677055/define-my-own-reader-macro-in-clojure

danm08:08:48

Maths in clojure/lisp is just reverse polish notation isn't it?

danm08:08:09

Took me a bit to get my head round it, but once I did I find it clearer and easier to work with than the normal notation

qqq09:08:11

hmm, so the idea is that as long as my 'math dsl' also happens to be a syntacticaly valid sexp, I can just convert it from infix to prefix via . a macro? the limitation to this approacy is that instead of writing x+y I would have to write x + y right? basically sprinkle in many spaces to make it all separate into the right symbols

sundarj10:08:39

something like this works: (map (comp symbol str) (interpose " " (name 'x+y)))

sundarj10:08:58

i guess using a string would be better than a symbol

sundarj10:08:09

oh right, it works without the interpose too

kwladyka10:08:09

What is your experience about profiling Clojure app on production. I mean solutions like https://sentry.io for logs but to answer how much memory, CPU etc. it use. I was very excited about https://stackify.com/retrace/ because of good marketing articles, but on the end in my experience when i sent to it logs and got “success” no logs appear in panel. I was trying and trying… so in my opinion this tool is not enough mature at that moment. Can you recommend something?

kwladyka10:08:35

*i need something free/cheap for small hobby project and the best if i don’t have to host it myself, but on the end i can 🙂

delaguardo11:08:02

@kwladyka https://newrelic.com/java try this one, really good like a monitoring service and also they can do stress tests for java-based applications

kwladyka11:08:11

thx, do you recommend other alternatives to compare?

delaguardo11:08:09

I’m using only newrelic and it is more than enough for me) if i remember something i will update this thread

kwladyka11:08:22

hmm ok new relic looks really good, thx

delaguardo11:08:58

How can I forget) There is another service - https://www.datadoghq.com/

kwladyka15:08:51

i am trying to understand why datadog has integration with new relic. Why? I want to understand which tool is for me, but this make me even more confuse.

kwladyka15:08:53

now i use http://sentry.io for logs and alerts, today i started search solution to know how much memory and CPU app needs and soon i will do metric about how long operations take

delaguardo16:08:26

datadog is more generic service for collecting and visualizing data from different places. One of this places can be NewRelic. It makes migration from NewRelic to Datadog much more simpler. Both of them can cover use cases that you just described but requires different integration strategies in you code. But as long as you are using http://sentry.io (which has an integration with datadog) I can recommend to check datadog first. For your case you should install, configure and start datadog agent on your servers (information about CPU, memory, IO and network traffic will be collected without any additional steps). For the second part you should check for the library that can send events to datadog servers directly from your code and follow the guides about integration on code level.

kwladyka17:08:21

thank you, BTW i am going to use docker swarm (first time) with it

kwladyka17:08:43

all this infrastructure things are some kind of new experience for me

sandbags12:08:12

If I wanted a version of postwalk that treats maps as a discrete element rather than a set of key-value pairs is there any better way than creating my own version of walk with a clause like (map? form) (outer form)

sandbags12:08:43

(my use case is labelling nested vectors a la (but not actually) hiccup markup)

sandbags12:08:52

that seems to work fine and, other than the minor duplication & variation of a couple of library functions, not be a lot of hassle

sandbags12:08:05

if i'm missing a trick though, that would be worth knowing

juhoteperi12:08:39

@sandbags postwalk calls the function for the maps also, so just check inside the fn if the value is a map

sandbags14:08:23

@juhoteperi my experience is that it uses map to walk the map and so i get a call to the function for each keypair, as a vector, which is indistinguishable from any other vector

juhoteperi14:08:34

@sandbags It calls the fn with both

juhoteperi14:08:45

Check (clojure.walk/postwalk (fn [v] (println v) v) {:a 1 :b 2})

sandbags14:08:15

Right but i don't want it calling the function with vectors of keypairs from inside maps

juhoteperi14:08:22

Oh right. Yes, kw pairs are indistunguishable from vectors.

sandbags14:08:47

i could perhaps of been clearer about that

dominicm14:08:09

Bit confused by some behaviour I'm seeing for sorted-map-by with maps w/ namedspaced keywords

bronsa14:08:43

namespaced keywords are sorted lexicographically on their namespaces, then on the names

bronsa14:08:23

same as if you were sorting tuples of ns, name

dominicm14:08:39

@bronsa In 1.9 alphas, there seems to be a bug:

boot.user=> (sorted-map-by (fn [k1 k2] (doto (compare k1 k2) (prn k1 k2))) :db/ident :foo :db/valueType :db.type/string :db/cardinality :db.cardinality/one :db/index true :db/unique :db.unique/value)
13 :db/valueType :db/ident
-6 :db/cardinality :db/ident
10 :db/index :db/ident
-13 :db/index :db/valueType
12 :db/unique :db/index
-1 :db/unique :db/valueType
6 :ident :cardinality
6 :index :cardinality
10 :index :ident
12 :unique :ident
12 :unique :index
13 :valueType :ident
13 :valueType :index
1 :valueType :unique
#:db{:cardinality :db.cardinality/one, :ident :foo, :index true, :unique :db.unique/value, :valueType :db.type/string}

bronsa14:08:01

that looks sorted correctly to me

dominicm14:08:31

@bronsa The function starts receiving non-namespaced keywords to sort with.

dominicm14:08:12

This behaviour is different from 1.8:

boot.user=> (sorted-map-by (fn [k1 k2] (doto (compare k1 k2) (prn k1 k2))) :db/ident :foo :db/valueType :db.type/string :db/cardinality :db.cardinality/one :db/index true :db/unique :db.unique/value)
13 :db/valueType :db/ident
-6 :db/cardinality :db/ident
10 :db/index :db/ident
-13 :db/index :db/valueType
12 :db/unique :db/index
-1 :db/unique :db/valueType
{:db/cardinality :db.cardinality/one, :db/ident :foo, :db/index true, :db/unique :db.unique/value, :db/valueType :db.type/string}

bronsa14:08:14

@dominicm nah, that's just the repl printing

bronsa14:08:31

user=> (def a (sorted-map-by (fn [x y] (println x y) (compare x y)) :a/b 1 :a/c 2 :a/d 3))
:a/c :a/b
:a/d :a/b
:a/d :a/c
#'user/a
user=> a
:c :b
:d :b
:d :c
#:a{:b 1, :c 2, :d 3}

dominicm14:08:07

@bronsa Took me a minute to get my head round what you were saying there. I get it now, thanks!

alice15:08:42

Anyone have any experience using clojure to stream video files w/ http? preferably with ranges and stuff taken into account as to not force the browser to download the whole video

nathanmarz15:08:05

@sandbags you can walk data structures with more precise semantics and performance much greater than postwalk with specter. Here's a path that replicates postwalk:

(def walker
  (recursive-path [afn] p
    (cond-path (pred afn) STAY
               coll? [ALL p]
               )))

nathanmarz15:08:34

and here's one which for maps only walks the values:

(def walker2
  (recursive-path [afn] p
    (cond-path (pred afn) STAY
               map? [MAP-VALS p]
               coll? [ALL p]
               )))

nathanmarz15:08:05

to do something like increment all numbers, it would just be (transform (walker2 number?) inc data)

nathanmarz15:08:19

to get a vector of all numbers, it's (select (walker2 number?) data)

jeff.terrell16:08:25

@alice - Haven't done anything with Clojure for that, but I know a lot of HTTP streaming (including Netflix) is done with the "DASH" protocol. https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP

alice16:08:59

Thanks for the tip! I'll get digging.

lepistane16:08:43

Hi guys! not clojure specific request but community one i'd appreciate it if you participated in vote https://www.reddit.com/r/Clojure/comments/6vxqro/lets_settle_this_vote_slack_vs_other_platforms/ So many complaints and useless posts i wanna do something about it and a lot of people want something to be done. Lets vote and see how we all stand on the topic and consider our options 🙂 thank you

hiredman17:08:29

I dunno, as much as I prefer irc, I find this sort of thing ridiculous. A vote presupposes some kind of body able to move with a single purpose. Which the "Clojure Community" is not. The clojure slack wasn't created because of a vote, it wasn't created by rich or anyone in clojure/core (as far as I know). It was created by some guy, and then people joined. In effect people have already voted.

hiredman17:08:59

of course, in variably these polls are put up by people who prefer whatever isn't being used at the moment

hiredman17:08:39

"oh, we aren't using the tool me and my friends prefer, so the will of the people is being flouted, we need a vote"

theeternalpulse17:08:23

having used slack day to day at work I find it incredibly frustrating, but that's mainly because I'm expected to rely on it despite the massive noise for actual pertinent information. Also no one uses threads at work -_-

theeternalpulse17:08:08

but for information gathering I think it's fine, though forums make it more visible to people that tend to answer more than ask

sandbags17:08:28

@nathanmarz thanks, yes, I had forgotten about specter even though I have played with it a little and think its a good thing.

sandbags17:08:12

i confess i don't follow your examples or how i'd translate to my use-case but probably i should go back and read the Specter docs again

alex-dixon17:08:18

Anyone care to offer advice on using Kafka? In particular with the Streams API

seancorfield17:08:51

A general reminder -- to @lepistane and anyone else who cares about which communication platform is used -- that discussion belongs in #community-development (and has been happening there, on and off, for close to two years).

nathanmarz17:08:21

@sandbags yea recursive paths are more advanced usage but easy once you learn the concepts

nathanmarz17:08:41

happy to walk you through it in #specter

borjarus17:08:55

When I try to call this in repl (.sleep (Thread/currentThread) 2000) O've got an error : IllegalArgumentException No matching method found: sleep for class java.lang.Thread clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:53) After read java api about Thread I was found that there are 2 sleep method to call First one with one long argument and second with long and int arguments. When I try to cast to long I get same effect:

borjarus17:08:16

(.sleep (Thread/currentThread) (long 2000))

mfikes17:08:59

(Thread/sleep 2000)

mfikes17:08:36

(It is a static method)

borjarus17:08:25

what should i do to call in clojure this one in java : Thread.currentThread().sleep(2000)

joshjones17:08:38

(Thread/sleep 2000)

eriktjacobsen17:08:08

How does Thread/sleep work differently than Thread.currentThread().sleep()? It looks like they both sleep on the current thread.

joshjones17:08:23

no need for currentThread, since javadoc for sleep says: Causes the currently executing thread to sleep

hiredman17:08:24

they are identical

joshjones17:08:27

said another way, it makes little sense for a thread other than the currently executing thread to sleep

hiredman17:08:38

the java compiler lets you call static methods on instances

the-kenny18:08:08

which is quite stupid in this case. otherThread.sleep(2000) would still cause the current thread to sleep.

borjarus18:08:02

i have this in book exemple .. and i thougt why i can't do this in clojure if i can in java (even it's a stupid idea to use it)

mfikes18:08:26

@borjarus Yeah, you have to act like the Java compiler, which statically knows the type of Thread.currentThread(). Another way to think about this in Java is that this won’t work:

Object t = Thread.currentThread();
t.sleep(1000);
while this is OK, IIRC:
((Thread) null).sleep(1000);
In other words, the Java compiler is just employing static type information, which you can do in Clojure via (Thread/sleep 1000)

cpmcdaniel18:08:29

Has anyone here used java.text.MessageFormat.format() in their Clojure apps instead of clojure.core/format, which calls String.format()?

rgm19:08:17

I'm doing a lot of physical-units structural-engineeringy stuff at the moment. Does anyone know of something for clj/cljs like Python's Pint (http://pint.readthedocs.io) to get some guardrails up around dimensional calculations?

rgm19:08:40

(looked at clojure-toolbox and frinj / minderbinder don't quite seem to be in the same space).

rgm19:08:53

best case it could work with Javelin and subsume a whole host of crazy brute-force Excel work.

bwstearns19:08:35

Calling the following code works when entered in the REPL but when compiled and required into the repl it fails. Any idea why?

(defn parse-file [file-name]
    (-> file-name io/resource io/file xml/parse zip/xml-zip))

hiredman19:08:26

you cannot io/file a resource

hiredman19:08:05

which is to say, io/file will, unfortunately, let you do it some times, but given the way resources work you should never expect it to work

bwstearns19:08:25

hm. ok. So what is the proper way to go about this?

hiredman19:08:42

io/input-stream

hiredman19:08:09

(with-open [x (io/input-stream (io/resource ...))] (-> x xml/parse ...))

hiredman19:08:22

well actually, it depends

hiredman19:08:34

you might be using io/resource when you shouldn't be too

hiredman19:08:07

a resource is something you load from the classpath, which essentially means some kind of static data you distribute with your code

bwstearns19:08:26

I thought io/resource was the correct thing to use because I am using :resource-paths ["resources"]

hiredman19:08:31

if you intend to deal with arbitrary files on the classpath, you can't use io/resource

hiredman19:08:37

yes, io/resource is correct for that

bwstearns19:08:10

ok cool. the parse-file call is really only for testing (usage will have xml passed to it as a string)

hiredman19:08:13

the reason it isn't working for you isn't the compiling, it is the jaring, once you are running out of a jar, a resource is just a stream of bits read from that jar, no file on the file system

hiredman19:08:31

xml/parse will explode if you pass it xml as a string

bwstearns19:08:33

ohhh. that makes sense

hiredman19:08:52

xml/parse will try to treat a string as a url to read xml from

bwstearns19:08:13

ah, yeah, not string, was it bytestring or something? (I had it working that way earlier but for testing purposes it was easier to have it read a file)

hiredman19:08:49

it could be all kinds of things

bwstearns19:08:16

cool. thanks a ton for pointing out the issue with file btw. was thoroughly confused at the gap between repl-entered behavior and ‘require’d behavior.

hiredman19:08:26

yeah, the whole resource thing presupposes some knowledge of jar files and classloaders, which clojure docs tend to studiously avoid talking about

bwstearns19:08:44

I’ve used clojure for some stuff before, but I haven’t used Java for anything so a lot of times I run into issues when the underlying java concepts get unearthed.

csm19:08:16

is it possible to get the running test name from within (use-fixtures :each ...)?

noisesmith20:08:10

you get the function itself, which likely has a name hiding in it

noisesmith20:08:32

(in the printed repr that is) - but it’s messy and it’s an implementation detail

hiredman20:08:34

the function you get can be just about anything, I wouldn't count on getting anything from it

hiredman20:08:40

in fact, I am pretty sure the function it is passed in some anonymous closure created inside clojure.test that actually calls test-var which eventually calls your test

csm20:08:42

what I figured (and yes, the passed-in function is anonymous)

hmaurer22:08:28

Hi! Question: what is the easiest way to write a clojure library (with lein) that I can re-use in other projects? I read some stuff about creating a private Maven repo but surely there is an easier way?

seancorfield22:08:57

@hmaurer You can just do lein install and it'll build the lib and put it in your local Maven cache on that machine. Is that sufficient?

seancorfield22:08:34

(i.e., do you only need the lib on your own machine?)

hmaurer22:08:58

@seancorfield mmh that might be enough for local dev, but for deployments I might need the CI server to have access to my lib

hmaurer22:08:15

I could have it clone the lib and run lein install before running lein uberjar though

hmaurer22:08:43

thank you!

seancorfield22:08:38

If you have a CI server (in-house?) then it's not a stretch to stand up Archiva or something similar for an in-house Maven repo and deploy to that. Or an S3 bucket if you have access to AWS (S3 Wagon is a reasonable deploy/repo option I've heard).

noisesmith22:08:28

s3-wagon-private would be perfect if it didn’t print stack traces when somebody queries for an artifact that isn’t in the repo

noisesmith22:08:46

it still behaves properly, but the stack trace printing is super annoying

seancorfield23:08:29

Submit a PR? 🙂

noisesmith07:08:11

oh I tried fixing it, and I couldn't figure out what was going on in that code at all

noisesmith07:08:35

it's weird dependency injected java inheritance stuff

hmaurer23:08:42

@seancorfield @noisesmith thanks; s3 wagon looks neat, I’ll try it out

hmaurer23:08:53

@seancorfield unrelated question: I seem to remember you use ProtoREPL?

seancorfield23:08:17

@hmaurer Yes, I do use #protorepl !

hmaurer23:08:35

Ah, there is a channel for it!

noisesmith23:08:49

my workaround for the s3-wagon-private stack trace issue is to override :repositories so that the s3 wagon is the last one checked, otherwise you get spammy stack trace printouts for every artifact that isn’t yours

hmaurer23:08:33

@noisesmith why are those stack traces such a big deal?

hmaurer23:08:56

oh right, makes sense

noisesmith23:08:35

It's like 30 lines of noise, looking like an error, for every individual dep. Many thousands of lines with a fresh m2, and no actual erroneous condition.