Fork me on GitHub
#clojure
<
2017-10-02
>
razum2um09:10:34

hi, I read another http server comparison on twitter https://twitter.com/playpausenstop/status/914218443443929089 and decided to reproduce and add clojure to it: https://github.com/razum2um/hello-http-bench and wrk said clj was even faster. I suspect something is wrong, there shouldn’t be such difference. What am I missing?

rauh09:10:49

@razum2um The python code has a router it seems. Which ruse and clj don't have

razum2um10:10:29

@U051SA920 ok, but to tell the truth, I’m not a python-guy, took it straight from twitter, can you PR?

rauh10:10:23

Sorry no, I don't think such trivial benchmarks are meaningful. The big techempower benchmark tries to benchmark more real world usage patterns.

razum2um10:10:37

@U051SA920 but still, what about such big difference even with rust?

razum2um13:10:05

who’s here ownes oss_clj in twitter?! why tweeting before getting understanding?

wilcov13:10:52

well.. the maintainer is mentioned in the twitter account description. But it seems to be using a daemon that tracks twitter mentions with http://github.com in it, so it seems to be an automated tweet

ertugrulcetin14:10:49

Guys I created Clojurecademy I hope you like it, feel free to provide feedback

cch116:10:12

I’ve been programming in Clojure for years, but this one stumped me for a while: Without giving away too much of the solution, why do (pf) and (f) return different ranges?

cch116:10:20

(let [csrng (java.security.SecureRandom.)]
  (defn pf
    ([] (pf csrng))
    ([rng] (java.lang.Math/abs (.nextLong rng))))

  (defn f
    [] (java.lang.Math/abs (.nextLong csrng))))

cch116:10:05

To confuse the issue, try taking out the call to Math/abs.

bteuber16:10:52

pretty sure you mean

(pf csrng)
there - and I have no clue what's going on ^^

bronsa16:10:36

you're closing over the same random generator so getting a new random value every time, independently of which function you invoke

dpsutton16:10:07

but the question is about why they return different ranges?

bronsa16:10:17

i don't understand the question then

dpsutton16:10:39

@cch1 can you explain a bit more what you mean?

dpsutton16:10:57

are pf and f returning values that are in different ranges?

bfabry16:10:07

ok that is pretty weird

bfabry16:10:17

boot.user=> (pf)
545391547
boot.user=> (pf)
1976462804
boot.user=> (pf)
1535096817
boot.user=> (pf)
825017953
boot.user=> (pf)
596938158
boot.user=> (f)
7871239576498841695
boot.user=> (f)
8955889148814316217
boot.user=> (f)
8975201899528191775
boot.user=> (f)
3340060048302527866
boot.user=> (pf)
768712039

reborg16:10:59

It has to do with some type information no propagated properly during the function call. If you type hint they return results of the same magnitude:

(let [csrng (java.security.SecureRandom.)]
  (defn pf
    ([] (pf csrng))
    ([^java.security.SecureRandom rng] (java.lang.Math/abs (.nextLong rng))))

  (defn f
    [] (java.lang.Math/abs (.nextLong csrng))))

bronsa16:10:21

ah, I guess Math/abs reflective call is picking the abs(Integer) arity rather than abs(Long)

cch116:10:51

@reborg: correct. But I find it bizarre that in the polymorphic function reflection fails, but not in the “normal” function.

cch116:10:06

Thanks, @bthouret for catching my typo.

bronsa16:10:46

reflection is not failing anywhere

bronsa16:10:54

refleciton is being used in pf and not used in f

bronsa16:10:14

because f knows the type of csrng, because it is statically determinable, while rng can be any Object

cch116:10:53

@bronsa Yes, failure is too strong of a word. But the inability to determine the type, and the unfortunate guess of integer leads to very surprising results. In this case, the range of a crypto function suddenly loses 32 bits of randomness…

cch116:10:16

…silently, unless you have *warn-on-reflection* set, which is how I debugged this.

moogey16:10:13

I have this for getting fields from a pdf document

(defn get-fields [document]
  (->> document
       (.getDocumentCatalog)
       (.getAcroForm)
       (.getFields)))
If I wanted to be able to pass in any of those things to get the fields, what would be the clojure way to do that?

bronsa16:10:49

@cch1 well guessing the type as anything but object would be a mistake - - that arty is public so you'd be free to invoke it with anything exposing a nextLong method

bronsa16:10:02

even a class that returned an integer from it :)

bfabry17:10:12

isn't reflection a runtime decision? doesn't it already have access to the object returned by nextLong at that point?

ccann17:10:56

tossing this out there because I don’t understand: why does first work on a set? what utility does that have?

bfabry17:10:19

it lets you get the object out of a set that you know to be of length 1

bfabry17:10:01

though I imagine the reason it works is just because the types fall out that way. but that's one use for it

reborg17:10:28

@bronsa but isn't the Reflector here making a dubious choice? It looks like Math/abs int is taking precedence over the long version, probably because both are "congruent". Shouldn't be the conservative choice to go long?

ccann17:10:09

also I suppose the seq abstraction uses first and rest to iterate over seqs so it is used there, you would just never use it on set a set with more than one element and expect to care about which element you get back

bronsa17:10:37

@reborg there's no good pick in this case, you're assuming that the Integer version will behave the same as the Long version but there's no guarantee

bronsa17:10:44

so clojure just picks a random one

bronsa17:10:03

with a different clojure/Java versions you might get the long arity

razum2um17:10:46

How reading through jetty-server source and thinking about how much mutable is all the javax/servlet api. I think about http, sockets, and view templates: given we have immutable structures and (to tell the truth) a lot parts of a http response doesnt change between requests - cannot we leverage the fact and do less allocations flushing into the socket same and same strings(bytes) again (+ probably even doing a writev)?

chalcidfly18:10:14

So I have some code in a -main class that works in the REPL but not with lein run. Is that normal?

noisesmith18:10:09

@chalcidfly does your -main return a lazy-seq where the repl forces evaluation? (eg. a call to for or map)

chalcidfly18:10:47

Oh, does for return a lazy seq?

chalcidfly18:10:58

I switched to doseq and it works. Huh.

noisesmith18:10:07

it’s a list comprehension for generating data, and yeah, doseq is a drop-in replacement

chalcidfly18:10:15

Thanks again

chalcidfly18:10:22

I think that’s the third time this week

noisesmith18:10:55

at this point, the main thing I have to offer is recognizing the most frequent problems people have when first starting to use clojure, it’s a pretty predictable list, I should probably make a faq for clojure learners or something

seancorfield18:10:31

And lazy seqs would be the 1st, 2nd, and 3rd items on it 🙂

chalcidfly18:10:34

Yeah that would be seriously helpful

chalcidfly18:10:00

Official clojurians FAQ would be nice

chalcidfly18:10:04

Or unofficial

noisesmith18:10:38

> 1) lazy-seqs > 2) lazy-seqs > 3) lazy-seqs > 4) data structures are immutable

seancorfield18:10:39

A good rule of thumb to bear in mind is that doseq is for side-effects, for/`map` etc are for pure functions.

seancorfield18:10:53

(it's a bit more involved than that but...)

chalcidfly18:10:08

Yep, 4clojure put me in the 100% functional mindspace, but I’m working on something now with a bunch of side effects

chalcidfly21:10:53

Now I’ve got something that works with lein run but not as a standalone jar.

chalcidfly22:10:41

This is the code for anyone who might be able to help: https://pastebin.com/vRjvKZNc

chalcidfly22:10:53

Has to do with setting ns

seancorfield22:10:40

I think you'll need create-ns / in-ns instead of that nested ns call.

chalcidfly22:10:40

in-ns causes the same problem

seancorfield22:10:37

Hmm, I didn't realize in-ns was also tied to the static namespace management of compilation.

seancorfield22:10:49

@chalcidfly Looking at the docs for in-ns, yeah, you're not going to be able to use that either. I think you're going to find that you can't switch namespaces in compiled code.

seancorfield22:10:06

If you stopped using AOT, you could probably do this (maybe)?

chalcidfly22:10:06

Interesting, I’ll try that! Thanks for your help!

seancorfield22:10:10

Set :main to clojure.main and remove any :aot and gen-class stuff.

seancorfield22:10:38

Then build your uberjar and run it with the -m option to specify your -main namespace and see what happens.

chalcidfly22:10:55

Yep, that did it

chalcidfly22:10:14

@seancorfield Is there any I can package it up as a standalone jar without having to do the -m option?

seancorfield22:10:24

What you could do is write your own -main in a standalone namespace that only refers to other namespaces dynamically (`require` / resolve) and have it be the only AOT-compiled piece, and then it can dynamically call your "real" main function.

seancorfield22:10:50

You need to ensure that AOT doesn't cascade into the code you still need to be dynamically compiled (at runtime).

seancorfield22:10:24

Clojure's AOT makes this a bit of a pain 😐

chalcidfly22:10:10

Ah, I see. I don’t know if that will be possible here. I tried simply putting the extra ns call in another namespace but I think the AOT somehow spilled over and compiled it as well.

seancorfield23:10:37

You can't have any static references from the namespace containing -main to the rest of your code.

seancorfield23:10:37

So it would be a small, standalone namespace that only contains your (new) -main, and that function only contains a require of your "real" namespace, and then a call via resolve of your driver function.

seancorfield23:10:26

(ns compiled.main (gen-class))
(defn -main [& args]
  (require 'real.main)
  (apply (resolve 'real.main/-main) args))
something like that

seancorfield23:10:45

And make sure only compiled.main is AOT'd.

chalcidfly23:10:08

Oh, that’s genius, let me try that real quick

chalcidfly23:10:20

I still get that pesky Can't change/establish root binding of: *ns* with set

chalcidfly23:10:25

FWIW in real.main/-main I still have a ns call inside a function

chalcidfly23:10:02

Well, turns out if you eval the second ns call inside of clojure.main/with-bindings, it works just fine!

bwstearns23:10:09

Does anyone have experience using the clojure.data.zip and clojure.data.zip.xml member functions together?

bwstearns23:10:18

I’m running into an issue where I’m always getting one element back when I’m expecting 14 elements and I’m having trouble inspecting it. I went to try to use clojure.data.zip/children but I got back ClassCastException clojure.lang.LazySeq cannot be cast to clojure.lang.IFn clojure.zip/node (zip.clj:67)

bwstearns23:10:56

Which presumably means that the object I’m getting back from the zip-xml/xml-> function isn’t a viable input to zip/children

bwstearns23:10:42

Do I need to re-parse it into something?

bwstearns23:10:12

ah. Semi-solved. it was a seq of locs