Fork me on GitHub
#clojure
<
2017-06-08
>
qqq00:06:46

is there a way to get ring/jetty to serve on https://localhost/ .... or does it make no sense since "localhost" is not a full domain name and thus it can't be https ?

weavejester00:06:07

The certificate wouldn’t match. You’d need to self-sign.

qqq00:06:23

okay, so it's doable then ?

qqq00:06:35

I self sign some cedrtificate, add it to my browser as an authority

qqq00:06:46

and then I can do https://localhost ?

delitescere00:06:00

That should do it.

melindazheng02:06:10

Calling all clojurians who are in SINGAPORE, please search #clojure-sg to join our Singapore clojure channel:grinning:😀

tap05:06:10

what’s is a good concise resource for someone outside community who wants to learn about clojure’s data-oriented programming style?

fbielejec07:06:38

@tap Brave Clojure http://www.braveclojure.com/ It's also amusing 🙂

qqq08:06:09

When writing a clj / cljs application, what is a good way to track (server side) "this user here is logged in with username foo" ?

qqq08:06:29

This must be a solved problem, and I absolutely do not want to roll my own custom possibly insecure solution.

danielgrosse08:06:20

When I add an atom from a different ns and set this to a local variable, why isn't the "remote" atom changed when I swap the content?

danielgrosse08:06:24

(ns test.db
  (def state (atom nil))
)
(ns test.core
  (:require [test.db :as db])

(def local-state db/state)

(reset! local-state 1)

@local-state ; => 1
@db/state ;=> nil
)

pesterhazy09:06:48

@danielgrosse the two vars are references to the same javascript object, so it should work as you expect

pesterhazy09:06:00

there's something wrong with how you test

pesterhazy09:06:29

why the var definition in the ns declaration? shouldn't it be test.db/state? etc

misha09:06:16

gentlemen, are transducers – a replacement of reducers?

dominicm09:06:14

misha: not quite

dominicm09:06:26

misha: reducers are for single values

dominicm09:06:54

Transducers are a faster way of operating on a sequence

dominicm09:06:42

Where a sequence can be a list or channel, etc.

misha09:06:07

@U09LZR36F can you give an example of a "single value" in this context?

dominicm10:06:16

A map, a sum, technically you could have another list, but it wouldn't be lazy

bronsa09:06:29

for most intents and purposes, yes. Altho there exist a non-empty difference between the feature sets those two provide

misha09:06:35

bronsa: is there a use case, where you would (and could) compose both? Or is it always "use one or the other" situation?

bronsa09:06:44

you can use non-stateful transducers using reducers' fold, but it's a bit tricky

bronsa09:06:58

transducers and reducers aren't designed to compose with each other

danielgrosse09:06:31

@pesterhazy I forgot one parenthesis in the example above. Also its in clojure not clojurescript, if this makes a difference.

pesterhazy09:06:59

Try it in the repl, it works as expected

danielgrosse10:06:44

@pesterhazy Hm strange. Will look into it, when I have time. currently I have to change the db/atom and the local var changes with it

danielgrosse10:06:41

Is there a macro to test a value on different predictions and only returning true if all succeed? Like some-> but without given the result to the next prediction?

danielgrosse11:06:00

@curlyfry I ended up using and:-)

danielgrosse11:06:29

Thought there was some other solution. Maybe and->

rubek12:06:34

and-> sounds like something you could homebrew with and and apply---but I’m not quite sure exacly how myself

misha12:06:05

@danielgrosse

(every? true? ((juxt keyword? qualified-keyword?) :foo/bar))
=> true
(every? true? ((juxt keyword? qualified-keyword?) :bar))
=> false

misha12:06:52

it is time to read clojure.core again

misha13:06:43

@rubek @rauh's every-pred is waaaay faster than juxt, and shorter, and build in too

rubek13:06:22

That is also really cool 🙂

lmergen13:06:17

if i have two sequences of the same length, a sequence of maps/objects, and another sequence of strings, and i want to assoc each string as the key :foo to each respective object, what function do i need ? i was looking at zipmap, but i’m not sure if that’s it

lmergen13:06:20

so basically, input ({:bar "1"}, {:bar "2"}) (3 4) -> ({:bar "1" :foo 3} {:bar "2" :foo 4})

lmergen13:06:20

i probably need to first transform (3 4) to ({:foo 3} {:foo 4}), and then use interleave ?

dpsutton13:06:03

(map #(assoc %1 :foo %s) col1 col2)

lmergen13:06:34

right 🙂

lmergen13:06:41

that makes sense!

lmergen13:06:50

and is a lot more readable :

dpsutton13:06:29

a for list comprehension could probably look really readable as well

lmergen13:06:21

you’re right

dpsutton13:06:34

although be aware that if you're sequences are not the same size you risk silently dropping data

dpsutton13:06:07

(map #(assoc %1 :foo %2) [{:a 1} {:b 2}] [:x])
({:a 1, :foo :x})

dpsutton13:06:12

if you have uneven ones you would lose the {:b 2} in this case. not sure which is more valuable but maybe wrapping the lesser important one with a (concat imporant (repeat :missing))

lmergen13:06:00

interesting behaviour of map

dpsutton13:06:39

yeah but understandable. you're giving it a two arity function and two sequences. when it runs out of one it cannot continue

dpsutton13:06:57

but yeah just something to remember

lmergen13:06:49

thanks for the info!

danielgrosse14:06:13

When I send an vector as string to the server, how can I transform it into an vector back?

bronsa14:06:29

read-string

bronsa14:06:45

altho if you're reading it from untrusted source use clojure.edn/read-string instead

captainlexington14:06:18

Ooh, I didn't know there was a secured version of read-string

bronsa14:06:18

so quite a few years :)

kah0ona14:06:21

Hi, is it possible to let an spec/fdef failure result log to, say, taoensso’s timbre?

kah0ona14:06:22

We are thinking about enabling instrumentation in our runtime (eventhough it’s not meant for that), occasionally, to hunt down some production bugs.

lxsameer15:06:15

hey guys, what do you think about pedestal ?

donaldball15:06:17

lxsameer: I’m using it. It’s fine.

lxsameer15:06:44

@U04V4HWQ4 did you have any problem with it ?

donaldball15:06:51

no, it does what it says on the tin. Server/servlet integration is easy, the interceptor chain construction is a little fiddly but much saner than an opaque stack of ring middleware.

lxsameer16:06:39

cool. Thanks man

sveri16:06:33

So I have a list of maps that contains a milliseconds field. What would be the best way to partition the list into sublists where each sublist contains all the maps for a given timespan, like one minute for instance? [{:t #inst"2017-06-08T15:46:39"} {:t #inst"2017-06-08T15:46:40"} {:t #inst"2017-06-08T15:46:41"} {:t #inst"2017-06-08T15:47:39"} {:t #inst"2017-06-08T15:48:39"}] would result in: [[{:t #inst"2017-06-08T15:46:39"} {:t #inst"2017-06-08T15:46:40"} {:t #inst"2017-06-08T15:46:41"}] [{:t #inst"2017-06-08T15:47:39"}] [{:t #inst"2017-06-08T15:48:39"}]]

trptcolin16:06:32

[is it pre-sorted? if not sort it, and then…] partition-by with clj-time’s within? and interval functions (look for within? on https://github.com/clj-time/clj-time

trptcolin16:06:01

hmm, actually i guess you don’t need within?, but clj-time is still good stuff, probably truncate/round the time to whatever increment you want instead

sveri16:06:24

Yea, my plan is to convert it to milliseconds,jeez and now I got the solution, of course I just have to cut the ms at the seconds part -.-

trptcolin16:06:26

this is getting to be a bit much for me w/ the point-free stuff but:

(require '[clj-time.core :as t]
            '[clj-time.coerce :as c]))

  (partition-by
    (comp 
      (juxt t/year t/month t/day t/hour t/minute)
      c/from-date
      :t)
    times)

sveri16:06:34

and then use partition-by

sveri16:06:06

or like that, thank you very much @trptcolin 🙂

martinklepsch16:06:22

Anyone an idea how to temporarily work around this issue? https://github.com/ptaoussanis/nippy/issues/98

reborg18:06:53

Also without clj-time dep: (partition-by (comp (juxt :year :month :day :hours :minutes) bean :t) t) or a shorter one (partition-by #(quot (.getTime (:t %)) 60000) times)

dealy18:06:59

is it possible to create a clojure library jar that can be used from Java, where the Java client is able to pass in an object.method for execution by the clojure function?

hiredman18:06:07

the specifics are going to depend on what you mean by object.method

hiredman18:06:03

in java methods aren't really first class values you can pass around, so you have to pick some representation of the method, of which there are a few possible

dealy18:06:13

I mean that if I have a class Foo { public int bar(String s) } and I want to pass in an instance of Foo and indicate that the bar function is to be called in the clojure world

hiredman18:06:51

one simple way to do that is to pass in both the instance of Foo and a java.lang.reflect.Method representing bar

dealy18:06:22

yea, I figured you might say reflection

hiredman18:06:31

I've also seen apis that pass strings of method names, in which case the clojure code would need to do the reflecting bits

hiredman18:06:48

you could also implement IFn in java and pass that

hiredman18:06:06

but that might be super painful

bfabry18:06:07

yeah I would create an anonymous implementation of IFn and pass that. going to be a lot faster

hiredman18:06:32

if I recall java requires you to implement all the methods in an interface, and IFn has a lot of methods

bfabry18:06:52

ah, yeah you're right. probably create a protocol that just accepts an arg array then

dealy18:06:59

would it be easier if the Java classes had to implement an Interface that my clojure lib was to expect?

hiredman18:06:42

well, I dunno

dealy18:06:25

I have to stay in Java 7

hiredman18:06:31

so the the java code constructs a Function and the clojure code gets a Function and can directly call the method to apply it

hiredman18:06:26

well, then you can create your own function interface, or use one from one of java libraries that provide that sort of thing, I think guava has one

dealy18:06:53

oh, I didn't know that about Guava, I'll check it out

hiredman18:06:29

if you aren't already using guava, functionaljava is a library specifically for this kind of thing

dpsutton18:06:47

non java guy. why is this more complicated than (.bar my-class)?

dpsutton18:06:06

with Foo { public int bar(String s) }

dealy18:06:32

yea we are already including guava I just looked at Guava's function, that may be Java 8 also, it looks like it expects a lambda

hiredman18:06:53

my understanding is he doesn't want to hardcode the method being called in the clojure code

dealy18:06:14

correct, the Java code needs to pass a method to the clojure code

hiredman18:06:23

he wants to hand the clojure code some kind of handle (a closure basically) that it can use to invoke a given method on a given object

dpsutton19:06:20

an interface of IMethodToCall and make each class you are going to pass implement this? and this is just a pass through?

dealy19:06:41

we have some crazy complicated threading and the associated problems in our Java code. I'm trying to help out by writing a clojure library that handles things in a more sane way

hoff19:06:09

hi, i'm playing around with some logical combinators. is there a library that can textually substitute function arguments into the body while observing shadowed variables?

tanzoniteblack20:06:20

hoff: If you use cider (emacs), you might try enabling the enlighten-mode (http://cider.readthedocs.io/en/latest/miscellaneous_features/#enlighten-display-local-values)

tanzoniteblack20:06:24

or just use the debugger

tanzoniteblack20:06:14

( enlighten-mode is a feature that was inspired by http://lighttable.com/ , which also does this quite well, if you’re not an emacs person )

hoff20:06:07

i was hoping for something more programmatic

hoff20:06:15

also i'm working with cursive

michaellindon20:06:28

Is there a verion of map that stores the output in a map? i.e. that would perform the following: (special-map f [a b]) returns {a (f a) b (f b)}

michaellindon20:06:57

(map vector coll (map f coll)) would work

noisesmith20:06:16

the usual idiom for that is (into {} (map (juxt identity f) coll))

noisesmith20:06:49

or (into {} (for [el coll] [el (f el)]))

bfabry20:06:52

specter is good for this. other than that there's about 4,127 utility libs that include a transform-keys and transform-vals variant

michaellindon20:06:04

sorry my suggestion would not work

noisesmith20:06:05

but the input isn’t a hash-map

bfabry20:06:36

(sp/transform sp/MAP-VALS f coll) imo is both pretty clean and generalisable

noisesmith20:06:49

but the input doesn’t have vals

fabrao20:06:56

Hello all,

(defmulti classifica
  (fn [vl-jitter vl-perda vl-latencia]
    (let [jitter 40
          perda 6
          latencia 150
      (if (and (= -1 vl-jitter) (= -1 vl-perda) (= -1 vl-latencia))
        [-1 -1 -1]
        [(> vl-jitter jitter) (> vl-perda perda) (> vl-latencia latencia)]))))

(defmethod classifica [-1 -1 -1] [_ _ _] :comunicacao)

(classifica -1 -1 -1) => ArityException Wrong number of args (3) passed to ... 
what I´m doing wrong?

bfabry20:06:57

oh I'm sorry

noisesmith20:06:57

@fabrao dispatch isn’t on the vector of args, it’s on the single value returned by the dispatch function

bfabry20:06:34

I would use noisesmith's original suggestion were I writing it

fabrao20:06:47

oh !!! sorry about my question so ... Thanks

hiredman20:06:01

well, that doesn't actually fix the issue

fabrao20:06:15

is there any way to do somthing like that?

hiredman20:06:16

because you wouldn't get that arity exception

hiredman20:06:38

my guess is your actual issue is you the defonce nature of defmulti

hiredman20:06:00

I would put a (def classifica nil) right above your defmulti, and you should be fine

hiredman20:06:55

for historical reasons, recompiling a defmulti form won't change an existing defmulti, so if you have defined a defmulti in a repl with a dispatch function that takes 1 arg, and realize that is a mistake and go to fix it by redefining the defmulti with the correct dispatch function, that won't fix it

fabrao20:06:11

@hiredman it worked, but WTF is that?

hiredman20:06:27

defmulti behaves similarly to defonce

hiredman20:06:51

putting the def before it makes it behave more like a regular def

fabrao20:06:13

So I can use multimple args in defmulti?

hiredman20:06:52

the dispatch function has to be able to handle however many args you pass, but each defmethod can handle just the args it needs to

fabrao20:06:37

=> (def classifica nil) is cleaning the definition for repl?

fabrao20:06:50

and defined it again?

hiredman20:06:30

it defs it as nil, so when the defmulti form is executed, it doesn't see an existing defmulti there

fabrao20:06:58

thanks for ur help

v3ga20:06:43

does anyone have any medium sized pedestal apps that run on top of boot?

donaldball21:06:24

decim: I have a pedestal app that is built with boot. I’m not sure what you mean by running on top of boot.

v3ga21:06:18

sorry built with boot is what i meant

v3ga21:06:09

is your app public on github by chance? I’m just looking for examples…i’ve gone through the few on http://pedestal.io but I’d like to see more fleshed out applications to get a grasp of things

donaldball22:06:57

Alas, it is not

hiredman20:06:24

if you dig back 7-8 years in the mailing list you will find some people complaining about defmulti's behavior back when it behaved more like def, so it was changed, and then shortly after that people started complaining about the defonce behavior, and it has acted like defonce ever since

nonrecursive21:06:56

hey y’all i’m trying to figure out something with clojure.spec/explain-data

nonrecursive21:06:07

Here’s what I’ve got:

(s/explain-data (s/map-of keyword? keyword?) {:x 0})
#:clojure.spec{:problems ({:path [1], :pred keyword?, :val 0, :via [], :in [:x 1]})}

nonrecursive21:06:24

what’s with the 1 in :in [:x 1]?

nonrecursive21:06:39

I can’t figure out why that would be part of the path

bfabry21:06:41

that's the index

nonrecursive21:06:30

I’m not following 😢 I’m not even sure how to ask for clarification

nonrecursive21:06:36

I’m thinking of :in as something you could pass to get-in

bfabry21:06:21

yeah... it almost seems like a bug? but I think at some point the keypair [:x 0] was checked against the predicates, and in that temporary tuple the number 0 is in position 1

noisesmith21:06:33

it’s element 1 of a map-entry composed of a key and a val

nonrecursive21:06:33

yeah that makes sense

bfabry21:06:52

^ that is a better explanation

nonrecursive21:06:13

aaaah ok so if the problem were with the keyword

nonrecursive21:06:22

then it’d probably be :in [:x 0]

nonrecursive21:06:41

ok there we are

(s/explain-data (s/map-of keyword? keyword?) {0 :x})
#:clojure.spec{:problems ({:path [0], :pred keyword?, :val 0, :via [], :in [0 0]})}

noisesmith21:06:12

interesting that it indexes the hash map as if it was ordered key/val tupls

noisesmith21:06:19

weird actually

bfabry21:06:49

it doesn't, it uses the key for that portion of the path I think (he made the key 0)

noisesmith21:06:12

ahh, it’s not get-in it’s find-in

nonrecursive21:06:14

ultimately I’d like be able to use explain-data and get-in to get some value from the value I’m trying to conform… is there some way to do that?

noisesmith21:06:17

(if such a thing existed)

bfabry21:06:24

the value is there already in :val though right?

nonrecursive21:06:37

I actually want the parent value

noisesmith21:06:03

haha, find-in is impossible because (find x 1) has a two element vector fixed point

noisesmith21:06:09

well, not impossible, but useless

noisesmith21:06:45

peregrine.circle=> (defn find-in [m [k & nks :as ks]] (if (empty? ks) m (find-in (find m k) nks)))
#'peregrine.circle/find-in
peregrine.circle=> (find-in {:a {{:b 0} 1}} [:a])
[:a {{:b 0} 1}]
peregrine.circle=> (find-in {:a {{:b 0} 1}} [:a 1])
[1 {{:b 0} 1}]
peregrine.circle=> (find-in {:a {{:b 0} 1}} [:a 1 1])
[1 {{:b 0} 1}]
peregrine.circle=> (find-in {:a {{:b 0} 1}} [:a 1 1 1])
[1 {{:b 0} 1}]

bfabry22:06:16

it looks to me like you could write a version of get-in that uses find for maps and get for other colls

noisesmith22:06:00

yes, that would resolve the issue

noisesmith22:06:16

for some reason I found the problem of using find recursively amusing

bfabry22:06:52

and then for @nonrecursive's use case, just call my-get-in on (butlast (:in explain-data))

nonrecursive22:06:33

I can imagine a function that would do the thing, but why would I use find?

noisesmith22:06:16

the output looked like find’s output

noisesmith22:06:22

perhaps I was mistaken

nonrecursive22:06:56

I didn’t realize that, that’s awesome

nonrecursive22:06:50

(defn get-in-spec-path
  [val spec-path]
  (if (empty? spec-path)
    val
    (get-in-spec-path
      ((if (map? val) find get) val (first spec-path))
      (rest spec-path))))

sophiago23:06:09

Can anyone tell me when a map switches from PersistentArrayMap to PersistentTreeMap in the case where it's heavily nested? I know normally it's >8 elements (which doesn't entirely make sense to me when switching to a 32-ary trie), but I'm working with a map that nests many levels and almost never has that many on the top one. It exceeds eight elements in the first level of nesting, but I wouldn't expect any implementation to be counting elements due to it possibly taking linear time. From playing around with it, I've found it switches when it reaches >8 levels of nesting, although I haven't tested to see if this is true with empty maps. Can anyone confirm and/or shed light on this? Thanks

hiredman23:06:29

the only checks are within the implementation of pam, and they only kick in if you assoc more than HASHTABLE_THRESHOLD, where HASHTABLE_THRESHOLD is 16. since the array map stores items in pairs 16 / 2 = 8

bronsa23:06:30

PAM never switches to PTM

bronsa23:06:36

they switch to PersistentHashMap

bronsa23:06:44

PTM is the impl backing sorted-map

sophiago23:06:31

@bronsa oops. Thanks for the correction.

joshjones23:06:17

looks like you can make it a PersistentTreeMap and keep it that way

joshjones23:06:19

user=> (into (new clojure.lang.PersistentTreeMap) {:a 1 :b 2})
{:a 1, :b 2}
user=> (type *1)
clojure.lang.PersistentTreeMap
user=> (assoc *2 :c 3)
{:a 1, :b 2, :c 3}
user=> (type *1)
clojure.lang.PersistentTreeMap

hiredman23:06:53

ptm never promotes to anything else

sophiago23:06:24

@hiredman with regards to your first comment, if that was the case I'd expect a top level map map holding three maps with >8 kv pairs to always be a PAM and that's not the behavior I'm seeing once the nested maps reach a certain level of depth

hiredman23:06:47

I would suggest you look harder, I guess?

noisesmith23:06:59

it has nothing to do with nested totals, it’s the number of elements at that one level

hiredman23:06:09

the code is pretty clear, that is what it does, if you are seeing something else, then you are doing something else

sophiago23:06:21

Ok, I'll look again

joshjones23:06:39

user=> (type {1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8})
clojure.lang.PersistentArrayMap
user=> (type {1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9})
clojure.lang.PersistentHashMap

sophiago23:06:47

I'm also using an int-map although I'm pretty sure it implements the same semantics regarding that

hiredman23:06:02

there is also nothing that says maps with fewer elements are always PAMs

hiredman23:06:20

the reader just creates pams by default

noisesmith23:06:25

what is an int-map?

hiredman23:06:38

yeah, so you are using a completely other kind of map

hiredman23:06:48

it will behave completely differently

sophiago23:06:32

I am pretty sure int-map behaves the same with this regard as the built-in hash-map

hiredman23:06:34

there is no such thing as a pam for int-map

noisesmith23:06:41

based on what evidence?

hiredman23:06:57

and the reader doesn't create int-maps by default like it does for pams

sophiago23:06:31

Hold on, I better pull up a repl

hiredman23:06:24

the only mechanism that switches from pams to phms at a threshold is built in to pams, and hard coded to result in a phm

hiredman23:06:01

if you don't start with a pam, or you don't end up with a phm, then you are looking at something entirely and completely unreleated

sophiago23:06:15

Ah, ok. So I was mistaken about PersistentIntMap. It stays the same (which makes me wonder whether it's okay to reuse the same transient with it?). But I do see the behavior I was mentioning wrt to regular hash-maps. I'm going to examine the latter a bit further to make sure I'm not missing something.

joshjones23:06:22

reuse the same transient?

noisesmith23:06:30

why does the hashmap / arraymap distinction matter here anyway?

hiredman23:06:46

my guess is you are running some other bit of code you are failing to mention which is resulting in maps being transformed

hiredman23:06:32

you can never reuse a transient

hiredman23:06:56

the contract says you cannot bash on them in place

sophiago23:06:02

I was under the assumption the reason it's advised to never reuse transients is because of the switching of coll types as they grow

hiredman23:06:14

it has nothing at all to do with that

noisesmith23:06:46

it accidentally coincides - because the problem hits when reallocs / resize happen inside the map coll impl

noisesmith23:06:02

which is one thing that will happen with the arraymap hashmap transition

noisesmith23:06:12

but the real problem is not using the return value of transient ops

sophiago23:06:02

@noisesmith thanks for clearing that up. so even inside the same coll type it can create problems? Or are you saying there are additional reasons on top of that?

hiredman23:06:09

I would suggest not using transients

joshjones23:06:32

@hiredman in this context, in general, ?

noisesmith23:06:34

yes - because every persistent type eventually needs to alloc more space, and that’s when bashing transients in place tends to stop accidentally seeming correct

sophiago23:06:57

Got it. Thanks!

noisesmith23:06:16

there might be other gotchas too, you avoid all of them when you use the return value like the docs tell you to

hiredman23:06:18

@joshjones in general, I would suggest not using something unless you can make a cogent argument for using it

joshjones23:06:35

i see, agree

lxsameer23:06:44

@tbaldridge hi , did you published any other video on Pedestal series ?

joshjones23:06:01

@hiredman what do you think about creating a custom data structure, and to maintain persistent immutability, implementing functions that take the immutable structure, make it transient, modify it, and then return a persistent version? I'm doing this now. The transients guide seems to indicate that this is the primary use case for them. thoughts?

joshjones23:06:59

since going to and from a transient structure is O(1) both ways, it seems worthwhile

hiredman23:06:16

sure, that sounds just like what the docs describe

hiredman23:06:48

most people don't read the docs, don't understand the rationale, and just want a mutable collection with clojure magic sprinkled on it

hiredman23:06:27

a while ago someone wrote a redis client behind a transient interface

joshjones23:06:48

^^ sounds dangerous

hiredman23:06:54

no pretense of following the semantics at all

hiredman23:06:25

here are some methods that look like a mutable collection, I'll reify that interface

joshjones23:06:11

yes, i'm not crazy enough to try something like that 😉

joshjones23:06:37

@sophiago on the other hand, likes to live dangerously sometimes, iirc 😉 so you never know what she's up to 😄

sophiago23:06:40

I wasn't referring to anything odder than what @joshjones said above

sophiago23:06:59

But I won't deny that characterization either 🙂

husain.mohssen23:06:11

I have a question about the performance of calling Java funcgtions. I'm seeing really poor performance when calling a simple csv parsing library. Specifically I'm trying to reproduce the results here: https://github.com/uniVocity/csv-parsers-comparison In particular I'm calling uniVocity CSV parser. On my 2016 Macbook Pro I'm getting timings that are almost 10x slower than they should be. Behold:

(time (count (.parseAll (CsvParser. (CsvParserSettings. )) (io/reader "/tmp/csv-parsers-comparison/src/main/resources/worldcitiespop.txt"))))
returns:
"Elapsed time: 19995.803153 msecs"
3173959
It varies but the fastest I saw it was about 10 seconds

husain.mohssen23:06:19

what's going on?

husain.mohssen23:06:36

am I doing something dumb in the way I'm calling the java code?

husain.mohssen23:06:58

I ran the benchmark on my laptop and it indeed parses the 3million line CSV in less than 2 seconds.

sophiago23:06:15

Anyway, I'm trying to test my claim about nested maps switching to PHMs but am running into the whole "nested coll problem" (my phrasing). I think I knew how to work around stuff like this at one point, but forgot. Essentially these maps are way too large for me to even analyze visually after dumping into a text file, so I'd like to strip away everything that's not in the top level. It would be enough to filter for collections and then empty then out.

sophiago23:06:33

Nevermind, I got it.

hiredman23:06:51

@husain.mohssen all the time there is in the java library, which is going to have nothing to do with clojure. my guess is you are doing something slightly different from what the benchmark is doing, so you should double check

sophiago23:06:39

(map #(if (coll? %) (empty %) %) my-map) reveals I was wrong, as expected from all the comments and my code appears to be leaving one more kv pair at the top level for every time I nest another level.

hiredman23:06:41

either you aren't setting the same options, or you are doing more process before or after invoking the java

hiredman23:06:13

I don't know why people persist in working with big nested maps

hiredman23:06:37

flatten it out or use an in memory database

noisesmith23:06:59

@sophiago why would any key/value pair in a hash-map ever be empty?

hiredman23:06:11

I know people do it, and it has been great for spectr that people do use these big nested structures

noisesmith23:06:15

because that’s what that code is mapping over, is key/value pairs (if my-map is a map)

sophiago23:06:28

@noisesmith it's not testing for empty. It's setting them to empty so I can see what's going on.

noisesmith23:06:51

@sophiago correcting myself: why would any map entry not be a coll?

sophiago23:06:52

That code was to test my (false) assumption stated initially

sophiago23:06:15

Are you saying a kv pair is a coll?

noisesmith23:06:31

according to clojure at least

sophiago23:06:34

Ah, that better explains why I just get a map of nils 🙂

sophiago23:06:52

Although that's fine for my diagnostic purposes: nine nils

john23:06:54

MapEntry

noisesmith23:06:20

interesting that empty on a map-entry is nil

sophiago23:06:55

Well, none of them have scalar values.

sophiago23:06:03

They're either vectors or maps

noisesmith23:06:13

but your code doesn’t check any of that

sophiago23:06:40

But it should return an empty coll of the type it's called on. I assume that's what you're questioning?

noisesmith23:06:06

right, but I guess an empty map-entry is invalid, and the vector -> map-entry implicit coercion is one way

sophiago23:06:59

I'm not sure what you mean by that. I'm not testing for either of those cases explicitly

sophiago23:06:49

I suppose why (empty [...]) or (empty {...}) would return nil rather than [] and {}?