Fork me on GitHub
#clojure
<
2017-05-09
>
lvh00:05:52

do protocol impls have the same problem as multimethods where recompiling doesn’t necessarily cause your new impl to be used?

lvh00:05:56

I’m asking because I’m doing perf optimization

lvh00:05:44

Is there a way to deepcopy an IPersistentMap? (I understand that that’s pointless; but I’m benchmarking and I want to make sure no part of the benchmark is cheating with identical?)

tbaldridge00:05:05

@Ivh, one could argue that it isn't cheating to do 'identical?'

tbaldridge00:05:22

unless you're doing something crazy like putting atoms deeply nested inside maps, and trying to diff that, which I don't recommend.

tbaldridge00:05:07

actually that doesn't make sense at all....

tbaldridge00:05:41

@Ivh why would x != x if x is x?

noisesmith01:05:03

as long as it isn't floating point 😆

lvh01:05:52

@tbaldridge I agree it’s a valid optimization, but it also makes my benchmarks useless, because in practice that will never be true for my inputs

lvh01:05:42

so because it avoids most of the code paths artificially, it’s useless for optimization

tbaldridge02:05:26

Right, but wouldn't the best option be to mutate the leaves of the tree so that everything is different?

tbaldridge02:05:46

Mutate=update

tbaldridge02:05:52

When I've written diffing engines like this in the past, I've gotten the best results by crafting specific tests for what I'm trying to test. For example: going from a nil to a nested map. A nested map to a nil. Diffing two maps that are mostly the same, diffing two maps that are not the same at all, etc.

noisesmith02:05:36

post-walk with identity as the function should effectively be a deep copy (at least for all the types that walk/post-walk destructures then reconstructs)

john02:05:43

I don't see why we aren't able to leverage the versioned internals of persistent data structures for diffing. We'd have the fastest diffs on the market 🙂

john02:05:27

Well, presumably, we are able to leverage them... It's just difficult right now.

john02:05:33

oh, I guess that's not for an arbitrary diff. I was interested in it for incremental diffs.

noisesmith02:05:40

how so? equality checks already shortcircuit true for identity for our collections

john02:05:17

Nevermind. I was thinking about it for incremental diffs. A more specific use case.

john02:05:26

I basically want a copy of the last inode (or whatever it is called) in order to update a remote object, at the same time.

john02:05:46

Distributed persistent datastructures 🙂

tbaldridge02:05:15

@noisesmith I'm pretty sure post-walk optimizes for identical? as well

noisesmith02:05:47

my idea being that it would fill in the data, but construct new containers

noisesmith02:05:56

if you wanted to benchmark something and not let it short circuit

noisesmith02:05:24

(if your real case would never see identical objects?)

curlyfry07:05:42

I'm reading through the boot-new readme (https://github.com/boot-clj/boot-new). I'm reaching the following conclusion: a boot-template can only be used with boot/new, but a leiningen-template can be used with both lein new and boot/new. Is this correct?

borkdude09:05:51

What are people using here for monitoring exceptions in their Clojure applications? (if this question belongs to another channel, please say so)

mpenet09:05:18

sentry is not bad

borkdude09:05:24

@mpenet Cool, I’ll take a look

mpenet09:05:25

log at error level -> add sentry appender for these, done

robert-stuttaford09:05:48

we use sentry, it’s great @borkdude

borkdude09:05:37

Do you use the paid service or do you host your own instance?

mpenet09:05:04

both (it's complicated)

mpenet09:05:40

both are good options, it all depends on what resource you have to maintain your own server(s) and if that ends up costing more than the hosted plans

robert-stuttaford10:05:02

we pay for hosted @borkdude. the less we have to manage ourselves, the better. time is not replaceable, money is 🙂

borkdude10:05:30

Do you use the raw Java API or some library?

mpenet10:05:20

unilog + unilog-sentry here

borkdude10:05:00

There appears also to be a codahale/raven-clj

cpmcdaniel14:05:39

so, I am working on a Clojure webapp that does not control it’s own servlet container. Is it fairly common to embed nrepl in such an app (in dev environments only) to avoid the long feedback loop of redeploying into the container?

cpmcdaniel14:05:01

and would be interested in any gotchas people have experienced with this kind of setup…

Lambda/Sierra14:05:32

For development, yes.

cpmcdaniel14:05:47

going a step further, I will probably exclude the dep from my profile when building for production. So then I will need to resolve the namespace symbol and require it at runtime if it is there

noisesmith16:05:12

cpmcdaniel: you can use resolve, but that only finds things that have been properly required - what I find works is to check the deployment environment, and require conditionally (inside -main) if the environment matches. If you want it to strictly be based on presence or absence of the dep you could conditionally call require based on whether finds the file for the namespace (io/resource "my/ns/core.clj") will be nil if it isn’t on classpath.

cpmcdaniel16:05:14

that’s better than catching ClassNotFoundException

cpmcdaniel14:05:22

thanks @stuartsierra - always helpful!

borkdude14:05:34

@cpmcdaniel I do that all the time.

lambder15:05:49

Is it possible to ‘tap’ into a protocol method. I’d like to have ‘around’ wrapper for all such method calls. e.g there is a protocol clojure.spec.Spec which has conform* method defined. Some of the conform* implementations call conform* recursively. I’d like to wrap my logic around each such call.

mwelt15:05:05

isn’t it something like the monad way of doing things?

lambder15:05:09

well, if defprotocol would be implemented with monads and provide open for that then yes. My question is more practical than theoretical tho.

john17:05:16

@lambder I'm not sure, but I'd first start with a simple experiment, trying to reify some instance of clojure.spec.Spec and override the method in question, calling back to the original -method after doing my logic. That's just a hunch though. Haven't tried, personally. I usually just override a method.

lambder17:05:20

I’ll try that @john. Thanks

john17:05:53

for instance:

john17:05:26

eh, that works in CLJS

john17:05:20

4th times a charm 🙂

john17:05:40

Once you have it working with reify you can bake it in to something more involved with deftype or extend-type if you want.

qqq19:05:37

cloc src
      17 text files.
      17 unique files.                              
       0 files ignored.

 v 1.68  T=0.07 s (260.6 files/s, 25443.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
ClojureC                         8            155             39            877
Clojure                          5             96              0            354
ClojureScript                    4             20              4            115
-------------------------------------------------------------------------------
SUM:                            17            271             43           1346
-------------------------------------------------------------------------------
do I have way too many files? 1346 lines over 17 files seems too many files

dpsutton19:05:45

would combining some of your files make your code structure more legible and easy to follow?

linicks19:05:58

All, I'm trying to use the Monger driver for MongoDB to connect to the MongoDB Atlas service. The service is running MDB v3.4 and requires the 3.4 version of the Java Driver. Monger hasn't been updated to use the newer Java Driver yet. I would like to change the Monger dependency to use 3.4 instead of 3.3, but don't have a clue with Leinigen and Clojure. In Node/NPM land, I would to a npm install --save <lib name> of the lib, then go the local project copy under node modules and modify it, and force it to get the new deps. How can I do this with Lein/CLJ? Thx!!!

dealy19:05:13

I've never really felt like I completely groked the right way to deal with atoms and refs together in some instances. I have a state-atom for a component that holds many things. Sometimes I need to update one of those things in the atom as part of a transaction. So I put a ref inside the atom. However this feels like it is probably wrong. Any thoughts on how to properly organize something like this, since I don't really want the data in the ref to be held outside the context of the atom?

noisesmith19:05:24

dealy: wouldn’t it be simpler to use a ref if you need to coordinate with other refs? If it’s about the extra boilerplate of needing to alter it in a transaction that’s a relatively easy function or macro to write

noisesmith19:05:58

something like rswap! which calls dosync with an alter inside

dealy19:05:01

yea, that's what I've been doing for the past hour, its a big ripple effect in my app

noisesmith19:05:30

but having a ref inside an atom is a ripple too isn’t it? it just seems weird to nest like that

dealy19:05:37

not using rswap! tho, I didn't know about that

noisesmith19:05:55

oh no it doesn’t exist, I am just saying it would be simple to write

noisesmith19:05:02

it’s would be a one liner basically

noisesmith19:05:26

given that alter inside dosync does what you want

dealy19:05:28

it is weird, that's why I felt uneasy about doing it, by ripple effect I meant that I have these atoms and swaps in alot of places which need to be converted to dosync/alter

noisesmith20:05:24

right, right - my suggestion was to make something that is just like swap! but for refs - also I’m surprised to hear you have so many atoms, but that’s probably a whole separate conversation

dealy20:05:06

no there are only 3 atoms, one per component, but they are used all over the place

ghadi19:05:18

@linicks -- in leiningen you need to specify the explicit dependency on the Java Driver above your reference to monger

👍 1
ghadi19:05:42

that should take care of it (provided, of course, that it's source compatible with monger)

ghadi19:05:59

You can verify success by running lein deps :tree after your change.

linicks19:05:05

@ghadi Awsome; That worked! There was only a very small change to the driver in terms of compatibility (http://mongodb.github.io/mongo-java-driver/3.4/upgrading/). I definitely have allot to learn in the CLJ space, but I'm really liking it so far. Thanks for the help!!!

ghadi19:05:29

no problem.

arthur20:05:09

I haven't looked at clojure.tools.trace in a long time, it would come in handy today, Is it still the "cool" way to build call-graphs?

hiredman20:05:11

was it ever the cool way?

qqq20:05:47

how do I write a macro which returns '(foo bar) note the ' <-- not a typo

hiredman20:05:05

(macroexpand ''foo)

noisesmith20:05:11

`(quote (foo bar))
will do that I think

noisesmith20:05:24

wait, ` would add namespaces to foo and bar

noisesmith20:05:21

(ins)user=> (defmacro foo-bar [] '(quote (foo bar)))
#'user/foo-bar
(ins)user=> (foo-bar)
(foo bar)

qqq20:05:50

@noisesmith : thanks! I was trying (list quote ... ) and that did not work

noisesmith20:05:27

the longhand version of what I had would be (quote (quote (foo bar))) the shorthand would be ‘’(foo bar) but I think the example provided is the best balance of hte two

noisesmith20:05:47

since there are no channels for yourkit or profiling I’ll ask here: my yourkit mongo usage profiling is showing short bursts of network usage followed by what looks like extended usage of CPU - is this normal? should I look for something in my code that is too CPU heavy, or is it a mongo thing, or an artifact of how yourkit shows tracing that the full network io timeperiod isn’t shown properly?

noisesmith20:05:11

I should probably just go do some microbenchmarking of the nearby code…

qqq20:05:23

[(count lst) (count (into #{} lst))] <-- okay, I now have duplicates; what is anelegant way of finding the duplicates ?

qqq20:05:34

(filter #(> (count (second %)) 1) (group-by (fn [x] x) symbols))

qqq20:05:37

is the best I have so far

noisesmith20:05:48

you can use distinct, that just uses an atom under the hood

qqq20:05:29

I want the duplicates

qqq20:05:33

not the set of all possible values.

noisesmith20:05:44

oh, sorry, misunderstood

noisesmith20:05:54

yeah, group-by identity

noisesmith20:05:57

or just frequencies

qqq20:05:02

the best I have so far is to use grou-by

qqq20:05:05

and take the ones with > 1 count

noisesmith20:05:28

then you can filter by val > 1

dergutemoritz20:05:36

@qqq There's also frequencies

qqq20:05:54

(frequencies lst)
(group-by (fn [x] x) lst)

qqq20:05:59

I guess you win by like 5 chars 🙂

jeff.terrell20:05:01

Oh hey guys, did you know about frequencies?

noisesmith20:05:33

frequencies is one of the most frequent things I see my coworkers implement from scratch not expecting it to exist in the language

qqq20:05:25

it seems like the type of thing that wouldc e in a stat library, not language core

qqq20:05:36

for examples, I bet frequenches is not part of stdlib.h in C land 🙂

noisesmith20:05:04

nor would group-by, or distinct, or filter

qqq20:05:48

if we jumped up to c++ STL, I think there's filter

noisesmith20:05:21

on the other hand, lots of string functions in c stdlib that we don’t have in core

noisesmith20:05:33

my suspicion is that this is intentional

qqq20:05:42

yeah; like all the ones required for buffer overflow attacks

qqq20:05:11

I'm pushing this towards #off-topic; so I should stop now. Thanks for insight on frequencies 🙂

noisesmith20:05:23

(def strfry (comp (partial apply str) shuffle seq))

qqq21:05:13

you know what's weird why does groujp-by return a vector insteda of a map ?

qqq21:05:23

I bet it has to use a map interanlly to create the final result anyway

qqq21:05:36

oh, to preserve order of when stuff first paperas

noisesmith21:05:47

group by returns a map of the type of result of f, mapped to vector of the input type of f

noisesmith21:05:45

@qqq when has group-by returned a vector?

noisesmith21:05:32

see, that’s a hash-map

qqq21:05:49

clearly I'm wrong

noisesmith21:05:50

btw (fn [x] x) is identity

qqq21:05:39

I think I epxected there to be a function named 'id', it did not exist; then I never tried identity

waffletower22:05:50

Trying to understand why doall doesn’t realize my clojure.lang.LazySeq in this statement:

(def cr (doall (map first [[401 "Unauthorized"]
                                 [402 "Payment Required"]
                                 [403 "Forbidden"]
                                 [408 "Request Timeout"]
                                 [502 "Bad Gateway"]
                                 [503 "Service Unavailable"]
                                 [504 "Gateway Timeout"]])))

waffletower22:05:14

(type cr)
clojure.lang.LazySeq

waffletower22:05:46

mapv instead works fine of course…

noisesmith22:05:19

doall doesn’t change the type

noisesmith22:05:27

it just guarantees the data is realized

noisesmith22:05:38

a realized LazySeq is still a LazySeq

waffletower22:05:40

ah ok, so contains? isn’t supported on that type period

noisesmith22:05:53

no, because it’s not associative

noisesmith22:05:22

(ins)user=> (let [s (map inc (range 10))] (realized? s))
false
(ins)user=> (let [s (doall (map inc (range 10)))] (realized? s))
true

noisesmith22:05:33

realized? only tells you about the head of the list of course