Fork me on GitHub
#clojure
<
2017-09-12
>
lvh00:09:17

I have some serialization code I can’t touch that checks if something is an iterable before it checks if it’s a map. This is annoying with clojure’s persistent maps, because they’re also iterable. Is there a convenient way to get “exactly this object, but pretend it doesn’t implement this interface”?

lvh00:09:00

I guess the simplest way is to proxy Map

lvh00:09:25

because I don’t really want everything-besides-Iterable, I really just want Map, I think

didibus00:09:03

That serialization code sounds like its bound to get you in big trouble soon enough

didibus00:09:39

I don't think proxy would work, since it would just derive from PersistentMap, so the iterable interface would be inherited I believe

didibus00:09:44

I'm guessing the serialization logic has a way to serialize Iterable, and your Persistent Map would be serialized as such. Is the issue you want to unserialize it back into a PersistentMap and not an Iterable?

didibus00:09:06

If I was you, I'd convert the clojure map to a java map before serializing, and convert back after deserializing:

(java.util.HashMap. {"a" 1 "b" 2})

noisesmith00:09:51

it checks out

+user=> (instance? Iterable (java.util.HashMap. {:a 0}))
false

lvh00:09:15

@didibus That works too, but I think that constructor does more copying

lvh00:09:56

@didibus You’re right about proxy — I was thinking of reify

didibus00:09:06

Is that a problem? I'd be more worried about the brittleness of that serializer at handling Clojure objects, and corrupting your payloads, then a few ms of copying. Are you sure its not a case of premature optimization?

lvh00:09:37

well, the code doesn’t exist yet so by some litmus tests I’m sure it is 🙂

lvh00:09:59

it’s certainly shorter

noisesmith00:09:09

this would counterindicate copying

+user=> (let [a (into-array [0])
              h (java.util.HashMap. {:a a})]
          (aset a 0 42)
          (get-in h [:a 0]))
42

noisesmith00:09:56

of course you are creating one instance of HashMap$Entry for each k/v pair

noisesmith00:09:58

but that's it

didibus00:09:11

Well, the pointers are copied no?

didibus00:09:32

I read "copying" as in, that requires looping over the key/values of the Map

noisesmith00:09:34

right, a HashMap$Entry is a pair of pointers

didibus00:09:29

Anyways, I don't think there's a way to remove an interface from an existing class. And using a custom PersistentMap that is not Iterable sounds like a nightmare.I don't think you've got many more option. You could just let it serialize as an iterator too, and maybe reconstruct the Map from it when you deserialize it

didibus00:09:51

Or change the serialization to be smarter

didibus02:09:04

I've been doing some programming exercise like hacker-rank recently for fun, and I've noticed its quite challenging in Clojure to use index based algorithm. There's map-indexed and keep-indexed, but as far as I know, that's it, there's no other loop construct that is iteration aware. Anyone else felt this pain? Or tips how to get around it?

didibus02:09:51

I also felt the pain of immutability. Its hard to get around it sometimes, without falling back to using an atom

didibus02:09:57

In practice, when implementing software, those challenges never come up. But I find for algorithm and data structure type problems, Clojure made it extra challenging. Maybe I'm just not advanced enough in my FP kung fu yet though

noisesmith02:09:26

I think it's a fundamental difference in what's "basic" - in an imperative language you need to master all that modify-in-place stuff just to do idiomatic iteration on a collection, where we are more comfortable using things that do the mutation implicitly (and safely) and sticking to immutable apis on a higher level

noisesmith02:09:51

it's like if they taught basic hunting and gathering in elementary school - it's tough, and most of us don't need to start with that

madstap02:09:35

I like the indexed fn from medley for when I want to know indices. Can be used with any "looping" construct.

noisesmith02:09:44

@didibus also, the other index-aware iterator is dotimes (which only gives you indexes, not collection contents of course)

noisesmith02:09:48

how about using (pst) to get the stack trace?

noisesmith02:09:24

and why would it say your error was in :create-team! - the keyword? that's very odd

thomaslp02:09:41

yeah thats why im so stumped, ill try getting the stacktrace

didibus02:09:31

@madstap I'll check out indexed. I've been cheating with using (range (count coll)) for now.

noisesmith02:09:32

I use (map-indexed vector coll)

noisesmith02:09:52

then I have a new coll of [index element]

didibus02:09:20

@noisesmith I guess the primitive is recursion. But when you need nested loops, and you start nesting recursion it gets hard to read and reason about I find. And any kind of marking, like if you need to check you've been in this branch before is tricky without mutating a tracking state.

didibus02:09:57

Haha, interesting. Not as nice as an index binding, but I guess that's a gpod trick too. I'll keep it in mind

didibus03:09:29

I guess what starts to happen, is I find there's not enough functions in core lol, which is crazy, because core has so many. But I'm like, why isn't there a version of this that loops that way instead, why reduce doesnt have an indexed variant for example. In imperative, you don't ask yourself that, because you already accepted you'd do everything with a for/while loop and a few mutable vars.

didibus03:09:27

But somehow, if I drop to loop/recur in Clj, I feel like I was defeated :p

noisesmith03:09:50

(transduce (map-indexed vector) (completing (fn [acc [index item]] ...) ...) {} coll) works pretty nice though

didibus03:09:31

Hum, that's a good one. What's completing?

noisesmith03:09:36

it simplifies writing the function arg to transduce, because transduce expects to be able to call your function with one arg to "wrap up" after the reduction is complete

noisesmith03:09:51

eg. if you want to take a value out of the accumulator you put it in the one arg arity, or make a transient persistent

noisesmith03:09:11

but completing just returns the accumulator (identity) which removes boilerplate for the common case

didibus03:09:29

Ok, I see. I think I'll need to try it out. Like does that mean + implements the one arg arity here?

noisesmith03:09:15

yes, since it acts like identity it would just work

didibus03:09:06

Didn't know that, thought a reducing function would be have been enough

noisesmith03:09:07

+user=> (transduce (map inc) (fn [x y] (+ x y)) 0 [1 2 3])
ArityException Wrong number of args (1) passed to: user/eval19/fn--20  clojure.lang.AFn.throwArity (AFn.java:429)
+user=> (transduce (map inc) + [1 2 3])
9
+user=> (transduce (map inc) (fn ([x y] (+ x y)) ([x] "done!")) 0 [1 2 3])
"done!"

didibus03:09:50

(transduce (map identity) #(+ %1 %2) 0 [1 2 3])
Wrong number of args 1 passed to fn.
(transduce (map identity) (completing #(+ %1 %2)) 0 [1 2 3])
Okay, interesting.

noisesmith03:09:14

oh yeah, I should have shown that part too, but it looks like you get it now

noisesmith03:09:23

completing also takes an optional second arg (default identity) for a separate function to call on the penultimate value from the transduce

didibus03:09:43

Ya, so much more convenient then the overload arity syntax

thomaslp03:09:36

@noisesmith looks like the issue i with a null pointer

thomaslp03:09:52

but i have no clue why when i do it in some places im getting the error, but others im not

thomaslp03:09:59

something about functions im not understanding here

noisesmith03:09:18

why are you using resolve?

noisesmith03:09:33

are you aware that resolve will simply return nil if the thing you look for isn't defined?

noisesmith03:09:47

as technomancy likes to put it "resolve is usually a sign that someone is up to shenanigans" - it's valid but every place you do it is a place you are doing something odd and working around some limit in the language

noisesmith03:09:39

if you really need resolve you can use a check to ensure the resolved value isn't nil before calling it

thomaslp03:09:05

ill explain quickly what im doing. a user is uploading data from a webpage, and they choose what kind of data it is (aka what table it gets inserted into)

thomaslp03:09:23

so im using symbol/resolve to generate which function to use instead of using a case

thomaslp03:09:50

so if they want to upload to "team", itll symbol/resolve (str "db/create-" table "!")

noisesmith03:09:05

it's much safer to use a lookup map from name to function instead - it acts as a whitelist of which functions you are letting them look up

thomaslp03:09:32

ok, just thought i could be clever and save some code but sometimes trying to be overly clever bites you in the ass lol

noisesmith03:09:54

yeah, using resolve on user supplied data is giving them root access to your machine

noisesmith03:09:09

which - OK you might intend to do that, but it's kind of extreme

didibus03:09:30

What's the advantage over a case of the create -table fns are hard coded?

kirang06:09:35

Hi all, had a quick question: has Clojars removed support for signed releases?

didibus17:09:22

I think it never had it

didibus17:09:28

Use maven if you need this feature

niten.sagar07:09:20

Hi, I am using clj-http for API calls but i am getting very generic error message. ExceptionInfo clj-http: status 415 clj-http.client/wrap-exceptions/fn--1863 (client.clj:196) how can i see the full response from API?

niten.sagar09:09:10

@jumar thanks a lot it worked. 👍

firesofmay10:09:47

Hi, I want to find out time each test ns takes. Is there a cleaner way to do this? Or I have to wrap time inside each ns fixture? Even a library that does this would do. Thanks 🙂

roklenarcic11:09:21

has anyone tried to add "spring-boot run-script"-style script to an uberjar?

djtango11:09:05

does anyone have a favourite lein plugin for hotreloading?

djtango11:09:17

I've been trying lein-autoreload but it doesn't pick up new vars

djtango11:09:34

I have to declare them first in the repl if they've never been declared before

hmaurer11:09:36

@djtango doesn’t really answer your question, but as a beginner I found it a pain to set up autoreloading with Lein. I moved to Boot recently and started using https://github.com/danielsz/system. The bult-in autoreloading is brilliant

reefersleep12:09:28

Was the move to boot mostly painless? Do you expect inconveniences later on?

hmaurer12:09:01

@U0AQ3HP9U absolutely painless for me. I am a beginner but I found boot much more natural

hmaurer12:09:29

I don’t expect inconveniences, I think the general opinion is that boot is more flexible than lein

reefersleep12:09:51

Sweet 👍 really considering trying it out.

reefersleep12:09:21

Just concerned that a lot of discussions and documentation uses lein per default, but maybe that's mostly a concern for budding developers

hmaurer12:09:27

@U0AQ3HP9U I am not experienced enough to say if that would be an issue, but it seems to me that a lot of doc mentioning lein would use concepts that are easily translated to boot

hmaurer12:09:31

e.g. dependencies to add, etc

reefersleep13:09:06

:thumbsup: 😌 gotta give it a go! @U5ZAJ15P0

dominicm13:09:40

@U0AQ3HP9U totally a self-plug, but I recommend edge[1] for a complete boot setup. It's a little heavy as it includes the whole JUXT stack with examples & training exercises. [1]: https://github.com/juxt/edge

hmaurer13:09:19

@U09LZR36F oh thanks! didn’t know it

hmaurer13:09:09

have you tried danielsz/system? how does edge differ?

reefersleep13:09:05

@U09LZR36F does seem a little heavy, I think I’d like to port some of my small hobby projects with as few dependencies and build complexities as possible first, in order to learn iteratively

reefersleep13:09:21

But I’ll be sure to remember it 🙂

dominicm13:09:31

system is tied to component far more I think. Edge isn't as deeply integrated/coupled to component. I have a branch with integrant for example.

dominicm13:09:02

@U5ZAJ15P0 actually, system w/ boot-system is actually something edge could use.

pwrflx15:09:19

what's the easiest way to remove all those keys from a map where the value is nil?

didibus17:09:55

@pwrflx

(filter (fn [[k v]] (not (nil? v))) {:a nil :b false :c 1})
or
(remove (fn [[k v]] (nil? v)) {:a nil :b false :c 1})

ghadi15:09:15

(into {} (filter val) m)

ghadi15:09:24

N.B. that will filter false values

ghadi15:09:43

so you'd adjust to: (filter (comp some? val))

dealy15:09:35

Hi, I have a function which subscribes to a publication and sets up a listener loop. If I create the output channel within my function my events no longer work, however if I create the output channel outside of the function and pass it in as a parameter everything is fine. What am I misunderstanding here?

dealy15:09:53

Here is the func: (defn subl [publication topic output-ch handler-fn] (let [listener-ch (chan) ;listener-ch output-ch }

dealy15:09:06

oops, how do you get the code blocks in here?

dpsutton15:09:30

above and below, or use the + on the text input line for code syntax

dpsutton15:09:58

when you add it, in the top right there's a dropdown to select which language for syntax highlighting

noisesmith16:09:08

@dealy my suspicion is that there is something using the outside binding for output-ch and not the new one returned from subl. That said, it’s definitely better to define your core.async function to take chans as args rather than making channels itself (eg. that means your consumers can make decisions about buffering and transducing without having to add complexity to their consuming code)

dealy16:09:42

I think I finally figured it out, my tests call unsubscribe during the teardown, which I was thought was happening with each testing block. When I moved the offending test to its own deftest then it started working. I assume that the problem was my previous tests were using the same pub and therefore there was some leftover pubsub plumbing that caused all the havoc

dealy16:09:47

as far as taking chans as args yes I agree, the next phase is to make the chan optional so it can be supplied by the client with its own buffering when needed

tiagoantao20:09:57

Hi, I was wondering if one could do a (require-macros ... inside a cljs/eval-str?I am getting a error self__.rdr.cljs$tools$reader$reader_types$Reader$read_char$arity$1 is not a function

pwrflx21:09:09

is there a good linter or code style checker for clojure?

bfabry21:09:03

@pwrflx there's eastwood and kibit

dominicm21:09:46

@pwrflx there's also joker

aengelberg22:09:36

user=> (->> (take 5 (repeatedly (constantly 1))) (map #(doto % (println "<-"))) first)
1 <-
1
user=> (->> (take 5 (repeatedly (constantly 1))) (sequence (map #(doto % (println "<-")))) first)
1 <-
1 <-
1 <-
1 <-
1 <-
1

aengelberg22:09:49

is it possible to apply a transducer to a lazy sequence and keep it unchunked?

noisesmith22:09:55

the cleanest thing is if you care about side effects, don’t do them inside code that is lazy, do them to data that you generate lazily

noisesmith22:09:00

then chunking is a non-issue

noisesmith22:09:21

(doseq [e (take 5 …)] (side-effect e))

aengelberg22:09:27

I only used a side effect here to demonstrate the chunk behavior. In reality I want to ensure I'm only processing one element at a time for memory reasons.

noisesmith22:09:42

you can use reduce and build up a result instead, this avoids chunking as much as doseq is (and is just as non-lazy)

noisesmith22:09:32

laziness is an abstraction that doesn’t play nicely with situations where the speed of consuming the input is crucial (you could also look up unchunk, there’s a few versions of it floating around, but I sincerely think doseq and reduce are better ways to take care of the root problem)

aengelberg22:09:44

It's not really about speed, but I want to write an API that returns a lazy sequence of things that is guaranteed to only be dedicating resources to one thing at a time, to avoid memory or thread leaks.

noisesmith22:09:26

I know it’s not about speed - I meant “speed of consuming” in comparison to “speed of production”

noisesmith22:09:33

it’s was a sloppy way to express it, sorry

noisesmith22:09:48

if you want control of what is in memory, and for how long, lazy-seqs are tricky

noisesmith22:09:42

but making the expensive thing (cpu wise, memory wise) or the critical thing (side effect wise) non lazy, the rest can remain lazy

Alex Miller (Clojure team)23:09:26

You might want to look at an eduction with a transducer

lvh23:09:00

I’m trying to implement an interface (java.util.Map) with a record:

(defrecord JustAMap [m]
  java.util.Map
  (entrySet [this] #(.entrySet m)))
and getting the following error when evaling:
1. Caused by java.lang.ClassFormatError
   Duplicate interface name in class file
   compile__stub/unsiemly/stackdriver/JustAMap

lvh23:09:33

Heh, I tried to figure this out for a solid hour and of course as soon as I ask I figure it out: the problem is that defrecord gives you a map interface for free, so that’s not what you want.