Fork me on GitHub
#clojure
<
2017-07-14
>
brabster10:07:25

Hi all, I'd like to fan out the same input sequence to several transducers and collect the results into a single sequence... is there a function for that?

shidima_13:07:20

Any one experience with Lanterna? I'm looking into making a snake game, and would like to know if it can do a non blocking ui

manutter5113:07:04

@shidima_ I believe it does, haven’t used it in a while, but I see from the docs it has both get-key and get-key-blocking — the first returns nil if there’s no keypress waiting to be read.

alricu14:07:29

Hi all, Is there any way to mock a function in clojure. For example I have this function:

(defn a [x]
        (utils/count x)) 
I need to write a test, but I do not know how to mock the utils/count (for example)

alricu14:07:25

what if I have several functions inside the same function and I need to test it

alricu14:07:43

(defn a [x]
        (utils/count x)
        (utils/count2 x)
        (test/other x)) 

tankthinks14:07:15

@alricu, you can use with-redefs to "re-bind" a symbol

dpsutton14:07:39

but in that function there's nothing to test

dpsutton14:07:48

it sounds like you want to test utils/count

alricu14:07:01

it is an example!

dpsutton14:07:08

gotcha. just making sure

dpsutton14:07:42

how many dependencies are in it?

dpsutton14:07:56

i'm saying that because if its not too many, i would make a function that accepts those and calls them inside it self, and you can create your production one with (create-a [count count2 other] (fn a [x] (utils/count ...)))

dpsutton14:07:06

and then you can test that with just passing in your own mocks

dpsutton14:07:14

but this doesn't work if there are quite a few functions required

grzm14:07:06

I'm looking for a way to remove/clear protocol implementations during testing. Is there a common way of doing this? I came across mention in Stuart Sierra's LazyTest where he says his method is broken. https://github.com/stuartsierra/lazytest/blob/master/modules/clojure-language-tests/test/clojure/test_clojure/protocols.clj#L18-L28

grzm14:07:15

I'd rather not reinvent the wheel if someone has something for this already.

noisesmith15:07:29

@grzrm so your test needs to eliminate the extensions of your protocol?

grzm15:07:49

yup. The library I'm testing provides alternative implementations, so I'm testing each one. Right now I'm running separate test suites, but that's a bit of a hassle.

noisesmith15:07:42

why do other extensions need to be deleted? can’t the individual extensions be tested independently of one another?

noisesmith15:07:58

do you mean it’s two extensions of one type to a protocol?

grzm15:07:43

If I have two different implementations on, say, IMap, only one takes effect.

noisesmith15:07:51

on one type?

noisesmith15:07:25

my temptation would be to either replace the type or protocol when testing

noisesmith15:07:56

because clearly both implementors can’t own both the type and the protocol, which would be good reason to make it modular…

grzm15:07:19

I'm not following. In this case the I'm trying to test specifically the implementation on that type. Swapping it out, like for a mock, would defeat the purpose in this case.

noisesmith15:07:36

it could be as simple as a proxy that uses proxy-super for every method you call

grzm15:07:01

Can you elaborate?

grzm15:07:45

Or point to a reference of what you're getting at?

noisesmith15:07:58

user=> (str (proxy [Object] [] (toString [] (proxy-super toString))))
"[email protected]"

noisesmith15:07:12

you extend the supertype, and call the supertype’s method on method calls

noisesmith15:07:22

it’s a different class, but has the same behaviors

noisesmith15:07:56

another alternative is to use composition (a deftype with a Foo, which re-calls every method on that Foo)

grzm15:07:37

Thanks for the alternatives. I'll ponder those for a bit to see if I can make them work. Cheers!

octahedrion16:07:40

what's wrong with this spec expression that it doesn't return an explanation ?

(s/explain (s/cat :this (s/* (s/coll-of :q/o)) :that :q/test-map) '((7 3 1) (9 7 3) {:q/b [2 4 6] :q/c ["abc" "xyz"] :q/d [:q :w] :q/a [1 3 5]}))
IllegalArgumentException Argument must be an integer: [:q/b [2 4 6]]  clojure.core/even? (core.clj:1383)

octahedrion16:07:20

...each part of the s/cat works:

(s/explain (s/cat :this (s/* (s/coll-of :q/o))) '((7 3 1) (9 7 3)))
Success!

octahedrion16:07:52

(s/explain (s/cat :that :q/test-map) '({:q/b [2 4 6] :q/c ["abc" "xyz"] :q/d [:q :w] :q/a [1 3 5]}))
Success!

octahedrion16:07:07

(where

(s/def :q/test-map (s/keys :req [:q/b :q/a] :opt [:q/c :q/d]))
and the various keys :q/a etc are simple
odd? even? string? keyword?

hiredman16:07:15

even? isn't total

hiredman16:07:44

it throws an exception if something isn't a number

hiredman16:07:07

so you need something like (s/and number? even?)

octahedrion16:07:56

sorry I should have said :q/a and q:b are

(s/+ odd?)
and
(s/+ even?)

hiredman16:07:18

what is :q/o ?

octahedrion16:07:32

(s/def :q/o odd?)

hiredman16:07:36

so the way s/* can tell it is done matching is something fails to match

hiredman16:07:41

yeah, odd? isn't total

serg16:07:49

Hey guys can someone help with this problem, I am new to clojure, may be I am missing something: https://stackoverflow.com/questions/45107883/clojure-reducers-foldcat-not-working

joshjones17:07:24

serg: I saw you deleted this. I noticed if you removed into [] and just used the seq instead of a vector, it works. Did you solve it?

serg06:07:34

@U1ALMRBLL Hi Josh, actually the problem was in my mac, prev I ran this func with long running async task in map, it was like fetch 6k items and insert into db values for each one, so seems like it was running all this time consuming all available threads, that's why it didn't work, once it has finished I got my code working as expected.

serg06:07:16

The strange thing is that I ran this in REPL and I am sure I have cancelled this task, but somehow it was still running in background consuming all available threads

hiredman16:07:52

(s/and number? odd?)

octahedrion16:07:08

what do you mean by "total" ?

seancorfield16:07:50

I suspect the problem here is that s/+ is a regex matcher, not a collection matcher...?

seancorfield16:07:19

Change :q/b to (s/coll-of even?)

seancorfield16:07:39

(you can specify a minimum length of 1 to require at least one element)

hiredman16:07:39

that'll be the next issue, which will get him a weird failing spec

hiredman16:07:23

right now the error from feeding a non-number to odd? is bubbling out and killing the checking

seancorfield16:07:40

Ah, I see where you're going now. I was jumping ahead.

seancorfield16:07:00

@octo221 "total" means "defined for all inputs"

seancorfield16:07:19

even? and odd? are only defined for numeric inputs.

seancorfield16:07:39

Hence @hiredman's suggestion to use (s/and number? odd?)

octahedrion16:07:22

ok a simpler version works:

(s/explain (s/cat :this (s/* (s/coll-of odd?)) :that (s/coll-of even?)) '((7 3 1) (9 7 3) [2 4 6]))

octahedrion16:07:46

using

(s/def :q/b (s/coll-of (s/and number? even?)))
does not work

octahedrion16:07:25

(s/explain (s/cat :this (s/* (s/coll-of odd?)) :that (s/keys :req [:q/b])) '((7 3 1) (9 7 3) {:q/b [2 4 6]}))
IllegalArgumentException Argument must be an integer: [:q/b [2 4 6]]  clojure.core/even? (core.clj:1383)

hiredman16:07:50

you need to do it for odd? too

octahedrion16:07:56

however,

(s/explain (s/cat :that (s/keys :req [:q/b])) '({:q/b [2 4 6]}))
Success!

hiredman16:07:06

if I recall odd? is just (not (even? ...))

octahedrion16:07:36

ah yes that works ok

octahedrion16:07:48

(s/explain (s/cat :this (s/* (s/coll-of (s/and number? odd?))) :that (s/keys :req [:q/b])) '((7 3 1) (9 7 3) {:q/b [2 4 6]}))
Success!

octahedrion16:07:58

please explain why!?

hiredman16:07:46

because odd? and even? as predicates aren't total, so they will throw exceptions when not passed numbers instead of returning false

hiredman16:07:00

s/and tries each predicate it order

octahedrion16:07:06

yes but why should they be passed non-numbers ?

hiredman16:07:29

because in order for s/* to stop matching it has to fail a match

hiredman16:07:38

otherwise it would match everything

octahedrion16:07:33

oh yes I see!

(odd? {})
IllegalArgumentException Argument must be an integer: {}  clojure.core/even? (core.clj:1383)

octahedrion16:07:04

i had misunderstood how s/* worked

octahedrion16:07:14

thanks so much!

lwhorton17:07:28

this is perhaps a little convoluted.. but now i’m curious just for curiosity’s sake. if I had a function that maps over some collection, invokes a provided fn, and returns the value of that invoked fn as a different shape, how does this work with compose?

(defn do-map [f coll]
  (map (fn [[foo val baz]] [foo (f val) baz]) coll)) ;; <- notice the [a b c] 
;; wont work unless xform-2 and xform-1 are "aware" of the [a b c] shape
(do-map (comp xform-2 xform-1) coll)  

lwhorton17:07:36

is there a generic way to compose transformations and, after each xform, re-shape the data?

lwhorton17:07:14

the do-map couldn’t really know whether f was composed or not, it just sees a function. so it would be hard to do something like (-> coll f0 reshape f1 reshape ...)

noisesmith17:07:31

@lwhorton sounds like you want (fn [f] (fn [[a b c]] [a (f b) c])

noisesmith17:07:41

then you can wrap your functions in it?

noisesmith17:07:41

and -> isn’t a composer of functions, it’s a rewriter of forms, if you use comp it’s easier to get the right behavior via wrapping functions without jumping through syntactic hoops

lwhorton17:07:55

not quite, but the more i think about it the less possible it seems. i essentially want an interface where I can pass in a single function (composed or not), and that function will be applied piecemeal. if the provided fn is a composed fn, it will invoke the first composed, reshape the results, invoke the second composed, etc… regardless of how many steps belong to that composed function.

lwhorton17:07:06

I think it’s impossible to do it from the “inside”; there’s no way to call 1 step of a composed function and “steal” the result before calling the 2nd step.

lwhorton17:07:20

i would have to make the interface something like (do-map [coll f & maybe-more-fns] (if maybe-more-fns (compose f reshape (first maybe-more-fns) reshape (second maybe-more-fns ...)) and do the composing on the inside, I think

lwhorton17:07:29

that is, unless i’m totally misunderstanding your answer (happens often i find)

noisesmith17:07:01

@lwhorton #(apply comp (map (fn [f] (fn [[a b c]] [a (f b) c])) (reverse %)) will take the functions in the order -> would accept them, and return a composed function that does what you describe

noisesmith17:07:05

or you could write a macro that inserts that function around or inside each function in the form

lwhorton17:07:36

that’s pretty neat. i considered macros, too, but thats a whole ‘nother can of worms i havent opened yet.

schmee17:07:07

ooooh boy, you’re in for a treat! 😄

noisesmith17:07:33

this becomes more clear if you name that function eg.

(defn to-middle
  [f]
  (fn [[a b c]]
    [a (f b) c]))

(apply comp (map to-middle functions))

lwhorton17:07:08

yea that’s much more clear. funny now that you point that out I realize this is expressed in a protocol, which doesn’t support variadic arguments. so either enforce a seq like (do-map v [f]) and use the fn above^, or I think something like (-> coll (do-map xform-a) (do-map xform-b) (though more inefficient) works too

lwhorton17:07:55

so much functional-ness, so little time

noisesmith17:07:22

that -> form will break

noisesmith17:07:46

that’s why I mentioned using comp to avoid syntactic hoops - since -> is a form rewriter, it will put the coll in the wrong place

lwhorton18:07:35

so beautiful:

(map (apply comp (map reshape fns)) coll)
thanks a bunch !

lwhorton17:07:36

hmm, ill have to look deeper at that. thanks for the warning

alricu18:07:08

I am trying to rename the keys from a hashmap with rename-keys I have

(def a {a "A", b "B"} 

alricu18:07:55

and use

(set/rename-keys a {:a :aa, :b :bb} 

alricu18:07:33

but nothing changes

alricu18:07:17

Am I doing something wrong?

jahson18:07:24

Try (def a {:a "A", :b "B"})

hiredman18:07:25

well for one, a and b are both unquote symbols which evaluate to some value where :a and :b are keywords that evaluate to themselves

alricu18:07:50

the problem is that I receive the Json as it is

alricu18:07:03

I can not change the a variable

alricu18:07:22

it was just an example; But i need to modify the names of the keys

noisesmith18:07:39

sounds like your real problem is that you aren’t using the return value of rename-keys

noisesmith18:07:06

nothing in clojure actually modifies the built in maps, vectors, lists, etc. - they return new ones you can use

alricu18:07:10

I did a println

jahson18:07:16

You’re probably getting string keys.

alricu18:07:18

but it printed the same a

noisesmith18:07:37

don’t use println to look at data, use prn

dorab18:07:47

If you are working with JSON, you're likely getting the map as {"a" "A", "b" "B"}

noisesmith18:07:49

I bet a is actually “a” yeah

noisesmith18:07:02

so you need rename-keys {“a” :aa} etc.

alricu18:07:12

mmm I tried but I got this error :

alricu18:07:21

java.lang.ClassCastException: clojure.core$map$fn__4781 cannot be cast to clojure.lang.IPersistentMap

noisesmith18:07:14

that means you used (map f) where it expected a hash-map

alricu18:07:56

wait, yes!!!!

alricu18:07:18

I was doing it to another functionality

alricu18:07:22

now it is working

alricu18:07:35

rename-keys with "a" as key

noisesmith18:07:46

for future reference, this is the difference between println and prn

kingfisher.core=> (println {"a" 0 "b" 1})
{a 0, b 1}
nil
kingfisher.core=> (prn {"a" 0 "b" 1})
{"a" 0, "b" 1}
nil

jahson18:07:15

There’s probably other option — you can get json with keys as keywords, just look to documentation of your json lib.

noisesmith18:07:47

that’s sloppy though - there are valid json keys that are not valid keywords, so then you end up creating keywords that shouldn’t exist

hiredman18:07:10

well that is a whole other thing

hiredman18:07:03

(are valid keywords only those keywords that can be produced by the reader?)

hiredman18:07:05

but generally, turning keys in to keywords all the time is something every project (I have worked on) does, and I hate it

hiredman18:07:29

but if you do it you are doing the same thing that everyone does

bfabry19:07:28

"valid keywords" are those that follow the rules in the docs imo, but the reader will create some keywords that don't if asked

hiredman19:07:04

which docs?

hiredman19:07:59

those a docs on keywords that the reader can read

hiredman19:07:12

(which is why they are under the reader section)

hiredman19:07:28

I've seen this discussion play out over and over again in chat, on the mailing list, and in jira issues, and it seems like the core team's general stance is what the reader excepts doesn't define what is valid

bfabry19:07:41

the reader accepts more than those docs, but those docs do give a guarantee of what the reader will accept. so for me they make sense as a definition of valid. not the same thing as "whatever the reader accepts"

bfabry19:07:13

eg

cljs.user=> :&
:&

bfabry19:07:43

but it's not documented that that will always work (or become reserved for some special system/core usage) so I wouldn't use it

hiredman19:07:11

the point is, just because the reader only constructs certain kinds of symbols and keywords, that doesn't restrict the Symbol and Keyword data types to only construct those types of symbols and keywords

hiredman19:07:25

etc etc etc

seancorfield19:07:42

And at one point, very specifically, the Clojure/core team made :1 (numeric keywords) illegal in the reader on the grounds that they weren't sanctioned by the docs -- and they broke a bunch of code out there in the wild so they reverted the reader change.

seancorfield19:07:55

I was affected by that change (one of the OSS libraries I maintain) but I would have been perfectly happy to change :1 to (keyword "1") -- which they said they would not break... but they still reverted the change 🙂

seancorfield19:07:53

So we have prior art for the reader being publicly known to accept things that are not considered "correct" and not getting fixed -- and a statement that keyword (and, similarly, symbol) being able to produce "junk" output from arbitrary input 😐

hiredman19:07:06

the issue isn't that the reader and the reader docs are not in 100% agreement about what is a readable keyword (that is another issue) the issue is that a valid readable keyword is not the same as a valid keyword

hiredman19:07:31

for example, the keywords json libraries create for map keys never pass through the reader

hiredman19:07:42

so they can be any crazy thing

seancorfield19:07:50

Exactly my point, yes.

hiredman19:07:36

which negates the purpose of turning them in to keywords in the first place, because the only reason people turn them in to keywords is since most maps in clojure use keyword keys, they more comfortable with them

seancorfield19:07:07

Speed of lookup over plain strings. In theory.

seancorfield19:07:42

(identical? :abc (keyword (str "a" "bc"))) => true

seancorfield19:07:24

So map lookup will be faster if it can check for identical? before falling back to "equal".

hiredman19:07:30

but do you want to be interning json coming in over the wire?

schmee19:07:43

what am I missing here?

user=> (defn foo [args] (apply hash-map args))
#'user/foo
user=> (binding [*data-readers* {'foo #'user/foo}] #foo [1 2 3 4])

             java.lang.RuntimeException: No reader function for tag foo
clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: No reader function for tag foo
             java.lang.RuntimeException: Unmatched delimiter: )
clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: Unmatched delimiter: )

hiredman19:07:07

the whole form is read before the binding is run

seancorfield19:07:18

Good question, given that converting arbitrary strings to keywords can open you up to a denial of service attack.

hiredman19:07:08

I mean, you will likely be fine, and keyword's intern a weakreference, so the they will get cleared

seancorfield19:07:27

Ya, but... 🙂

noisesmith19:07:04

user=> (defn foo [args] (apply hash-map args))
#'user/foo
user=> (binding [*data-readers* {'foo #'user/foo}] (read-string "#foo [1 2 3 4]"))
{1 2, 3 4}

schmee19:07:05

@hiredman ahh, so you need this? (binding [*data-readers* {'foo #'user/foo}] (read-string "#foo [1 2 3 4]"))

schmee19:07:14

haha, timing 😄

schmee19:07:34

while on the topic of tagged literals: is there anything special you need to do to get it to work with tools.namespace?

schmee19:07:20

I’ve placed a data_readers.clj in my classpath root, but when I try to use it in the REPL I get something like “Can’t call unbound Var”

schmee19:07:33

lemme dig up the exact exception

hiredman19:07:40

you need to require the namespace before you can use the reader

hiredman19:07:31

data_readers.clj sets up the tag, but doesn't actually cause the namespace where the function is defined to load

schmee19:07:00

I’m pretty sure I did that, but let me double check

schmee19:07:18

ok, so my data_readers.clj contains {foo kleinheit.pg.impl/foo}

schmee19:07:39

(defn foo [vs] (apply hash-map vs))

schmee19:07:25

wait a minute!

hiredman19:07:33

and every place where you read something using that tag, before that does a require of that namespace execute?

schmee19:07:42

okay, I see the problem: if I use (require 'kleinheit.pg.impl :reload) first thing in the REPL, it works

schmee19:07:01

but if I do clojure.tools.namespace.repl/refresh, it doesn’t

schmee19:07:13

yep, tool.namespace clobbers it somehow

schmee19:07:16

the prevailing wisdom on the internet when having problems with data readers is to call (#'clojure.core/load-data-readers), but then I get:

dev=>   (#'clojure.core/load-data-readers)

clojure.lang.ExceptionInfo: Conflicting data-reader mapping

noisesmith19:07:54

if you look at the ex-data on *e that should show you what went wrong, right?

schmee19:07:48

I’m getting a conflict with my own data reader file 😐

noisesmith20:07:18

OK - so the data readers are already loaded, you have to reload the code they refer to, right?

hiredman20:07:01

(which refresh would do if you properly required that code before using the readers)

schmee20:07:58

still get the error after calling refresh

hiredman20:07:14

right, because somewhere you aren't properly doing the require

hiredman20:07:43

so find all the places you use the tag, ensure in that namespace you require the namespace where the function the tag maps to is

schmee20:07:34

dev=> (dev/refresh)
:reloading (... kleinheit.pg.impl ...)
:ok

dev=> #foo [1 2 3 4]
        java.lang.IllegalStateException: Attempting to call unbound fn: #'kleinheit.pg.impl/foo
clojure.lang.LispReader$ReaderException: java.lang.IllegalStateException: Attempting to call unbound fn: #'kleinheit.pg.impl/foo

schmee20:07:08

and it works if I just skipped the refresh and do the regular (require ...), so I’m at a loss ¯\(ツ)

hiredman20:07:23

what kind of repl you are using?

hiredman20:07:04

lein repl can be kind of weird

schmee20:07:29

ehh… errhmm… how do I launch a REPL without lein? 😅

hiredman20:07:03

that depends,

java -cp `lein classpath` clojure.main
often works

noisesmith20:07:49

rlwrap is very handy when doing it that way

schmee20:07:42

still get the same behavior in that REPL unfortunately 😕

hiredman20:07:35

put a println at the top of the kleinheit.pg.impl namespace to see if it is actually getting reloaded

schmee20:07:36

yessir, it prints

schmee20:07:37

well, I’m gonna give this a rest for now, appreciate all your help folks

hiredman20:07:43

ah, I bet refresh breaks var interning, so you end up with two vars with the same name

schmee20:07:34

ohh, now that sounds like it might be it!

schmee20:07:49

time to dig into the source…

noisesmith20:07:34

yeah, it destroys namespaces which might also include doing something wacky with vars that other code has captured…

noisesmith20:07:52

you might just want to blacklist the namespaces that define readers from auto-reload

schmee20:07:27

yeah, I’ll see if I can figure out a way to do that, thanks!

schmee20:07:08

yeah, the problem with that is if I call disable-unload! in the file, the “do not refresh” metadata will be added on the first refresh and I’m stuck again

schmee20:07:25

can you add metadata to a namespace in the ns call?

noisesmith20:07:07

yeah- iirc it makes tools.namespace unhappy though (and various tools that use that, like eastwood)

noisesmith20:07:14

but maybe that’s old info…

schmee20:07:20

well, that didn’t work 😄

schmee20:07:56

talking about interning got me thinking about a great clojure talk I need to rewatch: https://www.youtube.com/watch?v=8NUI07y1SlQ

schmee20:07:35

(thanks for the preview Slack….)

bcbradley20:07:56

woot i just made a graphics library for clojure! if anyone interested in that kind of thing could run some of the examples, i'd be thrilled to know if they work for you or not.