Fork me on GitHub
#clojure
<
2015-12-29
>
crocket01:12:39

@seancorfield: I read boot documentation, and it looks like a sane approach.

crocket01:12:58

It's easy to define a task, and each task can be a pipeline.

kopasetik05:12:27

Which Clojure functions are lazy seq functions? You know functions that are lazily evaluated

pastafari05:12:19

kopasetik: map, reduce, filter, etc.

nowprovision07:12:45

reduce is lazy?

kopasetik07:12:22

I’m getting an error about a Chunk sequence for this code that I wrote...

rm07:12:47

nowprovision: reduce is not lazy, you're right. http://blog.altometrics.com/2015/02/lazy-reduce-in-clojure/ someone made lazy version

kopasetik07:12:15

The problem I’m trying to solve...

underplank08:12:43

Hi all, I’ve got a compojure app with the following middle-ware

(def app
  (-> routes
      wrap-json-body
      wrap-json-response
      wrap-request-logging
      init-data))

underplank08:12:25

When i use ring-mock to test it, the :body of the request is an InputStream. When I make a request from a browser its just a string.

underplank08:12:42

Im confused by this and not sure the best way to go about fixing it.

rm08:12:35

@kopasetik: your problem can be solved without any cycle :)

rm08:12:58

underplank: how do you mock?

kopasetik08:12:11

@rm: OK, clueless as to how I’ll do that

kopasetik08:12:57

Create an infinite sequence?

rm08:12:03

@kopasetik: I'll try to prompt a bit. Split your queue into pieces like ["sheldon" "howard" "penny"], ["sheldon" "sheldon" "howard" "howard" "penny" "penny"] ... in your head. Then use Math/log2 to find out in which piece your input number is. I guess that would be enough :)

kopasetik08:12:34

Cool, that should work

rm08:12:55

you're welcome

kopasetik08:12:48

But I also want to know why I have that PersistentVector$ChunkSeq error when I run my original code…any ideas?

underplank08:12:17

@rm using ring-mock?

underplank08:12:57

Is this a good idea? bad idea?

pastafari08:12:03

nowprovision: kopasetik apologies reduce is not lazy.

kopasetik08:12:43

@pastafari: All good, thanks!

kopasetik08:12:02

This looks like a good guide to Clojure’s laziness functions: http://clojure-doc.org/articles/language/laziness.html

rm08:12:27

@kopasetik: maybe because rest is internal clojure function and there is some conflict. Try other name

rm08:12:03

as well as first

rm09:12:28

@underplank: no, it's ok, but I can't remember if I got the same problem. How your mock looks exactly?

underplank09:12:15

@rm:

(deftest user-put-handler-test
  (init-data app)
  (let [new-user {:email "" :address "hello there"}
        req (mock/request :put "/user")
        req (mock/body req (generate-string new-user))]
      (is (= {:status 200
          :headers {"Content-Type" "application/json; charset=utf-8"}
          :body (generate-string new-user)}
         (app req)
         )))
  )

rm09:12:13

you can use map, no need to generate-string

rm09:12:02

also read request code

rm09:12:27

so you could do (mock/request :put "/user" new-user)

underplank09:12:20

ahh… but my body is json.. not urlencoded...

underplank10:12:36

@rm actually I think i’ve worked it out. Its the body in the response not the request that is an InputStream.

rm10:12:10

ah, good

jcomplex16:12:51

world answers: hello @jimmyramia

gtrak17:12:27

,(+ 1 2 3)

gtrak17:12:09

are we running an eval bot here? Trying to get one on our company slack: https://github.com/verma/clj-slackbot seems to need pr-str for lazy-seqs, which is annoying.

gtrak17:12:13

which includes better printing

rcanepa17:12:32

Hey guys! … Two questions for you: 1) Which library do you use for error handling?, 2) How do you structure your "handling error” code to keep it separated from your main code?… I found a library named Dire, which encourages the separation between this two worlds, however, is not clear to me how to accomplish this in the real world (I am thinking at folder/files level).

rcanepa17:12:49

Does anyone follow this pattern/idea?

mbertheau17:12:07

Am I missing something or is (into {} (filter #(f %2) my-map)) really the most concise way to filter a map with f on its keys?

roberto18:12:14

not sure what you are trying to do

roberto18:12:29

is f a function?

roberto18:12:42

you want all fields in a map with the value equal to (f value) ?

bronsa18:12:10

I think he wants filter-keys

mbertheau18:12:54

filter-keys sounds great. It's not in the standard library though, right?

roberto18:12:54

select-keys

bronsa18:12:08

select-keys doesn't apply a predicatr

bronsa18:12:41

there's actually a filter-key function in clojure.core, but it's private 😞

mbertheau18:12:50

Hmm, given that clojure is not so young anymore, is it fair to say that some "batteries" (i.e. functions commonly needed) are missing from core and one shouldn't expect them to be added?

roberto18:12:31

i’m fine with them not being there, because they are trivial to create

mbertheau18:12:29

With that reasoning when-not could be excluded from core as well.

meow18:12:15

Ah, the great debate about what belongs in clojure core, and why certain less useful things are already included... simple_smile

meow18:12:34

Here's one that's been getting me lately. I just wrote this function to simplify a pattern I keep running into:

(defn zipmapf [coll f]
  (zipmap coll (map f coll)))
Anyone know if something like this already exists, or what a better name might be?

meow18:12:45

One could argue that it is weird to have frequencies in core, but not something more generic. But I'm not starting a fight. simple_smile

meow18:12:40

Looks like flatland useful has the following:

(defn map-to
  "Returns a map from each item in coll to f applied to that item."
  [f coll]
  (into {}
        (for [item (distinct coll)]
          (map-entry item (f item)))))

verma19:12:08

Ah @gtrak I wasn't aware there was a fork with fixes, I'll try to get that merged in, fwiw that fork is pretty old and I've added realtime API bot since (which is far more responsive). I'll try to get some of the annoyances fixed.

kopasetik19:12:46

If this slack doesn’t have an evalbot, it should! That’s my vote! simple_smile

jrychter19:12:21

I started using core.async/pipeline and I'm not getting the CPU utilization I expected. It's a computational pipeline, the job is CPU-bound, I specified 16 as the number of jobs, and yet I'm maxing out at about 250% CPU load as reported by the OS. I expected to reach 600-800%, as I did when using tesser.

gtrak19:12:05

@verma I tried both, webhook works for us b/c it lets us use heroku free tier+sleep, but the print fixes are also a good thing.

gtrak19:12:14

does anyone know how to send /clj from IRC?

verma20:12:15

@gtrak: makes sense simple_smile

ajmagnifico20:12:51

In my lein project.clj, is there a convenient way to automatically increment the version number when I deploy?

ajmagnifico20:12:03

I've been playing around with lein-release plugin, but not sure yet if it's exactly what I want

meow21:12:48

Can anyone help with making my use of clojure.data.xml more lazy wrt outputting big xml files?

meow21:12:36

Specifically I have element attributes that are large strings that I want to be lazy.

meow21:12:19

It looks like data.xml is just going to call (str o) on whatever object is supplied as the attribute.

micha21:12:36

hi all, i'm looking for some documentation on best practices for incorporating logging into a library. the only docs i can find are for application layer logging, where logging configuration is controlled

meow21:12:44

I want to do something like (apply str (interpose " " (apply concat coll)

meow21:12:26

@micha: You can't find anything because your time would be better spent helping me with my problem. 😉

micha21:12:58

ahah, wouldn't apply force evaluation?

meow21:12:15

exactly, that's my problem - how do I delay evaluation?

meow21:12:59

I need to supply an object on which str will be called and have it return a huge string.

micha21:12:47

you can use delay

micha21:12:56

(delay (apply str ...))

micha21:12:13

then you force it later by dereferencing

gtrak21:12:17

in general you delay stuff by wrapping it in a function, delay does that for you, lazy-seq is used to delay a recursion

gtrak21:12:25

but also wraps your code in functions

micha21:12:28

the benefit of delay is that it also memoizes

micha21:12:43

or maybe not good if memory is a concern

jr21:12:48

reify toString for the delayed value maybe?

meow21:12:58

yeah, that's not going to work - basically I'm using clojure.data.xml to write an X3D file and it isn't lazy enough so with big files I'm running out of memory.

meow21:12:25

well, it is supposed to be lazy according to the docs

meow21:12:45

to date I've been feeding it strings - big ones

meow21:12:58

but if they get too big it fails

meow21:12:18

trying to figure out how to get around that

meow21:12:19

internally data.xml is going to call str on whatever I supply as an xml element attribute

meow21:12:23

I've been formatting these attributes using this function: list-format (fn [coll] (string/join " " (apply concat coll)))

meow21:12:36

which, obviously, isn't lazy

meow21:12:27

it just produces one of these big strings

roberto21:12:02

is join lazy?

micha21:12:22

i don't see how a string can be lazy

meow21:12:21

@roberto: no, join isn't lazy

meow21:12:36

well, the string can't be lazy, but rather than create all the ones I need, I thought if the string creation were at least delayed until data.xml calls str on it that it might take some of the memory pressure off

meow21:12:32

this stupid X3D file format doesn't give me any room to breath 😞

meow21:12:35

I've basically got 6 big strings I need to create.

meow21:12:45

(apply str (interpose " " (apply concat [[1 2 3] [4 5 6] [7 8 9]])))
=> "1 2 3 4 5 6 7 8 9"
(str (interpose " " (apply concat [[1 2 3] [4 5 6] [7 8 9]])))
=> "[email protected]"

meow21:12:51

apply isn't lazy, str on a lazy seq doesn't do what I want. What am I missing?

roberto21:12:15

would lazy-cat help?

meow21:12:15

the problem is that calling str on any lazy seq, which is what clojure.data.xml is going to do, yields a string representation of the lazy seq object, not a string of the elements of the sequence, if that makes sense

meow21:12:39

you get "[email protected]" instead of "1 2 3 4 5 6 7 8 9"

meow21:12:51

I may have to create a custom deftype with an implementation of Object.str that does what I want.

kopasetik23:12:50

@alanforr: Used different names instead of first and rest and it still didn’t work

noisesmith23:12:40

@meow: apply is exactly as lazy as the function you are applying

noisesmith23:12:56

that wouldn't return if it weren't lazy

meow23:12:04

I need to figure out in exactly what way clojure.data.xml is lazy when it comes to writing xml files.

fappy23:12:35

(oops you weren't talking about boot. yay scrollbar)