Fork me on GitHub
#beginners
<
2015-09-10
>
meow00:09:58

I'm pretty sure this doesn't need to be written as a tail call recursion because of the use of lazy-seq.

meow00:09:51

I just worked on some code that uses lazy-seq in a similar fashion and if you look at the source code for clojure core you will see quite a bit of use of lazy-seq this way.

meow00:09:19

I would change (when (not= [] coll) to (when (seq coll))

meow00:09:48

When you look at the use of lazy-seq in the source for clojure you have to sort of ignore all the testing for chunks, but partition doesn't have any chunk optimizations going on so its a pretty good example of what I'm describing: https://github.com/clojure/clojure/blob/clojure-1.7.0/src/clj/clojure/core.clj#L3051

meow00:09:58

You can also see that (seq coll) is idiomatic, along with (when-let [s (seq coll)]

meow00:09:37

@paulb: I think (not= [] coll) will always be true - (seq coll) is the idiomatic test for an empty collection as it will return nil

meow00:09:10

not the case, actually, but I would still use seq:

(def baz [1 2 3])
=> #'user/baz
(not= baz [])
=> true
(def foo [])
=> #'user/foo
(not= foo [])
=> false

escherize01:09:56

I mostly agree with what's been said. seq is surely the idiomatic way to check for empty-ness. I wonder why it works better from a performance standpoint

meow01:09:23

@escherize: I'm going to guess that it's a datatype issue and that with the larger coll given on the website that the type of coll is being changed under the covers by clj and then (not= [] coll) is returning true even though it is empty

meow01:09:57

otherwise, I have no idea why it times out

escherize02:09:28

@meow: That could be the case, esspecially given:

(map (partial not= []) [#{} {} [] '()])
;;=> (true true false false)

escherize02:09:00

but I don't think they would make it a set or a map

meow02:09:58

@escherize: it's actually really simple:

(not= [] (seq []))
=> true

meow02:09:11

rest calls seq on its argument - it even says so in the docstring

meow02:09:02

(type [])
=> clojure.lang.PersistentVector
(type (seq []))
=> nil
(not= [] nil)
=> true

meow02:09:55

(type (seq [])) is kind of dumb because (seq []) returns nil so that's the same as asking (type nil) but you get my point...

meow02:09:03

eventually mm is getting called with f nil

meow02:09:28

and nil is not equal to []

Alex Miller (Clojure team)02:09:42

sets, maps, and sequential collections are each "equality partitions" and won't compare as equal

Alex Miller (Clojure team)02:09:00

calling seq produces a seq from any of them, pulling all of them into the sequential collection equality partition

Alex Miller (Clojure team)02:09:51

seq is not (necessarily) the fastest way to check for emptiness - really depends on the scope of what you are checking (whether you are potentially checking for any collection or just a counted collection, etc)

meow02:09:30

@alexmiller: got a minute for a stateful transducer question?

meow02:09:05

I have a process and in one step I want to check an item to see if it is a list (that's just a convention I've chosen) and if so, it needs to have two elements, the first is like any other item and the second is extra parameter data of some kind that I want to store separately, so I'm storing it in a map as state in the enclosing function, like this:

(defn split-module-params-xf
  "Returns a stateful transducer that accumulates a map of module parameters,
   where the key is the index position of the module in the resulting word."
  []
  (fn [rf]
    (let [data (volatile! {})
          index (volatile! 0)]
      (fn
        ([] (rf))
        ([result] (rf result [@data]))
        ([result input]
         (let [v (vec (for [[n, module] (map-indexed vector input)]
                        (if (list? module)
                          (let [[module param] module]
                            (vswap! data assoc (+ @index n) param)
                            module)
                          module)))]
           (vswap! index + (count v))
           (rf result v)))))))

meow02:09:03

in the finalizing step I'm simply adding the deref'd map into the result.

meow02:09:38

So my question is basically if this seems like a reasonable approach.

meow02:09:49

It works as intended.

meow02:09:58

I need to keep this extra parameter data stored separately from the module, to keep things simple for the consumer of this process.

meow02:09:10

Plus this data is optional and I can be processing millions of items so I need an efficient storage.

meow02:09:21

Something like a sparse array.

meow02:09:37

I just kept it simple and used a map keyed by index.

meow03:09:39

The only gotcha is I allow additional transducers to be supplied and if one downstream removed an item then this data map would be screwed.

Alex Miller (Clojure team)03:09:06

for is going to construct a lazy seq which you then put into a vector

Alex Miller (Clojure team)03:09:25

you could make a vector directly by replacing (vec (for ...)) with mapv

meow03:09:30

Not a rousing endorsement - if the code sucks please say so... simple_smile

meow03:09:46

mapv - good idea

Alex Miller (Clojure team)03:09:59

it's just hard for me to understand the scope of what's being done

meow03:09:11

its my L-system code, revised to be transducerized to the max: https://github.com/decomplect/ion/blob/master/src/ion/ergo/l_system.cljc#L256-L274

Alex Miller (Clojure team)03:09:16

or instead of mapv, you could nest another transducer operation using (into [] ...)

Alex Miller (Clojure team)03:09:53

you could then chain map-indexed with the operation with the inner map operation?

Alex Miller (Clojure team)03:09:32

does map-indexed have a transducer arity?

Alex Miller (Clojure team)03:09:09

oh, yeah. I wrote it. :)

meow03:09:53

so this is a rewriting system that I've coded, so typically the replacement value has more items than what is being replaced, which is why I use that inner map-indexed

Alex Miller (Clojure team)03:09:33

you're creating vectors for every element of input when you don't know yet whether or not it is a list so it seems like you are potentially doing speculative work there

meow03:09:18

input will always be a vector, what I don't know yet is whether the item in that vector is a list

Alex Miller (Clojure team)03:09:48

oh right, but you always need that vector regardless, so never mind

meow03:09:52

so a common case would be the value :A gets replaced with [:A :B :C]

Alex Miller (Clojure team)03:09:23

if you're really trying to squeeze perf out of that inner loop, I would think about ways to refactor to avoid the destructuring via either reduce or loop/recur

meow03:09:11

or ['(:A {...}) '(:B {...}) :C]

meow03:09:01

to state my problem more generally, I need an effective way to associate parameters with replacement values - I just made up the idea of using a list, since it isn't a commonly used datatype, with a pair of values

meow03:09:02

I'm open to any other mechanisms that would work better

meow03:09:02

I can't imagine a need for anything other than a pair

meow03:09:33

especially since the first element can be anything that can be a valid key in a map

meow03:09:53

and the second element can be anything, and certainly you can do a lot with a map or nested maps

Alex Miller (Clojure team)03:09:19

for very specialized cases, then you could use a defrecord or a custom deftype that implemented the needed interfaces

Alex Miller (Clojure team)03:09:48

with records, you'd be constructing small objects and using field accessors, which is highly optimized on the JVM

meow03:09:02

I will note that the pair has to be stored in something unique, because an item without this parameter data could be a 2-element vector, for example

meow03:09:31

so I need something that tells me to look for parameter data

Alex Miller (Clojure team)03:09:32

yeah, you get a type too of course so that's useful

meow03:09:20

I haven't done anything with records, and was wondering when I would need to learn them

meow03:09:04

It also needs to be easy for the writer of the "grammar" or replacement rules to specify these replacements

meow03:09:14

like this:

(defn parametric-contextual-system-example
  []
  (let [axiom ['(:A {:age 0})]
        rules {:A (fn [g w d i m]
                    ['(:B {:age 0})
                     :-
                     (list m {:age (inc-age d i)})
                     :-
                     '(:B {:age 0})])
               :B (fn [g w d i m]
                    ['(:A {:age 0})
                     :+
                     (list m {:age (inc-age d i)})
                     :+
                     '(:A {:age 0})])}]
    (parametric-contextual-system axiom rules)))

(comment (take 5 (parametric-contextual-system-example)))

Alex Miller (Clojure team)03:09:52

if you had (defrecord P [a b]), then you'd just replace list above with ->P

meow03:09:39

(->P m {:age (inc-age d i)})

meow03:09:43

like that?

meow03:09:12

that would work simple_smile

Alex Miller (Clojure team)03:09:31

there's actually an example of creating a Pair deftype in Clojure Applied, by the way

meow03:09:51

sweet - I really need to get that book

Alex Miller (Clojure team)03:09:25

well, I may be biased, but I agree :)

meow03:09:46

lol, yeah

Alex Miller (Clojure team)03:09:26

I'd really recommend buying 5 or 10 and giving them away

Alex Miller (Clojure team)03:09:43

you could put them under chair legs too

Alex Miller (Clojure team)03:09:56

or use them as a paperweight

meow03:09:14

I hear you simple_smile

meow03:09:47

I really am interested in the book, based on the TOC

meow03:09:41

as far as my trick of tacking on the @data map at the end of the transducing process, is that reasonable?

meow03:09:10

my parametric process then looks like this:

(letfn [(process [w d]
                    (lazy-seq
                      (when (seq w)
                        (let [result (transduce (get-xf) (get-rf) w)
                              word (butlast result)
                              data (peek result)]
                          (cons [word data] (process word data))))))]
     (process jumpstart {}))))

meow03:09:17

and if I make the parameter splitting always come after any user-specified transducers (comp (f-xf) (split-module-params-xf)) then I don't have to worry about a downstream transducer messing up the indexing.

meow04:09:38

@alexmiller: tyvm for all the help - I really appreciate it. simple_smile

sergeylaptev11:09:16

Hello everyone. I have question

sergeylaptev11:09:39

I use RabbitMQ, and write little worker which use it

sergeylaptev11:09:18

I start it via

lein run -m workers.yandex-market

sergeylaptev11:09:57

am I should make .jar, .war file for it? Or should execute from source

sergeylaptev11:09:14

There will be more workers later

beppu12:09:44

@sergeylaptev: This article is for webapps written in Clojure, but you could adapt this for your RabbitMQ workers. http://adambard.com/blog/deploying-a-clojure-web-app-on-digitalocean/

beppu12:09:44

It shows you how to use lein uberjar and then set it up as a service for Ubuntu.

sergeylaptev12:09:50

am I should compile every worker as dedicated .jar?

beppu12:09:35

Yes. It's a nice way to make a deployable artifact that can be easily copied around.

beppu12:09:10

If you had to deploy your workers to a lot of servers, I think it would be convenient to have one .jar file to copy around.

sergeylaptev12:09:07

Both worker and my app with compojure use "generate-yml" function

sergeylaptev12:09:21

Are they can use it after decoupling?

sergeylaptev12:09:36

As I understood, all my workers will be dedicated in separate repo, and compiled into one .jar?

beppu12:09:57

I would put the workers and your web app in different repos. You'll have to make your generate-yml function available to both somehow.

sergeylaptev12:09:13

I learn Clojure about 2 weeks, and can't understand this schema:)

clojuregeek17:09:26

Any better way to create a function to return and return a collection of a number of things?

(defn make-sample-data [how-many]
  (take how-many (repeatedly #(g/generate ApiResult))))

clojuregeek17:09:40

g/generate is prismatic schema creating a map

gigasquid17:09:00

repeatedly takes a n arg too

clojuregeek17:09:19

(defn make-sample-data [how-many]
  (repeatedly how-many #(g/generate ApiResult)))