Fork me on GitHub
#clojure
<
2019-08-02
>
wei01:08:18

I have a lazy seq that's constructed from http calls. I noticed it always makes at least four calls no matter how few elements I take. any hints on what's going on?

hiredman01:08:16

it will depend a lot on how it is constructed

hiredman01:08:11

if you construct it from lazy-seq yourself or build it from a another lazy seq using map/filter/etc

hiredman01:08:04

a very common reason for that kind of thing is chunked sequences, which are process in chunks of 32 usually, but if you build it yourself using lazy-seq that is less likely

hiredman02:08:11

a lazy seq like

(defn f [i]
  (lazy-seq
   (let[a (inc i)
        b (inc a)
        c (inc b)
        d (inc d)]
     (concat [i a b c] (f d)))))
with the incs replaced with http calls would also exhibit that behavior

hiredman02:08:36

oh, actually, yeah, using concat anywhere can get that kind of behavior too

hiredman02:08:26

(I seem to recall concat doing something like that but the details escape me)

wei02:08:53

I'm using Stuart Sierra's unchunk function:

(defn unchunk [s]
  (when (seq s)
    (lazy-seq
      (cons (first s)
            (unchunk (next s))))))

(defn offsets [perpage]
  (unchunk (iterate (partial + perpage) 0)))

(defn get-all [api-fn perpage]
  (flatten
   (take-while seq
               (map #(api-fn {:limit perpage :offset %})
                    (offsets perpage)))))

(get-all #(get-messages account %) per-page)

hiredman02:08:54

never use flatten

✔️ 4
wei02:08:48

you're right, concat seems like the better choice here

hiredman02:08:47

but I don't see anything that immediately jumps out about the 4

wei02:08:18

fwiw changing flatten to concat fixed the issue for me. guess flatten doesn't work well with lazy sequences but not sure why it realizes 4 either

wei02:08:06

actually I take that back.. concat works to stop the 4 calls but it gives me a seq of seqs. mapcat gets me the right behavior but still incurs the 4 calls at the beginning

(defn get-all [api-fn perpage]
  (take-while seq
              (mapcat #(api-fn {:limit perpage :offset %})
                      (offsets perpage))))

wei02:08:13

found it: https://clojuredocs.org/clojure.core/mapcat

;; mapcat always evaluates the first 4 arguments.
;; it can be solved avoiding 'apply' to handle varargs

ghadi02:08:37

There is an alternative expression in that ticket

wei05:08:43

thanks! that's very useful. coincidentally I found the same function just now in this thread: https://groups.google.com/forum/#!topic/clojure/c55xIFS9YGM

roninhacker03:08:29

Watching Alan Kay talks...if TRON were to really happen, AK is my Kevin Flynn candidate. Beard? Jazz? Coincidence? I THINK NOT

quadron12:08:44

here is a function that is supposed to turn xf -> (xf but only invoked once) in other words the returned xf is supposed to "shut down" after first invocation is this correct? is there better way to implement this?

thumbnail12:08:48

are you trying to do memoization?

deadghost14:08:17

Are there any functions I should look at for diffing two maps and returning a map containing changed key value pairs?

deadghost14:08:50

@scknkkrer Looks like exactly what I want, thanks

Aleksander14:08:21

Hi all, are there any good ways to profile loading of a namespace? Seems that adding a require to user.clj makes starting a REPL much longer and would like to work out why …

dpsutton14:08:48

there's a known and fixed issue with this. Have you tried clojure 1.10.1 to see if this alleviates everything?

alexmiller14:08:49

this is a java regression that occurred around Java 8u202 and 11.0.2. Fixes have been going in to newer versions, particularly Java 13, but Clojure 1.10.1 includes a mitigation for it.

Aleksander14:08:58

I was using 11.0.2 because it seemed that the latest version of org.openjfx.* libs required by REBL is 11.0.2 Is there a way to check what Java version have fix for the regression?

alexmiller15:08:16

I don't know that it has all flowed back to 8 and 11 yet

alexmiller15:08:32

it's a fairly complex problem that has spawned a whole cluster of changes

alexmiller15:08:08

I think they believe it's largely fixed in Java 13 :) but it's hard to tell what the backport status is

g15:08:32

is there anyway to inject a require statement globally from a lein profile, meaning the function is available in any namespace?

g15:08:45

i tried that, but it seems it only gets loaded into the initial namespace

alexmiller15:08:47

the short answer is, no

alexmiller15:08:40

the longer answer is that various tools and libs have done magic hacks to make this work at times, but they are magic hacks and in general, it's probably better (for working with your teammates) to just not do that. :)

jjttjj16:08:13

I have code which is pretty much functionally a record and protocol, but I'm just using a map for now instead. I'm not 100% sure on the api I want to expose yet and a map just seems like less of a commitment. Should I definitely make this a record+protocol or is this an ok choice for something i want to release an alpha version for https://gist.github.com/jjttjj/29997ff463df1cbe5b5036becbc171dc

benoit16:08:43

Based on the code you linked, it seems that the consumers of your API are supposed to get a client and then call the functions in this namespace on it. If this is the case then the representation of a client doesn't really matter since consumers are not supposed to look into it.

👍 4
alexmiller16:08:51

a map is much less of a commitment, so therefore is a good place to start. if you move to a record, it still satisfies that contract

alexmiller16:08:30

unless you have a lot of control/input on the consumers of this api, I would only make it depend on a map

rgorrepati22:08:52

Hi, Can you please tell me why the following wouldn’t work?

lilactown22:08:42

it sounds like something is wrong with printing that particular typee

hiredman22:08:28

it has to do with how the compiler embeds objects in bytecode

hiredman22:08:13

the result of (read-string "#db/fn {:lang \"clojure\" :params [] :code \"\"}") might shed some light

rgorrepati22:08:00

I ge the same error when I do read-string

hiredman22:08:37

oh, then something is broken in the tagged literal reader for #db/fn

rgorrepati22:08:10

I don’t get any error when i do read-string … I forgot to include the form in a string

hiredman22:08:17

(type (read-string ...)))

hiredman22:08:04

maybe ask in #datomic

rgorrepati22:08:57

yep, will do.. thanks

hiredman22:08:52

my guess is the #db/fn tag literal and datomic.function.Function is intended to be data, and not compiled code, which is is a super fine line to walk in clojure and would have terrible ergonomics

hiredman22:08:45

I can trigger the same exception

hiredman22:08:48

user=> (defmacro f [] (delay 1))
#'user/f
user=> (fn [] (f))
Syntax error compiling fn* at (REPL:1:1).
Can't embed object in code, maybe print-dup not defined: [email protected]
user=>

hiredman22:08:32

user=> (eval (list 'fn [] (delay 1)))
Syntax error compiling fn* at (REPL:1:1).
Can't embed object in code, maybe print-dup not defined: [email protected]
user=>

seancorfield22:08:33

@(f) fails too...