This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # admin-announcements (27)
- # beginners (12)
- # boot (168)
- # cider (4)
- # clojure (27)
- # clojure-art (3)
- # clojure-austria (1)
- # clojure-russia (10)
- # clojurescript (50)
- # datomic (10)
- # editors (2)
- # hoplon (51)
- # ldnclj (26)
- # liberator (1)
- # off-topic (7)
- # om (23)
- # parinfer (6)
- # proton (7)
- # reagent (17)
- # yada (1)
So that's worrying. This exactly the situation that imperative programming wins hands down for me. I can't figure out where some resource is being held but in an imperative loop I would know categorically when that might happen. The sequence abstraction is bleeding implementation details!
Serious point tho. For a language that prides itself on simplicity you need to know an awful lot about how core fns r implemented too avoid this. I suspect the real issue was an
into in the lib I was using around hbase
altho to be fair - laziness is the thing obscuring the causes of your prob, and lazy vs eager isn't a functional thing - there are certainly eager functional languages (tho i'm not aware of lazy imperative langs)
I meant the side effects of laziness itself or in this case the side effects of eagerness
Basically something in the lib I'm using is eager not lazy but I need to understand which fn's are eager and which lazy just to use them safely. With an imperative loop I would explicitly know all this
My point is you have to have detailed knowledge of the implementation details of the fn's to use them and I thought it was supposed to be simple.
> At first I thought I could just wrap ResultScanner iterator-seq in a take. That seemed to work on small tables. However on a large table (defined here as anything over a few thousand rows), the scan would blow up. It turned out that in order to minimize the number of network calls, a scanner by default will try to get all the matching entries and shove them into a ResultScanner iterator. For this use case, there’s isn’t a limiting criteria on a scanner, so a scan will try to “fetch” the entire table into memory. That’s one large ResultScanner object! No good. What to do?
I think party of the issue will be interfacing with hbase and how jangling that resource all the way through the driver works so I think you left simple a while ago