Fork me on GitHub
#beginners
<
2017-12-27
>
Drew Verlee00:12:50

I assume you can reduce over a lazy-seq without any trouble. This code seems to running a bit longer then i would think though (about 3 mins so far)... The file is 10gigs though.

``````(with-open [in-file (io/reader (io/resource kf))]
(reduce
(fn [coll {:keys [customer]}]
(conj coll customer))
#{})
count
println))
``````

Drew Verlee00:12:05

thats semantic-csv

noisesmith00:12:09

you could replace the reduce with `(into #{} (map :customer))` (leaving the rest the same) and that should be faster

noisesmith00:12:44

since your reduce is equivalent to mapping :customer across the input

noisesmith00:12:49

(then putting in a set)

Drew Verlee00:12:17

oh, right. Why would that speed things up?

noisesmith00:12:05

because into uses a transient which is marginally faster for building collections, and it's likely (map :customer) inlines better (edit - for larger collections the speedup can be better than marginal)

Drew Verlee00:12:03

is it possible to leverage multiple threads over chunks of the lazy sequence? Kind of like a map reduce?

Sabbatical201701:12:33

What's the idiomatic way to do a `map` that has some state? I have a function `f` that basically takes `x` and returns `y`, but also takes in a "state of the world" `s` and returns a "changed state of the world" `new-s`, so that `(f x s)` is `[y new-s]`. Given an initial state `s0`, and some vector of `x`s like `[x0 x1 x2 ...]`, I would like to compute `y0` which comes from `(f x0 s0)` being `[y0 s1]`, then `y1` which comes from `(f x1 s1)` being `[y1 s2]`, and so on for `y2`, `y3`, etc. (Also I would like to get the final value of `s`.)

Sabbatical201701:12:21

If it wasn't for the `s` stuff, this would just be `(map f xs)`.

Sabbatical201701:12:32

There is probably some kind of Haskell higher-order function that does this, but of course I am interested in doing this in idiomatic Clojure.

noisesmith01:12:37

that's a reduce

noisesmith01:12:46

Sabbatical201701:12:06

I thought that maybe I can do something with `reduce` or `reductions`, but I could not see a nice way to arrange it.

noisesmith01:12:38

you can use a hash map as your accumulator, and hold your state-of-the-world plus your ys under two keys

Sabbatical201701:12:57

@noisesmith Ah yes, that works very nicely, thank you!

Naylyn04:12:33

What's the most computationally efficient way of mutating elements of a Java array? So far I've tried `doseq` + `aset` and it seems terribly slow.

donaldball04:12:23

I would guess use `loop/recur` to walk the index and make sure you type hint and deal with boxes correctly.

donaldball04:12:59

`dotimes` is probably a better higher level form than `loop/recur` on second thought

noisesmith05:12:45

@naylyn.gaffney you could check out amap and areduce - you can run into pitfalls with numeric boxing with loops if you are not careful but those functions abstract that stuff a bit

noisesmith05:12:02

also be sure to check for runtime reflection and numeric boxing in general http://insideclojure.org/2014/12/15/warn-on-boxed/

Naylyn05:12:38

Having no experience with Java prior to Clojure it feels like there are a lot of landmines I need to avoid.

noisesmith05:12:32

for performance of numerics there are a bunch of gotchas - honestly it's often easier to write java code and call it from clojure instead

noisesmith05:12:09

but for correctness of programs that use a lot of threads (and readability of that code) - clojure can really excell

noisesmith05:12:40

the reflection and numeric boxing are a tradeoff for using a language that can't be effectively statically analyzed - there are benefits too (repl based development, being able to define things piece by piece without restarting the whole thing) but it makes optimal numeric performance tricky sometimes

Naylyn05:12:45

yeah I'm seeing that, it's starting to feel like I'm writing something closer to java than clojure at this point

Sabbatical201716:12:23

What causes `Math.abs` to be available in my Clojure code? In one of my source files, the REPL says `CompilerException java.lang.ClassNotFoundException: Math.abs`. The same thing works in another file though. I have no idea why. I have not done anything to refer (or anything like that) `Math` in either file.

Alex Miller (Clojure team)16:12:11

All classes in java.lang are automatically imported for you in every namespace

scriptor17:12:01

@sabbatical2017 it should be `(Math/abs -123)`

Sabbatical201717:12:35

@scriptor @alexmiller Thank you! I've been blind to that `Math.abs` instead of `Math/abs` ... too much Python exposure š

Drew Verlee18:12:55

Does anyone have any good resources about modeling the abstractions of your system as a FSM or maybe something like state Charts? Iām curious about the idea if creating a more contractual model for designing systems and modeling them as a FSM seems like one way todo this. For example a simple game might have something like `player-one -> player-two -> game-over` It wouldnāt capture all the possible things that could happen to the program but would be a means to express the business domain.

noisesmith18:12:55

@drewverlee this might not be what you are looking for, but petri nets are a formal language for designing asynchronous stateful systems that has a well defined semantics where you can make proofs about eg. whether it halts or deadlocks - similar to fsm but with no shared clock

Prakash18:12:43

I think reduce-fsm does somehting simiar - https://github.com/cdorrat/reduce-fsm

Drew Verlee18:12:52

Wow lots of responses. Iāll take a look at these in a bit. Thanks!

jgh22:12:23

how would I take this object and turn it into something I can access the data of `#object[clojure.core\$future_call\$reify__8097 0x3f7533a9 {:status :ready, :val 0}]`?

`(def x (future 10))` --> `@x`

That will return the thing that `x` is defined to be

You can also use the `deref` function explicitly, `(deref x)`

noisesmith22:12:29

you can also use `realized?` to see if it is done yet (otherwise deref will block until it is)

noisesmith22:12:53

you can also provide an optional extra arg to deref to tell it how long to pause and wait for it to complete before giving up

jgh22:12:08

itās definitely been realized since this is part of a result on a blocking function call, im not creating the future

jgh22:12:21

and in the repl calling (realized?) returns true

noisesmith22:12:23

right, just mentioning that in general that is a concern

jgh22:12:41

im just not sure how to get at the data heh

noisesmith22:12:51

like admay said - @ aka deref

A common pattern with `realized?` is `(if (realized? x) @x ( ... ))`

jgh22:12:13

weird im just getting 0 when i do deref

jgh22:12:22

oh is the status part of the future itself

noisesmith22:12:25

thereās also future-cancel which will stop it (if it is sleeping or waiting on IO at least)

noisesmith22:12:48

but thatās a no-op if itās realized iirc

jgh22:12:29

ah yeah so @ is doing the trick, i assumed :status was part of the return data for the future but i guess itās an internal thing

jgh22:12:50

thanks guys š

justinlee22:12:07

Iām trying to figure out how `go` and `<!` behave in the context of the JS environment. The `go` documentation say that `any visible calls to <!, >! and alt!/alts! channel operations within the body will block`. First: what does āvisibleā mean? Second, does a function which uses a `go` block return immediately or does it only return once the `go` block finishes?

tbaldridge23:12:46

It returns a channel immediately.

tbaldridge23:12:00

No values will appear on the channel until the go completes.

tbaldridge23:12:50

@lee.justin.m The macro rewrites the body of the block into a state machine that with attach callbacks to the channels wherever theres a >! or <!

justinlee23:12:29

okay so visible literally means textually visible because go is a macro

tbaldridge23:12:43

although the "visible" bit is a bit strange, since <! and >! can be inside a macro and go will still see them

tbaldridge23:12:49

but it doesn't dig into function calls.

justinlee23:12:10

i see. so basically one needs to pass the channel around until ready to consume, then use it with a go block and the <! macro.

tbaldridge23:12:38

Well if you just want to consume, then use put! and take!

justinlee23:12:58

oh. i was just following the readme from cljs-http

tbaldridge23:12:59

the go blocks run in a task queue in JS. When the browser isn't busy the pending go blocks will run

justinlee23:12:30

iām trying to map all this stuff to promises and/or callbacks and having a hard time of it

tbaldridge23:12:22

Well, it's a lot of the same sort of thing. Let's say you have `(http-get url callback)` in JS.

tbaldridge23:12:31

You could write this as `(http-get url (fn [result] (reset! a result)))`

tbaldridge23:12:56

Or we could put that value on a channel: `(http-get url #(put! ch %))`

tbaldridge23:12:45

but we may want to notify something when that put is done: `(http-get url #(put! ch % done-cb))`

tbaldridge23:12:05

So what core.async is doing is transforming calls to `>!` and `<!` into calls to `(put! ch val done-cb)` and `take!`

tbaldridge23:12:29

And a simplified level, when we have a callback we have to run later, core.async is calling `(js/setTimeout .. cb)` to queue it up

justinlee23:12:58

I see. So ergonomically go blocks are more like promises or async/await and put!/take! are more like callbacks. The upshot seems to be that you need to use the right functions from `core.async` to interact with the channel (which of course makes total sense)

justinlee23:12:21

I get that they are not implemented the same, but thatās very helpful.

tbaldridge23:12:01

actually you're right, async/await in JS are very much the same sort of transform