Fork me on GitHub
#clojure
<
2016-01-23
>
Alex Miller (Clojure team)00:01:59

That's where things drop into the compiler

bradford00:01:40

Oh, I think I figured out my problem. I do a depth-first traversal then build the core.async channels 'upward'. Neato!

seancorfield01:01:04

Q re: clojure.core.reducers/fold — under what circumstances does it decide to use fork/join vs just doing a regular reduce?

seancorfield01:01:58

I was playing with this using a combinef that actually prints when it is called and so far I’ve only managed to see it called with zero arguments to get the seed value, never to combine the result of reducing multiple partitions...

seancorfield01:01:01

Never mind, I think I was just confused about reducible collections…

seancorfield01:01:36

Here’s why I got confused:

(r/fold add-r + (into [] (range 1000)))
;;=> produces two partitions and combines them — add-r prints the addition
(r/fold add-r + (into [] xf (range 2000)))
;;=> produces two partitions — xf filters on odd? so the range needs to be twice as long
(r/fold add-r + (eduction xf (range 2000)))
;;=> only one partition…

seancorfield01:01:38

Even if I do (r/fold 100 add-r + (eduction xf (range 200000))) I only get one partition. Since eduction produces a reducible collection, I was expecting that would still do some parallelization.

Alex Miller (Clojure team)02:01:21

the inputs must be maps or vectors and it chunks (by default

Alex Miller (Clojure team)02:01:02

eduction won't work for this (currently)

jaen03:01:21

@jonahbenton: kinda late reply (sorry, had to run) but anyway - It's basically event sourcing. I have an event store component which stores raw events as they happen due to - say - the user interacting with the system, for example something like {:event/type :created, :event/version 1 :aggregate/type :user :aggregate/id #uuid ... :event/payload {:username "test" :password "some hash"}} that are returned as a stream matching a given filter (by id, timestamp, type, whatever) and can be then folded into a representation of the current state of an aggregate by a repository component. The fold might need some minimal logic, like for example constructing derived values like means or something from the event data, hence the need for event handlers. I'm recently thinking if even that is not too much logic in event sourcing, but the same issue would be there for command handlers as well (things that validate user input and possibly persist events into the event source).

jaen03:01:29

Hope it makes sense.

nowprovision03:01:02

@Jean multimethod dipsatching on :event/type?

seancorfield05:01:22

I had a feeling that was the reason in the end. I look forward to parallel transducer operations.

jaen10:01:54

@nowprovision: yeah, but those can't be used with components effectively, since they are globals, see https://clojurians.slack.com/archives/clojure/p1453478907014726 and on. But as far as I understand a patch to decouple def-ing multimethods from creating them is welcome, so I'll put one together soon hopefully.

bcoop71317:01:57

any suggestions on getting a backend websocket server going? looking for more of a small library than a http://socket.io equivalent.

jaen17:01:10

Sente is pretty cool, but if you want small

jaen17:01:25

Then webservers like immutant or aleph expose websocket streams directly

rm17:01:21

oh sorry, I read it wrong, you're not looking for http://socket.io :) well, http-kit supports websockets too

jaen17:01:07

Sure, but it's comparatively less maintained these days, I think.

bcoop71317:01:07

aleph looks interesting, somehow never came up in my googleing

rm17:01:04

jaen, can I ask about web-servers? Why http-kit is not maintained? Because I use it, but only because it was on first lines in google (and it's ok in general). Why, for example, aleph is better?

niwinz17:01:18

It comes with a lightweight sente like approach https://funcool.github.io/catacumba/latest/#postal for client-server communication both for RPC and streams.

jaen17:01:15

@rm: a) as you probably can guess by I think, it's just my impression basing on how the commit activity graphs look for http-kit and how it looks immutant and aleph. It seems ptaoussanis stepped in to pick up some slack recently, but in general you can see it doesn't seem as active a project as the other two, b) it doesn't support SSL, immutant and aleph do IIRC, c) immutant can be better if you want to have additional services that come in a package (queues, schedulers, distributed transactions, though that last thing is kind of an ewww antipattern IMO) and ability to easily deploy into a Wildfly cluster; aleph can be better if you're interested in your webserver being based on a pervasive abstraction over asynchronous data streams.

jonahbenton17:01:32

http-kit's author (@shenfeng) unfortunately hasn't had much time to maintain http-kit recently. In an attempt to help out I'll be doing issue triage, accepting minor/obvious PRs, etc. My own time's pretty limited so contributors welcome: looking for pull requests, feedback/ideas, and help dealing with GitHub issues, etc. Please contact me if you'd be interested in lending a hand. Thank you! - @ptaoussanis

jaen17:01:45

I, for one, currently use immutant because I really like the additional libraries it gives for free-ish.

jonahbenton17:01:06

@ptaoussanis is an amazing maintainer

jaen17:01:10

Not saying otherwise, but the point still stands that the project is languishing. I guess he stepped in because originally sente was http-kit only, so he must be invested in this webserver.

jaen17:01:31

@niwinz - aware of your library, but I didn't yet have any need to go full async; I would certainly evaluate both it and aleph should I need to move in that direction.

lvh18:01:59

Yeah, I use http-kit because sente; at some point I should probably just make it use aleph

lvh18:01:05

I like aleph because ❤️ netty, and ❤️ manifold

jaen18:01:07

I think there currently are adapters for http-kit, immutant and nginx-clojure; I remember some issue re: aleph support, but it doesn't seem to have resulted in an adapter in the end - https://github.com/ptaoussanis/sente/issues/124

jaen18:01:05

There's some new discussion on an adapter it seems.

tomoram19:01:41

hi, brand new here - been dabbling with clojure for a few years now - love the language but still get baffled by package dependencies

tomoram19:01:15

can any please explain to me what [org.lwjgl/lwjgl-platform "3.0.0b”] is not found, yet it’s listed on http://maven.org?

jjttjj19:01:47

I have a bunch of events coming in which i shove onto a chan, which then has mult called on it, and the mult is a member of a record. I have a protocol which has one function response-chan which just passes it's args to async/chan and then tap the record's mult and returns the chan. dumb questions: 1) is this a bad idea for any reasons i'm missing? 2) I'm struggling to name the protocol for some reason. ResponseSource? ResponseManager? ResponseChanGenerator?

tomoram19:01:49

I’m guessing it’s something to do with os versions in the file name but it would be great to understand exactly what’s going on - thanks

niwinz19:01:48

@tomoram: very strange it is also happens on my environment :S

tomoram19:01:22

@niwinz: thanks for checking, I’m assuming there’s a good reason but I don’t really know enough about packages for JVM languages - trying to understand it all a bit better

niwinz19:01:14

I do not find any info about that 😞

dm319:01:09

it's a parent pom

dm319:01:13

you probably want a jar

dm319:01:21

[org.lwjgl/lwjgl "3.0.0b"]

tomoram19:01:45

@dm3: that sounds like it makes sense but I don’t really understand it. Looking on http://maven.org I can see it lists a pom and 3 os specific jars. Can I instruct lein to fetch one of those jars somehow?

tomoram19:01:10

@dm3: I’m assuming that’s what I’m trying to do 😉

jonahbenton19:01:09

@tomoram the artifacts on maven require an extra "coordinate" called a classifier

jonahbenton19:01:16

those particular artifacts do

jonahbenton19:01:34

[org.lwjgl/lwjgl-platform "3.0.0b" :classifier "natives-linux"]

tomoram19:01:02

@jonahbenton: ah thanks, let me give it a try!

tomoram20:01:31

@jonahbenton: that did it, many thanks!