I've decided that vim orgmodes agenda isn't good enough.

morning

Oi oi

saveloy

månink

I've always been scared of saveloys @otfrom, is that unreasonable?

@mccraigmccraig seems perfectly sensible to me

@mccraigmccraig what about cocktail sausages?

saveloys are imo inferior to regular chip shop sausages

and by extension battered sausages

but nothing to be afraid of…

i dunno, inferior to regular chip shop sausages seems like a reasonable foundation for gibbering terror

fair

I mean given that the average chip shop sausage only contains fractionally more meat than a veggie sausage and… well, where that meat comes from you probably don’t want to know…

now battered quorn sausages… that’s an idea I can get behind

I like to think of sausages as slightly more environmentally meat - it uses up more of the animals we kill, and kills us off faster. Win-Win

hmm... I'm thinking about core.async again today, not for concurrency as such, but as a way of decomposing systems. I build a lot of data cleansing pipelines where I then want to do some report on things and then do some calculations with some proportion of the data. Doing this with channels that have transducer functions, mults to have multiple things processing that data to create different reports and then core.async/transduce functions at the end feels like a really good decomposition to me. I know it isn't the fastest necessarily, but it is fast enough and with things like pipeline-blocking I can get the parallelism I'm after too and heat up those cores (so I can at least save wall clock time). I'd be interested in hearing from people whether or not this is a good or bad idea and what alternatives they'd propose (if any).

Not considered it, but it certainly sounds like it's got legs

perhaps from having too many run-ins with the node runtime, but my hand-wavey observation would be one does not simply do async

as in, if you’re looking to window over the data, there’s probably a way of sampling that still involves a good separation of concerns but doesn’t fundamentally change the expectations around return time

adding async also more strongly introduces the dimension of time into the code and ’tis not to be trifled with

@alex.lynham I think having go blocks sitting around and receiving events does that, but bolting channels together doesn't feel very async to me, which is partly why I'm a bit nervous. I'm really using it for things that are embarrassingly parallel

(with the exception of the bits where I'm using the pipeline-blocking, but even then we're not really pushing that hard on asynchrony)

it feels a lot more like using kafka for pushing this kind of thing around (w/o using kafka). It does make me wonder if I should just do some sugar around some java queue stuff, but then I'd lose things like pipeline and having all of my extra filtering and transducing functions making good use of machine resources.

I'd be doing this in clojure rather than in clojurescript and all on the jvm.

(so I've got real threads etc)

okay, well if it doesn’t need to be platform portable then that does make things easier, and that is a fair point about channels

the last large thing I did was heavily evented python, but using lambda so the async happened between these tiny lil handlers - so you sort of had the benefits of async while writing synchronous code… which I guess you can kinda get the way you’re describing

my workflow atm is to prototype things in a rich comment block w/transducers and then create the actual app itself my moving those transducers and reducing functions to core.async

interesting

I think the only reason I'd want to write a go block would be to tweak some atom, but even then I'd like to structure things so I didn't have to do that (like say processing the data more than once, which is fine with the data sizes I'm dealing with)

Morning

@jasonbell I'm interested in your take on the above as we spiked out those ideas together and I think you are in a team with different opinions now. 🙂 https://clojurians.slack.com/archives/C064BA6G2/p1545993724119400

@otfrom Let me think of a remotely coherent answer and I’ll pop it here 🙂

I’ve actually been thinking about it on and off.

lol, sorry, didn't mean to put you on the spot

I need to find a way to publish what some of this would look like

No need to apologise at all. It’s an interesting point to make.

@otfrom Okay. Firstly yup, new team and a new way of thinking, some of which I’m still wrapping my head around but ultimately it’s settling in my head. The way I’m seeing async now is really centred around futures and promises, “do this! bind it to this! oh it’s messed up I better handle it“, regardless of whether it’s http or db calls etc. Anything that has an associated response time is better handled that way, having seen it in action I’m fairly convinced of it. How that would apply to what we spiked out in a core.async sense I’m not sure without trying. I do distinctly remember us talking at length about a stream of data and the distinction between something like a Kafka like stream and what we were working on which was more a transform stream (my words, no one elses). Therefore core.async with transducers makes perfect sense when reducing numbers on usage data for example, perfect for reporting. As every report requirement is a function then it makes sense to have the channel <- mult -< n taps. Data goes in once and the report’s own functions extract what it needs to construct the report, transducers do the lifting. All makes perfect sense to me. Annoyingly the times I needed it I forgot most of the code from the spike 🙂

For data processing, transforms and reporting core.async now feels like a logical choice.

morning

@jasonbell New company? Or just new team within the current one?

New gig started mid december

@jasonbell I'm not sure your use case fits in with what we were doing. I know a lot of people use core.async to do the whole "make a bunch of requests w/o blocking and then deal with the results" but I think things like promises and (and promise monads?) do a good job for that. I'd expect that it would look something like

(let [foo (promise call to remote service)
     bar (promise call to other service)]
  (do stuff with foo and bar))

yup

The spike we did made perfect sense from a data transformation point of view. It stuck with me.

But yes, where you are coming from in the CSV -> loader put onto channel <- mult <- tap makes perfect sense for multi level reporting. I think that got proved in the spike.

a little offtopic but @jr0cket that lenovo came and it’s actually immaculate, you can barely tell it’s been used. Just making an ubuntu USB now to get cracking. Might leave windows on for now so my other half can use that… think I’ll go with ubuntu for now as I’ve used that the most in the past & I doubt I can get a stable arch setup to test on my lunch break

though having gone for one in better condition trading off against SSD size means I need to get a new SSD and switch it in, but that’s easy… ish

okay so loads of the ubuntu experience has gotten slicker since 2013 (he said, dumb-ly)

the window management is absolutely leagues ahead of osx

@alex.lynham Great to here the 'new' laptop is working so well. I love the keyboardy-ness of the ubuntu desktop, you can even do window placement by holding the Super and arrow keys - left/right shows on that half of the screen, up to maximise, down to unmaximise.

Yeah, that stuff is super slick. I had to get an app (spectacle) for that on osx and even then it doesn't work as well as this does. Plus the thinkpad keyboard is everything the mac's is not. Obviously the mouse experience on osx is superior mind you but hey

Amazingly my osx emacs config pretty much worked out of the box as well