Fork me on GitHub
#clojure-uk
<
2019-12-13
>
otfrom09:12:44

don't mourn. organise.

๐Ÿ‘ 16
thomas10:12:10

well.... that was disappointing...

Samuel10:12:21

Buckle up for another 15 years of the tories ๐Ÿ˜•

dharrigan11:12:48

It's like the ghost of christmas past - all - over - again.

jiriknesl11:12:51

There are no other people happy for Tories victory here? Borisโ€™s ideology seems good for me: โ€œ[I am] free-market, tolerant, broadly libertarian (though perhaps not ultra-libertarian), inclined to see the merit of traditions, anti-regulation, pro-immigrant, pro-standing on your own two feet, pro-alcohol, pro-hunting, pro-motorist and ready to defend to the death the right of Glenn Hoddle to believe in reincarnation. โ€”Boris Johnson, 2011[131]โ€ (source https://en.wikipedia.org/wiki/Boris_Johnson#Political_views_and_ideology ) So I am pretty happy for the result of these elections.

picard-facepalm 12
Edward Hughes15:12:14

I would be, were it not for the Tory base and party apparatus being fiscally liberal and socially conservative when Boris Johnson's libertarian principles are as solid and unyielding as jello. The Tories were deporting second. third generation legal immigrants back to Jamaica as illegals, and their electoral success has been built on promising more handouts in one form or another to a population that cannot understand there are more peoples in Britain than just the English. Even discounting new arrivals.

Wes Hall11:12:32

@jiriknesl You're quite brave ๐Ÿ˜‰

๐Ÿ˜„ 4
Samuel11:12:57

I'm just concerned about the possibilities of a free Conservative government (no longer tied to another party to hold office), and the policies and studies underpinned by functionalist sociology

jiriknesl11:12:03

As somebody for free-market, tolerant, ultra-libertarian, donโ€™t care about traditions, anti-regulation, pro-immigrant, pro-standing on your own two feet, pro-alcohol, pro-hunting, pro-motorist and pro whatever that Hoddle guy wants, Johnson is better for me than Corbyn, right?

Wes Hall11:12:03

@jiriknesl I'm pretty "centrist", but my experience is that the "tech community" generally leans left. Not sure why. Just one of those things. I don't think there could have been any result that made me happy last night, but you have now taken my title as, "most right-wing person in the channel", which is some compensation ๐Ÿ˜‰

jiriknesl11:12:07

@wesley.hall I donโ€™t understand why this is happening, but itโ€™s true. Before UK, I lived in California and most people were left leaning. At least those who wanted to discuss politics.

dominicm12:12:20

Maybe bravest right wing?

๐Ÿ‘ 4
๐Ÿ˜„ 4
Wes Hall12:12:30

@jiriknesl Yeah, think you have hit the nail on the head there. You generally get more "social kudos" from presenting left wing opinions than right so I think it's more a matter of people who have more right-wing opinions tending to keep quiet until they get to the polling booth. Definitely true when it comes to channels like twitter. If you had been living exclusively in a twitter bubble you'd have expected a 98% vote share for labour last night. Some people do live in that bubble, and boy do they seem surprised today.

jiriknesl13:12:03

@wesley.hall yes, but itโ€™s not only when discussing. I know CTO whoโ€™s hiding right-wing opinions (heโ€™s conservative, we libertarians usually donโ€™t care) as he has seen hostility against conservatives in Silicon Valley. In fact, it is absolutely ok to disciminate against conservatives in SV even when itโ€™s absolutely unacceptable to discriminate for any other human attribute (even when you have way more control for your body shape then your ideology, fatshaming is verboten, bashing conservatives is part of mainstream narrative). But I donโ€™t want to bitch against left. And I hope people here are ok with different opinion as well.

Wes Hall13:12:17

@jiriknesl I generally find it hard to apply labels to my own opinions (it doesn't usually stop other people from doing it). I am pathologically addicted to steel-manning and playing "devils advocate", which, I think keeps me quite stable and towards the centre because the more I move in one direction the harder I look for merit in the opposing arguments. It's like how they build passager aircraft. The more it rolls in one direction, the stronger the forces pushing it in the other, and you get some stability. Twitter and other social media sources strike me as the kind of places that build fighter jets. Extreme instability. You move one inch in one direction and you find yourself in a bubble containing only people who are going to pull you further in that direction. I like my strategy better honestly, but it's not the most fashionable. Twitter is a dangerous thing.

Conor13:12:19

My impression is that right wing people in the US are more bonkers than right wing people here, so I am totally fine with discriminating against them. As a non-English person, I love you guys but I don't love your abililty to make choices that are good for anyone but rich people

๐Ÿ‘ 8
jiriknesl13:12:48

@wesley.hall Twitter, etc. is build for brevity. Brevity kills nuance. So even moderate people I personally know seem more orthodox on Twitter. And not only about politics, it is the same with opinions on prog.langs, frameworks.

Wes Hall13:12:33

@conor.p.farrell Everybody in this channel is a "rich person". I'd argue that, "middle class people not understanding working class people", is a big reason for the result last night. As somebody who grew up pretty working class, I never had to look too far to see this misunderstanding. Middle-class labour activists entirely conflated "working class" with "people who rely on benefits" on a very regular basis. I saw these same people regularly say things like, "the working classes have abandoned the movement!", like they are owed something. Jeremy doesn't understand this demographic as well as he thinks he does. The weatherspoons crowd were never going to respect a life long pacifist. Boris (for all his posh pomp), is closer to the "lovable rogue" figure. The Derrick Trotter of the election. The labour party campaigned by constantly pointing out that Boris is "a bit dodgy", but these people love a guy who is a "bit dodgy". They say that Trump is a model of what poor people think rich people are. The labour campaign seemed to be based upon speaking to "who middle-class people think working-class people are", and they totally missed the mark. Like... entirely. Dan Jarvis is the kind of guy that will win back this group. A major in the parachute regiment. Somebody not afraid of a bit of a fight. Walk into a Weatherspoons in any working class town on a saturday night and tell them you are an ex-para and people will buy you your first pint. Tell them that you think the IRA might have had legitimate grievances and you'll probably get your arse kicked. Policies matter of course, but so does respect...

jiriknesl13:12:23

Truth. Also, I have seen multiple times left parties who abandoned lower class and protecting big vulnerable groups (single mothers, etc.) and started pushing topics, that are important, but affect traditional left electorate way less often or on smaller scale (LGBTQ+ rights, climate change, gentrification)..

jiriknesl13:12:36

@conor.p.farrell well, both left and right have very noisy radicals, who are totally bonkers. But does discrimination against others help? In fact, it might radicalize those, who are not that fanatical. I think, both left and right are doing lots of things, that are good for all groups of people. We might disagree on that, but in my native country (Czech Republic), people went from socialism to capitalism/welfare state, and people are way better off. 99 % of people are doing much better now, not speaking nature. Plus plutophobia isnโ€™t something worth pursuing.

โž• 8
โค๏ธ 4
otfrom14:12:57

so, I'm about to do some data processing again using core.async.

otfrom14:12:43

I know @jasonbell will be familiar with it as I'm stealing ideas from things we did together on the closed (sorry, client only stuff) witan.disco repo. Looking forward to getting it public in witan.cic

otfrom14:12:36

this is after a quick chat with Alex Miller about how to use core.async in data processing, where it is about creating conveyors of data rather than dealing with concurrency

otfrom14:12:29

I know that @mccraigmccraig has done quite a bit of work using cats and manifold (from http://github.com/ztellman)

otfrom14:12:46

but I'm still a bit unsure about how what I'm doing might be different than yapster

Wes Hall14:12:01

Wish I had a pound for every time I have started building something with core.async and ended up removing it as it turned out the code was simpler and worked just as well without it. I only tend to reach for it now when I have a need to move the back pressure somewhere else, and think about it, pretty much exclusively in those terms.

Wes Hall14:12:47

More of a general comment I suppose, but you wanted to change the subject ๐Ÿ˜‰

otfrom14:12:36

seems a reasonable comment. I'm pretty happy with how disco turned out. It made it really easy to add new data products onto the event stream I was processing.

otfrom14:12:42

what were you doing with it?

otfrom14:12:08

and do you have an example of what went wrong and how you fixed it?

otfrom14:12:34

I've generally been slightly less happy when I've used in in UIs to solve concurrency issues

Wes Hall15:12:10

@otfrom Yes, entirely possible that the things I was doing didn't really call for it. I've been in a few instances where the core.async stuff just sort of started to shrink and disappear under refactoring. Usually when I try to do some kind of event bus style things or CRP in web application handlers. I haven't much used it on the client directly, but I think that's probably because I tend to use pre-rolled frameworks that handle a lot of the these things already (stuff like re-frame). I think I would find the async stuff a lot more useful and relevant in things like ETL processing where you often do want to move the back pressure around, but as a kind of "application framework system", i've always just ended up refactoring it out. There is also a very real possibility that I just don't understand how to use it properly ๐Ÿ™‚

otfrom15:12:50

there is that possibility for me too which is why I'm asking around ๐Ÿ™‚

Wes Hall15:12:38

@tcsavage Might have some thoughts on this, but seems to be offline at the moment. Wake up Tom!! ๐Ÿ˜‰

tcsavage15:12:13

I really like Claypoole (https://github.com/TheClimateCorporation/claypoole) for a lot of data processing stuff. It re-implements things like pmap and doseq on more explicitly controllable thread pools. I find it easier to understand than a lot of core.async code I've seen

otfrom15:12:10

what are the bits you've not liked in the core.async code bases you've seen?

tcsavage15:12:26

I can't really point to anything specific. It's more that core.async is used in many different and inconsistent ways across several projects because it's more of a low-level library of primitives.

tcsavage15:12:13

As opposed to something like claypoole which is less flexible but potentially easier to understand for people who might be relatively new to clojure

otfrom16:12:05

ok, that makes sense. The biggest issue I see when people use it is that they do go blocks everywhere and unless I need to update an atom for some reason I don't see any need for go blocks in the type of core.async I'm writing

otfrom16:12:36

I use chan, mult, tap, reduce, and into with lots of transducers on the channels

otfrom16:12:37

oh, and <!! and >!!

Wes Hall16:12:08

@otfrom I remember doing a fairly early experiment with core.async (though not the specific details), where I just spat sequential numbers into a channel and picked them up on the other side. Found it very easy to generate an effect of slow down and eventual crawl to a stop doing this most simple of things. As I recall it was something to do with leaving loads of channels hanging and might well have been to do with the often overlooked behaviour that (go) returns a channel. You'd expect them to just get GCed away eventually, but at least at the time there did seem to be a bit of a problem with the notion of having lots of open channels active. This is another slight bugbear I have found with core.async. It's rather easy to get it wrong. I've not used the claypoole library that @tcsavage suggests, but I do rather like the idea of letting somebody else deal with all this complex subtlety because I am often too lazy to comprehensively test the behaviour of my hand rolled async code.

Wes Hall16:12:27

As I said, it usually ends up disappearing anyway, but while it's there, I don't usually have a lot of confidence in it.

otfrom16:12:13

ok, sounds like I'm using it a bit differently then. I do see go as being very low level and not something I need to use except when I'm doing UI work and even then I'm not happy with it

tcsavage16:12:37

What about the pipeline functions for data processing?

Wes Hall16:12:18

@tcsavage Didn't the async code get refactored out eventually?

tcsavage16:12:39

The stuff you remember was, yes

Wes Hall16:12:51

There you go... I delegated the refactoring ๐Ÿ˜›

otfrom16:12:13

those are OK if I've got something big that I want to map (or similar) over and I want to heat up the cores. I want do do a number of sub tasks on things and lots of different kinds of reducing functions on stuff I've scrubbed

Wes Hall16:12:39

Tom has had the job of fixing all my broken shit hence, the previous exchange ๐Ÿ˜‰

tcsavage16:12:57

job for life ๐Ÿ™‚

Wes Hall16:12:45

There we have it, i'm a job creator...

otfrom16:12:30

but I hardly ever use go, and worry about it when I do

tcsavage16:12:24

Because of the shared global thread pool?

Wes Hall16:12:50

@otfrom It's always difficult to give general advice on a specific problem, but one of the big revelations that we had at Signal which I am sure Mr Savage will remember, is that when it comes to these long processing chains, it's usually not necessary to scale it by allowing for each step on a single piece of data to happen on a different thread. We got a fair bit of simplicity mileage from running the entire set of processing on a single piece of data on a single thread of simply composed functions but just doing the distribution at the start. As long as your per data item processing is relatively uniform, this scales fairly well and generally ends up being quite a bit simpler.

otfrom16:12:46

I'm not really worried about the scaling. I'm more using it for decomposition

Wes Hall16:12:15

In which case, could a simple middleware style pattern not work? Like in ring?

otfrom16:12:49

I have a feeling that what I need to do is publish an example

Wes Hall16:12:07

Yeah, it's the general advice / specific problem issue of course.

otfrom16:12:44

it is slightly different from the kinds of problems I've had before where it is ETL and I'm trying to get the data into something (like a database or a single output). The need for multiple slices through the same data is what is really driving me

Wes Hall16:12:54

I think it is entirely possible I am trying to advise avoiding core.async in precisely the case where it should be used. It does seem to be quite handy for doing versions of what we used to call, "enteprise integration patterns". The stuff that we'd use Apache Camel for back in the day. There is a lot of that, "tap this channel", "push this to a broadcast channel" type stuff that certainly has it's place.

dominicm16:12:29

I liked Tim Baldridges talk on core async

Wes Hall16:12:31

This conversation actually is quite timely for me. Just looking at some old code that I need to somehow maintain that might be a pretty good example of how not to do this stuff. Nested go blocks, reads from channels where the result is ignored over and over, nothing dealing with the channels returned from the go blocks themselves. It's cljs so this is depressingly common. The complexity of callbacks / promises replaced with the complexity of crazy core.async work ๐Ÿ˜•

Wes Hall16:12:46

Politics is simpler, can we change back? ๐Ÿ˜‰

seancorfield16:12:49

@otfrom (or anyone really) feel free to engage my team mate @hiredman (in a non-UK channel) -- he's a heavy user of core.async here at World Singles Networks

seancorfield16:12:17

I'm learning a lot about core.async by reviewing his code. Still don't think I understand it enough to write much of it myself but definitely more comfortable with it than I used to be

Wes Hall16:12:43

@seancorfield Probably shouldn't use you as the messenger here but if he or you happen to have any examples of what you consider to be particularly good core.async work that is open, i'd love to take a look.

otfrom17:12:11

same ๐Ÿ™‚

otfrom17:12:19

I've joined core.async too ๐Ÿ™‚

seancorfield17:12:15

I should probably join #core-async too...

seancorfield17:12:26

@wesley.hall I'm off work until Monday. I'll try to remember to ask him then, unless you ask him before then ๐Ÿ™‚ I have mostly been working a four-day week for several months, trying to burn down an excess of PTO before year end.

Wes Hall17:12:01

@seancorfield I know that feeling. Going to be losing a lot of mine. Hasn't been an, "abundance of time for a holiday", kind of year ๐Ÿ™‚

mccraigmccraig17:12:38

@otfrom i've not been the greatest fan of core.async, and initially struggled to articulate it, but i think i have it now - it's mostly because it is often used to do something it's not very good at, namely: promises. it's fine at stream processing stuff, map, reduce, back-pressure etc

mccraigmccraig17:12:47

but stream processing stuff intersects cleanly with promises in a well thought out abstraction, and manifold does just that - the result of a stream/reduce is a promise (aka deferred)... the result of a stream/put! or stream/take! is a promise, you can easily convert a stream of promises to a stream of plain values, and promise chain evaluation has short-circuiting - it all works nicely together

mccraigmccraig17:12:38

'course, manifold is really not perfect - error propagation with streams is not even thought about

seancorfield17:12:00

We're allowed to carry over well over 200 hours but I had nearly five weeks built up, so I took almost every Friday off for three months. Wish I could continue doing that in 2020 but we're planning a three week trip to Australia and possibly a two week trip to China...

mccraigmccraig17:12:48

and the principal reason i use funcool/cats (post-hoc rationalisation again) is the monadic let/do abstraction - it lets us stop thinking about promises and concentrate on the underlying computations. it also let us move all our promise type async code from core.async to manifold/deferred a while back with almost no code change, which was nice

otfrom17:12:27

I'm doing error propagation by turning them into values and then filtering to separate channels, counting and seeing if I'm happy with the ratio

otfrom18:12:17

And given that my stuff is batch if something more serious happen then I just abort and shut down

mccraigmccraig18:12:42

@otfrom that's exactly what we ended up doing with manifold - we never use manifold/stream directly, we have our own wrapper with implementations of map and reduce and a bunch of other fns which deal with error propagation - so that e.g. if there's an error during a reduce the result is an errored deferred (rather than the manifold default behaviour, which is just the reduce value when the error happened)

mccraigmccraig18:12:38

rsn i've got to extend that wrapper to do chunking, much like the clojure.core lazy seq stuff, 'cos our stream-join ops are drowning in callbacks