Fork me on GitHub
#clojure-uk
<
2019-12-19
>
dharrigan05:12:09

Do you use the async facade, or the poll approach?

dharrigan07:12:06

ta. I see on their github page that they're thinking of removing/deprecating since they don't see to use async

dharrigan08:12:32

last comment

fmjrey08:12:06

I'm not on a computer atm, will comment on that ticket saying in my fork I added the ability to pass a response channel. I can't say I use it for prod only for a prototype, but so far it's been great for building processing flows that run without Kafka in demo/test while being able to change config and enable Kafka

fmjrey08:12:16

I'm currently rebasing my changes with better tests. We can always make it a separate lib if needed

dharrigan08:12:50

that would be fantastic! (if required!)

mccraigmccraig10:12:18

that kinsky async producer facade looks like a case of using streams where promises would be better suited

mccraigmccraig11:12:43

we have an async kafka producer protocol which is just (send [_ topic message]) and returns a manifold/deferred of the result

mccraigmccraig11:12:42

are you building async @dharrigan?

dharrigan11:12:11

Maybe. I'm doing two things - one is looking at different kafka libraries (I'm currently using Jackdaw) to determine if Jackdaw is good enough, or if there is something lighter and two, I'm experimenting with learning more async and doing things in parallel with kafka processing in Clojure

dharrigan11:12:50

I must admit, it's still not very clear when I must use threads or go-blocks (or threads inside go-blocks) etc... if I'm doing IO work, like 1. reading from S3 or 2. doing a REST call etc...

dharrigan11:12:53

so learning as I go

mccraigmccraig11:12:38

with core.async it's best not to do anything in a go block which will hog a thread from the single fixed-size threadpool - i/o or long-running computation

mccraigmccraig11:12:06

so if you are doing sync (i.e. blocking) i/o then move that to a thread, if you are doing complex calculation, move that to a thread

mccraigmccraig11:12:58

but if you are doing async (i.e. non-blocking) i/o and co-ordination then keep that in the go block

dharrigan11:12:45

Thanks! All the hints and tips I'm getting are indeed very useful

dharrigan11:12:52

I've set myself a goal this christmas to bottom this out 😉

fmjrey22:12:31

I just pushed some improvements to the kinksy async face in my fork, see https://github.com/pyr/kinsky/issues/42#issuecomment-568599400

thomas08:12:03

Mogge 😼

dharrigan09:12:15

Morning everyone!

otfrom10:12:11

heh, got bit by an infinite lazy seq yesterday (I think). It can make debugging quite fun

thomas10:12:00

could take forever @otfrom

jasonbell13:12:52

I’m glad to see that in my absence that @U052852ES is still willing to go for the low hanging fruit.

thomas13:12:35

someone has to...

thomas13:12:56

And I have learned from the best 😉

jasonbell13:12:19

I know but you’ll have to thank @otfrom later 😉

thomas13:12:02

ok, I'll come back later for that 😉

otfrom10:12:16

:drum_with_drumsticks: tish!

Eamonn Sullivan11:12:06

Hi all, you seem like a friendly bunch, so introducing myself: I'm a developer at a very large broadcaster (the biggest, according to wikipedia). At the moment, I'm mostly doing Javascript (React, on the front end) and Scala (for backend services), but I've been using Emacs (and tinkering with my config) since the late 1980s/early 1990s, so am somewhat familiar with lispy things. I've taken on a 10% time project to create a new online retrospectives site (a clone of http://funretro.io initially), and to make it challenging choose Clojure/ClojureScript (luminus, re-frame). It's particularly challenging because we have only a handful of Clojure programmers at my organisation (it's used as part of the pipeline to get an episode to iPlayer), but none around me. I'm in London (White City, usually). Are there any regular meet-ups around me? Any local resources I should be using?

Jcaw15:12:48

Where are the clojure developers based? Manchester?

Eamonn Sullivan15:12:33

I just checked the recent committers on a BBC clojure repo, and it looks like it, yes.

Jcaw15:12:16

That's interesting! Didn't realise there was much of a community up there.

Jcaw15:12:23

Thanks :)

alexlynham09:12:06

the bbc clojure devs are in mcr - there are a few former platform folks in this channel

alexlynham09:12:26

also I’m in mcr as are the swirrl gang

dharrigan11:12:47

Do you mean BBC?

dharrigan11:12:02

There are a few London Clojure shindigs - look at London Clojurians, also we just had a mini-conference re:Clojure a couple of weeks ago at The Crypt.

dharrigan11:12:25

There are quite a few companies in London (and surrounds) that use Clojure too

Eamonn Sullivan11:12:16

Yep, the BBC. Prior to that I was at Bloomberg. I'm an American transplant to London, but have been here 21 years now.

dharrigan11:12:12

I think if you hang out here a bit more, you'll get more people chiming in of what is happening in London presently.

dharrigan11:12:37

There's also the thoughtworks meetup

Conor11:12:44

I used to be one of that handful of Clojure programmers! Simply try completely pivoting languages because your team feels like it

Conor11:12:08

Only partially joking, there were also good reasons too

Eamonn Sullivan11:12:15

Yeah, I talked to that team and they told me it was because the language had good concurrency support and seemed to do pipeline-y things well (threading macro, e.g.). That team still likes the language, but it hasn't really caught on elsewhere. I like it (lispy-things like elisp) because it's such a small language that it fits in my head and I'm not constantly googling how to do things. I liked C for the same reason. With Clojure, I'm doing a lot of googling initially, but it still seems like the happy little language I fell for decades ago...

rickmoynihan12:12:42

Having used clojure for over a decade, I can attest that it’s still the happy little language it was back then too… Though a little bigger now.

jiriknesl13:12:32

In my 7 years of Clojure, it didn’t change a lot. And I love it. We got clojure.spec instead of prismatic/schema (I still love Schema) We got core.async, but I don’t see people use it Clojure Toolbox got much larger, there are lots of libraries Luminus is still the only backend framework, there was Pedestal, it died and people now use Duct and similar instead of frameworks (which I see as unhappy thing that blocks Clojure adoption) On frontend, it is completely different story with a wide choice of options (re-frame, Rum, etc.) Datomic got into plenty of production uses Datomic Ions seems without traction And it seems to me that that’s the thing.

jasonbell13:12:43

Luminus is still go to, I can just get stuff working quickly.

folcon13:12:58

We do use ions actually, but it’s still fiddly…

dominicm14:12:08

@eamonn.sullivan you balance out @seancorfield who went the other way. I always wondered if we were migration neutral in the Clojure community.

Eamonn Sullivan14:12:27

Yeah, that’s the more usual direction. I left Mike Bloomberg’s (well-paying) empire because I desperately needed a change after 15 years. Also, all my kids grew up and left home, so I didn’t need that much money any longer. I haven’t regretted it, but nor do I regret the time at Bloomberg.

seancorfield17:12:44

@eamonn.sullivan Whereabouts in the US were you based? New York? I moved from Hampshire (UK) to California, initially to work for a UK company that was setting up in Silicon Valley, then six years at Macromedia once I got my US work permit, then citizenship and some time as a consultant, until I landed at a boutique online dating company (where I've been for ten years and using Clojure in production for eight and a half of those).

Eamonn Sullivan18:12:07

Well, like many Americans, I’m from all over the place. I was born in San Diego , grew up (until high school) in New York, then around Boston. My wife and I were in Acton, Mass, before moving to London on a two-year assignment in 1998. One thing led to another and we haven’t left. I have Irish citizenship (via my mother) so work permits weren’t an issue. Sorry, I misunderstood the comment and thought you were now working for Bloomberg from the BBC.

seancorfield18:12:25

🙂 Americans seem so much more mobile than Brits. My wife was born in Tucson but moved all over the Midwest until she settled in the Bay Area in her mid-30's.

seancorfield18:12:57

I was born and raised in N. Ireland so we have been considering trying to apply for Irish citizenship since it is staying in the EU. I let my UK passport lapse over ten years ago.

Eamonn Sullivan18:12:20

I was a journalist for 20 years (including my first seven years at Bloomberg here in the U.K.), so I needed a nice, neutral passport for moving around with. (Software engineering is my second career -- long story.) The Irish passport has been very handy. I have been able to secure Irish passports for my five kids (so they retain the ability to work either in the U.S. or anywhere in the EU), but sadly not my wife (she'd have to live in Ireland for a time). She's going to wait until this whole Brexit mess settles down and then get her British passport. The world is a bit of a mess lately, isn't it? I'm waiting for the pendulum to swing back round toward sanity. Not holding my breath, but we'll eventually get there.

Eamonn Sullivan18:12:16

I love the bay area, and Arizona too.

seancorfield18:12:23

Aye, fingers crossed. Things have certainly gotten crazy over here -- just glad I live in a "liberal bubble" for now 🙂

mccraigmccraig18:12:45

you can always get your UK passport back @seancorfield , even if you renounced it: "You have a right (once only) to be registered as a British citizen if you previously renounced British citizenship in order to keep or acquire another citizenship." https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/788225/Guide_RS1.pdf

seancorfield18:12:00

Yeah, I know it's only a matter of going through the paperwork -- but my wife doesn't want a UK passport and we always travel together so there's not much incentive there, but an Irish passport (and access to the EU) is more appealing.

seancorfield18:12:44

(we have friends in SW Eire already so retiring to that area is a nice dream)

mccraigmccraig18:12:42

not been to SW eire, but i imagine it's quite (i.e. very) rainy

seancorfield18:12:02

It's beautiful and green! 🌳

mccraigmccraig18:12:09

haha, well the west coast of norway is beautiful and green too, but omfg the rain

dharrigan14:12:47

That manifold library is pretty neat

dharrigan14:12:09

I'm playing around with it, i.e., chain, catch, loop etc.

mccraigmccraig14:12:35

yes, it's good... there are a couple of issues with it (don't use let-flow and error-handling with streams of deferred's requires some work) but the foundations are solid

mccraigmccraig14:12:21

• promises (i.e. deferreds) have an error state and compose well with chain

mccraigmccraig14:12:21

• put! and take! and reduce on streams return deferred

hugo14:12:30

What's the issue with let-flow?

mccraigmccraig14:12:56

it might work for you, but apparently has bugs and was only ever really experimental... looking for a reference to where i saw that

hugo15:12:07

Hmm good to know

mccraigmccraig15:12:55

let-flow isn't necessary for anything, so no biggie

mccraigmccraig15:12:48

and the icing on the cake @dharrigan - streams of deferreds work very nicely - you can use stream/realize-each to turn a stream of deferreds (of operation results) into a stream of values, and you can use a buffered stream to then control the concurrency of those operations just by setting the buffer size

dharrigan15:12:37

Lots of things to play with over the holidays!

mccraigmccraig16:12:45

some manifold tips @dharrigan [1] write fns that take plain-value args and return promises of things, including promises of streams if you are using streams - that way you can very easily compose them with chain [2] be careful of loop - much like clojure loop it's quite low-level and you don't need it very often - many times you are better off making a stream and mapping and reduceing over that

dharrigan17:12:46

that's very useful

dharrigan17:12:50

I'm going to write that down 🙂

Gulli19:12:31

Lol, I just got a book for about Pedestal. Was wondering why there docs where so insufficient. Guess it's because it's dead

seancorfield19:12:57

I think the core of Pedestal lives on as a library, but the framework itself is dead, yes.

seancorfield19:12:39

Server-side Pedestal is actively maintained https://github.com/pedestal/pedestal

seancorfield19:12:27

And it looks like they have been keeping the documentation up-to-date too http://pedestal.io/ @glfinn83

seancorfield19:12:17

The front end part hasn't been touched for over five years https://github.com/pedestal/pedestal-app -- I remember when they launched it with a conference talk... it looked pretty slick back then.

seancorfield20:12:29

That is a bit sparse. You could always open issues asking for the guides to be fleshed out more https://github.com/pedestal/pedestal-docs/issues

seancorfield20:12:18

(I only looked at that repo to check activity -- I didn't look in the actual docs)

seancorfield20:12:04

But docs seems a long-standing issue with a lot of Clojure libraries... Some projects have great docs, some have very sparse docs 😞