Fork me on GitHub
#clojure-uk
<
2016-07-22
>
paulspencerwilliams07:07:43

Hope you're feeling better. Going to get chance to walk etc at all while across there? Where in the Lakes?

agile_geek07:07:58

I haven't really got the right gear with me to do much walking. Windermere BTW

agile_geek07:07:26

Travelling from Home -> London -> Windermere -> London -> Home all by train means I'm lucky I've managed to lug enough stuff to have my suit for the wedding!

korny09:07:15

It’s always interesting to try to explain obvious clojure idioms to people new to the language. Some of the things I love about the language are also definitely barriers to entry for new folks.

korny09:07:34

It’s one of the things that makes me fear that it will always be a niche language ...

dominicm09:07:08

I found that once you learn a few things, which turn out to be surprisingly simple, you are away. These things are just new. If there wasn't new things, it would be a fairly pointless language.

dominicm09:07:44

However, I am young and obsessed with learning new things. I may have a slight edge in this regard over someone who has written Java for many years.

korny09:07:30

True. An example might help - I have code that needs to wait for a JMS queue listener to go idle. The listener pushes all messages onto a list in an atom (this is a quick and dirty listener for testing!)

korny09:07:56

And I wanted code to say “block until nothing new has been posted to the atom for at least 10 seconds”.

korny09:07:29

ah, crap, I think this was the wrong bit of code - my brain isnt’ working this morning. Will try to remember what code I was showing to someone yesterday - the queue listener is actually pretty straightforward (and somewhat procedural):

korny09:07:08

(defn wait-for-queue-idle [queue-atom idle-ms]
  (let [now (System/currentTimeMillis)
        last-msg (latest-message-timestamp queue-atom)]
    (if (< (+ last-msg idle-ms) now)
      true
      (do
        (Thread/sleep 500)
        (recur queue-atom idle-ms)))))

korny09:07:03

Ah, that’s right - it was the code to drain a queue. How to say “get all remaining messages from a queue, as a list, stopping when the get message call returns null"

korny09:07:21

There are probably cleaner ways to do this, but my quick solution was:

korny09:07:35

(I needed doall as the queue connection was closed immediately afterwards.)

dominicm09:07:57

Yep, makes sense. Laziness is a slight pain in these situations.

korny09:07:39

However, it’s very neat to be able to use repeatedly and take-while to magically turn a possibly-infinite sequence of function calls into a list of results. I could have wrapped it in a (take N function to stop it going out of control and never ending, too.

dominicm10:07:33

Yep. It was clever. But I understood it fairly quickly too. The best kind of code.

korny10:07:16

My point I think was, this is easy and idiomatic for a person used to clojure. And a lot shorter and simpler than a similar function in a non-lazy language. But explaining it to a colleague, who is new to clojure, involved a bunch of new concepts and new ways of thinking. They did get it, but I’m not sure they’d be able to modify this code without a fair bit more learning.

thomas10:07:26

@korny have you considered using core.async for the waiting problem described above?

thomas10:07:36

you can timeout on a channel.

dominicm10:07:55

> These things are just new. If there wasn't new things, it would be a fairly pointless language. I think the premise of Clojure is that we need a large shift in the paradigms and concepts we talk within in order to make our code more stable (and all the other benefits) When you learn something new, with actually new concepts, it's difficult. You are juggling a lot of concepts, for an example of driving. Steering, awareness, gears, peddles, directions! All at once?! After a while, you get good at the particulars and they become second nature. Steering, gears, peddles, awareness. Then you focus on the problem, directions. I moved from C to Python then Javascript. The hardest part was callbacks when learning JS. When I learned Clojure: Parens, Immutability, Side Effects, Atoms... Then it clicked. And I was busy doing my domain logic. --- Whether or not these new paradigms will be too big of a blocker to prevent growth is something I'm not sure about. I think you could teach yourself most of the information in a few days, with a strong resource.

korny10:07:23

@thomas: definitely, but as this is just test code, appending to an atom was very straightforward. In a real system, core.async would be a natural fit. But I’d have to remember how it works 🙂

korny10:07:50

I did just this at a prior client, but don’t have the code anymore, as it was the sort of client that demanded they keep all IP 😞

korny10:07:32

(also I ended up scrapping the wait - I couldn’t find a good timeout that genuinely meant the queue listener had finished. Now I just kill the listener, and then call the queue drainer instead)

glenjamin10:07:59

shame there’s no take-until, it’d read slightly nicer as (take-until nil? ...

korny10:07:24

part of me wants an excuse to define functions take-on-me and take-me-on

glenjamin10:07:42

(defn take-until [f coll] (take-while (complement f) coll)) I suppose

agile_geek10:07:40

Re: Clojure as a niche lang. I think we need Clojure to continue to evolve to make difficult things 'explicit' and 'called out' in code. I want a slightly larger base of companies and developers than we have now but I don't want 'Clojure is the new Java'. Let's leave that dubious distinction to Scala shall we?

korny10:07:58

🙂 I think actually “java 8 is the new java” is more likely.

glenjamin10:07:57

@NotNull Optional<String> - what’s not to like? 👿

otfrom12:07:42

korny: you mean java 9 with its repl^H^H^H^H shell

otfrom12:07:11

dominicm: I'm old and like to learn new things (tho not as old as agile_geek, I think he likes to learn new things to make sure he isn't dead)

mccraigmccraig12:07:03

@otfrom @agile_geek : but how do you know you've learnt something new and not just uncovered a forgotten memory of an old learning ?

otfrom12:07:10

mccraigmccraig: like most computing practitioners I can't tell the difference. 😄

agile_geek12:07:09

@mccraigmccraig: like most old ppl I can't remember

dominicm12:07:46

@mccraigmccraig: to me, this new is relative to me. Not absolute.

thomas13:07:01

everything is new to me...

korny14:07:30

I had something very close to this… As part of my undergrad degree (in the 80s!) I had a project that was writing a lazy-evaluating functional programming language interpreter. I enjoyed it enormously (well, after some massive initial pain) but basically discounted it as impractical and of no real value. And then went to work in C and C++ and others over time, completely forgetting all that lazy FP stuff. And then a long time later I started looking at clojure and laziness and went “hang on, I vaguely remember some of this…"

dominicm14:07:06

@mccraigmccraig: I misread what you said. I'm just starting, so everything is new!

otfrom14:07:23

agile_geek: I see clojure.spec as part of the way to moving things towards being more explicit (and giving better Elm like error messages)

agile_geek14:07:13

@otfrom: I thought the same. I think Clojure's moving in right direction it just needs a little bit more adoption.

korny15:07:51

Partly, I think we need to break the “one true language” mindset. Clojure is a very sharp tool for a number of kinds of problems. There are other languages that are also good tools, that are suited for other kinds of problems. I’m happy to push clojure heavily for what it’s good for, but not try to sell it as the language.

otfrom15:07:07

korny: I suppose my question to you then is: what is it good for, or what do you think it is bad at?

glenjamin15:07:28

it’s pretty poor for unix-style CLIs

glenjamin15:07:36

although CLJS might be ok at that

otfrom15:07:32

I'd rather do node stuff in cljs esp with spec

otfrom15:07:41

(though I wonder if I'd do haskell instead)

korny15:07:47

I think it’s poor at quick adoption by anyone who hasn’t learned FP previously. Especially, un-motivated people. We’ve had a lot of success at bringing people into clojure projects, but it’s far more of a hurdle than (say) moving people between the procedural/oo languages - moving someone from Java to Ruby or C# to JavaScript or whatever, there are always some speed bumps. But nothing in my experience like getting people fluent at clojure.

glenjamin15:07:57

although I find cljs on node suffers somewhat from large numbers of cljs users hating node

otfrom15:07:31

I've found moving people from R -> clojure a bit more straight forward

otfrom15:07:33

less to unlearn

korny15:07:58

In a lot of cases, you can use clojure/clojurescript to do most things, but if you don’t previously know the language, then you might be as well off using something else. ClojureScript especially - it’s great if (a) you already have clojure people, or (b) you have a complex front-end that merits it. If you just want to do some d3 visualization, or build a react UI for a business website, I’d use es6 or TypeScript.

korny15:07:14

If you want to build a database or queueing system, on the other hand, I’d avoid the JVM entirely and go for an erlang-based system, or something in a more system-oriented language.

otfrom15:07:19

korny: I'm wondering who will port clojure.spec to LFE 😉

korny15:07:22

(this last is mostly my reaction to jepsen-ising ActiveMQ, where it keeps relying on Java functions that just don’t work all that well, like exclusive file locks. I note that Artemis, the “next gen” activeMQ, uses a linux system library for it’s low level stuff rather than trusting the JVM…)

otfrom15:07:14

tho I suppose you are talking about jvm problems rather than lisp/FP problems

korny15:07:36

My ideal is to have a microservices or similar world, where you can make language choices separately for each service/component. At one of our big projects we had 10+ microservices, initially 2 were in clojure and the rest in java. Now several more have apparently been migrated to clojure.

otfrom15:07:11

I think that is my ideal world too.

otfrom15:07:32

I can't think of many places where I'd want to use java rather than clojure other than interop shims

otfrom15:07:02

or things that are meant to be called from other polyglot jvm things

otfrom15:07:28

I can see places where I'd want to use c style executables, or erlang or other

korny15:07:16

A lot depends on the team and what they know (and who they want to hire). Smart companies hire motivated individuals who want to learn. The world is full of non-smart companies 🙂 And of course it’s also full of legacy code that can’t just be thrown away.

korny15:07:43

Relating to which - there are other great niches for clojure, even in places with no core systems in it. For example, current client is building in Java mostly - but for resiliency testing, we’re using Jepsen, definitely the right tool for the job

korny15:07:06

I’d also love to try getting lambdacd into a project, at least as a trial for the build pipeline

otfrom15:07:27

yeah, trying to move big companies is a problem all of its own. Startups are a trade off of terror vs the frustration of Big Cos

otfrom15:07:26

and saying you do clojure does change who applies to work with you and what they expect. I do expect that many people will have to be trained up though

korny15:07:20

If you can get commitment, the training is possible - but it needs motivation and time. We’ve moved a pile of devs at our Glasgow project to clojure, both client devs and our devs 🙂 But for some, it was weeks to get productive. For a couple, it was never.

agile_geek16:07:03

@korny I can completely see why some people never make it. I think it's learning curve is such that easy problems (like a simple CRUD web app) are probably best done in something like RoR unless you already know Clojure. But once you get the paradigm I think you can solve some really hard problems in a simpler manner. I've looked at Haskell but I wonder at whether it's eco-system supports what can be done on the JVM. Can't comment on Erlang as I've no experience.

mccraigmccraig16:07:08

@agile_geek @korny isn't half the problem that you need to unlearn stuff - rails has a huge learning curve too if you are starting from scratch. ruby scoping - good luck with that

agile_geek16:07:37

@mccraigmccraig: I agree. Unlearning was hardest bit for me. I guess the learning curve is steepest if you're already from a language with a strong C and OOP heritage?

dominicm16:07:52

I'd like to make a compendium of ideas, failures and such in computer science.

mccraigmccraig16:07:58

so perhaps that means that all lazy &| fp languages are going to be a difficult switch for imperative programmers, and since a significant part of the difficulty is unavoidable and inherent in the imperative->fp switch we should take care not to teach clojure as a "switch" language or it will get the blame, so instead throw some other language under the switching-to-fp-is-difficult bus and only teach clojure once the difficult bit is done !

dominicm16:07:09

So when I'm off inventing a library, I can see that it's done before.

agile_geek16:07:19

@dominicm: in my experience it's always been done before!

agile_geek16:07:01

but often not in the environment that you're operating

agile_geek16:07:19

@mccraigmccraig: I tend to agree. What other inherently lazy & FP lang's are there after Haskell?

dominicm16:07:52

@agile_geek: it's so hard to see though!

agile_geek17:07:15

@dominicm: yeah. I've created and implemented designs only to find out 10 years later someone else had done the same in another org. This was especially true prior to the internet.

dominicm17:07:17

@agile_geek: I think this lack of looking back actually holds us back as an industry. We're unable to learn, and repeat the same naive mistakes because of it.

otfrom17:07:50

laziness is one of the things where the REPL vs running in prod can be misleading, as the repl realises the lazy things (often) to print them out whereas that doesn't happen in a production env

agile_geek17:07:57

@dominicm: Sometimes. Often the environment has changed and we make brand new mistakes! I've been through 5 cycles of distributed systems architecture that share concepts with microservices for example.

otfrom17:07:47

all distributed tech dies when people say that there must be good static typing that can be looked up from a central service (SOAP, CORBA...)

otfrom17:07:21

the only massively distributed systems I know of that work well are SMTP, HTTP, POP, IMAP

otfrom17:07:28

(and I'm not sure about all of them)

agile_geek17:07:38

@otfrom: REPL vs run - this catches me out all the time. State change is called out pretty explicitly but for laziness you just have to remember which fn's are lazy

dominicm17:07:37

@agile_geek: I'm not convinced that microservices are that great anyway!

agile_geek17:07:45

Interestingly I've seen several message based async systems that have lasted. Although the messaging middleware is often a bit inflexible.

dominicm17:07:29

I think Wunderlist have made some mistakes. For example they ended up building their own sql transactions

agile_geek17:07:02

Re: microservice - you're trading independent deployment/single responsibility/unit simplicity/horizontal scalibility for a bunch of orchestration/choreography/consistency concerns. Just need to understand the trade offs

agile_geek17:07:27

Did Wunderlist implement 2PC?

glenjamin17:07:57

the best microservices are designed to avoid orchestration/choreography/consistency problems

glenjamin17:07:38

or put another way, the best services to create/deploy/maintain independently are those without orchestration/choreography/consistency relations to the rest of your system

agile_geek17:07:52

@glenjamin: OK I was about to challenge that first statement until you qualified. However to be useful there has to be some relations that have those concerns with rest of system.

glenjamin17:07:29

i have a theory that dynamic language programmers are more comfortable with not breaking external API contracts

glenjamin17:07:38

because they have to be careful with internal ones

glenjamin17:07:49

and those problems exist in a monolith too, if you have client/server in the same massive repo, and you want to do a rolling restart of many nodes, you have a consistency problem

glenjamin17:07:16

how many people have their rich javascript apps compare versions with the server after a server has deployed?

agile_geek17:07:40

Even if the client is dealing with them?

glenjamin17:07:15

which bit was that to?

agile_geek17:07:35

Sorry tunnels cutting off conversation... my bit about client related to delegating coordination/choreography/consistency to client

mccraigmccraig17:07:46

@glenjamin: i do (version compare) - but then i'm deploying a mobile app, so you kinda have to

dominicm17:07:45

I think a monolith can have functions and Namespaces with a single responsibility. They're also easier to debug, and quicker to build. For now at least, I think they're very far from a silver bullet.

agile_geek17:07:48

@mccraigmccraig: well you don't have to but it's dangerous not to. Not sure current client does.

otfrom17:07:20

dominicm: clearly there is no silver bullet, nor indeed a free lunch

glenjamin17:07:21

dominicm: have you seen Dan North’s “Software that fits in your head” talk?

dominicm17:07:46

@agile_geek: I'm not sure what 2PC is, the explanation I got was that they look for certain exceptions, and rollback in other places. And sometimes that data has moved, so it becomes difficult.

glenjamin17:07:22

he talks about “modules” with clear interfaces that separate concerns, and microservices being one way of drawing those boundaries

agile_geek17:07:30

Sorry - 2 phase commit

mccraigmccraig17:07:39

@agile_geek: if you are ever wanting to be able to deploy breaking changes you need to be able to give "you really wanna upgrade your app now" nags with an appstore link...

dominicm17:07:13

@glenjamin: Mmhm. Microservices are one way. They feel like they're being sold as the best/only way. But not everything is always awesome.

agile_geek17:07:46

I'd agree, they're useful but like all tools not the answer to everything

glenjamin17:07:56

Consider a simple javascript interaction: server rendered page makes an ajax call to populate a country dropdown. If someone in a microservice or a monolith changes the API response format, do you reckon they consider people who’ve already opened the page?

dominicm17:07:08

@agile_geek: I don't know the meaning of the practice. I'd Google, but a phone on the train isnt the best place. Could you summarise it in any way?

mccraigmccraig17:07:29

@dominicm: that sounds like a 10greenspun-2pc

mccraigmccraig17:07:24

@dominicm: 2pc is one way of databases and other participants agreeing on whether to commit or rollback a transaction

agile_geek17:07:51

@dominicm: yeah what @mccraigmccraig said (I'm also on train going thru tunnels!)

dominicm17:07:47

@agile_geek: you've either passed or will soon pass me. I'm on my way to Leicester

otfrom17:07:49

every useful thing is oversold by people who will make money out of that useful thing. That doesn't stop it from being (potentially) useful (in some cases)

agile_geek17:07:24

@dominicm: nope wrong side of country. I'm heading to Windermere for my friends wedding tomorrow.

agile_geek17:07:02

but I guess I may have passed you on line north/south

dominicm17:07:18

@mccraigmccraig: yes, its some form of doing that across microservices. Its done using a crap ton of conditionals, not the most reliable system apparently.

dominicm17:07:08

@agile_geek: ah, less brotherhood today then.

mccraigmccraig17:07:43

@dominicm: if it ain't 2pc or paxos or the other one then it's probably broken and aphyr will torment you

dominicm17:07:12

@mccraigmccraig I don't work there 😃 just relaying information from a chat. It sounded informal. Possibly repeating past mistakes.

mccraigmccraig17:07:45

pass on the warning 🙂

korny19:07:47

The best microservices systems I've seen were more async message based, than synchronous with transactions and orchestration. If you can put events on a transactional queue, you don't need distributed transactions. You just need business flows that can handle asynchronous change.

korny19:07:23

@mccraigmccraig: these days I tend to like the "bff" model, where a ui talks to a custom facade, rather than directly to a generic rest interface

dominicm19:07:27

I'm currently trying to work on an async-based CQRS/ES system in Clojure. Though I only want one server for now. But I suspect the work to make it a microservice/consumer based would be centered completely on my use of shortcuts.

dominicm19:07:59

But shortcuts are important early on. And Clojure lets you avoid the worst pitfalls of them, and makes it easy to correct them later. I think.