Fork me on GitHub
#other-languages2016-05-06
>
borkdude15:05:40

(hmm, I should probably change language settings in Powerpoint)

tjg15:05:15

@borkdude: What's the link to that? Closest thing I've found is a handwritten version without Scala/Clojure: https://channel9.msdn.com/Blogs/Charles/Simon-Peyton-Jones-Towards-a-Programming-Language-Nirvana

bvulpes16:05:02

borkdude: and where praytell is Ada on that chart?

bvulpes16:05:27

i'd squeeze clojure and scala significantly to the left, and scala significantly down on the usefulness spectrum

borkdude16:05:18

@bvulpes: Interesting, your opinion is that Scala is more useless than Clojure?

borkdude16:05:28

Please elaborate

bvulpes16:05:36

i've seen a lot of time wasted trying to pave over paradigmatic incompatibilities between scala libs and even differences between author styles /within an org/

borkdude16:05:06

@bvulpes: Have you used Scala professionally and for how long?

bvulpes16:05:09

ofc "clearly they're doing it wrong", but otoh, "no true scotsman"

bvulpes16:05:39

no, watched a scala org evolve through a friend who ultimately left in disgust at how the language had crippled their work.

bvulpes16:05:35

yeah i mean it gives me hives, i won't touch it. but he liked the team, and so there went three years of his life.

bvulpes16:05:56

and re clojure's "usefulness" well the compiler is not particularly useful in a wide swath of applications i'd like it to be.

borkdude16:05:13

I'm using it at work now (also moved there because I liked the team/company). So far I must say that despite that bad noise, I quite enjoy it

borkdude16:05:05

We use microservices, maybe that helps to prevent growing an unmaintainable mess

borkdude16:05:19

of course that's not a silver bullet either

borkdude16:05:48

I'd like to hear experiences from bigger Clojure teams vs bigger typed FP teams

borkdude16:05:53

The real question is, is it types that make scaling easier, or mainly other things like immutability, people in the team, documentation etc.

bvulpes16:05:56

over my pay grade.

bvulpes16:05:18

i generally solve medium-sized problems for small orgs on my own. last mega codebase i was in was ios and let me tell you the challenges on that project were of the human factor drop-shadow "like this other interface but different in inane and complexity-inducing ways" kind.

seancorfield17:05:55

I’m glad @martintrojer is here — hopefully we can ask questions about your articles?

seancorfield17:05:18

We have 30K+ production lines of Clojure and 10K+ lines of test code. We aren’t really seeing the problems cited in that Prelude article — but we do have only a small team (two full-time Clojure devs, we had a part-time third dev for a while).

seancorfield17:05:05

We certainly don’t find the nil-punning to be as problematic as it seems some people do.

seancorfield17:05:36

We do find we have a lot of questions around best practices for actually organizing and structuring code across namespaces — and we’re constantly refactoring and refining that organization as we learn lessons over time.

borkdude18:05:42

@seancorfield: do you use Schema and pre/post a lot?

borkdude18:05:34

I think one of the important things is to have a team that is fluent with one technology and also knows how to deal with the downsides of it, regardless the language.

seancorfield18:05:56

We’ve tried core.typed and Schema multiple times. We did not feel the benefits were worth the costs.

seancorfield18:05:13

Schema’s validation is only as good as your test suite so it’s only going to catch errors in structure that don’t cause your underlying code to fail the test anyway.

seancorfield18:05:36

core.typed is better in that regard: it checks the types not the (runtime) structures.

seancorfield18:05:34

But retroactively adding core.typed to a large codebase is an exercise in frustration due to the relatively poor type inference it does and the fact that even for a single namespace, it’s essentially an all-or-nothing affair due to the amount of "warning" noise that a partially annotated namespace provides.

seancorfield18:05:51

I think we’d gotten around 5% coverage with core.typed before we gave up. It required too many changes to code to satisfy its analysis, or else you just slapped metadata everywhere saying "Trust me — don’t check this body!" which sort of defeats the whole purpose.

seancorfield18:05:32

We actually started with core.typed, switched to Schema, gave up, tried core.typed again, tried Schema again, then switched back to core.typed. Then gave up again.

seancorfield18:05:58

Re: Scala — We came to Clojure from Scala. Lots of reasons. Scala 2.7 -> 2.8 was such a painful transition we were very gunshy about future upgrades. Compiler was very slow. Toolchain was fragile and quirky. Error messages were horrible, esp. when you ran into the combined buzzsaw of implicits and builtFrom (or whatever that weird collections library signature thing was).

seancorfield18:05:44

We had a legacy dynlang system anyway so integrating Scala was painful on many levels (calling into Scala from other JVM languages gets messy unless you’re very careful about your API).

martintrojer18:05:59

@seancorfield: nice to hear it's working out for you :)

martintrojer18:05:57

My experience based on many things

martintrojer18:05:11

Having to read code to figure to argument types is pretty horrible

martintrojer18:05:45

All the NPEs is death by a thousand cuts

martintrojer18:05:17

But the biggest thing for me is that I'm at lest 10x faster refactoring is a Lang like Haskell

martintrojer18:05:35

With confidence as well, that haven't messed something up

martintrojer18:05:38

Tracking side effects in types is also a pretty big thing for code quality

martintrojer18:05:05

Biggest teams I've been in is 8 clj devs

martintrojer18:05:27

Total code weighing in at 40k perhaps split over many services

martintrojer18:05:19

Steady stream of totally unnecessary bugs pushed to prod

martintrojer18:05:35

Anyway, on a journey to find something better

martintrojer18:05:11

Front end is pretty clear, Elm has win written all over it

martintrojer18:05:19

Backend still looking

seancorfield19:05:25

I agree Elm is a great choice for the front end at some point. Not sure it’s really production ready yet beyond bleeding edge adopters (I followed Elm very closely for a long time during its development — and seeing ElmConf ahead of Strange Loop makes me want to attend that more than Clojure/conj this year).

seancorfield19:05:07

I have a love/hate relationship with Haskell. I was part of the UK university research crowd that created our own FP language in the early 80’s as part of our language research. When Haskell appeared, I truly hoped FP would go mainstream — it was so obviously better than the status quo! — but Haskell remained an academic curiosity and OOP took over everything (unfortunately).

seancorfield19:05:36

So I’ve worked with Haskell-like languages since the early 80’s and with Haskell itself on and off for about 25 years… and, much as I love many things about Haskell, every time I actually use it to try to build a real "thing" I get so incredibly frustrated with its whole ecosystem and, often, by the attitudes of many Haskellers 😿

seancorfield19:05:40

As for Clojure’s NPEs… I just don’t encounter them much. I know some people run into them a lot. I don’t know why some do and others don’t.

borkdude19:05:43

@seancorfield: one of the best thing of Clojure is the community, the people are so nice compared to other more elitist languages (my words)

borkdude19:05:41

@seancorfield: on the other hand, I'm jealous of the documentation and effort some other communities are putting in their artifacts

seancorfield19:05:19

I can understand the refactoring argument. The type system means you can’t break things in so many ways that a dynlang would allow you to break things during refactoring. Although I don’t find refactoring in Clojure to be particularly problematic so it isn’t a pain point I feel strongly about.

seancorfield19:05:13

@martintrojer: Remind me, what was your language background before Clojure? Much FP? Much dynlang, or mostly typed languages?

borkdude19:05:22

With 3 months of Scala, I can say that in my experience refactoring is much easier than in Clojure (2.5 years commercial experience)

borkdude19:05:29

Maybe some tools make it easier now (like clj-refactor), but often these tools are not mature and crash (CIDER, etc).

borkdude19:05:39

Running lein check and lein test in the background helps

borkdude19:05:42

I wonder if Cursive makes a big difference here, I don't have enough experience with that

sveri19:05:14

Interesting discussion so far and I agree with a lot of the statements made here I tried haskell a few times and always gave up because of the tooling, emacs being a big part of it (dont want to start IDE flamewar here, I just dont like emacs). It seems like stack solves some of these problems, but, the last time I looked into it the emacs setup stated that it still had problems with stack (which may be figured out inbetween) Also I have a hard time figuring out parameters and shapes of maps in standard clojure. Although I found some peace in schema. But again, here I agree, a better type system, which is optional, would fit my needs more.

sveri19:05:38

@borkdude: I never used cursive for refactorings in large codebases, but for my small ones it did not fail once

sveri19:05:07

I refactor parameters and function names, but lately saw that it can also refactor keywords, I wonder how reliable that is

martintrojer19:05:49

@seancorfield: before clj: c, c++, Python, c#, f#

borkdude19:05:27

My favorite languages were Common Lisp, then F#, then Clojure and now I don't know anymore 😉

sveri19:05:03

Anyone here looked into Rust? I went through the docs and like the idea about ensuring data safety through the type system

borkdude19:05:13

I think theses 'favorites' were always in the context of what I could do with them at work. So I chose F# when I was doing .NET.

seancorfield19:05:15

@sveri: I spent a fair bit of time playing with Rust last year. It’s very interesting. Not so much for the type system itself but for the borrow-tracker aspect.

seancorfield19:05:18

If I were doing near-metal systems programming, I’d be very tempted by Rust. It’s what I had hoped Go might be. [I think Go is horrible having spent a fair bit of time playing with it the previous year!]

seancorfield19:05:03

If F# ran on the JVM (and therefore could interop with all my legacy code), I’d probably be using that instead of Clojure. Depending on tooling etc.

sveri19:05:26

@seancorfield: That was my feeling too regarding Rust. It seems like there is also a lot happening regarding game development with Rust, if anyone is into that

martintrojer20:05:17

@seancorfield: coreClr port will open the door for f# properly

martintrojer20:05:44

Very nearly there, and better than the JVM on pretty much every metric

martintrojer20:05:28

So much less boxing in .net it puts JVM to shame

martintrojer20:05:40

Also, startup not an issue

seancorfield20:05:29

But still completely irrelevant to me if I can’t interop with my existing legacy codebase! 😸

seancorfield20:05:34

@martintrojer: One thing I found very interesting about your Prelude article was how you were bored just working in Clojure day-in, day-out. I would assume that has also been the case in all the previous languages you’ve used?

seancorfield20:05:56

(well, assuming perhaps you left some of them due to frustration, rather than boredom 😆 )

borkdude20:05:23

What is CoreCLR?

borkdude20:05:09

Common Lisp and Clojure had the biggest impact on my enthusiasm for programming, but yes, even Clojure can feel like work when you're doing it day in day out. It also depends on the system you're building. If you're making a CRUD app and every feature is more or less the same old stuff, it gets boring.

seancorfield20:05:57

(and a year of playing with Clojure before that)

seancorfield20:05:41

And I do a bunch of hobby / FOSS Clojure in my spare time too. And I help non-Clojure devs learn Clojure (I’m doing my best to convert the world, one developer at a time!).

seancorfield20:05:34

I look at that monadic getPlayer’ function and I think of (some-> params :player get-player) or similar...

seancorfield20:05:48

I hope we’ll be hiring at some point, maybe this year, but probably a junior dev… maybe someone who’s fairly new to Clojure but seems to "grok" FP in general...

sveri20:05:44

I would also love to do some clojure as part of my job. But here it is hard to find one in my City... I have not heard of one clojure Job...

borkdude20:05:31

I had a job with a couple of Clojure projects, but it was a software services company and every project was only a couple of months. I was missing working in a team and building up deeper knowledge of something.

borkdude20:05:31

What is it called in English when a company builds software and also maintains it for other companies (as opposed to help the developer team at some other company started and then leaving)

ericnormand20:05:38

sounds like a consultancy

borkdude20:05:43

there are two types of consultancies then: one that builds all your software and maintains it, and the type that can help you accelerate your software team

borkdude20:05:33

@ericnormand: I read the discussion on your board. You mentioned 'types are the future, but...'. I thought it was interesting. Why do you think types are the future, but still do mainly Clojure?

ericnormand20:05:37

they're the future, not the present simple_smile

ericnormand20:05:01

Clojure still rocks, and has a ton of great design decisions

ericnormand20:05:00

and I do want to expand into other languages

borkdude20:05:49

You mentioned some Haskell experience. Some Clojure devs seem to go into that direction.

ericnormand20:05:04

seems really common when people get frustrated

ericnormand20:05:11

Haskell is a great language, up there with Clojure.

ericnormand20:05:38

Clojure has a huge advantage with the Java ecosystem

ericnormand20:05:54

Haskell is good, but the rhetoric is tiring.

ericnormand20:05:06

I wish there were more options of typed languages to compare.

ericnormand20:05:36

at the moment, saying "I want good types" seems to mean "I want Haskell".

seancorfield20:05:50

pssst! There’s a Leiningen plugin that lets you do mixed Clojure/Frege projects

borkdude20:05:04

I think Rich Hickey once said in one of his talks: it's either this or all the way to Haskell

ericnormand20:05:36

what does "this" refer to?

borkdude20:05:00

@ericnormand: I should look it up, can't remember the exact phrase

borkdude20:05:19

@ericnormand: maybe it was about 'the language of the system'

borkdude20:05:36

@ericnormand: programming like you do with services, but in the small

ericnormand20:05:00

the next language I design (in like 30 years at this rate simple_smile ) will explore types

seancorfield20:05:08

Back when I was exploring FP language design and implementation (my unpublished PhD thesis in the early 80’s), I created a dynamic language based on the notation in W.H. Burge’s book and implemented it as a transpile to LispKit (that part was based on Peter Henderson’s book). But I used a Prolog program to analyze types across whole programs to verify behavior.

seancorfield20:05:31

(so, yeah, I had a transpiler to Lisp(Kit) and a transpiler to Prolog for separate purposes)

seancorfield20:05:36

Prolog could produce a sequence of all types that satisfied the program — and it was pretty easy to see when you had an illegal program because Prolog, bless its little cotton socks, would just act the two year old child and respond with No.

seancorfield20:05:25

It was also nice that if your program was essentially "any -> any" then you’d done something wrong 😆

seancorfield20:05:40

It was very interesting to hear Matthias Felleisen ragging on Hindley-Milner type inference at Clojure/West.

ericnormand21:05:04

what did he say?

seancorfield21:05:11

Oh dude, you have to watch the talk! It’s awesome!

borkdude21:05:24

yes, it was an awesome talk

seancorfield21:05:56

An hour and fifteen — it stands out like a sore thumb by virtue of being half an hour longer than any other talk at the conference! 😆 No one wanted him to stop!

borkdude21:05:34

"Everyone in Clojure thinks in types, but nobody is writing them down" was a sentence I remembered

sveri21:05:02

Isnt that how you do it in every language that has insufficient types?

borkdude21:05:10

Of course. But his story went on. At least you could write them down as documentation. But then it becomes a problem that they are not checked and the documentation isn't right anymore.

sveri21:05:45

Just like it always is with documentation

borkdude21:05:19

Right. So it would be nice if you could check it. And this is how he arrived at Typed Racket more or less, if I remember correctly.

sveri21:05:55

Yes, I watched it too, no objections from my side, I just wanted to point out that this problem is not specific to clojure

tjg21:05:14

I feel like a weirdo when mentioning my programming opinions (liking lisps doesn't help), but I avoid scaling problems by avoiding writing large programs.

tjg21:05:30

I believe there's lots of techniques that help you do this. (After all, using a high-level language is one.)

tjg21:05:37

Not just technically, but socially. Finding another programmer with whom you can synch well with imposing constraints on your mutual work... not so easy in my experience.

tjg22:05:30

To the extent that large programs are necessary (for example, accepting Tim Daly can't simply avoid a large Axiom codebase, given its 1.5 million lines of Common Lisp), it's a new game, and static types are only one of many techniques available.

borkdude22:05:42

@tjg What other techniques can we use in Clojure?

tjg22:05:59

Guess I'll write about this in the http://discuss.functional.tv discussion, but Tim Daly frequently mentions literate programming. http://lambda-the-ultimate.org/node/3663#comment-51878

tjg22:05:09

So for example, he edited a literate programing book of Clojure. http://daly.axiom-developer.org/clojure.pdf

borkdude22:05:07

"Beyond a certain point of complexity you can only understand a program by talking to the author about why they wrote it the way they did." I think he's right about that, types or no types

borkdude22:05:34

He proposes literate programming for software that must survive beyond 5 years

seancorfield22:05:01

Are there really any large systems in production written using literate programming?

borkdude22:05:15

And doesn't literate programming suffer from the same problem as other documentation, that it isn't checked and therefore is incorrect a lot of the time?

tjg22:05:58

BTW, I don’t personally advocate literate programming, just one of many perspectives I’d consider if I ever entered the world of Large Clojure Programs. As I understand Martin Trojer’s article, a sufficiently large Clojure system requires you to spend “more than 50% of the total line count” on maintaining its architecture.

tjg22:05:02

Analyses like that are one of the reasons I simply try to avoid large systems... like I avoid hard computational complexity classes. (For example, “Small Things, Loosely Joined, Written Fast, Discarded Readily.” http://blog.cognitect.com/blog/2016/3/17/the-new-normal-protected-asset-or-disposable-inventory )

tjg22:05:07

I have no idea about literate programming, but I personally devote much time to documentation: - an org-mode notebook in git, where I try to boil a domain down to a conceptual language, and constantly describe project case-studies in terms of it. - “Battle Stories” (which I first developed in a company with 15 devs, using narrative structure so people read my boring tech notes) - video walkthroughs (timestamped), where developers walk me through their systems as I ask questions