Fork me on GitHub
#off-topic
<
2019-10-05
>
hmaurer18:10:09

Hello! Out of curiosity, is there anyone reading this message who has decent experience in an language with an advanced type system (e.g. Haskell) and yet find themselves preferring the untyped nature of Clojure for most (or for some specific) tasks?

schmee18:10:09

here’s a nice blogpost on that topic: https://lispcast.com/clojure-and-types/

emccue20:10:35

@hmaurer Ive used rust and scala if that counts

emccue20:10:54

i prefer dynamic languages for tasks that involve "real" data primarily

hmaurer20:10:09

@emccue most definitely counts 🙂

hmaurer20:10:18

why so? (out of curiosity again; I’m not opinionated on the subject)

emccue20:10:39

part of it is that the "advanced" typesystems are a cognitive burden

emccue20:10:54

which isn't a bad thing neccisarily

emccue20:10:04

rust is a fantastic language

emccue20:10:35

but if most of my work is just "take data, shape it, shove it, twist it, bop it"

emccue20:10:14

then simple stuff with the typesystem like "declare fields and optionality" starts to become a dev time burden and imo outweighs the "safety" benefits

emccue20:10:49

at one of my coops i had a scala trait with 15 Option[...] fields

emccue20:10:21

there is more but my stupid monkey brain can't think of it

emccue20:10:31

(and obviously all of that is opinion)

emccue20:10:41

there's also the concrete benefits of being able to "hack it"

emccue20:10:18

tech debt is worth taking on if it lets you move at the pace the business needs to

emccue20:10:42

if you attempt some kind of abstraction with rust and it doesn't work out then you might be stuck doing a refactor to ensure memory safety/etc

emccue20:10:50

which is perfectly fine, at least for rust, and doing refactors in advanced typesystems kind of languages are as painless as is possible

emccue20:10:45

but a dynamic language lets you take on that tech debt of accessing a "private" field or writing something that dispatches on a class and some string together, etc

emccue20:10:34

scala in particular kinda stinks at doing that kind of stuff without brandishing a phd and the sword of artorius

hmaurer20:10:21

@emccue thanks for writing all that out! the optionality part echoes some personal experiences as well

emccue20:10:46

something i've noticed, at least with clojure

emccue20:10:04

if i don't have a specific task its hard to write "skeleton code"

emccue20:10:32

like, pre-coding abstractions

emccue20:10:19

if im writing in a typed language its alot "easier" to keep my fingers typing before i even know what im supposed to be doing

emccue20:10:00

thats not necessarily a condemnation of boilerplate - you might end up with the same interfaces and system boundaries at the end of a project regardless - but it is what I use as a sign that a project is a good fit for a dynamic language

emccue20:10:32

if i can sketch it out in great detail beforehand in a dynamic system it might be a good fit for doing statically

emccue20:10:40

if that makes sense

emccue20:10:45

i dunno, im kinda just riffing

hmaurer20:10:42

Yeah that perfectly makes sense

hmaurer20:10:22

I am going through a similar experience right now where I am re-writing a project from Clojure to Haskell, and it strongly fits in the category of “can design in great detail beforehand”

emccue20:10:34

im also not the most experienced person out there, so take anything i say with a mountain of salt

ericnormand20:10:32

@emccue's points resonate with me

ericnormand20:10:56

if you have a model that you know ahead of time, being able to encode that in the type system is a big win

ericnormand20:10:17

but if it's a changing/unknown model, untyped is easier

ericnormand20:10:56

you won't get the benefits of static typing

hmaurer20:10:58

The author of the blog post himself! Thanks for chiming in @ericnormand 🙂

ericnormand20:10:00

and you still pay the costs

ericnormand20:10:33

Haskell's type system's "painless refactorings" assume that a large portion of the types are unchanging

ericnormand20:10:48

which might be a valid assumption

ericnormand20:10:07

but if it's not valid, the refactoring isn't painless

ericnormand20:10:44

and you may be talking to an API that doesn't have the same level of consistency as Haskell

ericnormand20:10:54

(which is almost by definition the case)

ericnormand20:10:57

if you're trying to type that API's JSON response, good luck

emccue20:10:27

oh i just noticed the blog post

ericnormand20:10:34

there could be a ton you know about the JSON, but to describe the type in Haskell's type syntax would be hell

ericnormand20:10:36

and undocumented, so it's not like you could just bite the bullet and do it

ericnormand20:10:52

it's a matter of incremental discovery

ericnormand20:10:04

which really sucks when you have to type out your expectations ahead of time

ericnormand20:10:22

you'd rather make the API call and print out what it gives you

ericnormand20:10:08

anyway, I think the Haskell type system is really valuable, but it's not the end of the story

ericnormand20:10:20

I can imagine better type systems (and so can many Haskellers)

borkdude20:10:20

Interesting comment by someone I used to work with: https://twitter.com/GHengeveld/status/1180191410676469761

ericnormand20:10:27

so it's not dynamic vs static

ericnormand20:10:36

it's more about Clojure vs Haskell

seancorfield20:10:12

I share Gert's view, but for Haskell. I only used Scala for about 18 months, before switching to Clojure.

ericnormand20:10:14

I'm working on an episode of my podcast where I talk about why I prefer Clojure over Haskell, even though I think Haskell has some benefits

ericnormand20:10:43

as far as type systems go, I think they're the future but we have to live in the present

ericnormand20:10:58

a type system that I would use would have to be way more expressive and cost less

borkdude20:10:04

it's nice when you get type checking for free without having to change your coding habits

seancorfield20:10:08

Back at university, I worked with ML and Miranda and SASL while I was doing my research work and designing/building yet another FP language so I could experiment with syntax and semantics (and implementation issues around GC).

seancorfield20:10:40

When Haskell appeared, I had really high hopes for it to go global and take FP into the mainstream.

ericnormand20:10:47

Type systems are a whole design space that a language like Clojure can avoid

ericnormand20:10:00

type systems need to be designed, and Haskell's is 50% not designed

seancorfield20:10:06

Every few years, I go and try to re-learn Haskell, but I just get frustrated with it each time.

ericnormand20:10:11

since it's used for research

ericnormand20:10:37

and the half that is designed is designed for research applications

ericnormand20:10:02

Haskell is a rather well-designed language overall, but there's still so much room for figuring out the type system

ericnormand20:10:17

Clojure is well-designed, too

borkdude20:10:26

I have spent a fair amount of free time in 2018 learning Haskell, just so I could have a better judgement of this. I liked it, but more as an intellectual challenge than something I actually want to work with. Having my SQL queries type safe isn't the most important thing to me.

ericnormand20:10:09

that gets into one of the things I don't like about the Haskell ecosystem

ericnormand20:10:16

but I'll save that rant for my podcast

seancorfield20:10:49

Where I had most frustrations with Scala was around database interactions. I suspect I would find similar frustrations with Haskell.

borkdude20:10:02

It wouldn't surprise me if this was more or less the same problem as spec1 where you have to make up a name for a specific set of key/vals (like is very common with query results).

borkdude20:10:02

So in a type system you have to make a new type for every slight variation of a query result is what I mean.

seancorfield20:10:58

Yeah, or you construct these giant, complex, compound types with lots of optional fields.

borkdude20:10:36

oh yes.. Maybe everything 🙂

hmaurer20:10:58

yeah, from my experience that’s a pain-point in Haskell to which I haven’t found an answer to

borkdude20:10:32

I think row polymorphism could be an answer to that. They have this in PureScript and Elm. I think that could work for Haskell, but they haven't figured it out yet as a language extension. They do have libs but they tend to be complicated and slow.

hmaurer20:10:03

yeah the libs in haskell for row polymorphism are not.. practical

hmaurer20:10:06

let’s put it this way

borkdude20:10:41

I asked a question to a couple of Haskell folks at a speaker dinner last week: what's the state of row polymorphism in Haskell nowadays? One of them replied: that's a troll question.

borkdude20:10:36

oh, Clojure thing on HN front page right now 😛

hmaurer20:10:00

your clojure thing 🤔

borkdude20:10:45

it is. sorry for the plug 😛

borkdude21:10:11

Another reply was: when are you going to remind us that spec is not a type system?

ericnormand21:10:11

that's a real troll question

ericnormand21:10:21

I think Haskell is going to have the same effect Lisp did

ericnormand21:10:35

it's going to be a great experimental language that other languages borrow features from

borkdude21:10:25

Which reminds me: y'all should try this podcast episode with Hal Abelson: https://corecursive.com/039-hal-abelson-sicp/

borkdude21:10:38

He has a nice perspective on JS as a progression of Lisp and why it became popular

hmaurer21:10:30

progression?

borkdude21:10:51

maybe not the right word

emccue21:10:38

you know how that one guy did the experiment where he had some people go into psych wards and pretend to hear a voice that said "thud"

emccue21:10:39

and that revealed that practicing psychologists were basically inept and not doing things in a scientific way even slightly

emccue21:10:08

then there was a push to "standardize" stuff which is where we got the DSM

emccue21:10:37

and then there was a news story about how alot of early psychology experiments aren't reproducible

emccue21:10:51

thats what I wish would happen with programming

emccue21:10:58

i want either something absolutely terrible to happen or someone to indisputably shame everyone so that everyone gets scared into actually trying

emccue21:10:45

def. the wrong crowd for this rant, but "industry standards" and "best practices" feel like they are mostly gobbldeygook

emccue21:10:53

I've met so many people who say "I want to use the best tool for the job"

emccue21:10:00

but it honestly feels like a cop-out

emccue21:10:03

I know that screams "hey we've tried to make standards before and design by committee is terrible"

emccue21:10:55

there is no guidance on how to pick the best tool

emccue21:10:26

there is no actual adult in the room who knows

hmaurer21:10:25

that was my main revelation in my early 20s

emccue21:10:01

I keep thinking about one of the recent talks by the guy who made rails

emccue21:10:46

where he talks about how open source is a gift, not a commitment, etc

emccue21:10:00

but he brings up how he had to market rails

emccue21:10:28

because, technology aside, the way to win people over was through marketing

lilactown03:10:16

"Tech is pop culture" 😫 I feel this so hard

emccue21:10:12

it never mattered if you had tool that was good for solving a problem

emccue21:10:00

Java won over C++ people back in the before-i-was-born's, but only with the help of a massive marketing effort

seancorfield21:10:48

When Java appeared, I was the editor of a C++ journal in England and I wrote a very skeptical op-ed about this "new kid on the block". And then several years later, I switched from C++ to Java as my primary work language...

bherrmann23:10:59

Marketing helped, but so did not having to track every free and malloc... java was card out of that jail...

seancorfield00:10:10

The other thing I liked about Java was that it's not punctuation-heavy -- after years of C++, it was a breath of fresh air to write (and read) code that looked a lot more like English than line noise 🙂

Vincent Cantin13:10:04

I loved Java at that time because it would explain why it would crash and where, thanks to the stacktrace.

emccue21:10:24

and it appealed to their personal experience, not to any real yardstick of "good"

emccue21:10:00

and languages are only half of it

emccue21:10:10

what database should I use?

emccue21:10:15

mysql? why?

emccue21:10:25

postgresql? why?

emccue21:10:42

whats the difference and why does it matter?

emccue21:10:28

"oh postgres can let you query over a blob of json" or "mysql can't do transactional data declarations"

emccue21:10:34

honestly, i would just pick postgres because ive "heard good things"

emccue21:10:43

but i can't tell you what any of those things are

emccue21:10:13

ive been on a team that used a nosql database in a way that required joins

emccue21:10:48

and it wasn't just lack of foresight, it was a lack of understanding why we would use something to begin with

emccue21:10:18

nobody who learns sql is first shown why people even do the normalized stuff

emccue21:10:53

how some old dudes in the past concluded that graph databases and object databases weren't flexible enough and now its just momentum (even if they are right)

emccue21:10:22

its just "the standard"

emccue21:10:49

the javascript situation is a microcosm of this kind of stuff

emccue21:10:00

react/redux were a good set of things

emccue21:10:19

but people used the tools by reputation, not on their merits

emccue21:10:33

and boom react codebases move to vue, or dojo, or yam

emccue21:10:39

(i made that last one up)

emccue21:10:15

or from whatever tool that was high in the hype cycle to something else

emccue21:10:14

but really the programming level of stuff is such a small part of the actual engineering needed to make a thing exist

emccue21:10:32

how do I run a thing in a way that people can access?

emccue21:10:52

There is a huge part of my small brain that still thinks of the internet as read only

emccue21:10:12

like a computer in my house cant use the internet, not really

emccue21:10:39

i have an ip, and i can expose a port, but my ip changes

emccue21:10:59

its the "internet people" who can put stuff online

emccue21:10:54

i should go through blogspot, or wordpress, or some minecraft server hoster, or aws, or whatever

emccue21:10:27

even as a developer there isnt an escape

emccue21:10:32

how do i run code?

emccue21:10:46

well you write it, and run it on your own computer - simple!

emccue21:10:30

but if you want to run it in a way that can support a buisiness you need to pick a way to do logging, a place to host the code, a way to scale to more people, how to deal with storing data

emccue21:10:58

and there are people who think about that stuff constantly

emccue21:10:17

but I heard splunk was good, so i am going to just google how to get splunk running on aws

emccue21:10:41

grafana is great, but how did you set it up?

emccue21:10:07

did you download it onto a node manually?

emccue21:10:17

did you run a docker container?

emccue21:10:26

why docker?

emccue21:10:52

why does docker have a central thing you need an account with an external company to use?

emccue21:10:15

i don't know where im going with this

emccue21:10:24

all I know is that I feel emotionally exhausted

futuro01:10:15

Analysis paralysis is real. There’s a lot here, but one of the ways I’ve found to deal with this is to recognize that there are no universally right answers, because what makes something the “right tool for the job” depends on your constraints.

futuro01:10:09

More often than not I’ve found that simply beginning to build something immediately clears away a lot of choices that don’t work.

futuro01:10:58

In the worst case scenario, I spend a long time building something only to discover I didn’t understand the problem domain properly, and I have to begin again from the start. But even then, I only understand that about the problem domain precisely because I wrote that code I later had to throw away.

futuro01:10:28

So having to begin again isn’t a good rubric for whether I made the right choice or not. I know more about the problem and can build a better solution now, which strikes me as strong evidence to having made the right decision.

futuro01:10:07

Also, depending on the industry you work/hope to work in, you’re unlikely to hit many of the failure cases for a particular piece of tech until you’re pushing its limits, which isn’t the worst kind of problem to have.

futuro01:10:05

So some questions, like what kind of data store to use, don’t have a lot riding on the answer. Should you use Postgres or MySQL? Pick whichever one strikes your fancy. A document store versus relational? Read a couple blogs and if you still can’t decide, pick at random.

futuro01:10:46

I don’t know if this will help you at all, but it’s what I’ve learned the hard way so I hope it can ease your path.

emccue21:10:47

there isn't a programming language or anything in tech that makes me happy anymore

emccue21:10:28

every company has their own homegrown solutions to stuff

emccue21:10:41

and open source won't fix the problem

emccue21:10:46

and I am not confident in my ability to make a rational decision anymore

seancorfield21:10:30

Yeah, decades ago, it was all a lot simpler. You could actually run an internet service on your computer at home. I wrote my own mailing list server software in C and ran it on my home computer connected via dial-up -- and I managed mailing lists for fans of Peter Hammill (and a few other things).

seancorfield21:10:15

Everyone using the mailing list knew that the server was only running from 7pm to 7am weekdays and then all weekend. It was too expensive to stay connected during "peak" day time.

seancorfield21:10:23

(I paid per-minute for that connection)

seancorfield21:10:11

I don't know how you'd even do that these days -- but of course why would you? There are so many mailing list services out there.

emccue21:10:25

its kinda like how video game consoles are locked now

emccue21:10:42

and even when they aren't, how can people even begin

emccue21:10:06

maybe thats not the best example actually

emccue21:10:16

unreal and unity do give a starting point for sure

emccue21:10:19

and its a good one

emccue21:10:35

maybe what the CS world needs is a religon

emccue21:10:49

not just fads

emccue21:10:52

im still in university for another year before im thrown on the job market

emccue21:10:31

and one of our assignments for the software development class was to write some text specification for a toy problem

emccue21:10:35

3 functions max

emccue21:10:48

(they gave us a problem - it was just a trivial one)

emccue21:10:07

and we picked our language for the class so we are supposed to say "do it in language X"

emccue21:10:16

and it gets sent to another team

emccue21:10:35

we got back a flimsy "we didn't know clojure so we wrote it in python"

emccue21:10:05

which hurts their grade, but it kinda feels like people don't want to learn stuff

seancorfield21:10:14

When I did CS, we did some pretty substantial final year projects. Mine was to write an APL interpreter. A friend wrote a Lisp interpreter (that had full native system call integration -- so you could use Lisp to write system utilities which was cool).

seancorfield21:10:49

I wanted to write mine in Algol 68, which I loved, but none of the lecturers knew it well enough to review my project, so I ended up writing it in Pascal instead.

emccue21:10:46

everyone in the class picked java, python, kotlin, typescript, or javascript

emccue21:10:53

vast majority java and python

emccue21:10:26

which isn't wrong but its telling that those are the only 2 languages other than racket that our college includes in the curriculum

emccue21:10:41

so not many people actually learned stuff outside of lecture

borkdude21:10:45

regarding types, is TypeScript winning territory or are there still a lot of JS devs who prefer "vanilla" JS?

emccue21:10:37

@borkdude my impression is that it is growing still, though no clue what the upper limit is

borkdude21:10:04

I see https://twitter.com/brandonbloom sometimes tweeting about his gripes with TS

hmaurer21:10:25

@borkdude I work with a JS dev who is vehemently against TS

hmaurer21:10:30

from my perspective TS is really nice

seancorfield22:10:26

TS is a pure superset of JS tho', right? So why would a JS dev be so against it?

emccue22:10:03

@seancorfield yes

hackeryarn22:10:32

In my experience, most JS devs are fine with TS as long as people don't start writing C# or Java like code with it (heavy class and enheretance use). This especially become a problem when C# or Java shop move to a node with TS backed, or devs with mostly experience in those languages start setting the direction for front end projects.

hackeryarn22:10:53

JS makes it hard to bring over those practices. TS seems to encourage it.

ericnormand23:10:19

I have seen a lot of pro-type arguments like this: "48% of bugs in production would have been caught by the type system"

ericnormand23:10:30

those are true positives

ericnormand23:10:38

what about the false positives?

ericnormand23:10:53

how many non-bugs would the type system have identified?

ericnormand23:10:20

after using types, do you really have 48% fewer reported bugs in production?

ericnormand23:10:55

that would be nice to know

ericnormand23:10:49

or maybe a better metric: how many hours total have you saved?

hackeryarn23:10:49

Also how many of the same bugs would a good test suite catch?

ericnormand23:10:09

meaning time saved from debugging column minus time spent coding the feature in the first place

ericnormand23:10:25

i wonder if it just moves the cost around

ericnormand23:10:55

does it have any effect on velocity or similar measures?

hackeryarn23:10:20

I've wondered if it detracts from the focus on the bigger domain problems by focusing too much time pleasing the type checker.

ericnormand23:10:58

these questions are never answered

ericnormand23:10:53

having worked in a haskell codebase, I did feel like it caught stuff

ericnormand23:10:06

but then it was something that wouldn't have been an error in clojure

ericnormand23:10:18

whereas maybe those things are problems in javascript that a type checker could really help with

ericnormand23:10:34

all this said, I still have dreams of writing type systems for clojure

ericnormand23:10:42

static analysis is useful