Fork me on GitHub
#off-topic
<
2018-02-14
>
ackerleytng22:02:40

i thought clojure interop with python would be really useful, I'm thinking something like clojurescript, but for python instead. anyone else think this is a good idea?

bronsa22:02:11

I bet @tbaldridge has some thoughts about that :)

noisesmith22:02:34

I forget how many versions of that he’s done - at least a few if I remember correctly

noisesmith22:02:49

pixie doesn’t do python interop though - it only uses a python vm

noisesmith22:02:08

(I guess it can do python interop on a compiler level? not in the runtime)

ackerleytng22:02:51

but it's not updated

ackerleytng22:02:15

but it doesn't interop with python...?

noisesmith22:02:00

if you mean pixie - it uses rpython to make a vm, but doesn’t have access to python at runtime

ackerleytng22:02:12

oh oops, it does!

tbaldridge22:02:19

yeah there's a lot of differences between python and clojure that makes it hard

tbaldridge22:02:26

I used to maintain clojure-py back in the day

tbaldridge22:02:45

it had a lot of problems though

ackerleytng22:02:01

is there one problem that you remember?

ackerleytng22:02:46

one of clojure's strengths is really the interop. every few days someone asks on r/clojure about clojure and data science

ackerleytng23:02:06

i think these questions would go away if there was a clojure that could interop with python

tbaldridge23:02:16

mutable closures, reference counting, reference counting that defults to borrowing (everything holds onto the head)

ackerleytng23:02:31

sort of like clojure for the CLR

tbaldridge23:02:08

python is also rather slow, so if you want something liek clojure-py you need part of it to be in c

tbaldridge23:02:13

or make everyone use pypy

noisesmith23:02:42

what about a clojure vm that knew how to consume the python wrappers of c and fortran stuff? (I ask, knowing nothing about how python ffi works or whether that question even makes any sense)

noisesmith23:02:30

perhaps the real killer app for python in data science isn’t the wrapped libraries from c/fortran but the higher level stuff in python that consistently works together and builds on that - if so competing is going to be a lot more work

tbaldridge23:02:26

The other thing that makes this really hard is that it's hard to define what Clojure is.

tbaldridge23:02:55

Stuff like multi-arity functions don't exist in Python. So do we need to implement something like IFn and pay the cost of two dynamic dispatches on every function call?

tbaldridge23:02:34

What is even the core protocols needed by Clojure? CLJS and CLJ are really different in the protocol area, and some of the CLJS stuff is pretty JS specific.

tbaldridge23:02:55

so the few times I've looked into it I got bogged down in trying to figure out how to map Clojure semantics to Python

tbaldridge23:02:26

For example in Clojure falsey is nil and false. In Python it's: None, False, [] and 0.

ackerleytng23:02:01

would you be interested in working on this again if you had someone to help you? 😛

noisesmith23:02:05

I bet you could get some traction with a docker image set up to run jvm clojure using blas and some good matrix and stats libs out of the box

ackerleytng23:02:42

i think javascript is different if it's falsey too

ackerleytng23:02:59

i'm glad i talked to you, you have so much experience!

tbaldridge23:02:06

@ackerleytng I'd be willing to give people pointers. But at this point if I'm going to write a language, there's other non-Clojure concepts I want to explore.

tbaldridge23:02:25

Effect systems, fully immutable VMs, etc.

ackerleytng23:02:00

but then you'd have to reimplement numpy

jsa-aerial00:02:13

That's basically what core.matrix is. And neanderthal already wraps this stuff in a very strong robust API. Generally, things in the data science front are starting to settle and accelerate. #data-science is the 'de facto place' to hang out for this.

ackerleytng23:02:08

@tbaldridge ok first question! how did you even start? by reading the clj/cljs implementation first?

tbaldridge23:02:51

pretty much, the stuff in clojure.lang is pretty simple. A good way to start is by looking at clojure/core.clj and seeing what stuff like conj calls, then dig into clojure.lang.RT, and that then takes you into all the other code.

tbaldridge23:02:48

Oh, that reminds me of something else that's a bit hard to implement:

tbaldridge23:02:04

No Clojure implementation out there has first-class Vars and protocols on the bottom.

tbaldridge23:02:17

CLJ has vars, but not protocols at the bottom

tbaldridge23:02:25

CLJS has protocols at the bottom, but not vars.

bronsa23:02:01

heh, that is tricky indeed

tbaldridge23:02:07

Python could have both, but that's a really nasty chicken-egg problem. To get vars you need hashmaps, and some other structures. To get that you need protocols. Protocols are stored in Vars....

bronsa23:02:22

I've played with fully self-hosted clojure a bunch of times and that's always been the biggest headache

bronsa23:02:04

@tbaldridge you don't actually need hashmaps tho, there's a few tricks you can play to defer hitting code paths where those would be needed, until they are defined

bronsa23:02:49

but they don't make for pleasant or very understandable code

bronsa23:02:05

I found that a bigger headache than the circular dep between vars and protocol is the one between vars, namespaces and symbols

bronsa23:02:18

which makes me think -- I've heard RH say that if he was to redo the var/ns system he would make it significantly less reified

bronsa23:02:29

I'd be curious to know which parts he'd do away with

bronsa23:02:40

@alexmiller any ideas on ^ ?

Alex Miller (Clojure team)23:02:12

I just want to get rid of Namespaces as mutable Java objects :)

bronsa23:02:25

that would be amazing

bronsa23:02:35

but also a massive breaking change

bronsa23:02:52

mostly for tooling I reckon

Alex Miller (Clojure team)23:02:12

Maybe, figuring that out would be step 1

Alex Miller (Clojure team)23:02:32

Someone did a spike on immutable namespaces a few years ago iirc

bronsa23:02:36

immutable namespaces come with their own sorts of problems

bronsa23:02:10

it's not the awesome happy land people think they would be :)

Alex Miller (Clojure team)23:02:11

But imagine that refer-clojure was just adding a pointer

bronsa23:02:40

i've spent long evenings imagining that

noisesmith23:02:13

oh I know - namespaces made of datoms

bronsa23:02:29

there's kiss from mikera that has immutable namespaces

bronsa23:02:40

but it's just a long-running experiment as far as i can tell

Alex Miller (Clojure team)23:02:52

Maybe that’s what I’m remembering

bronsa23:02:37

now called magic, just like ra's compiler

bronsa23:02:40

to add to the confusion

bronsa23:02:59

ocaml is the closest you'll get to having "immutable namespaces"

bronsa23:02:06

and redefinition in ocaml is just not a thing

bronsa23:02:25

and mutual recursion needs to be compiled in a single compilation block

Alex Miller (Clojure team)23:02:32

Well if it’s indistinguishable from magic then it must be sufficiently advanced

bronsa23:02:58

(altho the lack of redefinition in ocaml is considered an intentional feature)

tbaldridge23:02:38

@bronsa it blew my mind when I found out that F# kept all those "features"

tbaldridge23:02:47

one of the reasons I learned Clojure instead.

noisesmith23:02:03

brb implementing OClojml

bronsa23:02:11

you're kinda forced to do that in a staticly typed lang

bronsa23:02:25

otherwise redefinition would invalidate all your types

bronsa23:02:30

what a mess that would be

bronsa23:02:33

I've never looked at F#

tbaldridge23:02:18

It's pretty much what you'd expect if you built Clojure on .NET and used OCaml as a base instead of lisp.

bronsa23:02:00

that doesn't actually sound too terrible :)

dpsutton23:02:30

Microsoft have really good tooling and language designers as well

tbaldridge23:02:54

If Clojure didn't exist I'd probably be doing F# instead, I rather like it.

dpsutton23:02:02

I like it but never liked the non windows environment for it although I hear it's better now

andy.fingerhut23:02:15

I love the contrapositive of Clark's third law: Any technology distinguishable from magic is insufficiently advanced.

fellshard23:02:49

F#'s tooling was pretty rough last I tried. It also ends up being pretty verbose at times.

bronsa23:02:04

TBF so is ocaml's tooling IMO

fellshard23:02:12

Want to like it, but it def. suffers from MS' documentation problems

seancorfield23:02:15

I really like F# (what little I've used it). The providers stuff is awesome.

bronsa23:02:22

using ocaml's tooling feels like using common lisp before quicklisp was a thing

bronsa23:02:49

with the only upside that there's not 15 different incompatible implementations

noisesmith23:02:09

have you used opam? it’s a little clunky but at least it does switchable version isolations

bronsa23:02:13

yeah I have

bronsa23:02:32

opam2 is a massive improvemnt over opam1

bronsa23:02:36

but still quite clunky

bronsa23:02:44

compared to what I'm used to at least

bronsa23:02:24

jbuilder is also quite nice (I forget what they renamed it to)

noisesmith23:02:56

I haven’t made a new OCaml project recently enough to try jbuilder