Fork me on GitHub
#boot
<
2015-10-13
>
alandipert01:10:49

@cfleming: re: kotlin, does it have a runtime beyond JRE?

cfleming01:10:47

@alandipert: Yes, but pretty lightweight:

~/d/cursive (leiningen-fix)> ls -lah lib/kotlin-r*
-rw-r--r--  1 colin  admin   2.1M Oct 13 10:43 lib/kotlin-reflect.jar
-rw-r--r--  1 colin  admin   1.2M Oct 13 10:43 lib/kotlin-runtime.jar

cfleming01:10:55

Not sure if reflect is required or not.

cfleming01:10:15

So it’s not a runtime, as such, more a stdlib

cfleming01:10:24

It’s well worth it IMO, for interop-heavy code I like Kotlin more than Clojure

alandipert01:10:48

i am definitely interested in a "system jvm lang"

cfleming01:10:00

It’s the only option IMO

cfleming01:10:22

Mirah could have been ok but went nowhere

cfleming01:10:44

And IMO Ruby syntax is more of a bug than a feature

cfleming01:10:57

Fleming’s Law is: never write code in a language you’d be scared to parse, because the tooling will suck simple_smile

alandipert02:10:21

i would say you are debunked by scala in intellij

alandipert02:10:36

been living there recently, i can't believe how good it is considering how insane scala is

cfleming02:10:16

Well, I don’t use Scala, so I’m still following Fleming’s Law

cfleming02:10:41

And it’s taken both Scala IDEs years to get decent

alandipert02:10:43

fleming's 0th law, don't use scala

cfleming02:10:16

But yeah, reimplementing the Scala type system in the editor takes balls of steel

alandipert02:10:53

do you have a moment to discuss analyzers?

alandipert02:10:13

i remember you saying you made your own for cursive, is that right?

cfleming02:10:17

Do you want to Skype or something, or is chat ok?

alandipert02:10:05

is there a particular organization you subscribe to? and what format do you thread through your passes, if it's architected like that?

cfleming02:10:25

So I’m somewhat constrained by IntelliJ

cfleming02:10:39

It requires a classical lexer/parser, so that’s what I use for parsing.

cfleming02:10:22

I also have to use their AST classes, unless I want to duplicate everything from there into Clojure data structures, which I don’t.

cfleming02:10:56

So I have essentially a mirror of Symbol, Keyword, etc in my AST, but the AST is basically what the reader would return, not what something like core.analyzer would produce.

cfleming02:10:42

Then analysis is generally per-form, and the various functionality extensions are keyed off the head symbol.

cfleming02:10:48

So I set it up like this: (resolve/register-locals :clojure.core/refer-clojure refer-clojure-symbols)

cfleming02:10:47

Where refer-clojure-symbols is a function that returns a map of data describing the locals for that form. That will be invoked and passed the form itself, like a macro, and is responsible for its own parsing.

cfleming02:10:16

These extensions are generally called back asynchronously by IntelliJ during inspection passes etc.

cfleming02:10:44

So I don’t really have passes as such.

cfleming02:10:51

Does that answer your question?

alandipert02:10:12

do you have your own reader also?

alandipert02:10:06

oh sorry, you answered that already

cfleming02:10:11

Yes, it’s a JFlex lexer which I’d love to replace with something less quirky, and a recursive descent parser

cfleming02:10:46

I’m going to be speaking at the conj about the form parsing bit.

cfleming02:10:57

And a bit more about all this in general.

alandipert02:10:01

do you have a sense for how, ideally, you'd do it?

cfleming02:10:17

It really depends on the use case - what are you thinking of doing?

alandipert02:10:35

well, i have a toy lisp compiler going... compiles to java

alandipert02:10:20

ha, yes, this is like the nth toy lisp, but there are some particular things i want to test with it

alandipert02:10:42

the macros that receive arguments expanded thing, principally... but also different semantics for nil

alandipert02:10:01

anyway, my first go was a very tools.analyzer-esque AST, maps of maps

alandipert02:10:16

and the first mildly tricky thing is closure conversion

alandipert02:10:32

and there are various bonus things with that, like optimization passes to warn/elide unused locals, combine closures when possible, etc

alandipert02:10:58

but my goal was just to get it end-to-end first, and come up with a reasonable way to add optimization and lint passes later

cfleming02:10:27

Are you AOTing, or how dynamic is this?

cfleming02:10:34

What’s your compilation unit?

cfleming02:10:47

Top-level form?

alandipert02:10:48

top level form

alandipert02:10:45

you can either emit the java or compile/load in-process with javax.tools.JavaCompiler

cfleming02:10:57

So it’s quite different to how Cursive works. IntelliJ will call me for arbitrary forms, saying “run this inspection on this form”, and I inspect the form.

cfleming02:10:10

So I have to work inside out, i.e. trying to work out what my local context is.

cfleming02:10:31

It’s much easier to work top-down, and it’s not something I’ve done a lot of.

alandipert02:10:05

are you familiar with the concept of "nanopass"?

cfleming02:10:47

Yeah, but fairly vaguely.

cfleming02:10:05

Essentially very small passes run over local sections of the AST - is that right?

alandipert02:10:44

i'm not sure if it's the best thing though

alandipert02:10:58

the direction i'm going is, the way is nanolisps

alandipert02:10:10

basically every pass is an interpretation, not just a pattern match/replace

cfleming02:10:38

So I can imagine how that would work for unused locals, for example - when you have a form that you know declares locals, so a let or an fn once you’re expanded, you can walk the scope where the locals are valid and see if they’re ever referred to.

alandipert02:10:51

altho i'm not sure if it holds water. basically i just got excited that with the right interpreter, fn* dynamically binds its arguments and symbols evaluate to the fn* syntactically above them by looking at this binding

alandipert02:10:07

so the AST evaluates, under the interpreter, to some language for interpretation by the next pass

cfleming02:10:11

Why does it have to dynamically bind them if you’re recursing down?

cfleming02:10:35

I see, so the compiler is essentially executing the code as it compiles?

alandipert02:10:41

(defn eval0 [[op & args :as form]]
  (case op
    let0 (let [id                (genid)
               [bindings & body] args
               locals            (zipmap (take-nth 2 bindings) (repeat [:let id]))]
           (binding [*env* (merge *env* locals)]
             (template
              (let1 ~id
                    ~(map-nth 2 eval0 bindings)
                    ~@(map eval0 body)))))
    fn0  (let [id            (genid)
               [args & body] args]
           (binding [*env* (zipmap args (repeat [:fn id]))]
             (template
              (fn1 ~id ~args ~@(map eval0 body)))))
    ref0 (let [sym (first args)]
           (if-let [closure-id (get *env* sym)]
             (let [[type id] closure-id]
               (template (~(get {:let 'let-ref1 :fn 'fn-ref1} type) ~id ~sym)))
             (template (global1 ~sym))))
    lit0 (template (lit1 ~@args))
    (pass eval0 form)))

cfleming02:10:44

Can’t *env* just be an arg to eval0?

alandipert02:10:36

i believe so, yes

alandipert02:10:54

although i'm not sure what the interface to these passes should be yet

alandipert02:10:06

i only have 1 pass so far lol

cfleming02:10:11

It depends whether they’ll need to mutate the bindings, I guess.

cfleming02:10:23

That’s a nanopass compiler!

cfleming02:10:50

You could check out how MLton works

cfleming02:10:31

I think it’s reasonably well documented, and they have a variety of intermediate languages. I believe all their intermediate langs are immutable, but I don’t know the details of how it works.

cfleming02:10:40

Do you read Andy Wingo’s blog?

alandipert02:10:56

oh yeah, i think i've run across their docs whilst googling

alandipert02:10:15

and i do, on and off... but every time i get a new computer, i forget to restor my rss feeds

cfleming02:10:39

He talks a lot about how this works in the guile compiler, really interesting stuff.

alandipert02:10:55

oh yeah! i read with great interest his stuff on register vs. stack vm

alandipert02:10:02

the dude is clearly an animal

cfleming02:10:42

Yeah, and he’s a great guy to boot

cfleming02:10:45

(Matthew Fluet and Stephen Weeks that he refers to are the MLton guys)

cfleming02:10:57

So, the short answer to your question is: no idea, sorry simple_smile

alandipert02:10:55

oh wow, this is really juicy, thanks for the pointer!

cfleming02:10:23

Yeah, it’s pretty hard core

alandipert03:10:44

this continuation numbering process... very reminiscent of exploding maps into quads for datomic

alandipert03:10:40

same thing, i suppose. represent a tree as a series of structures instead of a nested map by numbering paths

cfleming06:10:52

I’m not familiar with the datomic technique, I’ve never really used datomic.

cfleming08:10:48

@alandipert: kotlin-reflect is only required if you’re using Kotlin’s reflection capabilities

alandipert14:10:34

@cfleming: good to know about kotlin, thanks!