Fork me on GitHub
#parinfer
<
2016-02-17
>
cfleming01:02:07

PARENS does require the bidirectional mapping, but that can be fixed trivially.

shaunlebron03:02:00

@cfleming: modifying parinfer for the lexer may be more trouble than it’s worth, especially since kotlin parinfer is really fast

shaunlebron03:02:46

chris and I will maintain the ports as the canonical one is updated

shaunlebron03:02:00

might be difficult to continue updating the kotlin fork

cfleming03:02:09

@shaunlebron: Yeah, I’m not sure if it’s worth it yet. It does have some nice features - everything is simpler because the state tracking is easier, and it works better with the existing infrastructure (everything is tracked using source offsets rather than line/col, allowing minimal updates to happen)

cfleming03:02:22

I’m going to evaluate tomorrow and see where it’s at.

shaunlebron03:02:32

oh okay, that sounds cool then if you got it figured out

cfleming03:02:40

But merging in future changes is definitely a concern.

cfleming03:02:58

Well, I have it figured out but not finished yet simple_smile

shaunlebron03:02:30

you gave an example I didn’t address

shaunlebron03:02:38

(let [x 10]<caret>)

shaunlebron03:02:01

just run paren mode when pressing enter

shaunlebron03:02:32

and you’ll get

(let [x 10]
  <caret>)

cfleming03:02:33

To ensure that the paren stays on the caret line?

shaunlebron03:02:54

mike fikes does this in replete

cfleming03:02:56

The problem is that parinfer is run asynchronously after the event happens.

cfleming03:02:15

It’s actually surprisingly hard to detect “this change is as a result of this action"

cfleming03:02:50

I’d need to detect cases like “a change that only inserts whitespace including a newline” which would catch that case.

shaunlebron03:02:46

not sure about that heuristic

cfleming03:02:19

The async nature of this integration makes a lot of the edge case rules pretty hard.

shaunlebron03:02:45

do you at least have a guaranteed order?

shaunlebron03:02:03

i.e. key event comes before document update event?

cfleming03:02:42

Yes, but it’s tricky due to the multiple document model.

cfleming03:02:26

I may have to try something more complicated where I try to detect key events and work out whether doc changes are as a result of them.

cfleming03:02:14

More or less what Dmitry suggested in the forum.

cfleming03:02:53

But events like cut & paste are tricky, since someone could paste a carriage return in there instead of pressing enter.

cfleming03:02:41

And doc changes as a result of other events like code reformatting are hard too, or at least have potential for lots of edge cases.

shaunlebron04:02:58

yeah, these edge cases seem difficult

shaunlebron04:02:25

that is my fear when I think about straying from mode invariants

cfleming04:02:53

yeah, I can totally understand that.

cfleming04:02:29

But things like the paren clamping are already changing those invariants to make things more intuitive/correct.

cfleming04:02:06

BTW I really like the idea of having the tests in Markdown - I wish I’d thought of that.

cfleming04:02:29

Currently I have a ton of test cases with the before/after cases in strings, which is really ugly.

cfleming04:02:59

I was going to have them in separate files, but then you end up with thousands of them.

cfleming04:02:09

I’ll be stealing that one simple_smile

shaunlebron04:02:16

I just hate escaped syntax

shaunlebron04:02:48

what are your test cases for?

cfleming04:02:33

Tons of things - all the paredit stuff, for example.

cfleming04:02:36

I create files in syntax like (some test <caret> code) which I use to create an editor with a caret, then I run an action (e.g. slurp, barf, whatever) and then check that the output matches (some test) <caret>code or whatever.

cfleming04:02:04

But lots of them are multiline, and the formatting is important, so I can’t just use quoted forms.

cfleming04:02:08

IntelliJ actually has very few unit tests, they’re mostly of that type - set up a scenario, do some interaction, check the result. It’s much more robust to refactoring the underlying code than unit tests are.

cfleming04:02:21

I have a mix but probably 90% are interaction tests like that.

shaunlebron04:02:27

cool, seems like a lot to test then

cfleming04:02:41

Yeah, I have around 2000 tests I think.

shaunlebron04:02:44

i’m a bit spoiled by parinfer testing, it’s nice just thinking of pure functions

shaunlebron04:02:02

rather than interactions I mean

cfleming04:02:19

Right - lots of the tests are quite complex: Set up file1 to look like this, file2 to look like this, then make file3 look like this, run autocompletion, select the element called “foo” and make sure that the output of the files looks like file1’, file2’, file3'

cfleming04:02:55

Even with all that, things still break - it’s just so complicated.

shaunlebron04:02:02

really interesting

cfleming04:02:06

And lots of UI stuff is difficult to test.

shaunlebron04:02:19

I’m really curious about how others do UI testing

shaunlebron04:02:34

it seemed like a mess when I tried it

shaunlebron04:02:40

so many different layers to it

cfleming04:02:41

It generally is a mess.

shaunlebron04:02:05

and also trying to decide at which level to test

cfleming04:02:27

JetBrains basically don’t test the UI, nearly all their tests are model based as I described, and they assume that the UI works.

shaunlebron04:02:29

and brittleness of tests is a big issue for UI, unless design is done

cfleming04:02:37

They probably do some manual testing too I guess.

shaunlebron04:02:04

i was handed a spreadsheet of things to check before our last deploy at stripe

shaunlebron04:02:13

made me think, hmmm

cfleming04:02:18

yeah, I think that’s really common.

cfleming04:02:55

For web stuff, Selenium is really good I think, but it doesn’t help with design/layout bugs.

cfleming04:02:11

There are tools that compare image snapshots, but I think they’re very brittle.

shaunlebron04:02:45

stripe uses bitmap testing for the style guide, i think that makes sense

shaunlebron04:02:57

but for UIs in development, i don’t see much value in that

shaunlebron04:02:29

i was looking at a stub/mock/spy framework that we use

shaunlebron04:02:59

so much effort going into stubbing out function paths and verifying if certain things are being called, and how many times

cfleming04:02:39

Those are really brittle too, in my experience.

shaunlebron04:02:40

and my last job had me writing tests in english, using cucumber

shaunlebron04:02:55

so the PM and QAs could write them

shaunlebron04:02:00

just didn’t work in practice

cfleming04:02:06

Did you like cucumber? A few people have asked for it.

shaunlebron04:02:08

for what I tried anyway

cfleming04:02:11

For Cursive, that is.

shaunlebron04:02:32

I didn’t read the full cucumber book, so I may not have used it correctly

cfleming04:02:35

I think there’s a lot of value in them being able to understand the tests.

shaunlebron04:02:39

I really liked the philosophy of it

cfleming04:02:43

Not necessarily write them.

shaunlebron04:02:28

I think it’d work great outside of UI actually

cfleming04:02:54

In my last job we made a graphical UI and language for modelling transaction flows. The idea was that technical business analysts would be able to design the flows, but that turned out to be a pipe dream. But it was really valuable that they could understand them.

shaunlebron04:02:57

even though this idea might fail a lot, i think it’s important to keep trying for sure

cfleming04:02:09

Parinfer, you mean?

shaunlebron04:02:22

oh, invisible segue

shaunlebron04:02:50

it reminded me of visual representation for designing flows

shaunlebron04:02:25

nasa tried something similar for creating their software, so that requirements could be specified in flow, and then directly executed

cfleming04:02:36

Interesting.

cfleming04:02:47

We found it didn’t work that well.

cfleming04:02:34

We would have been better off with a language or DSL that forced a strict development model on actual code, which you could then introspect to produce the graphical flows from for consumption by BAs or whoever.

shaunlebron04:02:48

it turns out that text is better for skimming large code bases, and visual flowcharts lose their benefit once they get too large

cfleming04:02:50

One thing it did work extremely well for was selling the project.

cfleming04:02:06

Customers love a pretty picture.

shaunlebron04:02:28

yeah, that DSL idea seems ideal

shaunlebron04:02:41

everyone loves a pretty picture

cfleming04:02:15

Not the developers when they’d rather express what they’re doing in code simple_smile

shaunlebron04:02:21

did you design cursive website?

cfleming04:02:29

yeah, I did

cfleming04:02:33

It could use a little love

cfleming04:02:57

A lot, in fact. I’m about to start re-vamping the doc, which is sadly out of date.

shaunlebron04:02:08

nah, I think the typography is good

cfleming04:02:31

I’m actually considering trying to pull the whole user guide into the IDE as an interactive tutorial.

cfleming04:02:44

Thanks, that’s mostly Skeleton’s doing.

cfleming04:02:02

Although I swapped out their font, I don’t like Raleway

cfleming04:02:39

The animations are neat, though, I stole the idea from the competition: http://www.sublimetext.com/~jps/animated_gifs_the_hard_way.html

shaunlebron04:02:45

i wonder if mp4 is supported enough

shaunlebron04:02:54

and if it would look as sharp

cfleming04:02:09

Probably not, that’s basically lossless true colour

cfleming04:02:38

And even if you rely on canvas and ditch the fallback, it still works pretty much everywhere (including phones etc)

shaunlebron04:02:05

ah, there you go

cfleming04:02:37

I found the images got a lot bigger when doing non-trivial demos including autocomplete dropdowns etc - the diffs got a lot bigger.

cfleming04:02:07

If I did this as an interactive tutorial I could do something similar to what you did with codemirror.

cfleming04:02:50

Which would be much easier to keep up to date - some of those animations on the website are from IntelliJ 12 and a version of Cursive from 2 years ago.

cfleming04:02:46

I have about half a system written that would allow me to record them as macros, but there’s still a bunch of manual steps I’d have to do for things you can’t script easily.

cfleming04:02:53

(much like testing).

shaunlebron04:02:55

right, but not even sure how you would simulate cursive in the browser...

cfleming04:02:22

Well, if i did them as an interactive tutorial they’d be inside Cursive itself.

cfleming04:02:32

And I wouldn’t even have an online user guide.

cfleming04:02:02

I have some ideas about something like a typing tutor for paredit, for example, that could work well with that.

shaunlebron04:02:23

that makes more sense, haha

shaunlebron04:02:36

yeah, that’d be great

cfleming04:02:50

Ok, got to go. I think this change is probably getting too big. I’m going to try a smaller refactoring tomorrow which would create a small set of diffs instead of replacing the whole thing, and a few related changes.

shaunlebron04:02:14

alright, sounds good

cfleming04:02:36

I think I’m going to modify paren mode to return the lines which need reindenting, and then I’ll reindent them accurately using the Cursive engine.

shaunlebron04:02:02

yeah, auto-indent has to happen somehow like that

shaunlebron04:02:42

you might be able to get away with just auto-indent the parent form altogether, unless speed is really that bad

shaunlebron04:02:49

but yeah, talk to you later

cfleming04:02:27

That’s what I’m thinking, I’ll identify the top-level forms and wholly reindent them. Cursive will have to do the expensive part of the reindentation for that anyway.

shaunlebron19:02:09

i got parinfer returning the cursor position this morning

shaunlebron19:02:21

gonna add more test cases tonight to confirm

shaunlebron19:02:09

thanks cfleming for the push. way simpler than i thought. really important feature too

sekao21:02:54

@shaunlebron: that’s pretty cool, is that mainly to make it easier to reposition the cursor after hitting enter? i’m adding parinfer to paren-soup and already got that working but it would still be useful

shaunlebron22:02:58

@sekao: oh hey zach 🙂

shaunlebron22:02:49

yeah this actually takes care of a few other cases. not just pressing enter

shaunlebron22:02:41

for example, it will keep your cursor from moving if you insert a close-paren that parinfer removes

shaunlebron22:02:30

any case where some part of the line before the cursor is added or removed

sekao22:02:17

hey greetings shaun — that sounds perfect, i’ll keep an eye out for that update. i only spent last night shoehorning it into paren-soup and it is already way nicer to use than the current release version. i definitely want to add this to nightcode as well.

cfleming22:02:02

@shaunlebron: Oh great, thanks for that! I’ll look at the change and if I have time I’ll port the changes over to the JVM version.

cfleming22:02:34

I’m going to see if I can get the tests running in the JVM version using Gradle too, to avoid having to install before testing.

shaunlebron23:02:22

great, i'll post a note in here about the change when i release

shaunlebron23:02:27

luckily the change is really small, so we can update ports pretty easily

shaunlebron23:02:35

@sekao: excited to see parinfer in night code! also glad parinfer-jvm is already taken care of for that