This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-02-04
Channels
- # aatree (5)
- # admin-announcements (37)
- # alda (1)
- # announcements (4)
- # architecture (1)
- # aws (3)
- # beginners (82)
- # boot (230)
- # braid-chat (14)
- # cider (48)
- # cljs-dev (8)
- # cljsrn (31)
- # clojars (47)
- # clojure (72)
- # clojure-austin (2)
- # clojure-russia (396)
- # clojurescript (72)
- # community-development (3)
- # component (6)
- # core-async (6)
- # cursive (26)
- # datomic (42)
- # emacs (6)
- # events (35)
- # hoplon (57)
- # immutant (3)
- # jobs (2)
- # jobs-discuss (10)
- # ldnclj (16)
- # luminus (2)
- # off-topic (50)
- # om (181)
- # parinfer (285)
- # proton (68)
- # re-frame (19)
- # reagent (2)
- # ring-swagger (23)
- # yada (36)
awesome
thanks for setting that up
I was planning on playing around with it, but will focus on something else
haha - I like that you kept lots-o-clojure
man - JS is beating the pants off this JVM version
but now we know that with confidence
and can optimize
the values in the table here: https://github.com/shaunlebron/parinfer/pull/88
indent mode for really_long_file is around 12ms, 23ms for paren mode
sweet
I am going to set up travis-ci to run the test suite on every commit
I think that should be not-too-hard to set up
wonder what happens to the performance if you put SENTINEL_NULL back in?
I might give that a try; going to set up travis-ci first
~/d/p/parinfer-test (master)> lein with-profile bench run
Evaluation count : 4920 in 60 samples of 82 calls.
Execution time mean : 12.537784 ms
Execution time std-deviation : 436.185958 µs
Execution time lower quantile : 11.935113 ms ( 2.5%)
Execution time upper quantile : 13.447069 ms (97.5%)
Overhead used : 1.707955 ns
Evaluation count : 5280 in 60 samples of 88 calls.
Execution time mean : 11.775496 ms
Execution time std-deviation : 408.468619 µs
Execution time lower quantile : 11.253086 ms ( 2.5%)
Execution time upper quantile : 12.753575 ms (97.5%)
Overhead used : 1.707955 ns
Found 5 outliers in 60 samples (8.3333 %)
low-severe 5 (8.3333 %)
Variance from outliers : 20.6507 % Variance is moderately inflated by outliers
I replaced a few of your functions with stdlib equivalents that are more efficient, especially around repeated String concatenation.
oh man 😄 😄 😄
this is quite exciting
Is it true that http://result.ch is always a character, or empty?
I believe so
I'm not sure; Shaun would know the answer to that question
I know he had to make some change in order to support Racket
there might be
it's his first week working at Stripe, so he's not around as much
although I spoke with him last night; he is pretty excited about parinfer-jvm as well as the JS speed-up
here's hoping
gotta add parinfer to cursive first!
how hard do you think that will be?
there can be some trickiness adding parinfer to editors
I can probably do a naive impl relatively easily, but there’s a few bits around the edges making it user friendly.
yeah - are you familiar with the "parent expression" hack that atom-parinfer uses?
Shaun and I were talking about this last night; with the performance we're at now we might be able to just take it out
it's so fast already
Yeah, I am. I’m actually planning to benchmark just parsing the top level forms from the top of the file until, say, the last one in really_large_file, and then parinfer’ing that.
haha - mental slip
you named a folder "paredit"
I will fix it; I have to commit something in order to test travis-ci
there might be a flurry of small commits in order to test that actually - nothing major
is it a convention to use the performance
namespace?
I was going to change that to parinfer-perf/core
?
wasn't sure if there was a convention there; I assume there is for tests
it would be faster if it were just a Char on the JVM?
so sweet
that test takes forever to run on my systme
but I get similar results
and your commit incidentally tested the build script
which worked perfect the first time
so we now have an automated build / test
should we mark a v0.2.0 with your changes?
do I need to add that number to build.gradle
?
in JS and Python people just include the file in their own builds
in Java people use actual packages and .jar files, right?
and is it ok to follow semver format there?
version '0.2.0'
I suppose another question I have is: what should the public API look like?
right now it's ParinferKt.indentMode()
drop the Kt ?
because it has to pass the type system I suppose
that sort of thing is pretty important in JS
also I went with 3-arity options instead of a hashmap of options
since Java doesn't have literal data types
I guess the question is: if this already existed and you were just adding it to Cursive, would what you want the API to look like?
lots of firsts for us both here
I think we should probably create the object explicitly so we can control the name, rather than have it be autogenerated.
yeah - I saw there was a way to do that in Kotlin
@Jvmname
or something
but I couldn't figure it out
yeah - I was going to look at that tonight
that discovery yesterday gave me the itch to squeeze out more performance
it's not too often that that sort of thing is the best way to spend your time
working on perf like that vs working on features or design
(at least in my experience)
but in this case it's worth it
parinfer has to run constantly as you're typing
speed is essential
Yeah. I’ll have to check how fast it is in Cursive because it’s not just manipulating strings, it’s fiddling with documents that have locks and so forth.
Will you make that API change? I think I prefer just Parinfer
to ParinferKt
In Atom, are files ever changed in the background? There’s nothing like refactorings or anything like that, right?
So these changes are only ever applied to a document while a user is interactively editing it?
Actually, what about if a file is updated from VCS, which is effectively in the background as far as Atom is concerned?
hey guys, just started catching up on messages here
@snoe: I like that idea you had yesterday! tracking here: https://github.com/shaunlebron/parinfer/issues/89
@thomas: by “block comment” do you mean “comment the selected lines” or “comment the lines of the cursor’s sexp”?
@cfleming: parinfer looks at all parens outside of comments, strings, and character literals. so it defaults to treating #{
, #[
and #(
correctly.
it doesn’t actually know anything about the preceding #
, so it’s treated as if it wasn’t there
and secondly what you said here is good
I’m actually planning to benchmark just parsing the top level forms from the top of the file until, say, the last one in really_large_file, and then parinfer’ing that.
hey man
like the activity happening here!
Colin is kicking total JVM ass; the benchmark on parinfer-jvm is cut in half from my original implementation
I saw it was around 40ms for indent-mode in really_long_file?
that's old news; now it's down to ~12ms
well then
that is good news
i can’t find the confetti emoji
I'm playing around with more JS optimizations
I'm a little obsessed with it now
little worried that the "clean implementation" is going to have more JS-specific stuff
that might not translate as easily to other languages
but that's not really a big deal; we can always link people to an older version
i think it’s fast enough
just seeing very little difference between indentMode and parenMode on the JVM
and thinking that probably that can be achieved in JS too
I think in reality locating the parent expression is the important part
and I feel like that circuit is finally closing
colin can just use his lexer to locate that
emacs already has fast support for that as well
and for editors without their own native support for locating it, I can put the fast reader back in the API
parinfer without all the transformation business can run through really_long_file in about 7ms
idk why i didn’t think of this earlier. chatting with the emacs guy today helped I think
https://clojurians.slack.com/archives/parinfer/p1454558963000385 atom-parinfer only ever looks at the current buffer
@shaunlebron: I added a couple examples to the issue.
@cfleming: in atom, if a file is changed in the background, atom-parinfer won't run until the user does something to trigger it, like press any key on the keyboard
ah, thanks it’s :tada:
🎉
I think we’re moving into paredit territory
from your example @snoe, you want an indent-next-sexp
function, not a indent-next-line
function, since it indents the child expressions with it
secondly, you want it to indent two spaces
the offset from the tabstop would have to be configurable
I think the parinfer API can provide some primitives that allow parinfer plugins to handle this behavior themselves
@shaunlebron: !!!!!!!
I just figured out the problem with paren mode
parinfer.js is about to get faster
I agree, maybe parinfer could return something like indentMode(..., {...}) => {:text "..." :tab-stops [1 4 8 11]}
?
The make edit, goto next line, select form, indent/dedent selection dance is still something I find myself doing all the time. I wonder if others do the same thing?
@chrisoakman: sheeeeeiiit
that must have been preventing some v8 optimizations
@snoe: I think that info will be enough for one-space indentation
you’ll need more info for (
to be two-space, and for (expr arg
to be n-space indent
yeah, the important part is making the structural change. formatting can/should be handled outside.
and whatever rules required for determining that stuff
I see what you’re saying, so do a separate auto-indent operation after parinfer moves it to one-space indentation
@snoe, seems like we need either indent-selected-lines
option instead of a single line
or some way to specify the lines of an sexp
Maybe : 1) go to next line -> 2) find first char -> 3) keep going down until you find a char before step 2
gah - I am super-happy about that find
I had a gut feel it was a JS thing
parinfer.py and then parinfer-jvm showed that the algorithm should be returning similar result times for both functions
something was clearly happening in the JS version
I'm not sure we're going to get faster than that; I was playing with replacing the if/else with dispatch and such
but doing object key lookup in JS isn't free; in most cases it was hurting perf
@chrisoakman: cool, I wish we had something Criterium for node
it's v8 optimizations
that's what this boils down to
i can’t figure out why the last two tests are any faster than the previous
I considered changing the perf.js to run multiple iterations and then show the average, etc
I only included one result on that PR, but ran it several times on my machine
it might be saving some data from the first test
and it was consistently faster when Object.preventExtension
was removed
some caching I mean
to be honest, I "discovered" that earlier today on a Windows machine running an older version of node
saw a huge bump when I removed that
yes - but I can't seem to make heads or tails out of the .log files it produces
in both of these last cases of "discovering" big speed improvements, it's clear they are v8 optimizations
not fundamental algorithm or data structure changes
I did see some perf improvement in parinfer.py by changing that if/else chain to a Dict lookup
but when I did that in JS it was slower across the board
I'm looking at Python string concatenation now
@chrisoakman: you were asking about the next language to tackle maybe if you do C/C++ you could then use emscripten for js (or c node module)
Shaun already went down that road a bit
with c++
and decided to scrap it
because the JS version is fast enough now
I'm thinking emacs lisp should be the next target
I don't use emacs, nor terribly wish to write the plugin
but I think writing the parinfer algorithm would be fun
I also want to do it in F#
but only because I want to do something in F# 😉
I don't know what editor would benefit from having an F# implementation; I don't think there are too many people using MS Visual Studio to edit Lisp code
another benefit of writing Parinfer in emacs lisp is that I get to use parinfer to write it in !
which is cool and recursive
not sure if I added enough context about finding the parent expression in a file
but cursive, emacs, and parinfer stripped down to its reader form only, all of these can locate a parent expression very quickly and reliably
so it’s like using atom-parinfer’s parent expression hack, except without the corner cases
I am seriously considering dropping the parent expression hack for files under a certain N length
is what I said clear
I mean - parinfer.js can do a 2800 line file in under 10ms
that's the debounce rate
just to put that in perspective
yeah for what it's worth I haven't bothered with expressions in vim because the whole file is pretty much fast enough
I am willing to wager that 99% of lisp files that people regularly work with are under 1000 lines of code
there are obviously exceptions to that, like if you're working on cljs.core
what i’m saying is that it’s now possible to hit 100% reliably and quickly
are you going to add another function to the API?
is what I said about the parent expression thing clear?
vim's select top form: https://github.com/guns/vim-sexp/blob/b4398689f7483b01684044ab6b55bf369744c9b3/autoload/sexp.vim#L1054-L1211 hehe
maybe not?
you use regex
emacs and cursive can do the same as your regex, but reliably and as fast
so can parinfer’s reader
by just keeping a running data structure of the whole file?
i’m not sure how emacs and cursive do it
but emacs’ paredit and smarparens plugins have fast implementations for it
the point is that data is available for parinfer to use
I think atom uses regular expressions for the grammar engine
actually, I know it does
for syntax highlighting and whatnot
which is probably a decent decision for most languages, but probably the wrong one for lisp-based languages
where it would be cheaper to just keep the actual AST
right, that will not work for parinfer
breaking syntax highlighting will not corrupt your code, only its coloring
thanks @snoe! this looks how most editors would locate the parent expression: https://github.com/guns/vim-sexp/blob/b4398689f7483b01684044ab6b55bf369744c9b3/autoload/sexp.vim#L1054-L1080
i want to test that in vim to see how well it works
heading out for the night
@snoe, I’ll keep thinking about the indent thing
did you bump parinfer.js on npm?
@chrisoakman: thanks for speed improve, will publish
I don't think we're going to get too much faster than this, btw
would be interesting to see how Chakra or SpiderWhatever performs
thanks man
Last I saw, the strategy was to rewrite parinfer in emacs lisp, instead of using the lib