Fork me on GitHub
#joyride
<
2022-05-26
>
pez09:05:56

Before I settle for returning a string from the Calva evaluateCode API. What are the trade-offs between returning something potentially structured and a string? The consumer of this API might not be a ClojureScript program. It makes some sense to return a JS object, but it can't represent reader tags, for instance. And then also we might have a non-EDN result, even if right now, I'm not sure when that would be?

borkdude09:05:45

A string is the only viable option imo

🙏 1
borkdude09:05:04

You can of course return a map with a string and add more things later.

borkdude09:05:15

But the evaluation result should be a string

pez09:05:46

If you could elaborate a bit on this, that would be swell. I trust your judgment here, of course, but still would like to know about why it is the only viable option.

borkdude09:05:22

There is a reason why all of these tools use a string. The io.prepl also uses a a string:

$ clj
Clojure 1.11.0
user=> (clojure.core.server/io-prepl)
(+ 1 2 3)
{:tag :ret, :val "6", :ns "user", :ms 13, :form "(+ 1 2 3)"}
I'm sure if EDN would be possible, they would have done this. But you just can't represent non-data things (objects, JS functions, Clojure functions, etc) as data.

borkdude09:05:41

Maybe returning a format similar to io-prepl could be nice, for extensibility later

pez16:05:09

Here's a pull request on Calva adding the evaluateCode API: https://github.com/BetterThanTomorrow/calva/pull/1740 Install the PR VSIX build once it is built to test this. Docs here: https://github.com/BetterThanTomorrow/calva/blob/5b155d58035a7779aedeb95f8664ba6f74a29abb/docs/site/api.md (Slightly weirdly rendered, since it is made for MkDocs Material.)

pez19:05:56

The evaluateCode Calva API now released. https://calva.io/api/ cc @orestis @seancorfield Thanks to @alpox for helping me with it! I had a lot of fun in that pairing session.

🎉 1
seancorfield20:05:13

Nice! I think I know what I'm going to be doing this afternoon (as soon as I get off this call with our vendor 🙂 )...

joyride 2
pez20:05:28

Looking forward to see what you do with this! And I am also curious what your next need for a Calva API will be.

seancorfield21:05:46

@pez What is [z-joylib.editor-utils :as eu]? I tried following the ignore_form example but I get Could not find namespace: z-joylib.editor-utils

seancorfield21:05:42

Ah, it's in a subfolder! Sorry...

borkdude21:05:29

@pez What if you are connected to multiple REPLs, what part of the API controls which REPL you are sending a form to?

borkdude21:05:02

Shouldn't you be able to query the currently open sessions and get a unique id from those to indicate where to send?

borkdude21:05:34

E.g. if you are simultaneously connected to a joyride, JVM and nbb REPL

alpox21:05:18

From what I gathered the current design only allows for one clj repl and one cljs repl connection at one time cutting down the selection (for the moment) to "clj" and "cljs" which can be passed to evaluateCode as session key. Although im not quite sure if the joyride connection is treated as a normal cljs repl session :thinking_face:

seancorfield21:05:41

Yeah, currently, if you have the Joyride nREPL connected and try to invoke Calva's evaluate API, it'll execute it via Joyride.

seancorfield21:05:36

I developed most of my script interactively via the REPL, but then had to switch back to my Clojure nREPL connection to run my new script completely, since it relies on executing in a Clojure JVM environment.

orestis03:05:11

Nice!! I'll give it a spin today. I also wonder how should I detect which session key to use? Daily development has me connected to a deps.edn + shadow.cljs so I jump between CLJS and CLJ. I could probably use the filename extension for this, but not sure about cljc :)

seancorfield03:05:40

Yeah, my life is way simpler for only doing Clojure, but I do sort of look forward to a time where I might be doing cljs again (after seven year break at this point).

seancorfield03:05:42

It sure would be nice if Calva supported a) more than two possible REPL connections and b) automatically figured out the right REPL based on the currently-active editor 🙂

orestis04:05:37

The other immediate need that comes to mind to do interesting things is having access to constructs like current form, top level form etc etc.

seancorfield04:05:36

Yeah, I was a bit disappointed that it would not be easy to get "current form" -- executing paredit commands and trying to figure out ranges is way too messy. We need all of the substitutions that customREPLCommandSnippets can use, accessible as API methods.

orestis04:05:26

My thoughts exactly. I think adding a few more top level APIs like get current session key, get the current/top level form as text etc etc would allow for interesting combinations. Presumably Calva could also use the same API to do what it does today.

seancorfield04:05:19

Early days. This is just the first API -- and it's pretty powerful -- so I'm sure a lot more will follow 🙂

pez05:05:30

@U04V15CAJ, about the ”which session”. So what @alpox says, Calva only has two at most. See this old issue for a discussion around this: https://github.com/BetterThanTomorrow/calva/issues/76. So the unique id:s are clj, or cljs, (and also cljc, as it happens, which is an alias for either of those, that the user controls).

seancorfield05:05:27

Sure would be nice if it had more than two and had a different ID for Joyride 🙂

➕ 2
pez05:05:53

I choose to decouple this API from the ”magic” of automatically figuring out which session should be used. Thinking that an API for querying this makes more sense. Also thinking that in a world where Calva supports more configurable REPL connections, this whole story changes and less magic will help then.

2
seancorfield05:05:18

I'm all for "less magic".

pez05:05:33

As for API for getting snippet-variables. That's on my todo. For now to get e.g. the current form text, we can do something like this:

(defn top-level-form-text []
  (p/let [_ (vscode/commands.executeCommand "paredit.rangeForDefun")
          selection vscode/window.activeTextEditor.selection
          document vscode/window.activeTextEditor.document
          code (.getText document selection)
          _ (vscode/commands.executeCommand "cursorUndo")]
    code))
I tuck such things away in editor-utils and try to forget about all the mutation going on. 😃 But, yeah, eventually this will beak in some situation.

seancorfield05:05:44

That's fine for top-level form. What about "current form"?

seancorfield05:05:32

For my symbol/expression needs, "current selection" is OK since you really kind of need to specify just how much text (code) should be evaluated, but having all those "custom REPL command snippet" substitutions easily available would be awesome!

pez06:05:26

As it happens, I think all those calls are synchronous, so we get away without the promise. A more general function:

(defn text-for-selection-command [selection-command-id]
  (let [_ (vscode/commands.executeCommand selection-command-id)
        selection vscode/window.activeTextEditor.selection
        document vscode/window.activeTextEditor.document
        code (.getText document selection)
        _ (vscode/commands.executeCommand "cursorUndo")]
    code))
The two main examples:
(text-for-selection-command "paredit.rangeForDefun")
(text-for-selection-command "calva.selectCurrentForm")

pez06:05:32

For some reason it doesn't work for "editor.action.selectAll", I don't understand why...

pez06:05:41

To find the command ids, an easy way is to use the keyboard shortcuts editor:

pez06:05:55

With that particular command you can recreate calva's evaluate-thread-to-cursor command. You'll have to close the form, of course. Calva has code for that. I guess that's a candidate for an API function as well.

orestis06:05:18

So cool. I will spend some time today hacking with all this.

pez06:05:50

Awesome. When you start playing with the evaluateCode API and thus run into the limitations with Calva's REPL session managament, there's a recipe over at ClojureVerse for how to give yourself some evaluate code in Joyride commands: https://clojureverse.org/t/say-hello-to-joyride-the-clojure-repl-in-vs-code/8957/3?u=pez

seancorfield06:05:03

Thank you for this:

(text-for-selection-command "calva.selectCurrentForm")
This makes my Joyride scripts work much better!

metal 2
seancorfield06:05:54

(my vsode-calva-setup has been updated)

seancorfield06:05:23

Seriously, this is really f'ing amazing 🙂

❀ 2
pez06:05:32

An alternative to calva.selectCurrentForm, is paredit.sexpRangeExpansion, which in many cases does the same things, but if you run it an even number of times it can be used for selecting enclosing forms N levels up. You'll then have to shrink the selection back an equal amount of times, of course, (or Undo Selection, which is equivalent for this purpose.)

pez06:05:33

When exposing these as API function we'll not be using Paredit really, because those commands are very much the same as what you get with the executeCommand function. We'll need to expose the underlying TokenCursor (similar in concept to a rewrite-clj zipper) in some nice way. Will be quite exciting to see what people do with that one. It often gives me a feeling of unlimited powers using it. 😃

seancorfield06:05:19

I'd be interested in ways to parameterize my two scripts to reduce duplication.

seancorfield06:05:49

I know I could refactor to a "library script" in subfolder, like you have with z-joylib in the examples but that seems a bit artificial. Providing that sort of stuff directly inside Joyride itself seems reasonable, in terms of providing a Calva-specific ns that we can use.

seancorfield06:05:01

Maybe a joyride.calva namespace with some of the common stuff in?

pez06:05:08

It's not a perfect fit in Joyride, I think. But Calva could maybe provide it somehow... Right now Joyride only has the user and the workspace script folders on its class path, But if we provide a way to add to the classpath, then a calva joyride library could live in the calva extension folder. WDYT, @U04V15CAJ?

seancorfield06:05:00

Ah, so it would be automatically available if you were running Calva? Interesting.

seancorfield06:05:36

I can imagine several calva.* commands being available as direct functions, to make life easier.

borkdude06:05:38

Hm yeah, that's interesting

pez06:05:51

It could be general, Joyride checks all extensions for if they provide a library and adds to the classpath.

orestis06:05:26

That would be nice also for the portal extension.

seancorfield06:05:32

This feels like boilerplate, for example:

(def ^:private calva (vscode/extensions.getExtension "betterthantomorrow.calva"))

(def ^:private calvaApi (-> calva
                            .-exports
                            .-v0
                            (js->clj :keywordize-keys true)))

(defn- evaluate [code] ((:evaluateCode calvaApi) "clj" code))

seancorfield06:05:22

Even this:

(defn- selected-text []
  (vscode/commands.executeCommand "calva.selectCurrentForm")
  (let [editor ^js vscode/window.activeTextEditor
        selection (.-selection editor)]
    (.getText (.-document editor) selection)))
select-current-form would be a good name for this, providing the text of selecting the current form.

borkdude06:05:27

@pez Is it possible to determine the loading order of extensions? E.g. extensions could also actively register extra joyride namespaces

seancorfield06:05:51

And maybe a higher level execute function for this:

(-> (p/let [code (clojuredocs-url (selected-text))
            resp (evaluate code)
            url  (edn/read-string (.-result resp))]
      (vscode/commands.executeCommand "simpleBrowser.show" url))
    (p/catch (fn [e] (println (str "Evaluation error: " e)))))

borkdude06:05:00

There's always a boilerplate/flexibility balance to strike

seancorfield06:05:25

(joyride.calva.evaluate (clojuredocs-url (joyride.calva.select-current-form)))

pez06:05:38

@U04V15CAJ, so if Calva want's to provide a library it would rather tell Joyride that, than that Joyride checks Calva for it?

borkdude06:05:12

Yeah, potentially it could call (joyride.api/add-namespace 'foo.bar {'dude (fn [] :hello)})

borkdude06:05:33

but this would only work if joyride would be loaded before calva I guess

pez06:05:20

I think it might be. But Calva could also wait for it to be active before it does it.

borkdude06:05:14

Is there a hook in VSCode to wait for all extensions to have finished loading, or extension("joyride").ready(fn ...) or so

pez06:05:35

Not that I know of.

pez06:05:54

Doesn't mean there isn't one, of course.

borkdude06:05:53

We could also do it another way. Whenever a namespace is loaded, we first search the classpath and then all extensions, lazily.

calva
.-exports
.-joyride_namespaces
.-the_namespace_you_were_looking_for

pez06:05:12

@seancorfield if Calva can provide a library to Joyride, it could deffo be used to reduce boilderplate.

1
borkdude07:05:33

Another idea:

(require '["extension-name$namespace" :as calva])
(calva/evaluatecode "...")

borkdude07:05:04

And then Joyride searches within the extension-name exports for the namespace object (and treats this like any other JS library)

borkdude07:05:02

extension-name would be calva here for this example

borkdude07:05:46

So the full example would be:

(require '[""betterthantomorrow.calva$joyride" :as calva])
(the dollar sign is already a convention in CLJS for getting a property out of a JS library)

pez07:05:02

Yes, I think that is very similar to what I had in mind starting this discussion. Even if I was thinking it would be a joyride/cljs library rather than a JS one.

borkdude07:05:59

It doesn't really matter, but in the end, the extension exports a JS object, so we can just treat like a JS library. Then other extensions not written in Clojure can also contribute functions we can use in joyride

borkdude07:05:10

I think as it stands, you'd have to do:

(require '["betterthantomorrow.calva$v0" :as calva])
And then you could immediately do:
(calva/evaluateCode ...)
This would work with the current Calva extension

borkdude07:05:00

Yeah, I like that

pez07:05:21

I'm thinking more that it would work very similar to what we do with user and workspace scripts. It would basically be joyride wrappers around the extension JS api.

borkdude07:05:28

There is no need to wrap, imo. You can just require the functions from the extension and use them

pez07:05:47

Ah, there could be other reasons why you would want to wrap things though. Some things only make sense to Joyride and the general API is not a good place for that.

borkdude07:05:10

That is why you can select a certain part from the extension that is the specific joyride API:

(require '["betterthantomorrow.calva$joyride" :as calva])
This also allows you to have those functions pre-compiled (as JS) and thus would avoid interpretation overhead.

borkdude07:05:48

I think it makes sense to have these functions as compiled JS so any extension can participate, not only those built with ClojureScript

borkdude07:05:04

E.g. one could make a TypeScript extension and functions from those could be used from joyride as well in this way

pez07:05:56

I don't think it would limit things to extensions written in ClojureScript. Calva is written mostly in TypeScript. Anyway, I opened up a discussion on the repo for this: https://github.com/BetterThanTomorrow/joyride/discussions/70

borkdude07:05:02

I already responded to https://github.com/BetterThanTomorrow/joyride/discussions/69 which a couple of points. One point you seem to overlook is that extensions can provide functions directly, not code that should still be interpreted.

borkdude07:05:18

It seems 70 is a duplicate, let's close that one.

pez07:05:01

Ah, Github gave me errors so I didn't realize I had opened #69. Will update that one then.

pez08:05:58

I know I replied to your comment on the #69 discussion, @U04V15CAJ, but now Github seems to be hiding them. It says 2 replies in one place, and 0 replies in another, and the latter seems to be what it is using. 😃

seancorfield21:05:46

@pez What is [z-joylib.editor-utils :as eu]? I tried following the ignore_form example but I get Could not find namespace: z-joylib.editor-utils

seancorfield22:05:35

This was the "missing piece" after my switch from Clover to Calva: with a symbol or expression highlighted in your Clojure code, running this Joyride script will: a) evaluate the selection in the context of your Clojure nREPL b) which includes the wrapper to produce a JavaDoc URL based on the type of that evaluated code c) and the result is then used to open SimpleBrowser inside VS Code at that page. I used this a lot with Clover where I could highlight some Java class type (or expression), hit a hot key (I have this bound to ctrl+alt+o j to match my other config which is all custom REPL snippets!), and get the JavaDocs directly in VS Code. Now I'm off to make one for clojuredocs which was the other wrapped up URL renderer I had in Clover!

🎉 2
borkdude22:05:08

Very nice. Feel free to PR this to the joyride examples directory

seancorfield22:05:46

Ah, good point! Yup, I'll clean it up and add more comments and send a PR, along with the clojuredocs.cljs that I just wrote.

seancorfield22:05:25

(they are very similar)

seancorfield22:05:56

I've updated my VS Code / Calva setup repo with these scripts for the time being -- https://github.com/seancorfield/vscode-calva-setup