Fork me on GitHub
#rewrite-clj
<
2020-08-24
>
lread20:08:38

Ok, I should figure out what I need to do for namespaced elements for rewrite-cljc. (It feels like I might have said this a year ago 🙂). Right now I am trying to guess if a correct sexpr on an auto-resolved namespaced keywords and maps would have real value to rewrite-cljc users. For reference, I am calling ::mykeyword1 current-namespaced and ::alias/mykeyword2 alias-namespaced.

lread20:08:33

@vemv you had some https://github.com/lread/rewrite-cljc-playground/issues/7#issuecomment-667743070. I’m wondering if your issue has more to do with the awkwardness of working with the node than any issue specific issue with sexpr.

vemv09:08:35

I reviewed my usage of the technique I mentioned over GH I use sexpr, but as an intermediate step, that might as well be discarded with an alternative implementation. In the end I care about being able to replace ::foo/bar with ::quux/bar. In my codebase that is a Node to Node transformation, with some hacks in the middle. So, answering to your question (`if your issue has more to do with the awkwardness of working with the node than any issue specific issue with sexpr`), I think it's the former :)

lread16:08:19

Thanks @vemv, I’ll take a look and see what I can do to improve in that area.

borkdude20:08:33

I guess the most important things to consider are: 1) is such a node distinguished from other rewrite-cljc maps/keywords -- can you tell if it's current-namespaced, or alias-namespaced by looking at some property? btw. I don't think rewrite-cljc can determine properly if the a in ::a/foo is an alias or full namespace name 2) can it round-trip properly, yielding the same string

lread20:08:03

Thanks for the reply @borkdude. I was thinking of maybe adorning the node with a namespaced-type which would be :literal , :alias or :current.

lread20:08:35

For ::a/foo you are saying a could be a namespace name or the namespace alias, right?

borkdude20:08:25

I'm just trying this out and perhaps it's always an alias

borkdude20:08:35

$ clj
Clojure 1.10.1
user=> (require '[clojure.string])
nil
user=> ::clojure.string/foo
Syntax error reading source at (REPL:3:0).
Invalid token: ::clojure.string/foo
user=> (require '[clojure.string :as str])
nil
user=> ::str/foo
:clojure.string/foo

borkdude20:08:43

so in that case it would be a good name

borkdude20:08:34

What is an example of a map with :literal namespaced-type?

lread20:08:05

Oh that’s my terminology sorry, I did not find a single definitive terms so made some up. An example would be :my-prefix/mykeyword3.

borkdude20:08:45

So the namespace and a boolean indicating if it's an alias

borkdude20:08:23

:foo/bar -> foo, false ::foo/bar -> foo, true :bar -> nil, false

lread20:08:28

I looked your version. It was helpful. Do you ever sexpr on, for example ::bar in clj-kondo?

borkdude20:08:45

I guess I have some weird keyword for representing the current namespace: :__current-ns__

borkdude20:08:14

since symbols and keywords are separate namespaces you can do that

borkdude20:08:36

let me try sexpr and see what happens

borkdude20:08:01

user=> (p/parse-string "#:foo{:a 1}")
<namespaced-map: #:foo{:a1}>
user=> (p/parse-string "#::foo{:a 1}")
<namespaced-map: #::foo{:a1}>

borkdude20:08:45

I guess sexpr is not a valid operation on these nodes since it will depend on your aliases

lread20:08:50

For your 2nd point on round-tripping I think that won’t be challenging and is not an sexpr concern.

borkdude20:08:51

in your runtime

borkdude20:08:29

I solved this problem in edamame by allowing the user to provide a function to resolve the alias

borkdude20:08:04

since you won't break the API of rewrite-clj, you could consider a dynamic var for the same purpose, or extra args in sexpr

borkdude20:08:04

it's only problematic with aliases. I think blindly resolving it to the runtime's aliases is not good

lread20:08:11

Yeah… rewrite-clj current has some partial support in for alias-namespaced maps where it does a (ns-aliases *ns*)…

lread20:08:27

Which implies that the code being parsed has been loaded.

borkdude20:08:24

which also implies a coupling between the tool parsing the code and the code being parsed, which does not make sense at all.

lread20:08:09

I’m not a big fan of the current approach.

lread20:08:52

I noticed that cljfmt actually parses the ns declaration to get aliases and considered that as an option.

borkdude20:08:02

I think that logic belongs somewhere else

borkdude20:08:30

you should just be able to parse #::foo{:a 1} with rewrite-cljc and provide an option how to resolve the aliases

borkdude20:08:34

before you know it you're writing a clojure interpreter. consider:

(ns foo (:require [clojure.string :as str]))
(in-ns 'bar)
#::str{:a 1}

lread20:08:47

I was thinking that rewrite-cljc might provide a rudimentary parser/resolver, but you could also plug in something else. But you are thinking such a thing does not belong in rewrite-cljc?

borkdude20:08:49

I can probably come up with a number of other examples where this will break down

lread20:08:00

Yes, that’s the rabbit hole I am struggling with.

borkdude20:08:36

I would do it in the reverse: start with the pluggable option, don't go down the rabbit hole until people actually want this to be part of the library

lread20:08:20

Yeah, that’s good advice.

borkdude20:08:37

The option before that could be: don't support sexpr for these nodes.

borkdude20:08:02

Can people implement sexpr themselves via some protocol?

borkdude20:08:49

(sexpr nsm {:resolve {'str 'clojure.string}})

lread20:08:50

Yeah, that was one consideration, but I think that would break some rewrite-clj fns. I think I’ll have to return a technically incorrect value for sexpr for auto-resolved namespace elements. This result could be improved by a plugin.

borkdude20:08:58

In edamame:

(parse-string "[::foo ::str/foo]" {:auto-resolve '{:current user str clojure.string}})
;;=> [:user/foo :clojure.string/foo]

borkdude20:08:31

I'm using this in sci where I already know the aliases because of handling the previous forms

lread20:08:11

Yes, at a minimum I’ll provide some mechanism for people to provide current namespace and aliases via hand-coding…. if that’s what you mean.

borkdude20:08:23

hence where I'm getting the idea from that an interpreter and a parser/rewrite-tool are separate things

lread20:08:51

I’ve always been half-annoyed that rewrite-clj includes sexpr at all.

borkdude20:08:38

well, it's convenient, but I think in 99% of cases I call it to determine the type of thing

lread20:08:44

I can see that it makes usage more convenient though… but in some cases, like this one, well.. it is not..

lread20:08:49

what you said!

borkdude20:08:11

so it would have been cool if rewrite-clj supported something like (type nmp) -> :namespaced-map

lread20:08:05

There’s (tag nmp) -> :namespaced-map

lread20:08:28

But tag granularity is not likely what you might expect.

borkdude20:08:55

yeah, tag often returns token which can be a lot of things

lread20:08:09

Anyway, I greatly appreciate the chat. I’ve been stuck in a bit of analysis-paralysis and I think you thawed me out. simple_smile

lread20:08:01

Is adding (type node) worthy of a git issue? I haven’t given it any real thought…

lread20:08:50

Ok will update the issue.

borkdude20:08:29

It doesn't have to be in the initial release I think, since it's not part of the original

lread21:08:28

Agreed! The only reason it would make it into the initial alpha release is if I found a real need to procrastinate on a must-have alpha feature/fix simple_smile.

borkdude21:08:42

I think parcera doesn't have sexpr right?

lread21:08:55

I haven’t dug deep into parcera, but I think you are right. Parcera seems simpler in many ways.

sogaiu22:08:49

i am not sure about the no sexpr bit -- i don't remember well enough what sexpr does atm (just woke up). but parcera does have quite decent successful round-tripping afaict (have tested against large number of samples).

sogaiu22:08:29

btw, wrt terminology (naming of constructs), i think there might be some benefit in examining what various grammars are doing. if nothing else it might make talking about some things easier.

sogaiu22:08:34

for example, it may just be my ignorance, but in the parcera grammar i encountered the term "macro keyword": https://github.com/carocad/parcera/blob/master/src/Clojure.g4#L157 i might have mentioned this before, but i have a version of the tree-sitter grammar that tries to merge some ideas from parcera and the naming is part of that. i don't know if anyone is up to making a glossary of terms, but in the matter of grammars, it seems like it could help reduce confusion (e.g. some people say "discard" while others say "ignore"). i also started saying "metadatee" to refer to the thing the metadata is supposed to apply to.

borkdude22:08:46

@sogaiu sexpr transforms a parsed node into a real clojure thing

sogaiu22:08:00

ah thanks -- i don't think parcera does that.

lread22:08:43

In the other direction rewrite-clj also supports coercion form real clojure things to rewrite-clj nodes.

lread22:08:05

For terms I’m kind of stuck with what I’ve got for rewrite-cljc. But where I have wiggle room, for Clojure elements, I am going to try to use what the Clojure core team uses. For example, I just asked on #clojure about namespaced element naming and got a response form Alex Miller.

sogaiu23:08:41

cool you got a response! i think it totally makes sense to use what the core team uses -- but afaict, they don't have terms for everything.