yamlscript 2024-06-03

I'm working on organizing my ~200 to-do items for YAMLScript (big and small) that are in various places. I want to get most of them into public GitHub issues and label/prioritize/organize them appropriately. Many of these are in a GitHub Project that is private. I was able to export to TSV which is nice. GitHub API access requires GraphQL. This got me to thinking that maybe GraphQL usage is a great targeted domain for YS. I found this very nice Clojure library to build off of: https://github.com/walmartlabs/lacinia Excited to explore this a bit.

Well I'll be... I was talking about having implemented tag functions in YS like !str/upper-case: and !merge*: But then I couldn't get it working so thought maybe I dreamed I wrote that 😄 Found it: https://github.com/yaml/yamlscript/blob/main/core/test/compiler-stack.yaml#L649-L687

$ cat foo.ys 
--- &a-map
foo: 111
bar: 222
baz: 333

--- !yamlscript/v0/

- !merge*: [*a-map, {derp: 444}]

- !merge*:
  - *a-map
  - foo: 444
    derp: 555

- !merge*:
  - *a-map
  - foo: !str/upper-case: hello
    derp: 555

$ ys -Y foo.ys 
- foo: 111
  bar: 222
  baz: 333
  derp: 444
- foo: 444
  bar: 222
  baz: 333
  derp: 555
- foo: HELLO
  bar: 222
  baz: 333
  derp: 555

I just implemented a very useful thing I've wanted for some time. The ability to write raw Clojure code as part of a YAMLScript file.

$ cat foo.ys 
!yamlscript/v0

=>: !clj |
  (defmacro qq [x]
    `(quote ~x))

say:
  qq: (1 2 3)

$ YS foo.ys 
(1 2 3)
YS doesn't yet have the syntax for defining Clojure macros (backtick, tilde, etc). This will be very useful for doing things in YS where the language isn't quite entirely ready for.

👍 3
🎉 1

Note this isn't released or even pushed yet. I'll probably play with it on a branch for a couple days...

Interesting that it -c compiles with macros expanded.

$ YS foo.ys -c
(defmacro
 qq
 [x]
 (clojure.core/seq
  (clojure.core/concat
   (clojure.core/list 'quote)
   (clojure.core/list x))))
(say (qq (1 2 3)))
Not entirely sure I want that.

pushed to branch clj-raw

Very cool that all that these giant infrastructures are composable enough to make your language come to fruition. I haven’t needed to use YAML much yet. But I appreciated the excitement and comparative history in your talk. And it’s always nice to see clojure semantics extended to other runtimes/projects! Any it also seems historically momentous how YS seems to rhyme with the reoccurring desire for infix lisp, vs a cleaner python.

🤠 1

I share that with appreciation of how yamlscript smuggles in a “real” (not greenspun) lisp.

Thanks (I think 🙂 ). YAMLScript has 2 musts: • Input code must be valid YAML • Output code must be valid Clojure Everything else is up for discussion.

YS doesn't yet really have a target audience, but its certainly not Lispers who cringe at YAML 🙂 It's likely people who already use YAML and want it to do more; not just in k8s or GHA but anywhere they use it.

Every day I see more potential syntax idioms that on their own look much cleaner in YS than the Clj they generate. I struggle on each to decide if its worth it.

Because really, at the end of the day, every accomplished YAMLScript programmer needs to become an equally accomplished Clojure programmer.

So how worth it is each special transformation they need to remember?

To make it into YS, a transformation needs to hold some substantial weight. That said, if I see that weight meets my threshold, I have no problem introducing it.

Because, on the other hand, YS is extremely flexible. You can write code the way it feels best to you:

say: 1 + 2 + 3 + 4
say: (+ 1 2 3 4)

In that regard, YS has so far avoided (like lisp) operator precedence.

$ ys -e 'say: (3 + 4 * 5)'
Error: java.lang.Long cannot be cast to clojure.lang.IFn
$ ys -e 'say: (3 + 4 * 5)' -c
(say (3 + 4 * 5))

vs:
$ ys -e 'say: (3 + 4 + 5)' -c
(say (+_ 3 4 5))

One goal of YS is to see it used in domains like k8s and GHA. To do that it must work in combination (at least for now) with the one-offs those techs have created for their own specific needs.

I'll finish by saying that I have been thinking for the past 15 years how to make YAML programmable in a way that was clean, robust and accomplishable. It wasn't until less than a year ago that all the right lightbulbs turned on at once and I saw the golden path forward! Lisp, Clojure, Java, Leiningen, SCI, GraalVM. GIANTS that make the path almost trivial. YAMLScript doesn't serve Clojure etc it makes great use of it. But I sincerely hope in a way that leads its power users deeply back into the giants that created it.

> It's likely people who already use YAML and want it to do more; not just in k8s or GHA but anywhere they use it. I think the entry point could be data templating, ie. "this YAML is quite repetitive, how can I reduce it?"

By year end I'd like to see YS be the preferred YAML loader (for plain old YAML usage) in every major language. The challenge will get be to get major players like GHA (CircleCI etc) to adopt it natively. But people can do things like this in the mean time...

https://github.com/BetterThanTomorrow/calva/blob/2952822ac64a4e85d0e28031749ab44d1df8aac4/.circleci/config.ys yes Every context using yaml ends up with hacked in "include", "for", templating, etc. I've done it myself