yamlscript 2024-03-01 | Slack Archive

Ingy döt Net17:03:44

One property of YAML that many people are not familiar with is that a YAML stream (file) can contain multiple documents.

foo: bar
baz: 42
---
- abc
- xyz: 99

The --- starts an entirely new YAML document. Another property of YAML is that YAML anchors/aliases can't work between those documents. That's kind of sad. In YS I realized that there is all kinds of use cases for using multiple documents, some data, some code, some both... Any YS code in a document should be able to access any part of the evaluation of a previous document. I've made it work. Here's a silly example:

$ ys <(echo '
- Alice
- Bob
- Charlie
--- !yamlscript/v0
name =: $.rand-nth()
say: "I heard that $name like YAMLScript!"
')
I heard that Alice like YAMLScript!

There are 3 special symbols to access things: 1. $ the evaluation value of the previous doc 2. $$ a mapping of evaluations of all docs 3. $# the number of doc evaluations so far Of course this code like all YS compiles to Clojure and then is evaluated in SCI. Here's what the compilation Clojure code looks like:

$ YS --compile <(echo '
- Alice
- Bob
- Charlie
--- !yamlscript/v0
name =: $.rand-nth()
say: "I heard that $name like YAMLScript!"
')
(+++ ["Alice" "Bob" "Charlie"])
(+++
 (def name (__ ($) '(rand-nth)))
 (say (str "I heard that " name " like YAMLScript!")))

The +++ is a macro that updates the 3 special atoms above. The only problem is that we can't load-file in there and then use the newly introduced symbols because of how clojure evaluation order works. +++ tries to do something like:

$ clojure -M -e '(def $$ (atom {}))  (swap! $$ assoc :1 (do (def x 123) x))  @$$'
#'user/$$
{:1 123}
{:1 123}

which works but then this doesn't:

$ clojure -M -e '(def $$ (atom {}))  (swap! $$ assoc :1 (do (eval (read-string "(def x 123)")) x))  @$$'
#'user/$$
Syntax error compiling at (REPL:1:40).
Unable to resolve symbol: x in this context

From this #C03S1KBA2 thread https://clojurians.slack.com/archives/C03S1KBA2/p1709311184357329 I learned that you can't wrap top level code in anything except do. I have another idea about how to do this. I'll report back later.

👍 1

Ingy döt Net19:03:41

Good news! I solved the problem!

Ingy döt Net19:03:52

You can't wrap the whole code block, but you can rap the last form in that block.

Ingy döt Net19:03:24

I already implemented it. Here's the input YS code, the result from running it and the compilation to Clojure code.

$ cat foo.ys 
- a
- b
---
- c
- d
--- !yamlscript/v0
a =: $$.1
b =: $$.2
concat: a b

$ ys -l foo.ys 
["a","b","c","d"]

$ ys -c foo.ys 
(+++ ["a" "b"])
(+++ ["c" "d"])
(def a (__ @$$ 1))
(def b (__ @$$ 2))
(+++ (concat a b))

I didn't need to create the a and b vars but I put them there to show how I wrap the last form in each compiled yaml doc with the +++ capture function.

phill22:03:26

It reminds me of the "plumbing" by which monads(?) connect the bits of overt source code they benignly supervise. In place of globals that none of the code blocks owns, there could be a supervisor that invokes each code block in a closure where $ and $$ are appropriately bound. By avoiding global state you could envision a single process with more than one of these pipelines going on... a typical testing scenario, but it is also the ind of "aha" that happens when you suddenly picture your program as a useful routine in a bigger program..

Ingy döt Net22:03:44

#Also sent to the channel

One step at a time I guess.

Ingy döt Net22:03:49

By which I mean that sounds fascinating, and my concept of how much ground ys can handle is regularly still expanding. Like 2 days ago I learned about bb pods (external module system) and realized that whole set of libraries almost instantly benefits ys. The multi-doc thing I only realized a month ago and finally implemented today. Pod integration should be done this weekend. My imagined scope is still way past my amount implemented even though I work on this almost constantly. I need to keep up the implementing work.

Ingy döt Net22:03:05

that really was confusing, sorry

Ingy döt Net22:03:17

I'll get the multi-doc stuff into a release tonight. Maybe you can try it out and suggest how it improves.

Ingy döt Net22:03:52

In Perl land we used to have this technique we called fat-packing where you bundled all the pure perl dependency modules into the same file as the app code. YS multi-doc should make that a breeze.

Ingy döt Net22:03:18

Well ys -N (changed to ys -b in next release) already compiles an app and deps into a native binary executable file. But multidoc provides a pure source single file option.

Ingy döt Net22:03:57

I don't have a clear mental picture of what you are suggesting. One thing I was thinking of was using futures to delay evaluation. Also if I'm not mistaken futures uses threads so we could get parallel evaluation.

Ingy döt Net22:03:44

replied to a thread:One property of YAML that many people are not familiar with is that a YAML stream (file) can contain multiple documents. foo: bar baz: 42 --- - abc - xyz: 99 The `---` starts an entirely new YAML document. Another property of YAML is that YAML anchors/aliases can't work between those documents. That's kind of sad. In YS I realized that there is all kinds of use cases for using multiple documents, some data, some code, some both... Any YS code in a document should be able to access any part of the evaluation of a previous document. I've made it work. Here's a silly example: $ ys <(echo ' - Alice - Bob - Charlie --- !yamlscript/v0 name =: $.rand-nth() say: "I heard that $name like YAMLScript!" ') I heard that Alice like YAMLScript! There are 3 special symbols to access things: 1. `$` the evaluation value of the previous doc 2. `$$` a mapping of evaluations of all docs 3. `$#` the number of doc evaluations so far Of course this code like all YS compiles to Clojure and then is evaluated in SCI. Here's what the compilation Clojure code looks like: $ YS --compile <(echo ' - Alice - Bob - Charlie --- !yamlscript/v0 name =: $.rand-nth() say: "I heard that $name like YAMLScript!" ') (+++ ["Alice" "Bob" "Charlie"]) (+++ (def name (__ ($) '(rand-nth))) (say (str "I heard that " name " like YAMLScript!"))) The `+++` is a macro that updates the 3 special atoms above. The only problem is that we can't `load-file` in there and then use the newly introduced symbols because of how clojure evaluation order works. `+++` tries to do something like: $ clojure -M -e '(def $$ (atom {})) (swap! $$ assoc :1 (do (def x 123) x)) @$$' #'user/$$ {:1 123} {:1 123} which works but then this doesn't: $ clojure -M -e '(def $$ (atom {})) (swap! $$ assoc :1 (do (eval (read-string "(def x 123)")) x)) @$$' #'user/$$ Syntax error compiling at (REPL:1:40). Unable to resolve symbol: x in this context From this #C03S1KBA2 thread https://clojurians.slack.com/archives/C03S1KBA2/p1709311184357329 I learned that you can't wrap top level code in anything except `do`. I have another idea about how to do this. I'll report back later.

One step at a time I guess.

Ingy döt Net23:03:11

Here's some fun stuff:

$ cat foo.yaml 
name:
- Alice
- Bob
- Charlie
- Diana
- Eve
- Felix

pets:
- ants
- bats
- cats
- dogs
- elephants
- gorillas
$ 
$ ys foo.yaml -e '"$($.name.rand-nth()) & $($.name.rand-nth()) like $($.pets.rand-nth()) and $($.pets.rand-nth())"'
"Felix & Charlie like cats and elephants"
$ ys foo.yaml -e '$.name.rand-nth() + " & " + $.name.rand-nth() + " like " + $.pets.rand-nth() + " and " + $.pets.rand-nth()'
"Bob & Charlie like elephants and cats"
$ ys foo.yaml -e 'str($.name.rand-nth() " & " $.name.rand-nth() " like " $.pets.rand-nth() " and " $.pets.rand-nth())'
"Charlie & Diana like gorillas and gorillas"

Quite a bit to unpack there...

Ingy döt Net23:03:32

-e gets added as another document in the stream here. We access the file data with $. The . operator in ys is something like the -> threading fn. In general (foo) and be written as foo() if you want, and (foo bar) as foo(bar). (a + b) becomes (+ a b) and a + b + c + d -> (+ a b c d). When used in those switcheroo contexts, + actually becomes _+ which is a polymorphic (nums, strings, seqs, maps) operator. _ are not allowed as symbols in ys, so you can't say (_+ a b) . It throws Invalid symbol: '_a' .

Ingy döt Net23:03:18

The first command shows off string interpolation with complex expressions.

Ingy döt Net23:03:16

The 2nd shows off a + b + c thing. The third is you typical clojure (str a b c) form.

Ingy döt Net23:03:03

This keenly reminds me that I need to write basic language docs very soon 🙂

2024-03-01

Channels