yamlscript

Ingy döt Net 2024-05-21T02:07:53.966269Z

I've been musing on what makes YAML a pretty good data syntax to write a programming language like YS in... When I started this project, my hopes for YAML being a reasonable syntax were pretty low. I started porting Clojure code to various made up YAML forms to see what I liked and what I didn't. To my pleasant surprise, it started looking pretty good. I've being thinking... could something similar be done in JSON, EDN or something else? If it came down to one thing, I'd give credit to YAML's "plain scalars" (unquoted strings), and the ability to use them vs the other scalar styles (quoted and literal). No other plain text data language that I know of has this (unquoted strings). To go a bit further, 3 of the other 4 scalar styles (single quoted, double quoted, and literal (heredoc semantics) play a big part in the language. Various scalar styles with various (code mode) semantics...

say: hello         # hello is a symbol here
say: 'hello'
say: "hello $name"
say: |
  hello $name,

  How are you?
The YAML spec asks you to not consider differences in scalar style (except for plain vs non-plain) when making a YAML data loader. YS is implemented as a specialized YAML loader, but it is loading data into a Clojure AST, not a data structure. In the example above, all the mapping keys are the same. YAML doesn't support a data model with duplicate keys, but again that (an AST) is not our targeted data model for this loader (the YS compiler).

🙌 1
2024-05-21T21:54:56.166919Z

Suppose I have a block of cloudformation configuration that I want to template, which contains a variable expansion such as ${ENV} that I want to expand, is there a way to do this in Yamlscript? Could I perhaps drop into code mode in the middle of my data here?

--- !yamlscript/v0/

&cloudformationBlock
dataStream: 
    Type: AWS::Kinesis::Stream
    Properties:
        Name: ${ENV}_DATA_STREAM
        RetentionPeriodHours: 24
        ShardCount: 1

--- !yamlscript/v0

defn main():
    ENV =: 'dev'
    say: *cloudformationBlock

👀 1
Ingy döt Net 2024-05-21T21:58:59.744719Z

sec

Ingy döt Net 2024-05-21T22:04:50.494969Z

There are many ways to do this with YS. Here's one: cf.ys:

--- !yamlscript/v0/

=>:
  ENV =: 'dev'

dataStream:
  Type: AWS::Kinesis::Stream
  Properties:
    Name:: "$(=> ENV)_DATA_STREAM"
    RetentionPeriodHours: 24
    ShardCount: 1
cmd:
ys -Y cf.ys > cf.yaml
cf.yaml
dataStream:
  Type: AWS::Kinesis::Stream
  Properties:
    Name: dev_DATA_STREAM
    RetentionPeriodHours: 24
    ShardCount: 1

Ingy döt Net 2024-05-21T22:07:44.372359Z

note that "$(=> ENV)_DATA_STREAM" can be "${ENV}_DATA_STREAM" in 0.1.59 when that drops

Ingy döt Net 2024-05-21T22:08:00.806179Z

code already in main

Ingy döt Net 2024-05-21T22:09:34.373609Z

Here's slightly different approach that you might like better: cf.ys:

--- !yamlscript/v0/

=>:
  data =: load('vars.yaml')

dataStream:
  Type: AWS::Kinesis::Stream
  Properties:
    Name:: "$(data.ENV)_DATA_STREAM"
    RetentionPeriodHours: 24
    ShardCount: 1
vars.yaml:
ENV: dev

Ingy döt Net 2024-05-21T22:09:45.942669Z

does same thing

2024-05-21T22:12:11.114109Z

This is really great!

Ingy döt Net 2024-05-21T22:12:49.321239Z

or... cf.ys:

--- !yamlscript/v0/

dataStream:
  Type: AWS::Kinesis::Stream
  Properties:
    Name:: "$(ENV.'MY_ENV_LEVEL')_DATA_STREAM"
    RetentionPeriodHours: 24
    ShardCount: 1
cmd:
MY_ENV_LEVEL=dev ys -Y cf.ys > cf.yaml

Ingy döt Net 2024-05-21T22:12:52.636209Z

🙂

Ingy döt Net 2024-05-21T22:23:06.893669Z

for completeness, you can put vars in a previous document in the same file and access them with $$: cf.ys:

---
level: dev

--- !yamlscript/v0/
dataStream:
  Type: AWS::Kinesis::Stream
  Properties:
    Name:: "$($$.level)_DATA_STREAM"
    RetentionPeriodHours: 24
    ShardCount: 1

Ingy döt Net 2024-05-21T22:24:22.566839Z

I was a bit confused at first because you named your var ENV which is a global mapping variable containing the environment in YS...

😁 1
Ingy döt Net 2024-05-21T22:25:29.445469Z

Also I doubt Name: dev_DATA_STREAM is the end result you would want, but now you know some ways to get whatever it is you want.

Ingy döt Net 2024-05-21T22:26:11.004839Z

I haven't used cloudformation but it's on my list to show how YS can help it.

Ingy döt Net 2024-05-21T22:26:38.161649Z

Let me know if you need more help.

2024-05-21T22:26:53.192439Z

Thanks!

Ingy döt Net 2024-05-21T22:27:04.476879Z

Also look at https://github.com/BetterThanTomorrow/calva/pull/2511

Ingy döt Net 2024-05-21T22:27:47.988699Z

it's a complete YS refactoring of a 650 line circleci yaml file, that turned out really well

2024-05-21T22:27:57.573509Z

This is motivated by my interest in alternatives to the Serverless framework

Ingy döt Net 2024-05-21T22:29:09.421299Z

URL?

2024-05-21T22:29:11.060619Z

https://www.serverless.com is going to change with v4.0 and will begin charging money it seems

Ingy döt Net 2024-05-21T22:29:31.015359Z

I see

2024-05-21T22:32:48.046349Z

I imagine that many people & companies will be interested in alternatives to the Serverless framework

Ingy döt Net 2024-05-21T22:33:58.274369Z

never heard of it until now

2024-05-21T22:36:31.297049Z

It’s pretty popular but there are some things that it’s not very good at, like as a small example you have to use a plugin to get if/then/else in your Serverless yaml

2024-05-21T22:37:40.082299Z

Cloudformation compiled with yamlscript would be like totally free to do programmatic things