Fork me on GitHub
#aws
<
2020-05-19
>
steveb8n04:05:40

fyi: I used clj-cdk and then switched to typescript. it’s a really nice experience, even without the lispyness. most importantly, gets the job done quickly with lots of compile time help. sacriledge, I know, but it’s a practical solution

viesti11:05:17

I've grown to like Terraform, especially the 0.12 language improvements, HCL kind of suites for writing configuration

viesti11:05:55

and that it has module concept built in, it's easy to make library modules and link to stateful modules

viesti11:05:36

though still thinking that if Pulumi gets JVM support, then it would definately need a clj wrapper 🙂

viesti11:05:16

with specs for all the resources so that one would get autocompletion in editors (+ linting)

kenny15:05:50

@viesti We use Pulumi to deploy all of our infrastructure. We write all that code in Typescript. I really think Typescript is a good fit for this sort of thing. Having all of static help with the hundreds of AWS properties you need to configure is a win. You wouldn't get that with Clojure. In our infra code, there's no hardcore data transform. It's mostly all constructing objects that represent "physical" aws resources. Not sure what value Clojure would bring to writing this sort of stuff. Even using the REPL doesn't give you much -- the code is almost all mutations.

viesti15:05:47

I was remembering spec browser in cider, while hacking portkey-cloud/aws-clj-sdk it was nice to autocomplete and navigate namespaced keywords, was thinking that cloud resources (like in terraform) are a bit similar, attributes namespaced into resources under providers :)

viesti16:05:39

like aws.instance.instance_type

alidcastano19:05:54

@kenny i’m only using aws services so I found Pulumi to be an unneccessary layer over what Cloudformation already provides. and even with static types I found myself needing to reference docs to see descriptions of what properties existed. also, I prefer the declarativeness of edn over typescript, and reader literals make it easy to inject useful utilities. in above repo, for example, the literal `#eid` makes a resource’s name unique per environment and `#with-ssm-params` adds an SSM parameter for every configured resource. I feel like configuring this behavior via functions in typescript would be less clear. that being said, Pulumi did have a nice command line experience and gives more out of the box, so I’m definitely sacrificing that for my own hand-rolled Clojure solution, and project* I’m using it for is still WIP so yet to see what other things I might be missing out on

kenny19:05:10

CloudFormation is okay. Pretty verbose and you're limited to the features CloudFormation supports. Pulumi has packages built on top of the "raw" AWS resources. These packages are absolutely fantastic for doing things that would take 50+ lines of CF yaml. (see the packages: awsx, docker, kubernetes, datadog, new relic). They have other cool features like magically creating https://www.pulumi.com/docs/guides/crosswalk/aws/lambda/#register-an-event-handler-using-a-magic-lambda-function which is great. Pulumi generates documentation for all the properties so generally you get what you need inline. I do sometimes need to reference various free-form aws docs or the API references though. This isn't something a dsl or Clojure Pulumi wrapper would solve. The edn versus typescript comparison doesn't make sense to me. It sounds like you prefer a dsl over a language. That totally works for simple use cases. When you have something that perfectly fits the dsl, you won't have any issues. When you don't, however, it becomes very painful or impossible. 'Tis the nature of DSLs. On top of all of that, if you need to deploy resources to a separate cloud provider, you don't need to go learn that cloud provider's DSL -- you keep using the programming model you're used to.

alidcastano19:05:13

thanks for the info on Pulumi, I’ll look into the utilities you mentioned. to your point, it’s not that I prefer a dsl, it’s just that I prefer data. most (if not all?) aws service configurations can be specified as Cloudformation JSON data. that’s a map of information. in a Rich Hickey talk, he mentions how only thing you can do to information is ruin it. what I like about edn is that it lets you to create utilities that transparently reduce the boilerplate of clf templates without creating another abstraction layer. as useful as Pulumi may be, it is another layer and language to account for being boxed into a cloud provider is a valid concern I had, but since I’m using Datomic, which is bound to AWS for now, it’s not one I can avoid anyway

kenny19:05:23

Data all the things mustn't be taken to an extreme where you start writing functions in edn. Data and functions both have a purpose, else you'll soon find you've written yourself your own programming language. IMO https://github.com/juxt/aero is on the border there 🙂 Since I rarely need to write data transforms in Typescript, I really am only every working with the json data to construct resources. Pulumi has already built the abstractions for me. My line of business is not in deploying software, so I don't have a strong incentive to compete in the space. My guess is this was the primary motivator behind Stedi's business decision to drop cdk-clj. It's very difficult to build something that competes with AWS's CDK, Pulumi, or even Terraform when it's not your direct line of business. You'll find yourself fixing bugs in your own deployment code rather than working on revenue incentives. Our config files are written in edn. We read those in with the Javascript edn reader and use them in Typescript. There are a few cases where I find myself wishing for Clojure's data transform facilities. Some of the newer Typescript features make this a bit better.

alidcastano20:05:48

those are all valid points, and reasons for why I tried out Pulumi in the first place 🙂. if my configuration gets too complex I’d definitely reconsider my solution . using a js edn reader together with typescript is a nice idea. hadn’t considered that

mruzekw00:05:59

Would there be any reason to use Pulumi with CLJS -> JS/TS?

mruzekw00:05:39

You're probably have to write your own specs unless there's a converter, but I wondered if y'all thought there were an advantages to doing that

lanejo0121:05:08

@kenny For completeness I thought I'd mention that Cloudformation also lets you write inline lambda functions. It even let's you make macro transformations implemented as lambda functions in case you want to make your own sugar.

kenny17:05:15

@ Yes but, unless things have changed, you need to write your function's code in YAML 😵 I've used the macro transforms implemented as Lambda. It's a crazy amount of complexity for such a simple use case... Seems better off to use something purpose built (like an actual programming language 🙂).