Clojurians Log v2

Thanks for the update, now verify is happy, but the build run still seems to do nothing. It looked to me like the build script runs app tests so I expected something to happen with build run vs running tests of the build script with build test. I am likely missing some insights that did not click from reading the docs.

Also noticed description on Github links to https://monkeyci.com/ which does not work, while https://www.monkeyci.com/ works.

See my PM

the site doesn't load for me

Off to a bad start 🙂. Let me check it out

Is the code public (I guess not) or do you have anything to share about how it was built?

yes, it's public, https://github.com/monkey-projects/monkeyci

I wrote it in Clojure, including the static and docs websites, which is generated using markdown and hiccup

Didn't feel like writing html ;)

Site should work now, seems to be a temp hiccup on the provider side 🤷

What I'm interested in is how you load user scripts. It seems you're using using the Clojure compiler: https://github.com/monkey-projects/monkeyci/blob/87dbef691274114897ba6eb1e7092ed351526973/app/src/monkey/ci/script/core.clj#L59 So people can execute basically anything on your CI server then right?

Build scripts are running on agents, which start a container and load the scripts there

So it's not run on the main api server

ah right

pretty nice :)

Yes, security is a major issue if you allow anybody to run code in your system 🙂

congrats on your project, looks like a lot of work :)

Thanks, it was indeed more work than I expected. Especially the infra stuff, I underestimated that.

What would you say is the benefit of using a custom CI environment vs just writing your build in clojure (or bb) and run that script on a "normal" CI provider?

I think one does not exclude the other. Both can be used at the same time. You can write your build using bb, or clojure cli or whatever, and then execute those in jobs using monkeyci, which allows more control on when to run those jobs or how to configure them.

That is my main frustration with existing ci/cd tools, you don't have enough control when you have a complicated build script.

It's also aimed at non-clojure projects, which are often less flexible. But I'm going for clojure-lovers first 🙂.

My main gripe with all of the CI systems I've used so far is this: Docker seems to be the isolation mechanism we standardised on. This is fine. Build definitions can thus select a Docker image to run in, usually with a choice of either choosing from a handful of images provided by the platform, or using an image hosted elsewhere. What I don't understand is why none of them (that I know of, please tell me if I'm wrong here) has an option to say: here's my CI Dockerfile, right here in my repo. Please build this image, then use it to run my tests. I don't want to manage my own, separate build pipeline for Docker images on another platform. Apologies if that feels off topic, I've wanted to tell that to someone who makes a CI platform for a long time.

> has an option to say: here's my CI Dockerfile, I think the issue there may be that it needs to cache that build since it would get slow otherwise? Is it possible to upload that image to a container registry and then use that image in a CI environment? Also probably not since these CI providers need to have some agent running in the container maybe? Or maybe it is possible with CircleCI docker builds, haven't checked

yeah, they have docs on this: https://circleci.com/docs/guides/execution-managed/custom-images/

Yes, so this is exactly what I don't want.

Unless I'm misreading something, that page is telling me to, outside of my CI, build my image and push it to a registry myself. I'd like the CI pipeline to deal with that.

If you don't want this, wouldn't it be similar to just do the steps you would otherwise do in the Dockerfile, but just in your build?

Sure, but with caching.

In MonkeyCI (and probably, any other CI tool) you could add another job that creates the image, pushes it somewhere (possibly a private repo), and then uses that image in another job to execute the tests.

I bet you can do this with circleci too

Yes.

You could even create a https://docs.monkeyci.com/articles/plugins/ that does this automatically, and just include that plugin in all your pipelines where you want to use this feature.

But that means I have to set up a registry somewhere, and handle the synchronization of checking whether the image is available. I'd like to be able to say something like

docker: 

It sounds like you need MonkeyCI ;P

You could also just rent a VPS and install some git hooks that get run on push :P

MonkeyCI uses podman, which can run standalone, without a service, but I know from experience it's still not easy to build docker images inside containers, when that container has limited privileges.

I'd have to try if MonkeyCI would make this possible 😅

Hmm, maybe this also resembles your wish @gaverhae? https://docs.github.com/en/actions/tutorials/use-containerized-services/create-a-docker-container-action

> GitHub will build an image from your Dockerfile, and run commands in a new container using this image.

That does sound promising, let me take a closer look.

Oh, no, that's not what I want. 😢

Yeah, I realized this is for creating Actions you can use in other places, darnit

Gitlab seems to have docker in docker stuff which allows it

Anyway, I don't want to derail this further. Let's just consider that a feature suggestion 🙂

@wout.neirynck Your website mentions we can start for free, but there does not seem to be a "pricing" page to get some sense of how much it would cost beyond the free tier, and what might be included in the free tier.

> Reduce Debugging Time > Write unit-tests for your build scripts to avoid production issues. Note that opening up an nREPL port to the build script would also be great for debugging.

similar to how you can ssh into github actions and circleci

I guess it would be great if you could develop the whole build in nREPL incrementally while being connected to the CI env

Congrats, looks great! I really like the rationale. I wanted to try local build to get familiar. Inside the app/examples/clj-build I ran monkeyci build run which computes for a bit and exits with 0 exit code like it succeeds. But then I tried to add (is (= 1 2)) to the example test to make it fail. After running monkeyci build run again I would expect some output and perhaps a non-zero exit code, but still get no output and 0 exit code. Perhaps I misunderstand how local builds should work? Then I tried monkeyci build verify which prints:

Got 1 error(s)
build.clj - at 11:16: Unresolved symbol: test-all

Ah yes could be the example need an update, I 'll take a look tomorrow

To run tests, you need to do ‘monkeyci build test’, they aren't executed when you build

i've been experimenting with https://tangled.sh/, so i'm excited to try this out as an alternative CI system

@gaverhae See https://docs.monkeyci.com/articles/pricing/, it mentions there is no commercial offering yet, it's still a todo 😅

@borkdude With the cli you can monkeyci build test --watch which will continuously run the tests on changes. But the repl idea is a great suggestion.

@kloud examples updated 🙂