😳 I guess I only read a screen full of messages and should have scrolled up more

but yea, there's lots of really interesting things about erlang

It gives low GC pause, but then it also gives slower application performance since communication between the processes is now more expensive

i guess i should take a look at elixir

(or clojerl, but like - I'd prefer to understand the platform first before making that dive)

if nothing else, I think the erlang paper is worth reading and rereading, http://erlang.org/download/armstrong_thesis_2003.pdf

don't forget clip I've come to the opinion it's best to just use component + aero, unless you want to experiment with something exotic and build your entire application with pipelines or as a state machine

I concur with Ben. Clip (with built-in aero support) is a great addition.

I find the main "complaint" which is the rationale for clip a non-issue. You can pretty trivially write a reader which encodes component systems as data. Clip needs a metacircular evaluator to work

Meanwhile, you can write a multimethod which together with aero obviates the need for clip

{:system
    {:db :make-db
     :server :make-server
     :app {:fn :make-app
           :using {:database :db :server :server}}}
    :config
    {:db {:url "foo.bar"}
     :sever {:port 8888}
     :app {:id 3}}}

Or, just do all the bootstrapping yourself

I consider that an option maybe more than others

It's always a viable option. I'd probably do it on a personal project but on a team project I would default to component

With one extra artificial constraint - never pass the component's content as an argument to a function. No need to start swearing at 3am one year in the future

But it might be an overly-opinionated position

cool. doesn't sound like there's much outside of what I already knew about

curious why people keep trying to do the whole "system as data" thing a la integrant/clip over something more programmable

"It's just data"

We've used Component for a long time and now the protocols are metadata-extensible we like it a lot more 🙂

Easier to reason about immediately. Compare {:a 0, :b 1} to (into {} (map vector [:a :b] (range))). Being constrained to "just data" makes you think in a specific way - one that often results in a nicer architecture, IMO.

We started out without using anything and migrated to Component slowly over time (contrary to some claims that it is "all or nothing" -- it isn't).

I've the most experience with component and mount

I've kept looking at alternatives. I really don't like Mount. I think Integrant is interesting but having seven lifecycle extension points seems over the top, and the multimethod approach means you sometimes have to hunt around to find all the relevant parts of how a particular dependency works -- I don't find it very intuitive, compared to Component, and I'm not terribly obsessed with "Data all the Things" so that aspect doesn't sway me much.

I haven't looked at Clip yet...

I've used almost exclusively Component. I like it in terms of simplicity but https://github.com/weavejester/suspendable looks like its missing piece, which presumably is built-in into Integrant For example, what should a DB-backed component do on reset? Wipe its data? That would be unfriendly for daily development, but in a way it also makes sense and will be occasionally desired, so there's no universal answer That's where Integrant's granularity would seem an improvement.

There was a podcast about mount on the clojurescript podcast. I've never used it. There was also one episode about component. We use component at work and one project has integrant, but personally I don't really have a preference.

Suspendable sounds interesting but the problem it solves is not one that I actually have with my workflow.

I need to make an effort to listen to the ClojureScript podcast about Mount.

As the author said on the podcast, it has some "loud haters" in the community, but I think he explained quite well some of the problems it addresses differently than the others.

Mount has some fundamental limitations, lack of reified system means you can't test in parallel for example Of course anyone is free to trade off that limitation for some ergonomics etc

component also has that problem yeah?

He addressed that in the podcast by saying that he actually doesn't want multiple systems in parallel for testing, he wanted a separate process for testing. Ah well.

No, component can have multiple systems. The system is one value you can pass around.

But to be honest, in all the years of using it, I've only ever had one system at a time.

I haven't written a lot of systems where I mock all the components, etc. I tend to test the "real" system if I can

> He addressed that in the podcast by saying that he actually doesn't want multiple systems in parallel for testing, he wanted a separate process for testing. Ah well. yeah as hinted anyone is free to make tradeoffs. But the alternatives are simply superior in this regard, they have a superset of features regardless of what you choose to use In this case you don't exactly get to call critics 'loud haters'

If you work in a monorepo you can even bring up multiple systems of multiple services in a single process

And the ergonomics of mount fall on their face about a week after you build the system Implicit side effects, impure tests which redef everything, meh, the biggest argument against mount is someone else's code

Most of use don't work alone. When you factor in collaboration and programming over time, component's explicitness is suddenly very welcome

And now with :extend-via-metadata arguably component has gotten "easier" too

(although I haven't really used it in that way yet)

I've been toying with the idea of a generic component. It has two explicit fields, a state and a function. Its constructor takes a start-fn, stop-fn and the function The start and stop funcions take this as their only argument (records are open,both settings and dependencies will be folded in) Implement IFn with the function, state and arguments

https://clojurians.slack.com/archives/C03S1KBA2/p1627150245264200?thread_ts=1627148839.261400&cid=C03S1KBA2 captures my motivation for clip as data very much.

Re: multiple Components in parallel -- we have some tests that rely very specifically on being able to spin up a "cluster" of systems in a single JVM to test certain behaviors. We have quite a few tests that use mock Components of some description as well. So that's an important ability for us.

I've worked on a lot of projects, and as a consultant I jumped around projects, and unpicking people's clever tricks for assembling systems was no fun. I was explaining clip to someone yesterday, and there's a whole iceberg beneath clip of what I'm trying to achieve that's a little deeper than what's on the surface. Problems caused by the infectious nature of component, and also problems like request stuffing in ring (which I have blogged about already). I'd like to start marketing clip at some point in the future more. But I want to put AOT in place first. I've got about 5 different implementations for how AOT could work, but it's a little complex due to future extensions I'd like to support.

Re: metadata extension -- next.jdbc leverages that for its built-in Component support (without needing Component as a dependency, BTW): https://github.com/seancorfield/next-jdbc/blob/develop/src/next/jdbc/connection.clj#L318-L326

oh I have used this outside of component, just not with it

oh like that, that's nice!

@U09LZR36F Since you're here: it looks like Clip "evals" expressions, based on just the README. I assume it doesn't actually use eval, right?

^ this is a place I need to figure out with clip. Clip doesn't need any support from libraries. But that means libraries don't market clip for you 😛

@U04V70XH6 It doesn't use eval no 🙂. It's basically just apply calls under the surface.

I'll put Clip on my (long) list of projects to take a deeper look at. I just get the impression from some of the Juxt projects that you almost have to buy into "more Juxt" to get the best out of them since several of them seem to recommend each other for various things... entirely possible that's just my (mis-)reading of the docs though...

@U04V70XH6 https://github.com/juxt/clip/blob/3375cb1564cc1d0d887bf6ebfb530b656c0dc422/src/juxt/clip/impl/core.cljc#L149-L188 here's the implementation if you're curious. It's very similar to SICP's metacircular evaluator. My v1 of clip was literally just apply, but I found a handful of counter-examples where that was insufficient and went down a rabbit hole of implementing those. In practice, the implementation is simple enough.

@U04V70XH6 yada definitely implies bidi. bidi doesn't imply yada though. Crux is also standalone. clip was designed to play nice with edn, which aero happens to provide quite nicely. However, I've also used it with cognitect/fern successfully, and also in pure code for Cljs purposes. If you do give clip a spin, let me know. I'd be very open to your thoughts on it. Another benefit of not pushing it much yet is that I'm happy to break the API still.

I'm curious to get y'alls initial thoughts on an approach like this to constructing a system and managing start/stop of resources by encapsulating them in what I'm calling "modules" https://gist.github.com/lilactown/0d6c842e8a36c989c1f52ad06f1c9f97

Requiring things to be def'ed is bad, ideally your core functionality should be usable without introducing any global bindings. And then layer defs on top of that to improve the UX. Introducing a new def form is something of a red(or at least yellow) flag in my mind

This is one of the reasons the metadata protocol extension is such a great addition to component, gets you out of reliance on defrecord

@U4YGF4NGM https://gist.github.com/lilactown/0d6c842e8a36c989c1f52ad06f1c9f97#file-system-clj-L34-L35 this line looks dangerous to me. if (kafka) fails, then you've got a lingering (database) hanging out. That can lead to connection exhaustion in databases. I think the postgres limit is ~40 by default or something. So if your connection pool opens 8 of them, that's 5 resets not working before everything goes 💥

I experimented with the idea of a reversible let binding as a design for clip, something like:

(cliplet [db (database foo) (stop-db db)
          kafka (kafka db opts) (stop-kafka kafka)]
  ??? continue with program, run a test, whatever
)

@U09LZR36F In practice tho', if your components fail to start, your process is probably just going to exit and that will clean up pretty much everything.

@U04V70XH6 In production, yep! But not during development if you're using a reloaded workflow (which I find optimal for some kinds of changes)

Do you run in component start failures enough in dev for that to be a problem? It's not a problem I've run into...

(but I am not constantly restarting components -- I tend to start my system up in my REPL and leave it running for a long time while developing against it)

That's a tricky question to untangle. It might not be something you hit weekly. But on the day you do hit it, I expect it will take several iterations before you get something new right.

If you were willing to kill the REPL in the case your system failed to start, you could probably get away with:

(ns foo)

(def db ...)
(def kafka ...)

(def http-server (make-handler db kafka))

@U09LZR36F the idea have is that it would close the db conn, but the fix for that is hidden behind the api; the idea is that the (database) call behind the scenes adds itself to a buffer of resources that were started in the current load! of a module and if any of them fails it catches the error at the module boundary and stops any that are in the buffer

the idea is basically "what if react but for services"

😄 You're up to magic.

I was working with some services where the whole system should still be able to function at least partially if one component was/is not able to get going

Shutting down the whole system if one service cannot start is not always an option

Yep! Clip currently gives you a partially started system you can choose to stop.

Sadly clip doesn't seem to be much of an option for me to my current needs - but i'll keep it in mind for later! I have a case of distributed configurations that should be pulled in whenever necessary and can come from theoretically anywhere - and those configurations may have to start registered services or old ones have to be shut down. Currently the design of integrant fits my needs the most and I can get it going without much (just a little) fiddling. I may roll-my-own sometime to get rid of the fiddling.

@U6JS7B99S over integrant, clip is a small improvement. If integrant works for you, definitely stick with it. Clip has most features Integrant has. Is clip missing something you'd need to pull that off?

@U09LZR36F from what I see i'd need explicit :start and :stop annotations as well as direct service calls within the config which I really don't want to see in my configurations. The configurations can also be created in yaml or json. Configurations should best not have any "system" or "environment" knowledge except for their dependencies and what they represent. -> I'm a "mapping" component and this is my configuration. Thats it.

I may be missing something here though and have to take a closer look :thinking_face:

All that to say: What I'm missing with clip is the disentanglement of configuration and lifecycle

@U6JS7B99S you could totally write a json->clip transform though, no problem. Same way that you would for json->integrant really. But you're right, clip puts the function you need to run to start a component at the forefront of the api rather than behind the indirection of a multi-method or record.

@U09LZR36F Atm I still fail to see how to represent a clojure functioncall for start/stop into json apart from a clojure-containing string? As reference: My ideal model for a configuration would look about like:

{:foo/bar {:type :foo, .... <configuration keys>}
 :foo/baz {:type :bar, :some-key #ref :foo/bar}}

(map-vals #(merge % (case (:type %) :foo {:start `(foo)} :bar {:start `(bar)})))

Or you could look it up somewhere. There's no magic to going from :foo to (foo). You need some lookup system. In Integrant, it's multi-methods that run code. In clip it's a symbol that references a function.

Hmm true, but it would need a bit more than that as i'd need start/stop, runtime replacement and :foo/:bar probably not matching a function. I may give it a whirl to see if I can get it 🏃 with clip

I can see that I could probably transform it somehow in the right format

If you're being super dynamic, it's probably much of a muchness anyway 🙂.

With integrant it was actually only a very few lines (the only thing necessary was actually setting up derivations from keys to component types)

But its not really something I like to do. But for setting up composite keys I'd have to postwalk all configurations for fixing references so that was also something I wanted to avoid as with many configurations that is not optimal

@U09LZR36F So then thanks for the insights! I now have the idea of how to use clip as basis, that might actually work out quite well if I manage to make component replacements on running systems work (a'la integrant/resume). I believe it should be possible to build components through multimethods/records on top of clip.

👏

heh, I'm not sure if it's worthwhile, but you could definitely do something like:

{:something {:start (integrant.core/start-key :something {})}}

Add it to component, we shall build the ultimate library, InterClipCom

Powered by the -><-->>?as-|> macro

> Sorry, http://interclip.com is taken. :(

That's a bit of a random (and local) Q for #clojure but, yeah, I think everything stopped with the pandemic and never really got going online... #clojure-sanfrancisco is where announcements tend to get made.

Looking at the archive of events on Meetup, it looks like I signed up to attend in April and May but I don't remember how those went...

I doubt I would attend in-person -- it's too much of a hike into SF from the East Bay at the end of the work day for me but I've been attending virtual meetups in Seattle, Los Angeles, and for a while Provo (but that's stopped meeting since one of the organizers moved away I think) @UJHMA11DJ