måningyåls

mostly i just start!/`stop!` my app contexts, but it seems to happen occasionally, after some hours or days, that i get my repl into a broken state with some stale class references or similar, at which point i reach for refresh - it's nice to have a nuclear-lite option around, since restarting the repl takes so long

yup, its handy to have 🙂

Morning all!

@dominicm why remove integrant?

Integrant does add complexity to a project, as does any other component / reloading system. Theses tools are very useful when you have multiple resources you want to coordinate. However, if you are creating a simpler system or just a single micro service by itself, you can certainly feel the additional complexity.

For example, my second project with JUXT Edge broke because I added a simple resource via Integrant and the project stopped working.

I do think Integrant is very useful, however with great power comes greater maintenance 😃

If you are building a collection of micro services that you want to orchestrate together, then overall Integrant should provide a consistent way to manage them. I think it’s an interesting choice as to when to include a component management system.

https://github.com/riverford/objection/ I think this looks really interesting from the perspective of something that "scales" with you

it's also great for the REPL

Looks a very nice dynamic alternative to the static nature of Integrant,etc. One of the riverford team will be at ClojureX this year too

I've heard 🙂

(though not talking about this)

They are talking about ClojureScript native reactive apps which should be very interesting to see

Hopefully it'll be an approach I don't need a macbook for

morning

Very interesting. I will give it a thought. TBH, i was sorta feeling this too. As someone who uses Spring (Boot) a lot in order to bring home the bacon, I was trying to do things purely in clojure (with a few libraries for http and jdbc (thanks again sean!))

so, I do like integrant and I'm using it, but yes, John, my things I'm writing are quite small atm, so I'm wondering, do I need integrant 🙂

(now that I know how it works, I can easily use it again in the future)

btw, when you read integrant, I also mean any other "injection" thingymabob

Anything I have used in Clojure is less complex than Spring framework. Spring started off as a lightweight alternative to Enterprise Java Beans stack from Sun but then grew into an even larger framework. Spring boot is supposed to help, but it makes Integrant look incredibly simple 😀

Edge uses integrant quite invasively. Which is a good in an integrant system. I've worked on projects where poorly defined keys between functions leaves you looking at a function thinking "what is :x and where the f?? did it come from?!"

That does come at the cost of static description and up-front wiring though, which I get is a lot of work. Also, if you want to call a function, you have to fiddle around with re-provisioning those dependencies somehow (although I think a helper could take of some of that).

Certainly more projects are using component/state management and they are useful to learn and solve a real challenge. It’s simply a design decision you need to make as to whether there is benefit include a component/state management system. But that’s what we get paid for, right?

I don't think I've worked on a project without either component/integrant.

How small are we talking when you say you wouldn't include it?

I do lots without it, but then I spend most of my time processing csv/xls and outputting csv/xls/png

and some of those things are pretty large as far as klocs go

if I was building something that was only talking http to other services then I wouldn't use it

but if I had a connection to maintain (jdbc/kafka/other) then I probably would do

I do wonder if you could get away without it for jdbc using a memoized function

haven't figured out the .close part of that statement yet (might be a specialized memoize)

At that point, integrant becomes a tool for "places" (e.g. ports) rather than parameterized singletons.

although, in many ways. A web server is a parameterized singleton (e.g. by it's port)

I would tend to agree that there is a scale where Integrant (or similar) is too much structure for too little advantage, but I think that there is a lot of value in actively considering and defining the dependencies in your application and then also reaping the practical benefit of being in control of them and having a received method for adding and managing more should the need arise over time.

tbh, I've found the integration of integrant (ha!) very very lightweight

even for my small kafka/postgres/jdbc usage

It's one of "those things", right? I mean as someone said, isn't making that call what we are paid for as much as writing the code..?

and very little burden to work with. In fact, having the ability to also have "specs" against the "injected" functions is very nice to ensure the configuration is correct.

*nods*

it's definitely a lot lighter touch than spring 🙂

but then again, I like spring (boot) too 🙂

it's all good 🙂

welcome @jpreston

Howdy 👋

REPL workflows and refresh based ones aren’t mutually exclusive you can have both. I spend most of my time evaluating forms beneath the component (i.e. being REPL based) and when I need to change a component or something closed over in the system I use ig/reset (which does a c.t.n.r/refresh underneath. Most of our integrant defmethods are just a call to something like: (partial component-fn opts) for this reason… i.e. it allows you to test the component function outside of integrant, and you can test the integrant wiring by providing keys to ig/init. I do agree that people need to learn the REPL workflow first, as refresh ones bring some complexity, and are further removed from the true evaluation semantics.

I’m used to tracking the evaluation of forms in my head, and I agree with Stu’s comments somewhat; but it’s a little prone to staleness errors, which when you’re in doubt you can avoid with a reset.

Anyway just returning to update some ye-olde clojure apps I wrote that didn’t include a component system.

Is it an accurate to say that consensus is that existing reloaded workflow libraries hinder a repl workflow? Is it purely discipline that your project has a repl workflow?

I don’t think they hinder it all… they augment it.

> Is it purely discipline that your project has a repl workflow? I think there is some truth to this and all workflows for that matter.

There's something aesthetic about the presence of something like integrant, in which users of projects with it feel inclined to use it rather than plucking at integrant.repl.state/system in the middle of code. There's some negative feeling towards doing that. So the act of giving someone integrant means they will (usually) keep a good reloaded flow. I'm not sure I have something like that for the REPL though. It might be enough to use examples, but examples are hard because they tend not to scale.

I agree integrant with integrant.repl gives you a framework for a reloaded flow. I think component systems and reloaded are the tool for a scalable REPL flow. It’s the main reason component exists in the first place. Without it people would restart the REPL all the time… or be forced into manually unregistering multimethods etc.

My previous assumption was that people would work around it differently, e.g. (in-ns 'dev) (def db (db/connect "localhost:1234")) … (in-ns 'app.user) (get-user dev/db 1) etc. But you're right that at a significant portion of people were killing and starting a new REPL so either weren't aware of this idea, or it didn't work for them for some reason.

well those are exactly the sort of tricks I did for about 5 years prior to component and probably a year or two after component was released too

though there was tools.namespace prior to that which helped a lot

I think if you want to avoid something like component then the best pattern I’ve seen to achieving it is this: https://medium.com/@maciekszajna/reloaded-workflow-out-of-the-box-be6b5f38ea98 Which has one big advantage over integrant/component etc; which is that it’s guaranteed to tear down in the face of start up errors. However I’m a big fan of integrant because it gives you a principled means for configuring anything… I’d rather not build an adhoc config system, which I think this approach would yield.

Did you see objection above? https://github.com/riverford/objection/

No I missed it… but I have seen it before; though I must confess I didn’t dig into it much

though I do agree with some of what they seem to be saying. I have run into tensions with integrant systems where the life cycles are too static

I didn't dig in either until I started questioning integrant last week, and realized objection fit the description of what I was thinking of.

My big problem at the moment seems to be a tension between granularity and static dependency declarations. Edge encourages (by example) fine-grained dependency allocation, your get-user should only depend on the req and the jdbc db conn, it shouldn't get a kafka event bus just by virtue of the send-email endpoint needing to send stuff via kafka. My reason for this is about paging what I can into my head. But the boilerplate around this process is tedious. I'm trying to determine what a medium would be. Objection's approach certainly seems like it could fulfill a partial role in that.

> Edge encourages (by example) fine-grained dependency allocation, your get-user should only depend on the req and the jdbc db conn, it shouldn’t get a kafka event bus just by virtue of the send-email endpoint needing to send stuff via kafka. Not sure I follow your example here… What are those two components and their dependencies? I’d normally expect components to encapsulate their dependencies in a closure.

so get-user is something like:

(defn get-user
  [db req])

{:db #ig/ref :hikari}

But then my router is like this:

:bidi [["/user" #ig/ref :get-user] ["/email" #ig/ref :send-email]]
:http-server {:routes #ig/ref :bidi}

I think the http server should receive the db and close over it. These are application level components, its not a 'http server' its your http server, that provides its function - which includes proxying lookups to the db. Thats how I would think about it.

Sorry to butt in, been lurkin

I always found having to push everything up to coexist in the integrant system was a bit difficult to deal with, every new thing has to have its place in this shared structure, and the map gets absolutely massive.

This isn't a closed conversation 😄

Its all definitely interesting, its good to see what others are doing too, I've got a feeling that this stuff will be quite different in different companies.

when you say close over, what do you mean?

I think the route table and accompanying handlers should be encapsulated by the server, so the server constructor makes the db available to each handler. Maybe as an argument, maybe by merging it into the request.

I think only 'live' things need to have a presence in the system definition, and that things that can just be normal functions should be, and we shouldn't give up wiring functions their deps the old fashioned way.

in the small scale, just db works out. My experience with this was that map got very big and unclear about what is available. Sometimes other things would have merged keys in ahead of time, sometimes not.

About being unclear about what is available, sometimes a set of helpers (context/db req) (context/kafka-client req) may be enough for discoverability. Then you can ctrl-space in your ide as you implement your handler and see what sort of things you have at hand.

you may choose to seperate out the request and context, though I'm not sure it really matters

Having said this I still like to bottom out to functions that just take args e.g (get-user db user-id) as soon as possible as and not to rely too much on bags of stuff.

I'm trying to think of ways to avoid ever having a bag of stuff so it can never be passed around.

bags of stuff are truly a blessing and a curse

I think there are things that make good 'callables' these are your normal functions (get-user db user-id). Other things sit in a framework, are not readily callable (maybe with some difficulty) on the app I work on these are mostly graphql resolvers, but for most people its handlers. For these things it is important to have a shared set of facilities be available and for me that includes access to the same set of things, so that its not a worry when I extend the application with a new one. Although this does obscure ones ability to reason about individual handlers - the pay off is that there is no wiring friction, and that extension can ideally happen in fewer places.

Bags of stuff 4 lyfe

Of course nobody has a problem with the callable stuff, its the composite big ass gnarly things that people bike shed over

One option I've been considering is that you only have one bag of stuff in your application. And your lifecycle is two-tiered, one is for "places" and one is for "lifecycle things I will run out of". If integrant supported hierarchy that would look like this:

:http {:merge #ig/ref :sub-system}
:queue-listener {:merge #ig/ref :sub-system}
:sub-system {
  :db {:url "localhost"}
  :kafka-conn {:url ""}
  …
}

this way you at least have a very clear bag of stuff that's not abstracted and modified as it passes around.

But excepting the sub-system, the top-level would never be interdependent (or, I haven't thought of a use-case for it that wasn't a side effect like triggering a migration). Which makes me question the value of integrant for this case.

Not entirely sure what you’re getting at… but I’ve often wanted to add extra phases to the integrant life-cycle. I do wonder why this can’t be extensible? i.e. why are init and prep suspend and halt privileged? Why can’t I define my own? Also once you can define your own, you’ll likely want something more refined/nuanced than integrant… something that perhaps looks more like a state machine.

I think you can, it's just an additional multi-method 😄

yes but I’d like those phases to exist inside init. e.g. I have an app that loads reference data on startup… It would be nice if it could also validate reference data prior to loading it; however some of the validations can only be performed after all the reference data is loaded. I’d like for my app to start only after the validation has occurred.

the main additional thing i wanted from ig was support for cljs and async

it supports cljs now btw 🙂

integrant does support cljs — but because of limitations in cljs WRT to require being a special macro / not a function it can’t do ig/load-namespaces. I ran into this too.

ah, yeah. But that isn't mandatory to using it, as it isn't for component.

indeed

but it does mean you can’t componentise in cljs to the extent you can in clj

as one thing clj integrant does (with load-namespaces) is move your require tree into config, and make that dynamic.

@dominicm it supports cljs, but it doesn't support async - you can't have factories which return promises (well, you can, but they aren't a lot of use)

I've never understood why I'd want that 😄

'cos lots of js-land stuff returns promises, and when you then want to build things which depend on something which was acquired as a promise, you need it

When I first thought integrant + state machines; my next thought was possibly orchestrating such a thing with core.async channels. Which is I think similar to what @mccraigmccraig is after. Though I also think such a thing risks becoming too complex.

@rickmoynihan but don't other components just have to do:

Promise.resolve([a, b, c]).then(function(a, b, c){ return produceThing(a, b, c); })

uh, sorry @mccraigmccraig ^

yeah, but you end up with a context full of promises... it's quite unpleasant, and when you add something to the top of the dependency tree then suddenly everything below it which wasn't a promise has to become a promise and ugh

anyway, i rolled my own thing with a promise-based approach @rickmoynihan ... you put in a map describing your system of objects, you get out a promise of the system

works straightforwardly on cljs with async factories

ok sounds similar to what I was thinking

though I’ve not actually done it — and likely will hold off on doing it 🙂

the state machine thing sounds funky, don't have a clue what you mean. Can you expand?

@rickmoynihan not an easy read, but hereyago: https://github.com/yapsterapp/promisespromises/blob/master/src/deferst/system.cljc#L323

Morning

Morning

sigh

firefox

anyone using it? my add-ons won't update (just noticed)

some are months out of date

I do

I refuse to Google

I think I have to remove and reinstall

(add-ons)