having a bad day - am I the only one who swings between ‘yeah, I’ve got this down’ and ‘wow - what was I thinking?’

@colin.yates: Yeah, I recognize that cycle. Usually means it's time to split that task into smaller pieces. 😉 http://www.agileforall.com/splitting-user-stories/

@ericlavigne: It was more a reaction to coding/tooling in particular rather than a given use-case

Yup, when asking team to create one new thing, we split up into many individually digestible use cases. When learning something new, we try to swallow the whole watermelon at once. I think the same solution works for both problems.

And yes, I often make the same mistake, which is why I recognize your cycle. 😉

@colin.yates: I'm studying event sourcing at the moment, and it looks like you're ahead of me. Any tools you recommend for the storage/replay/query pipeline?

not really - I store the event log in an append only format in a SQL table (for non-functional reasons).

I then have a bunch of listeners which react to an event, hydrate the relevant entity from the event log and denormalise into their own structures

each one is a (Stuart Sierra) component

but I couldn’t find any decent prior art so it is all hand rolled

I would love to open source it but the great thing is that there is hardly anything to the infrastructure/glue code

were I to do it again I might utilise core async and use a document database but it is all trivially simple stuff

Best prior art I found was https://geteventstore.com/ but it is very incomplete.

That’s the one from Greg? He is definitely a guy I would listen to. But yes, I looked at that but it didn’t quite fit my workflow

Sounds like you are collecting events and interpreting them at the moment that you need to query. Sounds like an approach that would only work for very little data.

nope, I was processing the events when they happened (asynchronously) and updating denormalised read stores

@ericlavigne: That's what state snapshots are for, I think. You can snapshot the entity state at some point and then read a snapshot and query only later events when need to rehydrate.

yes

having gone through this experience I can’t imagine not using an event-based approach for anything other than trivially simple CRUD type apps. Even then…

but, for me at least, it really did require a shift in mental aptitude when thinking about the solution space. I also pulled in DDD at the same time which was a mistake on hindsight

all the good stuff of ‘time travel’ people are experiencing with Om and Reagent - imagine that for the whole application

I haven't read DDD but the discussion on event sourcing does tend to mix with that.

Hmm, why do you think DDD was a mistake?

I kind of got impression it would fit quite well

So your event store is a single table with thrift-serialized events?

so you know how everybody says “I do OO” but they don’t - from what I see (and experienced myself) that applies even more to DDD.

DDD isn’t a technical discipline it is a domain modelling discipline

And you have another table tracking how far into the event stream you have processed so far?

@ericlavigne: how far each component has processed so far yes. It is great because if my view changes, simply destroy its schema and reset it back to 0

then it replays all those lovely events to create the updated view

@jaen - they are a match made in heaven. I hadn’t anticipated though how much a proper DDD approach differs from the stuff I had been doing for 20-odd years

thinking about it, there is a lot of symmetry between micro-services and event-handlers

when this insane deadline I am under passes I mean to put some effort into a set of blog posts because there is so much sympathy between FP and event sourcing - a lot of it is just good common sense.

Ok, that is a point, when I tried to get how to DDD it was so different than what people usually do I was kinda stumped hot to tackle that.

Yeah, I couldn't understand ES/CQRS when reading about it on some OO-oriented (that's double the oriented xD) websites, describing them in terms of objects and whatnot

Until I read that current state is a left fold of events

And it just clicked

yes - exactly

however, the complexity (for me at least) of DDD is what events and exactly how to define the aggregate roots

you can get all the benefit of event sourcing without DDD by simply storing all state changes as an explicit event.

then the entity is literally just a (apply merge (event-store/load ar-id)

And most of the challenge in schema design is trying to future-proof for new types of data you might need to store in the future without creating denormalization issues. But with event sourcing most of that is in views and you can always just create more.

exactly

there are downsides though.

you don’t realise exactly how many business decisions are made in queries until you manually capture each decision in an event!

but performance is just blindingly fast because you literally can have one view per ‘widget’ on you UI optimised just for that.

But putting an "aggregate ID" in the event implies that you already know what views you will use it for. Aggregates are your views, right?

no, aggregates is what is needed to make a decision in the domain

It's basically a in-memory domain model, to oversimplify it a bit.

so Customer and all their orders might be one aggregate (with Customer being the AR), another part of the business (i.e. another bounded context) might view the Customer as a set of delivery addresses, yet another might view that Customer as a list of purchases - each ‘Customer’ instance is the same physical real world thing and has the same ID.

but each part of the app might attribute different properties to that Customer.

Strictly, aggregates in DDD are silos of integrity. You define them to ensure each business decision is safely and atomically made (where possible).

it really blows your mind.

Yeah, an interesting thought just occurred to me

Aggregates define integrity boundaries

yep

Sounds like I need to read DDD ASAP.

In part because of how it's rather hard to have in-memory transactions in OO

How would the ease of doing in-memory transactions in Clojure

Play into that

I think that could be interesting to think about

http://dddcommunity.org/book/implementing-domain-driven-design-by-vaughn-vernon/ is a great book. Eric Evans’ ‘Blue Book’ is the authority but pretty old now

it also seems to have taken off much more in Microsoft land rather than Java land

Yeah, I've been recommended Vernon as well, though that's a big thing to tackle

IIRC ES/CQRS was actually re-invented in C# land

I need to run; gotta snuggle up on the sofa with my four kids and watch ‘Penguins of Madagasca’. When I say ‘snuggle’ I mean ‘be sat on’

well, Greg coined the CQRS term (as an extension of CQS)

Hah; if you write something on ES/CQRS/DDD in Clojure I'll gladly read that; those are things that are better learnt from people that used it in practice

Instead of on your own

if you do want to get into DDD then I strongly suggest you go through the archives of the DDD google group - probably 9/10 messages are about “how do I do this simple thing in DDD” .

thanks @jaen - it is #1 on my TODO list once I can take a breath (current work pressure is pretty high)

see you in a few hours

That is a big book but might read anyway. Really want to switch to this approach.

This one - https://groups.google.com/forum/#!forum/dddcqrs?

Thanks @colin.yates

If so, I'll bookmark that, it will surely come in handy at some point.