Morning

morning

Morning

morn

Ah, HMRC, how I love giving you even more money

I really wish accountants would do their jobs properly

Morning all

morning

Good morning

For some reason HMRC have refunded me, I'm really confused.

I especially liked how HMRC sent me about 15 reminder emails / SMS messages

with no button to say “I’ve scheduled a payment dammit"

to be fair, the HMRC web experience is quite a lot better than it was just a few years ago

still frustrating that the data they provide needs correcting - but if you enter your data wrong they can penalise you

@yogidevbear you mean you didnt use Alternative Facts to get your HMRC rebate... I should have mine next week 🙂

This discussion reminded me to give them a ring... but the tape on the phone said if it isn't dead-line related call back later.

so now I am going to wait till the 2nd.

hello

regarding HMRC, they are so great because they use Scala

no wonder they are such a static organisation...

😱

typeical

as do I

too bad I can't sign it (me thinks)

goes up quickly though

For anyone interested in graph databases: https://neo4j.com/book-graph-databases/?utm_source=GPPC&gclid=CJPHu8mc6tECFW6x7QodDZ0DAQ

anyone know anyone who could tell me about battle-tested strategies for migrating large cassandra databases ? @otfrom ?

when we did it in hecuba (to solve some partitioning problems) we just wrote some code to copy from one table to the other IIRC

(it was a long time ago and on and older version)

probably did it in clj using alia

is this schema migration or something else?

schema migration

(and the data wasn't massive. Only about 500GB)

our current hacky approach has a distinctly limited lifespan as our data size increases...

so the new column family had a different name

ours was pretty hacky, but :cowboy:

are you running DSE?

yeah

you could use spark I suppose

esp if you need to do a transform

ideally i want to be able to move data to a new cluster, migrate the new cluster, point the api at the new cluster and then move any recent changes from the old cluster to the new cluster

so there is no downtime and no risk of a schema migration borking production

hmm... sounds like you want to write new data to both clusters while you do the migration

(which I'd do w/a code change)

and then migrate the rest using spark and then turn off writing to the old cluster

(I might hope that data coming in was on something like kafka to make this easier)

but that is me just trying to persuade tcoupland acron elise_huard and jasebell of the beauty of my architecture

haha, most of it is on kafka... some not - but i'm happy with making code changes

so, I'd take my advice with a :shovel: of 🧂

:emoji_fail:

if the messages are coming in on kafka then you might be able to run a 2nd version of your inserter that puts the code in the right place while you migrate and then turn off the old one

actually I’m with on this one

it sounds like you'd want to keep track of offsets at time of migration, to know which bits you'd need processing after everything is functional again

there is that.

might be easier if you are doing idempotent things

or have all the messages consumed from that point and pushed onto another queue

if you have the complete history in your kafka then it'd be pretty easy

or even if it was all in kafka at some point and your kafka stuff was archived in something like s3

😉

just need to union the two

but again, I'm thinking spark for this

can do unions on two topics with Kafka streams.

tcoupland we'll have to test that one out then 😉

i can have a complete history in kafka... might have some problems with ordering consistency between old & new databases

tcoupland I did similar to process today's data and then union it with the state produced by the previous day's job to create the new state

well if you have the whole history, then you can reconsume it into the new cassandra

there is that

I think I like spark from s3 as it means that my parallelism for the re-processing doesn't need to be the same as the normal daily processing

but I think it is all trade offs and trying things out really

food for thought - thanks all !

o, one other thought would be to look at getting casandra to start logging it's updates. Then you can load the new one from a back up, catch up by using the log, do a bit of a comparison to make sure everythings cool, then flip over.

yeah, i've got two thoughts at the moment - one to log app-level mutation descriptions and apply those to the new copy as soon as the schema migrations have run, and two is to log cassandra-level mutation descriptions along with their timestamps, which i can easily mod my c* client to do, and do similar