Hey 🙂 not seen anything in the wild yet, but it's definitely a good idea. You could probably re-use the document store protocol (+ implementations), or at least borrow from it heavily

Sweet. So this doesn’t sound outlandish at all, right? Thanks for the pointer. Let me go through the doc-store code 🙂

cc @U01AVNG2XNF

It's not outlandish at all. You may want to upload a file, store its content in a docstore and save the identity/URL in Crux. You could consider a Content Addressable Store to provide this. Personally I've been working on something very similar for streaming multipart uploads into Crux but it isn't ready for public use yet.

This sounds interesting. Are there plans to make your work public? If nothing else, it would be nice to read it :)

I'm trying to reason my way through how much of ActiveStorage is worth borrowing, ideologically. I think a lot (all?) of their motivation to decouple the actual domain models from "attachments" by removing the need to identify an attachment within the domain model itself stems from the fact that it uglies up schema migrations. I keep mulling it over, but I can't see why we'd bother... there's no real harm to tracking the id(s) of the attachment(s) directly in Crux documents. That also means we don't really need both models from ActiveStorage: Attachment and Blob become one thing for us, I think, though we still need one model to track original filename, file size, and other metadata. (I wouldn't want to barf all that into every Crux document that had an attachment.) A lot of the other abstraction from ActiveStorage is valuable once we've unified those two things: • separate the public URL from the internal URL • purging (though I don't see much need for purge_later) • convenience fns like attached? and url_for • permanent URLs vs. short-lived URLs • maybe the direct upload JavaScript from AS could be used verbatim, at some point?

I'd be curious to hear both your thoughts on any of these things keeping in mind that we'd like to release this ActiveStorage-equivalent as a library unto itself. In particular, if there's a reason to include the extra layer of indirection from the get-go, we might as well head down that route rather than trying to patch it up later. I just can't think of one.

@U050CTFRT Just to clarify: were you thinking of an existing general-purpose CAS? Or just a hash of contents (or equivalent) that we build ourselves? So far I'd only considered the latter.

@U01AVNG2XNF https://github.com/juxt/vext/blob/master/src/juxt/vext/content_store.clj

But please be warned, I've only tested with small uploads, when I tried with a large file (a movie) it hung, so I haven't quite got it working -but you can see what's involved. I probably haven't quite got the RxJava incantations correct.

The point of the code is to stream an upload to disk (but any streamable sink could be used, such as S3), but also hash the upload in parallel, so that you don't have to reread the file after storing it to generate the hash.

It's non-blocking with back-pressure