Fork me on GitHub
#clojure
<
2023-07-30
>
mx200016:07:08

Is there something like clj-nstools (https://github.com/clojens/clj-nstools) which is not from 2013? Or are there similar tools to manage namespace dependencies easier?

mx200016:07:55

For example I want to define on a package base not namespace based my aliases and refers.

vemv16:07:00

Probably you can hack clojure-mode, clj-refactor.el, clojure-lsp or the like so that on each new ns you create, a template is applied Not very fancy, but it should be very effective. Probably most Clojurists wouldn't be pleased at maintaining a .clj namespace that used a different convention from vanilla ns

mx200016:07:30

Did you look at clj-nstools? It basically does that, but for individual namespaces only. What you mean with template? To generate the namespace form to create it instead or refactor it automatically? I think this means we have certain namespace patterns which we could put into some kind of package manager

vemv16:07:44

I did look at it, which is why I compared my impression of ns+ with vanilla ns

vemv16:07:12

> What you mean with template? To generate the namespace form to create it instead or refactor it automatically? I think this means we have certain namespace patterns which we could put into some kind of package manager Yes, I meant that. For the two Emacs packages I mentioned, one can standardize the choices by setting them in .dir-locals.el

Jenny Kwan20:07:26

I'm looking for something that provides an abstract file system that holds mounts to other file systems that can be configured at runtime. I'd like it to be cross-platform, i.e. anywhere JVM runs. And easy to test with, ideally with a mount to an in-memory implementation. Oh, with the ability to mount popular cloud blob storage. Something that tracks file metadata off to the side to do change detection (to try to enforce file immutability / versioning). Does a lib like this exist? I haven't found anything in my searches.

p-himik20:07:14

You'll probably have to use FUSE: https://github.com/libfuse/libfuse There's this library that wraps FUSE for Java that I found after a lightning-quick search, so maybe there's something better: https://github.com/cryptomator/jfuse

Alex Miller (Clojure team)20:07:28

I'm not sure if it's useful, but the jdk itself contains a java.nio.file.FileSystem abstraction, maybe there is some implementation of that that is useful

Jenny Kwan20:07:28

I don't think I need a full abstract file system. Maybe abstract immutable blob store that can be backed by a variety of file systems is better terminology? Apache JClouds seems to fit the bill with in-memory and local FS providers. The only thing missing is mutation detection, which I'll have to build myself. Thanks!

Jenny Kwan21:07:38

Poo. JClouds is headed for the attic.

didibus23:07:22

Aren't any persistent DB backed by a variety of file systems?

didibus23:07:57

Are you asking for a DB that can use as it's storage an OS mounted file system, or run fully in-memory, or use instead a cloud backend like S3 or DynamoDB ?

Jenny Kwan23:07:29

I'm not asking for a DB, but a blob store interface. Apache Commons VFS works. Built-in, it has in-memory and local FS providers. And there's recently maintained projects providing AWS S3 and MSFT Azure Blob. I'm in the process of wiring this up now.

didibus00:07:00

Lots of nosql DB are blob stores nowadays

didibus00:07:44

For example xtdb document store has all these backends: https://docs.xtdb.com/administration/configuring/

didibus00:07:59

But I also see what you are looking for.

didibus00:07:28

But if you just want to store documents, XTDB will do, support many backends, is immutable, and also supports indexing the docs, and perform queries over them, etc.

didibus00:07:45

But. ya, if you want to make an app where the user can "Save" and normally choose his filesystem to save a file, but also you'd want the option to have them save to S3, or other cloud document store, that won't do

Jenny Kwan03:07:12

External manageability is important for this use case, I think. I haven't decided yet.

Jenny Kwan03:07:02

Or at least visibility. I'm building a data management platform, and most files will be Parquet files. Having them visible from the outside so they can be mounted into something like Delta Lake is attractive. IDK.

didibus04:07:03

Yup, just giving options. You know your use case best.

onetom04:09:19

for testing purposes, it's indeed worth using the java.nio.file.spi.FileSystem abstraction. https://github.com/google/jimfs provides an in-memory implementation of it for example. i've also just tried this S3 implementation and it seems to work quite well: https://github.com/awslabs/aws-java-nio-spi-for-s3 here is some example code, which demonstrates its usage: https://clojurians.slack.com/archives/C240SQ0NM/p1695094832748469

onetom05:09:08

here is a good intro about how to use JimFS: https://www.baeldung.com/jimfs-file-system-mocking