Fork me on GitHub
#leiningen
<
2021-06-19
>
Tom Helmuth18:06:11

I have a large dataset (2 GB) stored as EDN files and want to write a small library to make it easy to sample randomly from the dataset. I want to make it so you can download the dataset to your machine once along with the library, and then be able to use the library from different projects on the machine. It doesn’t feel like making it a library you can include as a dependency and pulling from Clojars is the right way to go, since that will require pulling copies of the 2 GB dataset for each project that uses it. Is there a way to tell leiningen to use a library from a specified location? Or any other suggestions on a better way to go about this? (I’m not sure if this is the right place for this question, so let me know if it isn’t.)

dpsutton19:06:21

Data like that seems a poor fit for being encapsulated in a library

dpsutton19:06:57

Having those files distributed in another way and a library that gives the access you like when provided a local path to the data sounds way more reasonable.

Tom Helmuth19:06:00

I was starting to think that direction too. Thanks!