Fork me on GitHub
#arachne
<
2018-01-30
>
jcf20:01:43

The number of files opened by Fileset is tricky to provide. I've upped limits to 8192 and still Arachne's falling over.

jcf20:01:24

I'm not sure if it's a leak per se, but Figwheel and Boot both build fairly large codebases without requiring the same resources as Fileset.

luke20:01:29

@jcf this is after explicitly using arachne-fileset 1.6.1?

luke20:01:47

1.6.0 definitely had a bug where it did not release file handles

luke20:01:59

It is intentional that Arachne keep a reference to all files in a fileset that is currently “in use”.

luke20:01:20

otherwise the problem was that temp files would get cleaned up out from under Arachne while it was running

luke20:01:36

but it should only be the “current” set of files actually available via the web server

luke20:01:44

intermediate filesets should be cleaned up

jcf20:01:16

I saw this comment, but not sure that's what you're referring to:

;; Idea: we could theoretically do garbage collection, if space in the blob store becomes an issue:
;; - find all instances of FileSet (would require registering in a weak map at creation)
;; - find all TmpFiles in all FileSets
;; - delete all blobs not referenced by a TmpFile
;; - but it's probably unnecessary

luke20:01:29

I’ll take a look at your project; this might be “expected” if you have more than 8192 files in your public dir.

luke20:01:44

@jcf no, that’s about file cleanup from disk, not open file pointers

jcf20:01:49

Most of the files come from Closure I think. Not me. 🙂

jcf20:01:58

And other deps.

jcf20:01:04

Me indirectly.

luke20:01:07

right, fair enough. I’ll investigate

jcf20:01:26

I'd like to help. I don't know the fileset code that well, but will have a peruse.

luke20:01:08

I need to run now, but if you think this is something you could get your head around and want to have a call to get context, I’d be happy to do so.

jcf20:01:05

A call would certainly help, but in the mean time I'll familiarise myself with Fileset.

jcf20:01:27

I've not directly used NIO APIs a lot yet so this'll be interesting. 🙂