Fork me on GitHub
#mxnet
<
2020-06-14
Frederik16:06:47

Hi, I wrote my own force rebind function because I switch between training on batch sizes of 16 and predict on batch sizes of 1. The function looks like this:

(defn force-rebind
  ; Rebind doesn't seem to work for mxnet, should work when check-pointing and
  ; loading back in.
  ; We need this function to be able to train and predict on different batchsizes.
  [model n-samples-per-batch]
  (save-model model "models/intermediate_for_rebind")
  (bind-model
   (load-model "models/intermediate_for_rebind")
   [1 6 7]
   n-samples-per-batch))
Where bind-model is just some wrapper around module's bind with some specific parameters hard-coded for my use. Unfortunately I have the following memory-leak:
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.NDArray was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.Symbol was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
WARN  org.apache.mxnet.WarnIfNotDisposed: LEAK: [one-time warning] An instance of org.apache.mxnet.Executor was not disposed. Set property mxnet.traceLeakedObjects to true to enable tracing
Reading this issue on github: https://github.com/apache/incubator-mxnet/issues/10867 I thought this would be fixed with the merge. Anyone any idea why I still have this issue with the example code I gave? I had some other memory leak issues with NDarrays, but those were fixed by using (.dispose array) . In this case I'm not sure how to fix it though.