Fork me on GitHub
#aleph
<
2017-07-14
>
spangler22:07:21

I have been wrestling with some kind of memory/reference leak using Aleph and JanusGraph in tandem

spangler22:07:33

It looks like Aleph works fine on its own, and Janus works fine on its own

spangler22:07:55

but when combined, memory increases without bounds

spangler22:07:11

I have done some research and it looks like part of the problem is that Janus transactions are thread specific

spangler22:07:28

So, if a transaction is opened on a thread, it must be closed on that same thread

spangler22:07:03

I'm wondering how do I even debug this problem?

spangler22:07:04

For instance, I am doing this

(stream/on-drained
     source
     (fn []
       (log/debug "query complete")
       (db/commit graph)
       (db/rollback graph)))

spangler22:07:33

But how do I guarantee this function is being called on the same thread that the transaction was opened on?

spangler22:07:05

Do the on-drained callbacks always get called from the same thread that the handler is using?

spangler22:07:25

If not, how do I ensure they do?

danielcompton23:07:32

@spangler I'm not exactly sure, but http://aleph.io/manifold/execution.html might be helpful

danielcompton23:07:37

> Almost all interaction with JanusGraph is associated with a transaction. JanusGraph transactions are safe for concurrent use by multiple threads. Methods on a JanusGraph instance like graph.v(...) and graph.commit() perform a ThreadLocal lookup to retrieve or create a transaction associated with the calling thread. Callers can alternatively forego ThreadLocal transaction management in favor of calling graph.newTransaction(), which returns a reference to a transaction object with methods to read/write graph data and commit or rollback. I think that might be what you need to do?

spangler23:07:41

@danielcompton Thank you! I now have some leads to follow. I will report back work how it turns out.

spangler23:07:16

So do you know if in general the ondrained callback is called in the same thread as the handler?

spangler23:07:36

I sort of assume not