@U0C6HAPFU You shouldn’t need to close the deferred. You can call .close on the underlying InputStream in :body to be polite, but 99% of the time, that’s a no-op. It should be fine to stop using them and let them go out of scope. Just don’t hold on to any references, and they’ll be GCed.

Ah, not sure why I hadn’t thought about closing the InputStream. Thank you very much @U10EC98F5 - much appreciated!

I gave closing the InputStream a go and it doesn’t look like it worked. After connecting to Twitter using

@(d/chain 
 (http/get (:stream twitter-urls)
           {:headers {"authorization" (str "Bearer " bearer)
                      "user-agent"    "Aleph"}})
 :body)

{"title":"ConnectionException","detail":"This stream is currently at the maximum allowed connection limit.","connection_issue":"TooManyConnections","type":""}

Yeah, .close on the InputStream probably does nothing. Aleph’s set up to completely manage connections behind the scenes for you. In theory, you could call aleph.http.client/close-connection on the stored connection in the default-connection-pool, but I couldn’t get that to work off the top of my head, since the underlying Dirigiste object pool is kind of opaque. You could use intermediate functions to directly get a conn, like copying and calling create-connection, but there’s a lot of intermediate code that would be bypassed that’s necessary to correct functioning. I tried returning the connection used as a field in the response. While it does give you a handle to close, thus freeing you up to make another call to twitter, it gets into tricky territory, since it means either you or dirigiste could close the underlying netty channel without the other knowing about it. I need to think more about this.

@U0C6HAPFU Did you ever try setting the header Connection: close?

That would be on the Twitter side, but I’m not sure that would suffice for Dirigste without some testing. Really just speculating here. Waitaminit. What about on the client side, using request directly instead of get, post, etc, and adding :aleph.http.client/close true to the Ring map when you want to close it?

Thanks for the suggestions. I’ll have a go but would this work if I want to start a request, get a few minutes of streaming data come back before ending it? If so would you be able to briefly explain how that’d work as I had assumed you send the request and then cannot send further data or update the request map after that.

Http requests are independent, but the underlying connections are almost always kept open to avoid startup overhead (tcp, tls). But the request map with :aleph.http.client/close in it won't actually be sent, it's just a sentinel value to close a connection in Aleph. Usually it's just a random internal detail. If the same connection processes the sentinel, it will close. However, I haven't checked to see that it works while the existing connection is still open. It might not. It should work if the reused connection got the request, but if the connection was reused, then you wouldn't have your initial problem. You might also be able to alter the default conn pool so there's a max of 1 conn per site, which could force the existing conn to process the sentinel…or just make it block/throw 😜