Fork me on GitHub
#funcool
<
2015-08-18
>
mitchelkuijpers10:08:34

Cool a funcool channel! 😁

mitchelkuijpers14:08:20

Can I only return a manifold deferred as body in a response but not as a return value?

mitchelkuijpers14:08:46

but not:

(d/chain (jira-api/get-user-groups host user-key)
             #(http/ok %))

mitchelkuijpers14:08:49

(defn my-async-handler
  [context]
  (let [result (d/future (Thread/sleep 1000) "hello world")]
    (d/chain result
             #(http/ok & {"content-type" "text/plain"}))))

mitchelkuijpers14:08:58

the second one is more obvious

mitchelkuijpers14:08:21

I have a handler in my chain that checks some authorization and does a call with aleph and I would like that to be async

niwinz14:08:53

Maybe some extend-protocol is missing for the manifold deferred

mitchelkuijpers14:08:07

Yeah I checked the code that seems to be the issue

mitchelkuijpers14:08:20

because catacumba says

mitchelkuijpers14:08:25

2015-08-18 16:11:58,966 WARN  [r.s.i.NettyHandlerAdapter] No response sent for GET request to /api/1.0/companies (last handler: catacumba.impl.handlers$eval17761$fn$reify__17763)

niwinz14:08:35

Because the return value is handled by one protocol and the body sending by other

niwinz14:08:28

Efectivelly....

niwinz14:08:04

the deferred imlementation is not found for IHandlerResponse protocol.

mitchelkuijpers14:08:43

Yeah, I'll take a stab at it

mitchelkuijpers14:08:54

Would you like a pull request for that?

niwinz14:08:00

Yes and No xD

niwinz14:08:24

Seems that the implementation of deffered for body handling not will work as you expect

niwinz14:08:30

This is because the body handling is just the content

niwinz14:08:46

but in your examples the deferred will be resolved in a response

niwinz14:08:54

not the body content

niwinz14:08:21

And I think that your use case is pretty common so this should be solved

mitchelkuijpers14:08:30

True, I want to delegate or return a forbidden

niwinz14:08:12

Give me some time for finish that I'm currently working on, and later I will try to fix this inconsistences

mitchelkuijpers14:08:28

Thanks @niwinz much appreciated

mitchelkuijpers14:08:48

Were deploying our app next week with catacumba!

niwinz14:08:06

Oh great! Let me know anything

mitchelkuijpers14:08:05

Sure will keep you posted, we are building an atlassian connect addon with it. It is in it's very early stage http://addons.avisi.com/relationsforjira/

mitchelkuijpers14:08:42

Having a lot of fun with catacumba it's pedestal with docs

niwinz14:08:50

I'm very glad to hear about this!

niwinz14:08:14

One thing... the way to attach a watcher

niwinz14:08:32

to deferred is using the (d/on-realized...), no?

mitchelkuijpers14:08:24

yes I think so let me double check it for you

mitchelkuijpers14:08:49

(d/on-realized d
    (fn [x] (println "success!" x))
    (fn [x] (println "error!" x)))

niwinz14:08:21

Ok, nice! I will modify almost all implementation of IResponseHandler for async value for make it consistent

niwinz14:08:32

async values returned as is, should return a proper response

niwinz14:08:42

and async value in body should return the body value

mitchelkuijpers14:08:02

Then you can basically make a non blocking chain of handlers ^^

niwinz14:08:55

I have it almost done

niwinz14:08:03

I will publish the snapshot release

niwinz14:08:36

The 0.6.0-SNAPSHOT is published with that inconsistences fixed and with more tests for that use cases

niwinz14:08:57

Please check if it works for you.

mitchelkuijpers14:08:07

Ill trying it out as we speak

mitchelkuijpers15:08:43

restarting the repl....

mitchelkuijpers15:08:47

ratpack.exec.UnmanagedThreadException: Operation attempted on non Ratpack managed thread 'aleph-pool-1-2'

niwinz15:08:05

very strange...

niwinz15:08:43

this can happens when you are writting directly to the ratpack from some other thread

mitchelkuijpers15:08:17

Aha I do a call with aleph to a webservice and then process the result

mitchelkuijpers15:08:34

so I should put this in another or a new deferred

mitchelkuijpers15:08:38

or a promise or whatever

niwinz15:08:47

hmm no, it just ok, it should work

niwinz15:08:25

let see this test case. It just execute something in an other thread..

niwinz15:08:01

The exception is not related to the response handling

niwinz15:08:01

I'm trying understand it...

mitchelkuijpers15:08:53

at ratpack.exec.internal.ThreadBinding$$Lambda$99/1786206212.get(Unknown Source)

niwinz15:08:55

Can you give me handler code? a little simplfied previw of it...

niwinz15:08:13

I want understand why it is happens

mitchelkuijpers15:08:19

Hmm I'll see if I can create a very small example

mitchelkuijpers15:08:04

(defn example
  [context]
  (d/chain
   (http/get "")
   :body
   #(cthttp/ok % {:Content-Type "plain/text"})))

mitchelkuijpers15:08:36

[manifold.deferred :as d]
            [aleph.http :as http]
            [catacumba.http :as cthttp]

mitchelkuijpers15:08:46

Also gives

ratpack.exec.UnmanagedThreadException

mitchelkuijpers15:08:29

yess threadlocals the root of all evil

niwinz15:08:35

I know, but ratpack tries to use them for control from that threads the response is written

niwinz15:08:16

I'll try to reproduce the error with your code

niwinz15:08:21

and try to fix it..

mitchelkuijpers15:08:39

If it's only a aleph error I could also switch to another http lib

mitchelkuijpers15:08:51

But aleph is the only async lib I know of

niwinz15:08:55

I don't want condition your code because catacumba does not handles well this situation.

niwinz15:08:52

It is just work in test cases

niwinz15:08:59

I'm clearly don't understand xD

mitchelkuijpers15:08:38

If I think of something I'll let you know

niwinz15:08:58

Your example as is is just works in tests

mitchelkuijpers15:08:51

Ow lol forgot todo (reset) ...

niwinz15:08:02

I suspect that in some previous steps to that handler something goes out of the ratpack managed threads

mitchelkuijpers15:08:08

I call this function:

mitchelkuijpers15:08:11

(defn get-user-groups
  [host user-key]
  (-> (get host "/rest/api/2/user" {:query-params {:key user-key
                                                   :expand "groups"}})
      (d/chain (fn [{:keys [body]}]
                 (let [groups (-> body
                                  bs/to-string
                                  (json/decode true)
                                  :groups
                                  :items)]
                   (mapv :name groups))))
      (d/catch (fn [e]
                 (log/error e "Error occured while retrieving user" user-key)
                 []))))

mitchelkuijpers15:08:29

and then do a chain in the handler on this result

niwinz15:08:15

The exception happens before

mitchelkuijpers15:08:22

so something along the lines of: ` (d/chain (get-user-groups ...) (if (foo) (forbidden) (delegate)))

niwinz15:08:40

the exception now happens before your code as I can observe in the stack trace

niwinz15:08:18

As you can observe this is a problematic line and is executed before the handler code

niwinz15:08:40

it is nothing related to the handler code instead to something that runs after

niwinz15:08:46

how you start the

niwinz15:08:07

server or do you have any other chain that wraps something in aleph context...

mitchelkuijpers15:08:26

(let [conn (database/context->connection context)
        host (:host context)
        permitted-groups (set (mapv :permissions/group-name
                                    (database/get-permissions conn host)))
        user-key (:userKey (:user context))
        user-groups (jira-api/get-user-groups host user-key)]
    (d/chain user-groups
             set
             (fn [_] (ct/delegate context)))))

niwinz15:08:47

here is the problem

niwinz15:08:29

you are calling the delegate,.. from the aleph context

niwinz15:08:43

Now, I'll try to reproduce the error knowing that

mitchelkuijpers15:08:49

Aha and then I am int he wrong thread

mitchelkuijpers15:08:56

That makes alot of sense

niwinz15:08:47

Yes, now I can reproduce it...

niwinz15:08:11

But I'm currently don't know how I call the delegate in the proper way

niwinz15:08:15

it need some research

niwinz15:08:33

I have an idea on how to fix it bu at this time I shoud go out 😞

niwinz15:08:51

I'll fix it later (in some hours 1 or 2)

niwinz15:08:44

As a temporal workaround, use blocking operation and then call the delegate

niwinz15:08:50

and when this issue is fixed

niwinz15:08:09

you can return to the fully asynchronous way

mitchelkuijpers15:08:44

@niwinz: That's what I am currently doing already simple_smile

mitchelkuijpers15:08:53

Thanks for all your help dude

niwinz19:08:52

@mitchelkuijpers: I just uploaded an other version to clojars (under 0.6.0-SNAPSHOT) that should allow the fully async delegation process

niwinz19:08:29

the only restriction is that the return value of (ct/delegate ctx) should be the response (synchronous or asynchronous) of the handler.

niwinz19:08:52

As I can see on your last code pasted here in channel, this is just how you are using. You are just return a deferred with the value of the execution of (ct/delegate...) so this code now should works as expected in fully asynchonous way.

niwinz19:08:11

The curren implementation removes some speed improvements introduced previously but is just for fix this issue. I think that for the next version I will have time for fix it definitivelly with proper optimizations.

mitchelkuijpers19:08:27

You Rock @niwinz will try it out tomorrow. Will also check out your code which fixes this, I am curious how you did it 👍

niwinz19:08:47

\o/ Let me know any result! Now I go to sleep

niwinz20:08:17

Take care that (ct/delegate...) function call signature is changed. It's no longer accepts context as first parameter. Now the ct/delegate is just a special response instead of special case.

niwinz20:08:38

Read the changelog for an other, maybe more detailed explanation.

niwinz20:08:54

I will update the doc for the next version about this breaking change.

niwinz21:08:39

This change simplifies a lot how the context delegation works, because now is just an other response (that can be returned in sync or asynchronous way)

niwinz21:08:20

And now. Definitively I go to sleep!