Fork me on GitHub
#onyx
<
2017-09-20
>
ben.mumford08:09:03

hi all, we're experimenting with onyx and loving it so far. one of our tasks needs to call an external rest api. how should we best handle the blocking call? should we park the thread or something else?

ben.mumford08:09:15

any help would be much appreciated

lmergen09:09:26

@ben.mumford you can batch multiple requests concurrently and block inside the function

lmergen09:09:04

that way you can do any number of requests concurrently from within a single thread

ben.mumford10:09:15

so i batch multiple requests, for each one send of the call to the external API and block?

ben.mumford10:09:28

so no parking and no go blocks required?

ben.mumford10:09:40

i think i see, the libraries referenced in the batch fn documentation make sense. make all the calls, resolve all the promises and return the list

lmergen10:09:33

exactly — so that way onyx treats the batch as a single blocking operation, and inside the batch you can still do things efficiently and asynchronous

ben.mumford10:09:23

with regards batch fn what do you do if one of the api calls fails?

lmergen10:09:36

that depends upon your application

ben.mumford10:09:41

you have to return a list the same length

lmergen10:09:44

what would you normally do if an api call fails ?

lmergen10:09:18

my personal flavour to handle this is to just crash the task (i.e. throw an exception) and let onyx deal with it on a higher level

lmergen10:09:47

but it might be more efficient to retry api calls N times

ben.mumford10:09:47

what happens to the messages then?

lmergen10:09:01

onyx will retry the entire batch another time

ben.mumford10:09:33

i need to read more on exceptions thrown from tasks

ben.mumford10:09:54

thanks for your help

lmergen10:09:26

you can choose to create a flow condition to catch the exception

ben.mumford10:09:19

are flow conditions like middleware then? i thought that was lifecycles?

lmergen10:09:06

a little bit yes — it’s more that it allows your messages/data to “flow” through your workflow based on predefined conditions

lmergen10:09:32

lifecycles are lower level, and always called at predefined conditions

lmergen10:09:19

e.g. lifecycles allow me to “inject” certain information into a task when it launches (e.g. start a webserver)

lmergen10:09:14

flow conditions allow me to define how a message goes through my workflow dynamically — e.g., i can filter all messages that do not match a certain criteria

ben.mumford10:09:48

no doubt more stupid questions coming soon

Drew Verlee20:09:46

What would be the high level sketch of how to make onyx part of a reactive web application? From the users perspective the webpage would just update automatically. input -> ?? -> user not input -> ??? <- user request The clients could either subscribe to onyx directly in some form or maybe to a kafka topic.

lucasbradstreet21:09:46

You would output to a topic which routes the outputs to the correct client

lucasbradstreet21:09:38

You would include an id in the message which would correlate to a user’s web socket, and then your topic consumer would push the messages to the right web socket

Drew Verlee21:09:42

Right. That makes sense, the area i’m less clear about is web sockets. But now I can focus on that area. thanks @lucasbradstreet