Fork me on GitHub
#clojure-uk
<
2017-07-12
>
thomas07:07:17

morning from a very :rain_cloud: :flag-nl:

Rachel Westmacott08:07:18

It’s pretty wet in Surrey too

agile_geek08:07:05

morning from ☀️ north east england

maleghast09:07:54

morning from Strathard…

agile_geek09:07:32

I think I've eventually finished filling in forms for my next contract...until I start...then I'll probably have a lot more to fill in!

mccraigmccraig10:07:41

gawd that looks just awfully depresseing @maleghast , how do you stand it ?

maleghast10:07:51

Clean living, prayer, the usual… 😉

agile_geek11:07:08

@maleghast looks terrible...I'd just like to find a role within 250 miles of my house atm!

maleghast11:07:35

@agile_geek - Well, that I can relate to as technically__ my role is London based, but I am being allowed to work remotely full time at the moment and into the Autumn I will be back to London 3 days, Trossachs 4 days 😉

agile_geek11:07:16

Looks like I'm doomed to spend 5 days per week away from home for at least another 3 months

maleghast11:07:23

Sorry to hear that mon ami - I did that every other week for a while when we were in the Philippines, going to Hong Kong, and it was no fun… 😞

maleghast11:07:20

OK… I am trying to get data from an API using the http-kit client… It seems__ as though the promise that I get back from the function is NEVER fulfilled… What have I missed (please - tearing my hair out at this point):

(defn rightrelevance-request
  "Function to request content from RightRelevance API"
  [terms start rows access_token]
  (http/request
   {:url ""
    :method :get
    :user-agent "Cervest Harvester"
    :query-params {"query" terms
                   "start" start
                   "rows" rows
                   "access_token" access_token}}
   (fn [{:keys [status headers body error]}] ;; asynchronous response handling
            (if error
              (println "Failed, exception is " error)
              (println "Async HTTP GET: " status)))))

maleghast11:07:54

(is it because I am evaluating a call to this function in my emacs buffer and for some reason that can’t wait for the response?)

mccraigmccraig12:07:46

@maleghast are you specifying a callback and expecting a result in the response promise ?

maleghast12:07:38

This part:

(fn [{:keys [status headers body error]}] ;; asynchronous response handling
            (if error
              (println "Failed, exception is " error)
              (println "Async HTTP GET: " status)))
is the callback, as per the http-kit documentation

maleghast12:07:55

I am expecting the code to run and wait for the promise to be realised

mccraigmccraig12:07:42

i've not used http-kit much, but using both (callback and promise) seems a bit unusual

mccraigmccraig12:07:51

does your callback get called?

maleghast12:07:14

I am not sure how to tell, but I am assuming no

maleghast12:07:53

That’s the “instructions” I am following… I am assuming that I have either missed something OR they are assuming that I know something that I don’t… 😞

maleghast12:07:39

(my options map is just inside the function instead of being def-ed outside and passed in)

mccraigmccraig12:07:51

ah, i see, it delivers the result of your callback to the promise - https://github.com/http-kit/http-kit/blob/master/src/org/httpkit/client.clj#L197

maleghast12:07:29

apparently, yes…

mccraigmccraig12:07:40

so if your callback is never getting called then you will never get a response - it looks like your http request is not returning for some reason

mccraigmccraig12:07:27

do you get a response if you curl the url ?

mccraigmccraig12:07:01

http-kit doesn't appear to have any useful logging in the java part of the client https://github.com/http-kit/http-kit/blob/master/src/java/org/httpkit/client/HttpClient.java

maleghast12:07:49

Ah, no I don’t any more… I just re-ran my test cURL (admittedly using a client - Insomnia) and that no longer works…

maleghast12:07:11

It does send back an HTTP 500 however, so the code should be printing that…

maleghast12:07:44

What I do get back is this: #object[clojure.core$promise$reify__9424 0x5c770e66 {:status :pending, :val nil}]

maleghast12:07:50

Yeah, I thought that it was

mccraigmccraig12:07:01

hmm - your callback is returning nil, which would lead to a nil response, though i would have expected the status to not be :pending

glenjamin12:07:10

is this on the repl?

mccraigmccraig12:07:17

what happens if you just remove your callback ?

glenjamin12:07:30

can you deref the promise again after the initial output?

maleghast12:07:50

@mccraigmccraig I just get the promise back in a pending state

mccraigmccraig12:07:01

@maleghast have you derefed it after that ?

glenjamin12:07:03

@*1 should do it i think

maleghast12:07:16

@glenjamin - I have done that too - never got anything other than pending

mccraigmccraig12:07:59

it's expected that you should get the promise back :pending immediately - and that the value should be delivered some time later

mccraigmccraig12:07:26

how about a different url @maleghast ?

maleghast12:07:35

I’ve taken the callback out and now I get this when I deref the promise:

{:opts {:url "", :method :get, :user-agent "Cervest Harvester", :query-params {"query" "agriculture climate change", "start" 0, "rows" 10, "access_token" "998647ad406b2870b6a738e0243db376f8daf6a05175fc2d4d164e43f257882e"}}, :body "", :headers {:connection "keep-alive", :content-length "0", :date "Wed, 12 Jul 2017 12:49:30 GMT"}, :status 500}

maleghast12:07:54

So the callback appears to be buggered in some way

mccraigmccraig12:07:54

so that's working now

mccraigmccraig12:07:04

your callback was returning nil

mccraigmccraig12:07:15

and the value of the callback is what gets delivered to the promise

mccraigmccraig12:07:27

so your callback would never deliver anything other than nil

maleghast12:07:39

@mccraigmccraig - Ok, I get that… http-kit REALLY need to update their docs in that case…

maleghast12:07:24

Thanks for helping me to see the wood for the trees… 🙂

mccraigmccraig12:07:31

but deliver works perfectly well with a nil value, so i can't see why your original response wouldn't have looked like : #promise[{:status :ready, :val nil} 0x2e7c756]

mccraigmccraig12:07:58

whereas @maleghast 's original promise :status was :pending

glenjamin12:07:51

mm, very odd

maleghast12:07:49

So, once I’ve got the API’s curator to fix their 500, how do I write a bit of code that derefs the promise once it’s been fulfilled?

glenjamin12:07:29

the idea is usually you’d use deref to block until fulfilled

mccraigmccraig12:07:33

more abstract question @maleghast - what are you planning on doing with http-kit ? plain clojure promises are a poor async abstraction

maleghast12:07:14

@mccraigmccraig - I am using it as an HTTP client to get data from an API

mccraigmccraig12:07:23

(because they don't support callbacks, so composition (aka doing something with the result) is not easy without blocking)

maleghast12:07:43

the responses - if 200s with bodies - will be parsed, put into a canonical DB and added to an ES index.

maleghast12:07:04

And I don’t mind blocking in the slightest in this instance.

maleghast13:07:22

Nothing needs to be in real-time and nothing will be blocking clients to the app, as they will get their results from the ES cluster and will be ok with potentially old / stale data.

mccraigmccraig13:07:58

if you don't mind blocking then plain clj promises will do fine... if you want to avoid blocking then https://github.com/ztellman/aleph is much easier, or you could use the http-kit callback to put results on a core.async chan

maleghast13:07:21

I have access to Aleph already as I am using Edge (juxt/edge) facepalm

maleghast13:07:37

Perhaps I should be using that instead? facepalm

Rachel Westmacott13:07:56

there’s a small learning curve around manifold with aleph, but I’ve found it to be pretty nice when I’ve used it

mccraigmccraig13:07:26

i'm a manifold+aleph fanboy, so i'm probably biased

maleghast13:07:13

Hehe - nothing wrong with liking library-x over library-y - at least you know your biases @mccraigmccraig

mccraigmccraig13:07:29

actually, i'm a cats+manifold+aleph fanboy - together those things are great

maleghast13:07:02

I am looking at the Aleph docs / examples, it looks pretty accessible…

maleghast13:07:07

I am wondering what the benefit is in using it over http-kit in my current use-case…

maleghast13:07:32

(aside from the weird behaviour from http-kit that started all of this)

mccraigmccraig13:07:52

no benefit if you will be immediately blocking on the response promise

mccraigmccraig13:07:27

if you want to experiment with making something entirely non-blocking though, aleph/manifold will be much better - https://github.com/ztellman/manifold/blob/master/docs/deferred.md#composing-with-deferreds

maleghast13:07:30

Ah, ok then 🙂 When I want to do that, that’s the way I will go… 🙂

maleghast13:07:36

Thanks! 🙂

maleghast13:07:03

Ok, so I how do I kill the REPL in my emacs (cider-connect) from outside - my emacs is completely blocked - the API that I am trying to use is having problems and the REPL is completely blocked and locked up.

mccraigmccraig13:07:36

ctrl-c ctrl-c not working ?

maleghast13:07:51

now I know why aleph + manifold

shan13:07:53

@maleghast you can try and kill -SIGUSR2 the emacs process

maleghast13:07:10

I will try that, thx

dotemacs13:07:24

how about just finding out the PID of the nrepl session, and killing just that

dotemacs13:07:34

rather than tearing down the whole Emacs

maleghast13:07:00

@shan That did the trick, thx!

maleghast13:07:21

@dotemacs - I’d already stopped / killed the nrepl proccess

shan13:07:21

No worries, I’ve had spacemacs freeze due to some helm bug and i’ve used it a few times

dominicm14:07:37

Just a thought from the sun bed: how can you lint clojure? Example rule: it's nonsensical to call = with one parameter (although valid) usually you've made a mistake. However, searching the code for (= elem) will catch:

(-> x
    (= 10))
Well, you say, run it after macro expansion! But what about the numerous macros that might generate that code? Cond variants are perhaps candidates for this. I know eastwood hates:
(cond-> 10
  true
  Inc
  Foo?
  Inc)
Because the generated code contains an always true check (`(if true ...)`)

conan19:07:16

Yes. In a biscuit factory