Fork me on GitHub
#clojure
<
2019-02-26
>
Joshua Suskalo05:02:32

Anyone who has experience with http-kit: how do you send "multipart/form-data" parameters? I'm currently working with the discord api and that's what it's asking for, but I can't get it to send properly. I've tried putting the data directly into the :form-params key of the request map but that didn't do it, and I've tried putting the data directly into the :body as well, also no luck.

Joshua Suskalo05:02:30

I think I've figured it out.

seancorfield05:02:01

@suskeyhose Let me know if you still need assistance with that. We use http-kit in our test suite to POST images etc into our API for processing.

Joshua Suskalo05:02:46

Oh, thanks! Only issue I'm running into now is actually getting file contents into it. Not sure what's causing problems with it honestly.

Joshua Suskalo05:02:25

I've gotten it figured out now though

5
ido11:02:24

hey guys can I get an extra eye please? I have this ns form failing with spec on clojure 1.10:

(ns af-rematcher.deeplink
  (:require [clojure.string :as s]
            [cemerick.url :refer [query->map]]
            [cheshire.core :as json]
            [version-clj.core :as ver]
            [com.taoensso/timbre :as timbre]))
generating: clojure.lang.ExceptionInfo: Call to clojure.core/ns did not conform to spec. {:clojure.spec.alpha/problems [{:path [], :reason "Extra input", What am I doing wrong?

bronsa11:02:47

com.taoensso/timbre

ido12:02:19

oh it should be taoensso.timbre

wei13:02:38

I have a tree traversal problem: I'm trying to substitute values in a tree according to a substitution map in the tree. for example, if the node [:expr {:x 1} ...] should substitute all :x child nodes with 1. here's an example:

(subst [:a [:b [:expr {:x 1} [:d :e [:f :g :x]]]] [:expr {:y 2} [:h :y]]])
=> ([:a [:b [:expr {:x 1} [:d :e [:f :g 1]]]] [:expr {:y 2} [:h 2]]])
I'm aware of clojure.walk but I'm having trouble using it with "memory". any pointers?

bronsa13:02:22

@wei you can use a thread local to propagate the context down the recursion

wei01:02:52

interesting, do you have an example of this I could refer to?

wei17:02:48

@rodsenra thanks, i was reading https://github.com/nathanmarz/specter/wiki/Using-Specter-Recursively but it'll take a while to get my head around it

ccann18:02:17

Iโ€™m trying to translate this from java to clojure:

int alpha = (pixel >> 24) & 0xff;
int red = (pixel >> 16) & 0xff;
int green = (pixel >> 8) & 0xff;
int blue = (pixel) & 0xff;
but I donโ€™t know how to do the & 0xff part, anyone point me in the right direction?

ghadi18:02:56

(bit-and ..... 0xff)

ccann18:02:44

sweet, I had just gotten there myself ๐Ÿ™‚

oVerde19:02:08

Hi fellas, this mind-boggling error is consuming me Can't make API call blobstore.CreateEncodedGoogleStorageKey in a thread that is neither the original request thread nor a thread created by ThreadManager It's happening with me trying to do an interop where the original is

BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
BlobKey blobKey = blobstoreService.createGsBlobKey("/gs/" + bucket + "/image.jpeg");
to
(.createGsBlobKey (BlobstoreServiceFactory/getBlobstoreService) "/gs/crowd-hm.appspot.com/users/5644647548321792/profile/0eb6082b-f67d-46b4-ad2c-d2ef2b737ee5.jpg")

noisesmith19:02:34

@overde sounds like that method whitelists specific threads for the call and you are trying to call it outside the allowed thread

oVerde19:02:46

Never stumbled in that before, how do I handle it?

noisesmith19:02:53

you could send a function (which is a Callable) to be executed in an app-engine thread pool / thread https://cloud.google.com/appengine/docs/standard/java/runtime#threads

noisesmith19:02:22

I think that would satisfy it (based on a common sense reading of the error message and docs at least)

noisesmith19:02:52

it wants to impose extra coordination over what you get by default with network calls and threads I guess(?)

noisesmith19:02:14

> if you want to call App Engine APIs (com.google.appengine.api.*), you must call those APIs from a request thread or from a thread created using the ThreadManager API.

noisesmith19:02:16

@overde to clarify about the above, if something hands you a Thread you can use (.run t f) where f is any clojure function taking no args -- not quite right...

noisesmith19:02:49

so I think the solution is to ask the google ThreadManager for a thread, and pass your 0 arg function to the run method of that thread

noisesmith19:02:22

the root issue might be that you are in eg. an http server or go block some other context where new threads are created or allocated automatically to call your function

oVerde19:02:48

And I thought that would be just a silly mistake, sounds I need to go deeper ๐Ÿ˜„

oVerde19:02:12

Your explanation was awesome @noisesmith so much thanks!

noisesmith19:02:10

@overde I need to correct myself - you can pass a function to the Thread constructor, but you can't pass one to its run method - so the real fix is different

oVerde19:02:27

๐Ÿ˜ฎ

noisesmith19:02:52

but I'm sure ThreadManager does have the thing you need

noisesmith19:02:44

so it's still straightforward, just not how I initially described it

oVerde19:02:42

@noisesmith so i've got to (ThreadManager/createBackgroundThread (BlobstoreServiceFactory/getBlobstoreService)) after the imports and had com.google.appengine.api.blobstore.BlobstoreServiceImpl cannot be cast to java.lang.Runnable

noisesmith19:02:58

so what you do is make that call (and ancillary code if any) inside a function

noisesmith19:02:31

(ThreadManager/createBackgroundThread (fn [] (some-other-step) (BlobstoreServiceFactory/getBlobstoreService)))

noisesmith19:02:39

where (some-other-step) might not exist, of course

noisesmith19:02:20

you may also need to explicitly call the Thread's run method once it has been created

oVerde19:02:36

Might be something like this (.run (ThreadManager/createBackgroundThread #(BlobstoreServiceFactory/getBlobstoreService))) ?

noisesmith19:02:32

and you probably don't want that at a top level of the ns (otherwise it runs every time the ns is loaded or compiled)

oVerde19:02:47

Sure, it isn't ๐Ÿ˜‰

noisesmith19:02:09

I was taking a literal read of "after the imports" :D

noisesmith19:02:30

I find assuming common sense in software discussions is dangerous

oVerde19:02:38

Awesome ahaha

oVerde19:02:48

Current thread is not associated with any request and is not a background thread

oVerde19:02:55

:smiling_face_with_smiling_eyes_and_hand_covering_mouth:

noisesmith19:02:03

oh, facepalm you want the .start method, not .run - newb error on my part

noisesmith19:02:37

it's a weird thing about the API that if you call the run method, the thread's code (if any) runs on your thread instead - I have never seen a design where that was useful

noisesmith19:02:54

but start actually uses the thread object like you'd expect

Alex Miller (Clojure team)19:02:06

I've wasted more time than I care to admit accidentally calling .run instead of .start in Java

โž• 5
noisesmith20:02:25

I'm glad I'm not the only one

oVerde20:02:03

well.. didn't budge

noisesmith20:02:51

is it about other methods that need to be on google owned threads, or still the same method?

xiongtx20:02:39

Came across this trick to avoid loading files at compile time: https://stackoverflow.com/a/31333821 Does that work b/c the defs are evaluated at compile time and runtime?

noisesmith20:02:37

yes, that's one work around - the issue is there's no compile time / run time distinction of that sort in clojure - every time your namespace is loaded, whether it's because you are compiling your app or running it, the code inside every def runs (as well as any other top level code)

noisesmith20:02:14

the cleaner fix is to put things that interact with the "outside world" or stateful objects inside functions, and arrange to call them via your main or other startup code

xiongtx20:02:20

Interesting...so why evaluate anything at compile time at all?

noisesmith20:02:24

because clojure allows you to use the full language while compiling - it's a common thing in lisps

noisesmith20:02:45

there's no "compile only" mode other than putting something inside an fn

noisesmith20:02:02

and yes, another work around is not using AOT, so that your code is only loaded when running the app (but this still causes problems when eg. linting or running tests, depending what sorts of side effects are in your def)

xiongtx21:02:37

Yes, I see. The interplay b/t clj 's dynamism and Java's static nature always gets me simple_smile .

oVerde20:02:57

Same, the snippet here is the same one you see as the first and second line over the link

noisesmith20:02:26

OK - I think that all the api calls related to using the blob store need to be inside the function you pass to the google thread manager

noisesmith20:02:49

if you are doing that and still see that sort of error, you might need help from someone who knows app-engine