Fork me on GitHub
#clojure
<
2021-01-20
>
borkdude14:01:45

Does this work by design or by accident?

(clojure.core/IllegalArgumentException.)
I came across it over here: https://github.com/clojure/clojurescript/blob/0c353f1947089cb8b1f010b4294b94ac109d4ef6/src/main/clojure/cljs/core.cljc#L2293 (while implementing a new linter for clj-kondo...)

borkdude14:01:46

(`clojure.core/IllegalArgumentException` does not resolve to the class, but the constructor does, it seems)

bronsa14:01:13

it's a bug in macroexpand

bronsa14:01:36

user=> (macroexpand '(clojure.core/Foo.))
(new Foo)

bronsa14:01:28

well I guess it's GIGO

bronsa14:01:50

the namespace segment is just ignored (but because of impl details, it must also exist 🙃 )

borkdude14:01:47

same as with protocol method impls basically

Alex Miller (Clojure team)14:01:00

I know there are some weird corners in the symbol resolution, was just in that code recently, jira welcome

bronsa15:01:24

fixing this would be a breaking change now though

bronsa15:01:50

a patch for this is trivial, lmk if you want one

borkdude15:01:48

oh eh hmm :) I'll close it

mischov15:01:09

Has anybody ever encountered issues using aliased keywords in .cljc files when both the alias and the alias keyword are inside of reader conditionals for one platform (say :clj) but you're compiling for another platform (say :cljs)? For example

(ns main
  (:require #?(:clj [ :as io])))

#?(:clj (defn io-world [] ::io/world)))
results in an invalid keyword error when compiling cljs.

emccue15:01:08

wow, that is strange

mischov15:01:28

I would have expected because the namespaced keyword is restricted to clojure it wouldn't modify cljs compilation.

bronsa15:01:50

autoresolution of keyword happens at read time

bronsa15:01:32

reader conditionals require all the branches to be readable

👍 3
mischov15:01:56

Well that's unfortunate. 😕

p-himik15:01:14

Same with comment. At least, in this case you can just use a fully qualified keyword.

mischov15:01:11

Oh wow, commented code has to be readable in all branches?

didibus21:01:33

Only code wrapped in the comment macro need to be.

didibus21:01:49

If you use ; it does not need to be

dpsutton15:01:11

comment is just a macro. there's no mechanism by which it couldn't be readable

mischov15:01:07

I'm just surprised things have to be readable for platforms they don't apply to. I guess I had expected the reader would just skip over sections for other platforms.

dpsutton16:01:36

yeah its a bit weird but to skip something it has to be read and discarded. to be structurally one form it has to be read. I think the autoresolve of keywords should be inhibited of branches not taken but that's not the case. I think there are some tickets about this kind of problem

👍 3
didibus21:01:43

I think if you use the discard macro: #_ that's not the case though

didibus21:01:57

But the following form needs to be well delimited

dpsutton21:01:08

Clojure 1.10.1
user=> #_(::invalid/discard)
Syntax error reading source at (REPL:1:21).

didibus23:01:55

I didn't even know that was invalid syntax

didibus23:01:06

Ok, seems I was wrong then

dpsutton23:01:07

yeah. the reader expands the aliases. if there is no alias, it barfs

zendevil.eth16:01:06

I’m making a formdata post request like so:

zendevil.eth16:01:16

using http-xhrio in a re-frame handler

{:http-xhrio (http-post "/api/upload-shot-video" {:video body}
   [:upload-success]
   [:upload-error])}
     

zendevil.eth16:01:24

where http-post is the following:

zendevil.eth16:01:48

(defn http-post [uri params on-success on-failure]
  {:method :post
   :uri (str "" uri)
   :params params
   :on-success on-success
   :on-failure on-failure
   :response-format (edn/edn-response-format)
   :format (edn/edn-request-format)
   })

zendevil.eth16:01:19

However, I’m getting the following error on the server:

zendevil.eth16:01:43

clojure.lang.ExceptionInfo: Malformed application/edn request. {:type :muuntaja/decode, :default-format "application/json", :format "application/edn", :charset "utf-8", :request {:reitit.core/match #reitit.core.Match{:template "/api/upload-shot-video", :data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :result #reitit.ring.Methods{:get #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :get, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :head #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :head, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :post #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :post, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :put #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :put, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :delete #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :delete, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :connect #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :connect, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :options #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :options, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :trace #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :trace, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :patch #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :patch, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}}, :path-params {}, :path "/api/upload-shot-video"}, :reitit.core/router #object[reitit.core$lookup_router$reify__11769 0x6741233b "reitit.core$lookup_router$reify__11769@6741233b"], :aleph/request-arrived 150075890045367, :aleph/keep-alive? true, :cookies {}, :remote-addr "0:0:0:0:0:0:0:1", :params {}, :flash nil, :headers {"host" "", "user-agent" "Humboi/1 CFNetwork/1206 Darwin/20.1.0", "content-type" "application/edn", "content-length" "42", "accept" "application/edn", "accept-language" "en-us", "x-forwarded-for" "110.225.238.196", "accept-encoding" "gzip, deflate"}, :server-port 3000, :form-params {}, :session/key nil, :query-params {}, :uri "/api/upload-shot-video", :server-name "localhost", :query-string nil, :path-params {}, :body #object[java.io.ByteArrayInputStream 0x7f2c96a7 "java.io.ByteArrayInputStream@7f2c96a7"], :multipart-params {}, :scheme :http, :request-method :post, :session {}}}
	at muuntaja.core$fail_on_request_decode_exception.invokeStatic(core.clj:132)
	at muuntaja.core$fail_on_request_decode_exception.invoke(core.clj:126)
	at muuntaja.core$create$_decode_request_body__8194.invoke(core.clj:424)
	at muuntaja.core$create$reify__8216.negotiate_and_format_request(core.clj:491)
	at muuntaja.middleware$wrap_format$fn__8285.invoke(middleware.clj:72)
	at humboiserver.middleware$wrap_formats$fn__9418.invoke(middleware.clj:38)
	at reitit.ring$ring_handler$fn__12248.invoke(ring.cljc:326)
	at clojure.lang.AFn.applyToHelper(AFn.java:154)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.lang.Var.invoke(Var.java:384)
	at ring.middleware.reload$wrap_reload$fn__4463.invoke(reload.clj:39)
	at selmer.middleware$wrap_error_page$fn__4478.invoke(middleware.clj:18)
	at prone.middleware$wrap_exceptions$fn__4719.invoke(middleware.clj:159)
	at ring.middleware.flash$wrap_flash$fn__8523.invoke(flash.clj:39)
	at ring.middleware.session$wrap_session$fn__8890.invoke(session.clj:108)
	at ring.middleware.keyword_params$wrap_keyword_params$fn__8936.invoke(keyword_params.clj:53)
	at ring.middleware.nested_params$wrap_nested_params$fn__8994.invoke(nested_params.clj:89)
	at ring.middleware.multipart_params$wrap_multipart_params$fn__9126.invoke(multipart_params.clj:171)
	at ring.middleware.params$wrap_params$fn__9150.invoke(params.clj:67)
	at ring.middleware.cookies$wrap_cookies$fn__8841.invoke(cookies.clj:214)
	at ring.middleware.absolute_redirects$wrap_absolute_redirects$fn__9338.invoke(absolute_redirects.clj:47)
	at ring.middleware.resource$wrap_resource_prefer_resources$fn__9186.invoke(resource.clj:25)
	at ring.middleware.content_type$wrap_content_type$fn__9286.invoke(content_type.clj:34)
	at ring.middleware.default_charset$wrap_default_charset$fn__9310.invoke(default_charset.clj:31)
	at ring.middleware.not_modified$wrap_not_modified$fn__9252.invoke(not_modified.clj:61)
	at ring.middleware.x_headers$wrap_x_header$fn__8486.invoke(x_headers.clj:22)
	at ring.middleware.x_headers$wrap_x_header$fn__8486.invoke(x_headers.clj:22)
	at ring.middleware.x_headers$wrap_x_header$fn__8486.invoke(x_headers.clj:22)
	at humboiserver.middleware$wrap_internal_error$fn__9412.invoke(middleware.clj:17)
	at aleph.http.server$handle_request$fn__19409$f__14156__auto____19410.invoke(server.clj:158)
	at clojure.lang.AFn.run(AFn.java:22)
	at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62)
	at manifold.executor$thread_factory$reify__14038$f__14039.invoke(executor.clj:44)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.lang.Thread.run(Thread.java:748)
Caused by: clojure.lang.ExceptionInfo: Malformed application/edn in :muuntaja/decode {:type :muuntaja/decode, :format "application/edn"}
	at muuntaja.core$on_exception.invokeStatic(core.clj:284)
	at muuntaja.core$on_exception.invoke(core.clj:267)
	at clojure.core$partial$fn__5839.invoke(core.clj:2626)
	at muuntaja.core$create_coder$decode__8144.invoke(core.clj:322)
	at muuntaja.core$create$_decode_request_body__8194.invoke(core.clj:422)
	... 34 more
Caused by: java.lang.RuntimeException: No reader function for tag object
	at clojure.lang.EdnReader$TaggedReader.readTagged(EdnReader.java:801)
	at clojure.lang.EdnReader$TaggedReader.invoke(EdnReader.java:783)
	at clojure.lang.EdnReader$DispatchReader.invoke(EdnReader.java:549)
	at clojure.lang.EdnReader.readDelimitedList(EdnReader.java:757)
	at clojure.lang.EdnReader$MapReader.invoke(EdnReader.java:680)
	at clojure.lang.EdnReader.read(EdnReader.java:145)
	at clojure.lang.EdnReader.read(EdnReader.java:111)
	at clojure.edn$read.invokeStatic(edn.clj:35)
	at clojure.edn$read.invoke(edn.clj:14)
	at muuntaja.format.edn$decoder$reify__7763.decode(edn.clj:12)
	at muuntaja.core$create_coder$decode__8144.invoke(core.clj:320)
	... 35 more

zendevil.eth16:01:43

How to fix this error?

dpsutton16:01:43

Malformed application/edn No reader function for tag object. You're serializing something that shouldn't be serialized

jtkdvlp16:01:34

probably your video body is of the wrong type

zendevil.eth16:01:29

I’m trying to upload a video to the server from its url in ios

zendevil.eth16:01:40

Here’s more code:

zendevil.eth16:01:44

(let [video {:uri (-> coeffects :db :shot-video-uri)}
         body (js/FormData.)]
     (.append body "video" video)
     {:http-xhrio (http-post "/api/upload-shot-video" {:video body}
  [:upload-success]
  [:upload-error]}
    ))

zendevil.eth16:01:58

@jtkdvlp body is of type FormData

p-himik16:01:01

IIRC FormData has to be the body itself - you cannot wrap it into an extra object like {:video ...}.

p-himik16:01:51

Seems like in your case above you can just replace {:video body} with body.

zendevil.eth16:01:08

@p-himik unfortunately I’m still getting the error after replacing {:video body} with body

dgb2316:01:41

You might want to check the mdn resources on FormData.

p-himik16:01:46

Ah, you also have :format (edn/edn-request-format) - don't do that.

jtkdvlp16:01:01

yeah maybe you try to send form-data binary array as edn?

p-himik16:01:46

@ps Try just removing the line with :format from http-post..

dgb2316:01:38

Here is a little JS snippet that might interest you:

const form_entries = (new FormData($form)).entries();
const params = [...form_entries];

// params is a JS-array now

dgb2316:01:30

Oops we’re in #clojure here. I thought this is related to #clojurescript

p-himik16:01:49

It doesn't have to be an array. You can just send a single FormData object.

p-himik16:01:03

The issue is that OP is trying to serialize it as EDN.

dgb2316:01:21

Yeah, that’s why you’d want to convert it first right?

dgb2316:01:34

So this is actually a clojurescript question?

p-himik16:01:18

You don't want to convert it at all - you just have to send raw FormData because the underlying platform (JavaScript) knows how to handle it. Yes, it ends up being a CLJS question. Or even a cljs-ajax question.

👍 3
dgb2317:01:41

Ah right, I assumed that the library used here strictly expects serializable data. Your other answer (changing to :body) explained it for me thanks

👍 3
zendevil.eth16:01:12

@p-himik removing format gives unrecognized request format nil

p-himik16:01:20

That shouldn't happen because the error is thrown only when :body is nil. Can you post here the full code that you have right now?

zendevil.eth16:01:37

(reg-event-fx
 :upload-shot-video
 (fn [coeffects _]
   (prn "uploading video")
   (let [video {:uri (-> coeffects :db :shot-video-uri)}
         body (js/FormData.)]
     (.append body "video" video)
     {:http-xhrio {:method :post
                   :uri (str "" "/api/upload-shot-video")
                   :params body
                   :on-success [:upload-success]
                   :on-failure [:upload-error]
                   :response-format (edn/edn-response-format)
                   ;;:format (edn/edn-request-format)
                   }}
     #_(POST "/api/upload-shot-video"
             {:body body
              :response-format :json
              :keywords? true
            :handler #(dispatch [:upload-success %])
            :error-handler #(dispatch [:upload-error %])})
     )))

p-himik16:01:06

Ah, I missed one thing. Replace :params body with :body body.

zendevil.eth16:01:08

@p-himik thanks the error has disappeared

👍 3
zendevil.eth16:01:39

but the params are empty in the server

zendevil.eth16:01:46

Here’s the handler:

(defn upload-shot-video [req]
(prn "video is! " (-> req :params))
  (r/response {:res "okay!"}))

zendevil.eth16:01:22

and I’m getting:

"video is! " {:video ""}

p-himik16:01:36

I don't know what you're using on your server so I can't really help. My assumption is that you have the content not in (:params req) but some place else. Maybe (:body req) - at least, that would be logical.

p-himik16:01:56

I will probably be a binary stream.

zendevil.eth16:01:54

that’s right. there’s a stream at (-> req :body)

p-himik16:01:12

Well there you have it. :) Seems like all is fine.

zendevil.eth17:01:53

@p-himik while we’re at it, do you know how to convert an input stream to a file?

zendevil.eth17:01:04

that can be saved to disk?

p-himik17:01:09

Look at the functions in the namespace. I don't exactly remember the way to do it.

zendevil.eth17:01:37

How to convert an inputstream to an outputstream?

zendevil.eth17:01:03

I try the following:

zendevil.eth17:01:17

but get the following error:

zendevil.eth17:01:19

No single method: make_output_stream of interface: http://clojure.java.io.IOFactory found for function: make-output-stream of protocol: IOFactory

noisesmith17:01:55

@ps I think the most straightforward thing is io/copy to copy the input stream to the ouput

hiredman17:01:34

there is no such thing as "converting an inputstream to an outputstream"

noisesmith17:01:36

@ps actually, thinking more about this - what is the output-stream for?

noisesmith17:01:46

an output-stream is something you can write to

zendevil.eth17:01:48

@noisesmith to convert to a file on disk

noisesmith17:01:59

how is copying an input stream to it meaningful?

noisesmith17:01:04

that's different, and yeah, use io/file and io/copy

zendevil.eth17:01:26

I have a video as an input stream on the server that I’m trying to load on aws as a file

noisesmith17:01:47

that's not creating an output-stream to be clear

zendevil.eth17:01:08

for that I’m saving it as a file. I thought to save as a file, it first needs to be converted to an outputstream

noisesmith17:01:26

the file provides an output-stream

noisesmith17:01:28

you write to that

zendevil.eth17:01:14

@noisesmith I have the following now:

zendevil.eth17:01:25

(with-open [o ( "./resources/public/video.mp4")]
  (.write o (-> req :body))
    )

zendevil.eth17:01:28

but this gives:

noisesmith17:01:38

$ scrot -s foo.png
$ clj
Clojure 1.10.1
user=> (require '[ :as io])
nil
user=> (io/copy (io/file "foo.png") (io/file "bar.png"))
nil
user=> 
$ cmp foo.png bar.png
$ echo $?
0

zendevil.eth17:01:40

java.lang.IllegalArgumentException: No matching method write found taking 1 args for class http://java.io.BufferedOutputStream

noisesmith17:01:44

all you need is io/file and io/copy

noisesmith17:01:51

don't use the .write method

noisesmith17:01:33

(if you don't understand shell, the above example copies a png file, and cmp verifies that byte for byte the copy is the same as the original)

zendevil.eth17:01:54

in the example you are copying a file

zendevil.eth17:01:00

I have an input stream

dpsutton17:01:04

(doc )
-------------------------

([input output & opts])
  Copies input to output.  Returns nil or throws IOException.
  Input may be an InputStream, Reader, File, byte[], char[], or String.
  Output may be an OutputStream, Writer, or File.

  Options are key/value pairs and may be one of

    :buffer-size  buffer size to use, default is 1024.
    :encoding     encoding to use if converting between
                  byte and char streams.   

  Does not close any streams except those it opens itself 
  (on a File).

🙏 3
noisesmith17:01:07

right, the first arg can be an input stream

noisesmith17:01:13

io/copy is smart enough to do the right thing

noisesmith17:01:37

so (io/copy (:body req) (io/file "resources/public/video.mp4"))

noisesmith17:01:07

also, be cautious about writing to resources/public, usually resources is where you want to put things that would go in a jar, and you can't write a file to it like that

zendevil.eth18:01:38

unfortunately the video is 0 bytes

zendevil.eth18:01:14

it’s supposed to be more than 0 bytes

noisesmith18:01:20

a problem I've frequently seen is some ring middleware consuming the request body before my code sees it

zendevil.eth18:01:17

@noisesmith as far as I know, I’m using two: middleware/wrap-formats wrap-multipart-params

noisesmith18:01:16

@ps wrap-multiplart-params already wants to put your upload in a file https://github.com/ring-clojure/ring/wiki/File-Uploads

noisesmith18:01:30

(I have found this annoying in the past, eg. when I'm running on a cloud server, all I'd ever do with that file is write it back to the network for storage elsewhere, but everyone seems to accept that and just read the file back to upload... pointless use of disk)

noisesmith18:01:17

> By default, uploads are stored in temporary files that are deleted an hour after being uploaded. This is handled by https://github.com/ring-clojure/ring/blob/1.6.3/ring-core/src/ring/middleware/multipart_params/temp_file.clj function.

🙏 3