Fork me on GitHub
#clojure
<
2021-01-25
>
zackteo02:01:36

Hello! I have a uni project to do a distributed app service (e.g. a distributed lock server) - with emphasis on the service A friend of mine approached me to perhaps take this chance to learn Clojure - but I'm not sure how to evaluate the feasibility of doing so in Clojure vs say Go. Not sure if anyone is able to offer any advice before I look into it further 😮 The requirements are as follows: - Maintain correctness of the application w/o faults - Should provide scalable service (#nodes supported) - Fault tolerance - Byzantine fault tolerance (Bonus) - Implement a suitable consistency model

Nassin04:01:27

There's way more literature and implementations about all of that on the Internet in Go for sure, and IMO, this in area where Go is used the most, low-level distributed network services

Nassin04:01:04

So depends more on if you value community/ecosystem resources.

Nassin04:01:41

There's lots resources for Java too I guess, the Clojure world just reuses the Java implementations.

zackteo04:01:44

Thanks @kaxaw75836, I think it was too tough to find more members who would do clojure anyway. Maybe something to explore, but not within my academics i guess 🙂

zendevil.eth06:01:26

I’m trying to upload a video to the server as follows:

(reg-event-fx
 :shoot-video
 (fn [coeffects [_ !camera]]
   (if (-> coeffects :db :shoot-video)
     (do
       (prn "stopping shoot video")
       (. @!camera stopRecording)
       )
     (go
       (prn "starting shoot!")
       (let [uri (.-uri (<p! (. @!camera recordAsync)))]
         (. MediaLibrary saveToLibraryAsync uri)
         (prn "dispatching with this uri" uri)
         (dispatch [:save-uri uri])
         (dispatch [:upload-shot-video]))))
   {:db (assoc (:db coeffects) :shoot-video (not (:shoot-video (:db coeffects))))}))

(reg-event-fx
 :upload-shot-video-server
 (fn [coeffects [_ blob]]
   (let [body (js/FormData.)]
     (.append body "video" {:name "movie.mov" :type "video/quicktime" :uri blob}  "video.mov")
     (.append body "key" "VAL")
     {:http-xhrio {:method :post
                   :uri (str server-uri "/api/upload-shot-video")
                   :body body
                   :on-success [:upload-success]
                   :on-failure [:upload-error]
                   :format (json-request-format)
                   :response-format (raw-response-format) #_(edn/edn-response-format)}}))
 
and the handler is the following:

zendevil.eth06:01:26

(defn upload-shot-video [req]
  (prn "uploading video")
  (prn "video is! " (-> req :multipart-params))
  (prn "video is " (-> req :params))
  (prn "video before is " (slurp (-> req :body)))
  (.reset (-> req :body))
  (prn "req full" (-> req))
  (prn "video after is " (-> req :body))
  (prn "video is! " (-> req :multipart-params))
  ( (-> req :body) ( "./resources/public/video.mov"))

zendevil.eth06:01:27

However, the request is not a multipart request, so I don’t see any entries in :multipart-params key

zendevil.eth06:01:56

and the entries I do see in the params key doesn’t have a :tempfile key so I can’t access the file that’s uploaded

zendevil.eth06:01:30

Instead I see

{"_parts":[["video",{"_data":{"size":2971246,"blobId":"D002459C-47C5-4403-ABC6-A2DE6A46230A","offset":0,"type":"video/quicktime","name":"DCDE604A-954F-4B49-A1F9-1BCC2C2F37BC.mov","__collector":null}}],["key","VAL"]]}
when doing (-> req :params)

zendevil.eth06:01:00

there’s a filename in the request map. Where is this file stored?

zendevil.eth06:01:42

I’m unable to extract the data from the request since there’s no tempfile key

zendevil.eth06:01:57

and the request is not multipart

hiredman06:01:05

You can't pass a clojure map like that to a function that expects a js object

hiredman06:01:15

I am no js dev but these docs for append on form data https://developer.mozilla.org/en-US/docs/Web/API/FormData/append don't seem to match your 3 arg call to that method

hiredman06:01:17

https://developer.mozilla.org/en-US/docs/Web/API/FormData/Using_FormData_Objects has a js example, using form data to do a multiparty file upload, and you can see they construct a blob object, with the mimetype part of that, and pass it directly to append

zendevil.eth07:01:49

@hiredman, after changing to this:

(reg-event-fx
 :upload-shot-video-server
 (fn [coeffects [_ blob]]
   (let [body (js/FormData.)]
     (prn "blob is " blob)
     (.append body "video" blob)
     (.append body "key" "VAL")
     {:http-xhrio {:method :post
                   :uri (str server-uri "/api/upload-shot-video")
                   :body body
                   :on-success [:upload-success]
                   :on-failure [:upload-error]
                   :format (json-request-format)
                   :response-format (raw-response-format) #_(edn/edn-response-format)}}))
)
I find that multipart params does have the video and key keys, but the value to the video key is “” and not the blob that’s expected

hiredman07:01:11

You should ask in #clojurescript, but I would start by reading some docs. I don't know what library you are using that is actually making the request, but docs on that, I also think setting the format to (json-request-format) seems suspicious

hiredman07:01:22

So check the docs on that

zendevil.eth07:01:08

@hiredman I’m using the cljs-ajax library

hiredman07:01:34

The readme for https://github.com/JulianBirch/cljs-ajax says it doesn't support file uploads

hiredman07:01:19

Oh, I misread it

zendevil.eth07:01:25

@hiredman it says it doesn’t have “doesn’t have any other support for file uploads”

zendevil.eth08:01:42

@hiredman I’m using :body and not :params as mentioned

zendevil.eth08:01:13

wrapping the map with (clj->js) worked.

zendevil.eth08:01:41

@hiredman Thanks a lot. You are a savior. Had been stuck on this for two whole days.

svt10:01:15

How do we test future in clojure?

svt10:01:46

I want to test if a function which is passed to to future has been called or not

borkdude11:01:14

(def state (atom nil))
(defn foo [] (reset! state true))
(def fut (future (foo)))
@fut
(is @state)

borkdude11:01:52

or if you want to test the return value of foo, just return it from the future

borkdude11:01:04

and then test the return value of @fut

svt11:01:12

Thank you for the quick reply 🙂

svt11:01:43

@borkdude Is there a way to do it without atom? Or just checking if the future is initiated?

borkdude11:01:15

it depends on your usage. your question was: test if a function has been called, so I assume you want to test some side effect. the side effect depends on your specific program

svt11:01:08

Yes, right

mpenet12:01:09

you can do the same with promise as well

mpenet12:01:25

but it's the same really

mpenet12:01:43

you can also use a lib like spy

mpenet12:01:16

I quite like just relying on promise or atom (if you need to mutate more than once) personally

Christian14:01:37

Just checking, if I get the current eco system correctly. I'm a beginner, so bear with me, if what I type sounds strange to you. • Clojure is a hosted language on the JVM, which makes it easy to use Java libs with interop. • There are implementations with JS and .Net, does that mean I can run .Net and JS code like the Java Interop? • There is clj-python, I understand, that it runs python code on the side and reimports the results back into the clojure environment. So I can call (slower than normal?) Python libs like Java interop? • With GraalVM that is hosting Python on the same VM as my Clojure, does that just speed up my Python calls because they don't need to be external? What is the benefit to the clj-python lib?

delaguardo14:01:01

clj-python can utilize an entire python ecosystem. GraalVM provides a Python 3.8 compliant runtime.

andy.fingerhut14:01:29

yes on the first bullet. On the second bullet, ClojureScript is compiled to JavaScript, and has ways to call JavaScript libraries from ClojureScript code, which syntactically are often similar to calling Java methods from Clojure. I haven't used ClojureCLR, but I believe it can make calls to .Net libraries via interop similarly to Clojure->Java calls.

Andy Wootton15:01:47

I wouldn't risk anything important on the continued future of the .Net version right now. It seems to have lost momentum, even at Microsoft :-)

👀 6
Mno16:01:36

(there’s also #babashka which is clojure as a scripting language (basically), and it has a variety of interop options)

didibus06:01:11

> Clojure is a hosted language on the JVM, which makes it easy to use Java libs with interop. There are implementations with JS and .Net, does that mean I can run .Net and JS code like the Java Interop? Clojure has more than one implementation. The main one is known as just Clojure, and its the Java based implementation that compiles to Java bytecode, thus runs on the JVM and can make use of all any Java libraries through interop. There is another implementation known as ClojureScript, and that one is the JavaScript based one, it compiles to JavaScript source code, and can use any JavaScript library through interop (though sometimes setting it up to do so can be challenging). Yet another implementation is knowns as ClojureCLR, and that one is .NET based. It compiled to CLR IL, and you can use .NET classic libraries through interop. There's a beta release of it that also adds support for .NET Core ane .NET 5 support. > There is clj-python,  I understand, that it runs python code on the side and reimports the results back into the clojure environment. So I can call (slower than normal?) Python libs like Java interop? Not exactly like Java interop, but close. It shouldn't be any slower than standard Python, you should expect similar speeds, but if you keep going back/forth between Clojure and Python it can have some overhead. This library to access Python from Clojure only works for the Clojure JVM implementation. > With GraalVM that is hosting Python on the same VM as my Clojure, does that just speed up my Python calls because they don't need to be external? What is the benefit to the clj-python lib? That's different, clj-python uses the C implementation of Python: CPython, which is the reference implementation of Python. GraalVM has an alternate implementation of Python which can run Python code on GraalVM. With it, you can have a Clojure JVM application running on GraalVM which can run some Python code and interop with it, but its a lot less integrated than clj-python, or the Java interop. It also won't support things like numpy and other Python things that are actually implemented in C and not real Python (I believe). But a big difference is GraalVM lets you embed Python inside your Clojure application, while clj-python lets you use Python libraries as if they were Clojure libraries.

Andy Wootton18:05:59

I think it's fair to say the .Net implementation is no longer in active development.

didibus05:05:42

That's really not fair to say. Last commit is April 17 2022, dmiller is still actively maintaining it: https://github.com/clojure/clojure-clr There's a beta out for 1.11, and it works with .net core, .net 6 and .net framework. Unless I missed an announcement to the contrary?

danielneal16:01:23

I know of one library for generating clojure documentation from tests - http://helpshift.github.io/hydrox/ are there any others? Preferably one that doesn't depend on leiningen

borkdude19:01:20

would this be "too clever", shortcutting cond's expansion?

(defmacro cond*
  [& clauses]
  (when clauses
    (let [fst (first clauses)]
      (if (keyword? fst) (second clauses)
          (list 'if fst
                (if (next clauses)
                  (second clauses)
                  (throw (IllegalArgumentException.
                          "cond requires an even number of forms")))
                (cons `cond* (next (next clauses))))))))


(require '[clojure.walk :as walk])

(walk/macroexpand-all '(cond (odd? x) x :else 1))
;;=> (if (odd? x) x (if :else 1 nil))

(walk/macroexpand-all '(cond* (odd? x) x :else 1))
;;=> (if (odd? x) x 1)

hiredman21:01:23

the clojurescript compiler does a lot of similar kind of, I dunno what to call them, peephole optimizations? I can't think of how this would break anything, but I am wary of that kind of thing in general because as a result cljs has more corner cases with core.async, and sometimes you can get weird double code evaluations

hiredman21:01:41

(I guess it isn't the compiler that does it, it is the macros themselves)

hiredman21:01:49

https://clojure.atlassian.net/browse/ASYNC-91 and https://clojure.atlassian.net/browse/ASYNC-128 are examples of cljs+core.async issues due to macros trying to be too clever

borkdude21:01:09

wow, interesting

borkdude21:01:22

> also or and and in clojurescript both run analysis as part of the macro expansion, which can lead to macros be expanded multiple times yeah, I can see that leading to unexpected behavior...

hiredman21:01:12

there is another issue somewhere, the cljs compiler I think tries(tried?) to optimize literal list construction in a way that caused multiple evaluations as well

borkdude21:01:12

I've also seen some similar things with side effects in CLJS spec macros

phronmophobic19:01:33

seems reasonable, but at least for the jvm and js, wouldn't the JIT also do this kind of analysis?

borkdude19:01:05

probably, but why rely on that if you can cut it out earlier

borkdude19:01:28

I would have expected Compiler.java to have done a similar trick, but can't find it

phronmophobic19:01:33

trying to think of a case where evaluating a keyword may have some sort of side effect that a program might intentionally or unintentionally depend on, but I can't think of any

👍 3
caumond19:01:19

Interested in the answer

phronmophobic19:01:35

it's sort of contrived, but a deepwalking macro or code analysis tool that uses clojure.walk/macroexpand looking for keywords would be affected by this.

phronmophobic19:01:41

so if you had a macro that would automatically produce defs for all keywords in a macro-expanded expression, it would then fail to find any keyword that is a test in a cond expression

phronmophobic19:01:40

as well as any code analysis tool looking for keywords that macro expanded first

phronmophobic19:01:51

so it may introduce some edge cases

Alex Miller (Clojure team)19:01:11

any compile-time constant object could be treated as an else there

borkdude19:01:40

sure, keyword? could be (constant? ...)

Alex Miller (Clojure team)19:01:47

keyword certainly is common, but could also be string, true , 42 , etc

Alex Miller (Clojure team)19:01:35

I mean the presumption here is that it's faster but probably doesn't matter with jit

borkdude19:01:05

it would be faster in an interpreter, less code to process during evaluation. just wondering of what could go wrong

andy.fingerhut19:01:36

Rather than changing cond , is it feasible for the interpreter you have in mind to notice (if <constant-expression> ...) forms and optimize those 'in place'?

borkdude19:01:00

yeah, that's also possible

andy.fingerhut19:01:09

That would cover this case, and potentially others.

borkdude19:01:03

maybe if a macro that was called later on had a side effect

borkdude19:01:30

but maybe one should not rely on that

dpsutton19:01:22

small paper testing this hypothesis: > Small, deterministic programs reach a steady state of peak performance. https://arxiv.org/abs/1602.00602

dpsutton19:01:03

cool paper about the reliability of the JIT and how long (if ever) it might take to warm up

borkdude19:01:30

like:

(cond (odd? x) x :else 1 :foobar (macro-that-launches-rocket))

ghadi20:01:26

@borkdude @alexmiller a ticket+patch exists somewhere for if’s with constant tests

bronsa20:01:31

why would you want to do this on a specific macro instead of a simple general optimisation for if in the compiler

borkdude20:01:31

@bronsa yes, I also wondered about the compiler, I didn't see specific code for that