Fork me on GitHub
#beginners
<
2020-11-02
>
0xclj07:11:49

What is a recommended way to parse a multipart form data? In python, using the requests lib, I am able to get the text data in print(request.form.get('text')) . However, using ring, when I do (println request) it prints:

{:ssl-client-cert nil, :protocol HTTP/1.1, :remote-addr 0:0:0:0:0:0:0:1, :headers {accept */*, user-agent insomnia/2020.4.1, host localhost:3002, content-length 22290, content-type multipart/form-data}, :server-port 3002, :content-length 22290, :content-type multipart/form-data, :character-encoding nil, :uri /, :server-name localhost, :query-string nil, :body #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x16fc318a HttpInputOverHTTP@16fc318a[c=0,q=0,[0]=null,s=STREAM]], :scheme :http, :request-method :post}
The body is #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x16fc318a HttpInputOverHTTP@16fc318a[c=0,q=0,[0]=null,s=STREAM]] When doing (println (slurp (:body request))) , it prints the multipart form data with --xYzZY as the boundary: Sample:
--xYzZY
Content-Disposition: form-data; name="headers"
...

--xYzZY
Content-Disposition: form-data; name="text"

Great! Stuff

On Tue, Oct 27, 2020 at 11:05 AM John <[email protected]>
wrote:

> Deleting the test endpoint

--xYzZY
Content-Disposition: form-data; name="attachments"
0

0xclj08:11:27

Got it. Tried this:

(defn handler [request]
  (let [dt (-> request
            wrap-params
            wrap-multipart-params)]
    (println dt)
    (println (:multipart-params dt))))
dt is:

#function[ring.middleware.multipart-params/wrap-multipart-params/fn--771]
and the last println is just nil , what am I doing wrong?

phronmophobic08:11:13

if you’re using it in a handler (rather than as middleware), try multipart-params-request

Jim Newton14:11:30

I can never remember the name of the function similar to slit-with and partition-by which returns a ?vector? of two sequences, those that make the given predicate true and those that make it false.

Jim Newton14:11:43

no, not group-by, that's similar but not the same. The function I'm thinking about partitions a sequence into two sequences according to the boolean value of the predicate.

Jim Newton14:11:59

of course you can build the function with group-by, but there's already a function

andy.fingerhut14:11:00

with all values returning true for the predicate going into one of the two sequences, and all values returning false for the predicate into the other of the two sequences? Or are you thinking of a case where the first element that returns true for the predicate is the dividing point between the two returned sequences?

Jim Newton14:11:48

Here is how to implement the function I'm searching for. But I thought It already existed.

(defn partition-by-pred 
  "Apply the predicate to every element of the sequence and return a vector of two
  values, each of which is a vector, the left vector is the set of the items
  in the original sequence for which the predicate returned a Boolean true value,
  the right vector are the other values from the given sequence."  
  [pred items]
  (let [g (group-by (fn [i]
                      (boolean (pred i))) items)]
    [(g true) (g false)]))

Jim Newton14:11:27

Maybe I once thought that split-with did what I wanted.

andy.fingerhut14:11:59

You can call filter and remove with the same predicate on the same input sequence, but that will traverse the sequence twice, and call the predicate twice on each of the sequence's elements.

andy.fingerhut14:11:30

I think lots of people would use the map returned by group-by rather than make a vector with two sequences, because with the vector now you need to remember (or look up) which of the elements is the true sequence, and which the false, whereas with the map you can just look up the map with the key true or false.

Jim Newton14:11:48

this set of functions is very similar between Scala and Clojure. In Scala the function I'm looking for is called span, which i've always thought was a very unintuitive name, and I have to look it up every time.

vlaaad15:11:10

(juxt filter remove)?

3
vlaaad15:11:41

elegant, but iterates twice...

Jim Newton17:11:10

Perhaps reduce is your friend?

(defn partition-by-pred   
  [pred items]
(reduce (fn [[good bad] item]
            (if (pred item)
              [(conj good item) bad]
              [good             (conj bad item)]))
          [[] []]
          items))

Jim Newton17:11:16

@andy.fingerhut wrt group-by vs vector, the call site is something like the following, so In my case I prefer the vector being returned. I understand your concern, however, that the caller has to remember whether it is returned in good/bad order or bad/good order.

(let [[goods bads] (partition-by-pred ...)]
 ...)

andy.fingerhut18:11:10

There is destructuring of Clojure maps, too, which could be used with group-by like this. It certainly is not obvious what is going on until you become familiar with map destructuring, but then vector destructuring syntax is not exactly obvious what the semantics are the first time you see it, either:

user=> (let [{goods true, bads false} (group-by is-good? [1 2 3 4 5 6])]
         [goods bads])
[[2 4 6] [1 3 5]]

pithyless17:11:07

@U010VP3UY9X if you don't mind an additional dependency, cgrand/xforms has x/by-key which will split by predicate and you can transduce the outputs any way you see fit. It's probably not worth it for just one example, but I find the xforms library pretty useful to have around whenever there is data munging to be done. https://github.com/cgrand/xforms#simple-examples

pithyless17:11:36

(just realized this thread is 3 days old...)

Jim Newton10:11:52

@U05476190 it's great to add such comments to these threads, even days late. The discussion helps posterity, when people search the text later, they'll find useful information. I'm making the dubious supposition, of course, that this text will stay around forever.

andy.fingerhut14:11:59

The text will not be searchable on the Clojurians Slack free option for more than about the last 10,000 messages worth, because that is what the free Slack channel option provides. Most of these Slack messages are copied automatically to a Clojurians ZulipChat free instance, which does currently support searching of unlimited message history.

andy.fingerhut14:11:04

Because if you are thinking of what group-by does with a predicate function (one that always returns true or false), I can't think of any other built-in Clojure function that can do that.

Jim Newton14:11:56

I'd like to know whether there is more than 1 element of a sequence which satisfies a given predicate. Here's what I'm doing. Is it the correct way? It seems obfuscated.

(= 2 (count (take 2 (filter my-predicate my-sequence))))

andy.fingerhut15:11:58

I do not understand why you have (rest my-sequence) instead of simply my-sequence there, but if you made that replacement, then it is one correct way.

andy.fingerhut15:11:10

There is rarely only one "the" correct way.

Jim Newton15:11:36

yes. that's particular to my application. the first element is special. but for the purpose of the question, I should remove it.. Thanks.

andy.fingerhut15:11:37

Another correct way (assuming my-sequence is the sequence you are interested in), would be (>= (bounded-count 2 (filter my-predicate my-sequence)) 2)

Jim Newton15:11:37

bounded-count, excellent! That's much clearer.

Jim Newton15:11:24

I suppose I could also do some sort of check on the rest of filter, rather than (take 2 ...) but bounded-count is better.

Jim Newton15:11:49

@andy.fingerhut why not = 2 rather than >= ... 2 ?

andy.fingerhut15:11:05

I am probably off by one in typing this up, and haven't tested it, but something like (nil? (next (filter my-predicate my-sequence)) probably also works, but seems even harder to tell what the intent was than either of the previous 2

Jim Newton15:11:12

can bounded-count return something greater than its first argument?

andy.fingerhut15:11:30

According to bounded-count's doc string, it can

andy.fingerhut15:11:52

user=> (doc bounded-count)
-------------------------
clojure.core/bounded-count
([n coll])
  If coll is counted? returns its count, else will count at most the first n
  elements of coll using its seq

Jim Newton15:11:11

I see, there are cases when counting the whole sequence is faster than counting part of it. in particular when it has already been counted.

Gang Liang17:11:08

A noob question on numeric calculation. To my surprise, I found that Math/abs is much slower than a self-defined function abs as below. The function is called 100K times, and Math/abs is about 3 times slower than abs. I know Math/abs has some boxed-math operations, but I assume the same for my self-defined function. Can anyone explain why the speed difference here?

(defn abs [x]
 (if (pos? x) x (- x)))

noisesmith17:11:06

I would expect that function to box actually

noisesmith17:11:54

also, in a quick test, Math/abs is significantly faster than that function

noisesmith17:11:39

but in another test your function is faster

noisesmith17:11:08

(cmd)user=> (def input (doall (repeatedly 1000 #(- 50 (rand-int 100)))))
#'user/input
(cmd)user=> (time (doseq [i input] (if (pos? i) i (- 1))))
"Elapsed time: 0.644555 msecs"
nil
(cmd)user=> (time (doseq [i input] (Math/abs i)))
"Elapsed time: 9.021566 msecs"
nil

noisesmith17:11:03

@dpsutton won't that end up testing the inlining of the input that never changes?

dpsutton17:11:51

its a chain of macros so i'm hoping its repeatedly invoking it

dpsutton17:11:17

(c/quick-bench (prn "hi")) is quite noisy 🙂

dpsutton17:11:59

oh i misread you

phronmophobic17:11:41

part of the slow down is probably reflection:

> (time (doseq [i input] (Math/abs i)))
"Elapsed time: 7.279327 msecs"
nil
> (time (doseq [i input] (Math/abs ^long i)))
"Elapsed time: 1.207027 msecs"
if you know the type ahead of time, I would use type hints to speed things up, https://clojure.org/reference/java_interop#typehints

💯 3
noisesmith17:11:13

yeah, on my machine the hint speeds up the Math/abs version enough to be a bit faster than the function, as I'd expect

metal 3
dpsutton17:11:29

i threw a call to double around it not thinking

noisesmith17:11:47

the clojure version would at best be able to hint "Number"

Gang Liang18:11:02

Thanks for checking it out. I will remember to type-hint functions especially heavy computation ones. All in all, clojure is not a lazy language.

dpsutton18:11:15

What do you mean about laziness in this context?

Gang Liang18:11:27

@dpsutton I just mean I cannot be lazy... sorry for the confusion.

👍 3
WWeeks17:11:48

I like the culture and philosophy of Clojure development but I have not coded before. It is crazy to start programming with Clojure first?

noisesmith17:11:20

there are many ways Clojure is a messy first language, it exposes details most languages wouldn't about the host, because it's explicitly designed for "experts" who want the flexibility

🙂 6
noisesmith17:11:49

that said, I did learn clojure before I learned java, and that wasn't so bad (I did need to learn to read javadoc though)

👍 6
phronmophobic17:11:15

I would say it depends on your learning goals. I wouldn't say it's crazy, but there are other languages that try to make getting started in programming more approachable. what kinds of programs do you want to write?

6
👍 6
WWeeks18:11:40

I plan to manage linux cloud servers with Bash and Clojure

practicalli-johnny19:11:04

Babashka is a tool for writing scripts in Clojure that would otherwise be bash scripts. That seems to be a great starting place for you https://github.com/borkdude/babashka I encourage you to dive in and experiment at the REPL, also try websites such as http://www.4clojure.com/ and https://exercism.io/ Its useful to know a bit about the tooling available, but as the repl gives you fast feedback on each line of code it is a very useful experience. There are lots of tutorials that cover a wide range of things you can do with Clojure and I am working on a range of practically oriented books to help people learn https://practicalli.github.io/

👍 6
WWeeks22:11:11

This is an great reply. Thank you for the guidance

👍 6
Endre Bakken Stovner20:11:00

In my cljs app, I accept user input like (def a "[MAP-VALS]"). When I edn/read-string this, it becomes [MAP-VALS]. I want cljs to understand that it is the symbol MAP-VALS from com.rpl.specter. I can do this with the hash-map {'MAP-VALS specter/MAP-VALS} , but this feels inelegant. Is there any other way to do it?

noisesmith20:11:51

using a hash-map explicitly is the best way to go from arbitrary user input to values to use in code, in clojure you would have the option of using resolve (which doesn't work in cljs) but using an explicit mapping removes a security risk

Endre Bakken Stovner20:11:53

Okay, thanks 🙂 But is there an elegant way to get all the uppercase macros from specter into a map? I could grep the source-code, but it would be more fun to learn a clojure way 🙂

noisesmith20:11:54

also, pedantically, you don't want "the symbol MAP-VALS from com.rpl.specter", you want the value that MAP-VALS resolves to in that namespace

Endre Bakken Stovner20:11:03

I am happy to be corrected. How else would I learn? 🙂

noisesmith20:11:44

in clj you could start with: (-> (the-ns 'com.rpl.specter) ns-publics keys) to get all the symbols defined

👍 3
noisesmith20:11:29

(ins)user=> (->> (the-ns 'clojure.core) ns-publics keys (map name) (filter #(re-matches #".*byte.*" %)))
("aset-byte" "byte-array" "bytes?" "unchecked-byte" "byte" "bytes")

Endre Bakken Stovner20:11:31

Thanks. I realized that I need all of the names, not just the uppercase ones 🙂 So the task is straightforward :thumbsup:

noisesmith20:11:41

in that case you might be able to use the hash-map that ns-publics returns directly - as it's already a mapping from the symbol (name for lookup) to the var (which you can use @ or var-get to properly resolve to the value)

😎 3
👏 3
noisesmith20:11:50

with the right regex it should be easy to get all the upper-case names

Darrell22:11:16

I’m trying to test out next.jdbc, going through the REPL tutorial in the docs but when I try to execute a query I get this error:

Execution error (SQLException) at java.sql.DriverManager/getConnection (DriverManager.java:702).
No suitable driver found for jdbc:

Darrell22:11:31

Here’s my REPL:

clj
Clojure 1.10.1
user=> (require '[next.jdbc :as jdbc])
nil
user=> (def db {:dbtype "mysql" :dbname "clj_test" :user "root" :password "foo" :host "127.0.0.1"})
#'user/db
user=> (def ds (jdbc/get-datasource db))
#'user/ds
user=> (jdbc/execute! ds ["SELECT * FROM address"])

Darrell22:11:35

What am I missing?

seancorfield23:11:53

You didn't add a MySQL JDBC driver to your deps @darrell

seancorfield23:11:49

Per the Getting Started guide *In addition, you will need to add dependencies for the JDBC drivers you wish to use for whatever databases you are using.*

seancorfield23:11:34

The Example REPL Session shows a dependency on the H2 database driver (because that is easy for everyone to play with, without any setup), but if you want to do it on MySQL you'll have to add this to your deps: https://github.com/seancorfield/next-jdbc/blob/develop/deps.edn#L20 and you may well run into syntax differences between MySQL and H2.

Darrell23:11:48

Bah, you’re right. I had done that and then must have accidentally removed it. Thanks @seancorfield!

j23:11:47

Where is a good place/site/channel to find fellow clojurians to pair/mob/ensemble program with to learn more Clojure?

mathpunk23:11:39

You might check out the podcast Apropos. The panel often does pairing, so maybe their listeners would be into doing some pairing as well

j23:11:46

Thanks @U0E9KE222! Putting the link here for reference (since there's multiple Apropos podcasts): https://www.youtube.com/channel/UC1UxEQuBvfLJgWR5tk_XIXA

teodorlu14:11:41

If you're after the apropos audience, they have a discord they hang out in: https://clojureverse.org/t/apropos-clojure-panel-live-on-discord/6601

j00:11:43

Thanks @U3X7174KS! I joined it yesterday, but they don't seem too active