You can use Java's facilities for those operations, so the definitive source would be https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/file/Files.html and nearby pages

If you are looking for a clojure library that makes working with java.nio a bit easier: https://github.com/babashka/fs The interop can be pretty gnarly

Thank you!! Babashka is so important Clojure-host! 😍

@U03D1PUDM6W :) The library works in both babashka and in JVM Clojure

Depends on your needs, but there are datascript, datahike, datalevin, xtdb, asami, relic, and probably a few others I forgot

Thnx!

https://www.juxt.pro/blog/introducing-crux/ is also very interesting

Many Clojure projects use relational databases like Postgresql, etc through https://github.com/seancorfield/next-jdbc There are also wrappers for MondoDB and Firebase (and many more data stores)

And also use SQL databases, in not-too-complicated ways, without wrappers because of the irony that Java's JDBC is much easier to use from Clojure than it is from Java.

(get (frequencies coll) nil)

alternatively,

(comp count (partial filter nil?))

(or (get (frequencies [1]) nil) 0) to watch for no entries being 0 rather than nil.

((frequencies coll) nil 0) should have the same effect

If you are worried about efficiency, I would guess that most approaches using frequencies on the collection would be less efficient, since they must go to the extra trouble of counting how many times each other value occurs in the collection, not only the nil values.

(comp count (partial filter nil?))

(defn count-nils [o]
  (let [nil-count (volatile! 0)]
    (->> o
         (clojure.walk/postwalk (fn [form]
                                  (when (nil? form)
                                    (vswap! nil-count inc))
                                  form)))
    @nil-count))

(count-nils {:a nil
             :b [nil]
             :c [{:d nil}]})

I think this is the most beginner friendly version as well as efficient? (def count-value (fn [v val] (count (filter #(= % val) v)))) (count-value [1 2 nil 3 4 nil] nil) Yes it is most efficient in above all answers, Yay 😂 user=> (time ((frequencies [1 2 nil 3 4 nil 5 nil 6 nil 7 8 9]) nil 0)) "Elapsed time: 3.319538 msecs" 4 user=> (time (count-value [1 2 nil 3 nil 1 4 5 6 nil 7 8 9 nil] nil)) "Elapsed time: 0.228846 msecs" 4 user=> (def count-nil (comp count (partial filter nil?))) #'user/count-nil user=> (time (count-nil [1 2 nil 3 nil 4 nil nil 5 6 nil 7 8 nil 9])) "Elapsed time: 0.359615 msecs" 6 user=> (time (count-value [1 2 nil 3 nil 1 4 nil nil 5 6 nil 7 8 nil 9] nil)) "Elapsed time: 0.255385 msecs" 6

Alternatively, (filterv nil? coll), since vectors are counted?:

(def xs1 (doall (repeatedly 1e6 #(rand-nth [nil (rand)]))))
;;=> #'user/xs1

(take 5 xs1)
;;=> (nil nil nil 0.49028712793772145 nil)

(time (count (filter nil? xs1)))
;;=> 499755
;;=> Elapsed time: 242.944069 msecs

(time (count (filterv nil? xs1)))
;;=> 499755
;;=> Elapsed time: 62.262708 msecs

I read the docs of filterv, it says it returns a vector whereas filter returns sequence, so filter is lazily evaluating the vector and filterv eagerly, right? So, filter is basically processing individual elements due to its laziness as it returns sequence which is not evaluated immediately, whereas filterv eagerly processes all elements immediately, thus, in this situation filterv is faster than filter? Did I get it right? Though I've always read about practice to use functions which does lazy evaluation using sequences over functions which uses eager evaluation, I'm learning clojure, so can you enlighten me your thought process on this to use filterv than filter? So I can learn the same.

The way i see lazy vs eager in clojure is use lazy ones when you are passing around the seq and there are multiple places and amount of consumption from it, i.e. caching behaviour of the seqs is helping me there. eager ones are better if you're gonna be consuming/reducing the full thing then and there. the overhead of the lazy seq and caching isn't worth it, like the case here with counting.

seq = cached values for the long run, eager vecs = fast short sprints

In this situation specifically, filterv is faster because vectors implement clojure.lang.Counted. Collections that implement clojure.lang.Counted keep track of the number of items they have. Calling count on a vector returns the value of the vector’s internal counter. Conversely, calling count on a lazy seq forces Clojure to iterate the entire seq to get the count.

Thanks for these explanation, @U7ERLH6JX about using lazy functions to use caching and use eager functions where consuming things instantly like here with the case of counting, and @U4ZDX466T about clojure.lang.Counted I didn't knew this, and now I feel I understood the performance difference due to these two factors in filter and filterv, as calling count on filtered vector is instant whereas on lazy sequence it is counting entire sequence again.

;; put object with server side encryption
(put-object :bucket-name "two-peas"
            :key "foo"
            :metadata {:server-side-encryption "AES256"}
            :file upload-file)

(put-object :bucket-name bucket1
            :key "stream"
            :input-stream input-stream
            :metadata {:content-length (count some-bytes)}
            :return-values "ALL_OLD")

@U0LAJQLQ1 thank you, I feel I've tried inputstream, I'm going to try with the named args

Super stupid question here, but where i'm so confused is how this response is being coerced. When I run this:

(type (-> request
      :params
      :image))

get the bytes from the string

This is the first bit of the file coming is: ;; => "u�Zj�e�ƭ�����^��-��m�눕�pС�����Ԕ�E �� ��+��`����������5$t

(.bytes my-str) or something like that

multipart should be able to give you back bytes, as that's what it works with

do-copy is defined in http://clojure.java.io you can extend it to handle your case, but i think it's just easier to get the bytes from your string

and it's better if you use the bytes directly from multipart

however, i wouldn't be surprised if string would be interpreted as a file path in the context of http://clojure.java.io

@U0LAJQLQ1 - Thank you, I really appreciate this. I got this far:

(io/copy (.getBytes (-> (stash/peek-stash)
             :params
             :images
             last)) (File. "temp2.png"))

I've tried with both getBytes and without, I can actually write both ways, but both can't be opened

"�PNG\r\n\n���\rIHDR��\f*���\b���\b������sRGB����..."

Is something wrong here to begin with or is this expected to come in as a string? I would have imagined that reitit and the ring middleware would have coerced this into a file

base64 is not an image

if you want the browser to render it, you need to give it more information at the start of the string

i don't know if you'll be able to easily render data-uri's (what you build with the base64 data) outside of a browser

you are doing something a bit advanced, you may want to avoid dealing with base64 data

Is it possible to look at the raw request sent from the client? I can't see exactly what they are sending. Reitit gives me a body of:

[io.undertow.io.UndertowInputStream
 713490225 
 "io.undertow.io.UndertowInputStream@2a86ff31"]

(defn chop-header [s]
  (nth (first (re-seq #"(data:image/.*;base64,)(.*)" s)) 2))

(defn b64-ext [s]
  (if-let [ext (second (first (re-seq #"data:image/(.*);base64.*" s)))]
    (if (in? ["png" "jpeg" "webp"] ext)
      ext
      (throw (Exception. (str "Unsupported extension found for image " ext))))
    (throw (Exception. (str "No extension found for image " s)))))

(defn decode-str [s]
  (b64-codec/decode (.getBytes s)))

(defn write-img! [file b64]
  (
    (decode-str (chop-header b64))
    (java.io.File. file)))

[clojure.data.codec.base64 :as b64-codec] is a dep

try using that inputstream

maybe it'll work with the s3 functions

i don't use reitit, i use yada, and had to patch some of the multipart file

so my multipart gives me bytes, but maybe reitit doesn't do that

The thing is it seems like it's being decoded, because supposedly this is what the client is sending as multipart: "iVBORw0KGgoAAAANSUhEUgAADCoAAAK4CAYAAAAI6P39AAAAAXNSR0IArs4c6QAAAJZlWElmTU0AKgAAAAgABAEaAAUAAAABAAAAPgEbAAUAAAABAAAARgEoAAMAAAABAAIAAIdpAAQAAAABAAAATgAAAAAAAACQAAAAAQAAAJAAAAABAASShgAHAAAAEgAAAISgAQADAAAAAQABAACgAgAEAAAAAQAADCqgAwAEAAAAAQAAArgAAAAAQVNDSUkAAABTY3JlZW This is base64. On the server, I get:

"�PNG\r\n\n\rIHDR\f*�\b\b���sRGB����eXIfMM*\b>F(�iN�������\f*��ASCIIScreenshotf��\tpHYs%%IR$��iTXtXML:com.adobe.xmp<x:xmpmeta xmlns:x=\"adobe:ns:meta/\" x:xmptk=\"XMP Core 6.0.0\">\n   <rdf:RDF xmlns:rdf=\"\">\n      <rdf:Description rdf:about=\"\"\n            ..."

you'll get bytes if the browser is giving you a file, though. if the browser is sending base64, then multipart isn't going to see that as a file, and may avoid giving bytes, probably because strings and bytes are not so different

yeah, your thing looks decoded

what does your file look like? does it have "s in it?

Not sure if that's just in the repl output or in the file

;; => "�PNG\r\n\n\rIHDR..."

well, file should look like binary stuff, not string stuff

the quote is malformed

no the file has no "

save the bytes to a file, don't save the string to a file

sorry I wasn't inspecting the file

oh

this is the file:

�PNG


IHDR*����sRGB����eXIfMM*>F(�iN�������*��ASCIIScreenshotf��	pHYs%%IR$��iTXtXML:com.adobe.xmp<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 6.0.0">
   <rdf:RDF xmlns:rdf="">
      <rdf:Description rdf:about=""
            xmlns:exif="">
         <exif:PixelYDimension>696</exif:PixelYDimension>

but the file doesn't render?

no

you give me an idea to compare the two now

the one that can open

well they both start with: �PNG

no idea why I can't render the image

you could compare each byte of both files

it's a little difficult debugging this sort of thing. i had to do this for the multipart bugs i found... was not easy

but, if your frontend could send you something non-base64, you may be better

personally, i haven't had a problem saving base64 files, and i use a lot of those files in my current app

Thank you, I really appreciate this, I am going to try and see what else can be done

Yeah, it really makes me want to solve this

I feel like I'm missing something basic here

if you really want this to work, it may be worth while looking at my yada multipart PRs

cus they work 🙂

would love to

my multipart doesn't decode for me (as you can see in my function for writing out a base64 file)

could you share a link be any chance?

https://github.com/juxt/yada/blob/master/ext/multipart/src/yada/multipart.clj

Thank you!

https://github.com/boxxxie/yada/blob/master/src/yada/multipart.clj this is the one i use, i'm not sure if it's changed from the yada repo

Excellent

> I got this far: > >

(io/copy (.getBytes (-> (stash/peek-stash)
> 

(.decode (java.util.Base64/getDecoder) base64-string)

(.getBytes utf8-string "UTF-8")

trying it right now

@U90R0EPHA thank you. Just trying a few things. Still no luck. I was able to do the following though. I was able to move to json instead of multipart, and I can grab the encoded base64, and render that directly in the browser, so I'm sure the base64 part is fine. But when I try to decode and save it to disk, that new file still doesn't render

I used your decode snippet and copy that out:

(io/copy (-> (stash/peek-stash)
               :body-params
               :images
               last
               decode
               ) (File. "sample-test5.jpeg"))

I got it! @U90R0EPHA

base64 increases the file size by a lot

but, double seems unreasonable

In the chain is was putting back in a string :rolling_on_the_floor_laughing:

Thank you guys so much!

no problem

I'm here if you need anything, no matter how far, I will be there for you all