This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-03-18
Channels
- # announcements (7)
- # babashka (4)
- # babashka-sci-dev (73)
- # beginners (101)
- # biff (4)
- # calva (33)
- # clerk (36)
- # clj-commons (23)
- # clj-kondo (3)
- # clojure (38)
- # clojure-europe (2)
- # clojurescript (29)
- # datalevin (15)
- # emacs (2)
- # fulcro (8)
- # gratitude (1)
- # hugsql (9)
- # hyperfiddle (43)
- # jobs-discuss (4)
- # lsp (47)
- # malli (7)
- # off-topic (14)
- # pathom (5)
- # practicalli (1)
- # releases (7)
- # shadow-cljs (4)
- # spacemacs (6)
- # sql (7)
- # tools-deps (7)
- # transit (8)
- # xtdb (6)
Hello, friends!
Is there good Clojure-book about working with Unix-like file system? Creating/moving files/directories etc... Thnks!
You can use Java's facilities for those operations, so the definitive source would be https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/file/Files.html and nearby pages
If you are looking for a clojure library that makes working with java.nio a bit easier: https://github.com/babashka/fs The interop can be pretty gnarly

Thank you!! Babashka is so important Clojure-host! 😍
Is there simple Clojure data-base? Or just Datomic?
Depends on your needs, but there are datascript, datahike, datalevin, xtdb, asami, relic, and probably a few others I forgot
Thnx!
https://www.juxt.pro/blog/introducing-crux/ is also very interesting
Many Clojure projects use relational databases like Postgresql, etc through https://github.com/seancorfield/next-jdbc There are also wrappers for MondoDB and Firebase (and many more data stores)
And also use SQL databases, in not-too-complicated ways, without wrappers because of the irony that Java's JDBC is much easier to use from Clojure than it is from Java.
What's the best way to count how many times is certain item contained in an array?
I have an array with various nil
s in it and need to know how many nils there are in total.
Of course I could write a reduce fn, but hoping for something more straightforward given how simple task it is.
(or (get (frequencies [1]) nil) 0)
to watch for no entries being 0
rather than nil
.
If you are worried about efficiency, I would guess that most approaches using frequencies
on the collection would be less efficient, since they must go to the extra trouble of counting how many times each other value occurs in the collection, not only the nil
values.
(defn count-nils [o]
(let [nil-count (volatile! 0)]
(->> o
(clojure.walk/postwalk (fn [form]
(when (nil? form)
(vswap! nil-count inc))
form)))
@nil-count))
(count-nils {:a nil
:b [nil]
:c [{:d nil}]})
count nils in nested datastructuresI think this is the most beginner friendly version as well as efficient?
(def count-value (fn [v val] (count (filter #(= % val) v))))
(count-value [1 2 nil 3 4 nil] nil)
Yes it is most efficient in above all answers, Yay 😂
user=> (time ((frequencies [1 2 nil 3 4 nil 5 nil 6 nil 7 8 9]) nil 0))
"Elapsed time: 3.319538 msecs"
4
user=> (time (count-value [1 2 nil 3 nil 1 4 5 6 nil 7 8 9 nil] nil))
"Elapsed time: 0.228846 msecs"
4
user=> (def count-nil (comp count (partial filter nil?)))
#'user/count-nil
user=> (time (count-nil [1 2 nil 3 nil 4 nil nil 5 6 nil 7 8 nil 9]))
"Elapsed time: 0.359615 msecs"
6
user=> (time (count-value [1 2 nil 3 nil 1 4 nil nil 5 6 nil 7 8 nil 9] nil))
"Elapsed time: 0.255385 msecs"
6
Alternatively, (filterv nil? coll)
, since vectors are counted?
:
(def xs1 (doall (repeatedly 1e6 #(rand-nth [nil (rand)]))))
;;=> #'user/xs1
(take 5 xs1)
;;=> (nil nil nil 0.49028712793772145 nil)
(time (count (filter nil? xs1)))
;;=> 499755
;;=> Elapsed time: 242.944069 msecs
(time (count (filterv nil? xs1)))
;;=> 499755
;;=> Elapsed time: 62.262708 msecs
I read the docs of filterv, it says it returns a vector whereas filter returns sequence, so filter is lazily evaluating the vector and filterv eagerly, right? So, filter is basically processing individual elements due to its laziness as it returns sequence which is not evaluated immediately, whereas filterv eagerly processes all elements immediately, thus, in this situation filterv is faster than filter? Did I get it right? Though I've always read about practice to use functions which does lazy evaluation using sequences over functions which uses eager evaluation, I'm learning clojure, so can you enlighten me your thought process on this to use filterv than filter? So I can learn the same.
The way i see lazy vs eager in clojure is use lazy ones when you are passing around the seq and there are multiple places and amount of consumption from it, i.e. caching behaviour of the seqs is helping me there. eager ones are better if you're gonna be consuming/reducing the full thing then and there. the overhead of the lazy seq and caching isn't worth it, like the case here with counting.
In this situation specifically, filterv
is faster because vectors implement clojure.lang.Counted
. Collections that implement clojure.lang.Counted
keep track of the number of items they have. Calling count
on a vector returns the value of the vector’s internal counter. Conversely, calling count
on a lazy seq forces Clojure to iterate the entire seq to get the count.
Thanks for these explanation, @U7ERLH6JX about using lazy functions to use caching and use eager functions where consuming things instantly like here with the case of counting, and @U4ZDX466T about clojure.lang.Counted
I didn't knew this, and now I feel I understood the performance difference due to these two factors in filter and filterv, as calling count
on filtered vector is instant whereas on lazy sequence it is counting entire sequence again.
Hi all. I receive a base64 encoded image using luminus and wrap-multipart middleware. I get out what looks like binary data. Can't figure out how to get it uploaded to an s3 bucket using amazonica. I'm stumped. Any ideas or repos that do a simple image upload from form posted multipart upload? Thank you
;; put object with server side encryption
(put-object :bucket-name "two-peas"
:key "foo"
:metadata {:server-side-encryption "AES256"}
:file upload-file)
looking at the amazonica docs, you can use a file with put-object. turn your bytes into a file. you can do this via http://clojure.java.io https://clojure.github.io/clojure/clojure.java.io-api.html#clojure.java.io/copy copy the bytes into a temp file, then give the file to the put-object
function(put-object :bucket-name bucket1
:key "stream"
:input-stream input-stream
:metadata {:content-length (count some-bytes)}
:return-values "ALL_OLD")
also accepts input-streams, so that should probably be what you use instead of file@U0LAJQLQ1 thank you, I feel I've tried inputstream, I'm going to try with the named args
Super stupid question here, but where i'm so confused is how this response is being coerced. When I run this:
(type (-> request
:params
:image))
it says ;; => java.lang.String
so when I try to use copy, I get this error
No method in multimethod 'do-copy' for dispatch value: [java.lang.String java.lang.String]This is the first bit of the file coming is: ;; => "u�Zj�e�ƭ�����^��-��m�눕�pС�����Ԕ�E �� ��+��`����������5$t
do-copy is defined in http://clojure.java.io you can extend it to handle your case, but i think it's just easier to get the bytes from your string
however, i wouldn't be surprised if string would be interpreted as a file path in the context of http://clojure.java.io
@U0LAJQLQ1 - Thank you, I really appreciate this. I got this far:
(io/copy (.getBytes (-> (stash/peek-stash)
:params
:images
last)) (File. "temp2.png"))
And I see the file on the filesystem, and it has bytes, but is not a valid image.I've tried with both getBytes and without, I can actually write both ways, but both can't be opened
Is something wrong here to begin with or is this expected to come in as a string? I would have imagined that reitit and the ring middleware would have coerced this into a file
if you want the browser to render it, you need to give it more information at the start of the string
i don't know if you'll be able to easily render data-uri's (what you build with the base64 data) outside of a browser
you are doing something a bit advanced, you may want to avoid dealing with base64 data
Is it possible to look at the raw request sent from the client? I can't see exactly what they are sending. Reitit gives me a body of:
[io.undertow.io.UndertowInputStream
713490225
"io.undertow.io.UndertowInputStream@2a86ff31"]
How could I get at the raw request before anything ahppens?(defn chop-header [s]
(nth (first (re-seq #"(data:image/.*;base64,)(.*)" s)) 2))
(defn b64-ext [s]
(if-let [ext (second (first (re-seq #"data:image/(.*);base64.*" s)))]
(if (in? ["png" "jpeg" "webp"] ext)
ext
(throw (Exception. (str "Unsupported extension found for image " ext))))
(throw (Exception. (str "No extension found for image " s)))))
(defn decode-str [s]
(b64-codec/decode (.getBytes s)))
(defn write-img! [file b64]
(
(decode-str (chop-header b64))
(java.io.File. file)))
here is some functions i use to deal with data-urisThe thing is it seems like it's being decoded, because supposedly this is what the client is sending as multipart: "iVBORw0KGgoAAAANSUhEUgAADCoAAAK4CAYAAAAI6P39AAAAAXNSR0IArs4c6QAAAJZlWElmTU0AKgAAAAgABAEaAAUAAAABAAAAPgEbAAUAAAABAAAARgEoAAMAAAABAAIAAIdpAAQAAAABAAAATgAAAAAAAACQAAAAAQAAAJAAAAABAASShgAHAAAAEgAAAISgAQADAAAAAQABAACgAgAEAAAAAQAADCqgAwAEAAAAAQAAArgAAAAAQVNDSUkAAABTY3JlZW This is base64. On the server, I get:
"�PNG\r\n\n\rIHDR\f*�\b\b���sRGB����eXIfMM*\b>F(�iN�������\f*��ASCIIScreenshotf��\tpHYs%%IR$��iTXtXML:com.adobe.xmp<x:xmpmeta xmlns:x=\"adobe:ns:meta/\" x:xmptk=\"XMP Core 6.0.0\">\n <rdf:RDF xmlns:rdf=\"\">\n <rdf:Description rdf:about=\"\"\n ..."
you'll get bytes if the browser is giving you a file, though. if the browser is sending base64, then multipart isn't going to see that as a file, and may avoid giving bytes, probably because strings and bytes are not so different
this is the file:
�PNG
IHDR*����sRGB����eXIfMM*>F(�iN�������*��ASCIIScreenshotf�� pHYs%%IR$��iTXtXML:com.adobe.xmp<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 6.0.0">
<rdf:RDF xmlns:rdf="">
<rdf:Description rdf:about=""
xmlns:exif=" ">
<exif:PixelYDimension>696</exif:PixelYDimension>
it's a little difficult debugging this sort of thing. i had to do this for the multipart bugs i found... was not easy
personally, i haven't had a problem saving base64 files, and i use a lot of those files in my current app
if you really want this to work, it may be worth while looking at my yada multipart PRs
my multipart doesn't decode for me (as you can see in my function for writing out a base64 file)
https://github.com/boxxxie/yada/blob/master/src/yada/multipart.clj this is the one i use, i'm not sure if it's changed from the yada repo
> I got this far: > >
(io/copy (.getBytes (-> (stash/peek-stash)
>
getBytes
without a param assumes system default encoding, which is UTF16.
Assuming the string is currently encoded in Base64 as you say, you can use a Base64 decoder to get a byte array.
(.decode (java.util.Base64/getDecoder) base64-string)
Otherwise, your net request was probably sent in UTF8, rather than UTF16. So you need to specify that for getBytes
.
(.getBytes utf8-string "UTF-8")
@U90R0EPHA thank you. Just trying a few things. Still no luck. I was able to do the following though. I was able to move to json instead of multipart, and I can grab the encoded base64, and render that directly in the browser, so I'm sure the base64 part is fine. But when I try to decode and save it to disk, that new file still doesn't render
I used your decode snippet and copy that out:
(io/copy (-> (stash/peek-stash)
:body-params
:images
last
decode
) (File. "sample-test5.jpeg"))
What I notice is the sample.jpeg that I'm using as a source is 3 kb and the output is 6kb when I resave it outI got it! @U90R0EPHA