This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-09-02
Channels
- # adventofcode (6)
- # announcements (6)
- # babashka (21)
- # babashka-sci-dev (18)
- # biff (6)
- # clara (4)
- # clj-commons (2)
- # clj-kondo (7)
- # cljdoc (4)
- # clojure (9)
- # clojure-berlin (8)
- # clojure-europe (23)
- # clojure-gamedev (3)
- # clojure-indonesia (1)
- # clojure-nl (1)
- # clojure-norway (10)
- # clojure-poland (1)
- # clojurescript (27)
- # community-development (1)
- # conjure (32)
- # etaoin (6)
- # events (20)
- # fulcro (5)
- # graalvm (1)
- # helix (19)
- # hyperfiddle (14)
- # introduce-yourself (2)
- # music (1)
- # nbb (24)
- # off-topic (37)
- # pathom (2)
- # polylith (14)
- # reagent (11)
- # releases (1)
- # remote-jobs (1)
- # reveal (22)
- # shadow-cljs (16)
- # sql (3)
- # squint (11)
- # test-check (2)
- # xtdb (36)
I have some sample JS code to read a S3 object that I would like to translate to Clojure but :man-shrugging::skin-tone-3: I'm too dumb so can anyone help me with my homework?
export const run = async () => {
try {
// Create a helper function to convert a ReadableStream to a string.
const streamToString = (stream) =>
new Promise((resolve, reject) => {
const chunks = [];
stream.on("data", (chunk) => chunks.push(chunk));
stream.on("error", reject);
stream.on("end", () => resolve(Buffer.concat(chunks).toString("utf8")));
});
// Get the object} from the Amazon S3 bucket. It is returned as a ReadableStream.
const data = await s3Client.send(new GetObjectCommand(bucketParams));
return data; // For unit tests.
// Convert the ReadableStream to a string.
const bodyContents = await streamToString(data.Body);
console.log(bodyContents);
return bodyContents;
} catch (err) {
console.log("Error", err);
}
};
run();
what have you tried? I'd write it like (.on stream "data" #(.push chunks %))
etc.
I think https://roman01la.github.io/javascript-to-clojurescript/ can also be useful to get up and running.
Error: AwaitExpression is not implemented
Couldn't compile JavaScript code into ClojureScript :(
Here’s one version that could work.
(ns s3
(:require ["@aws-sdk/client-s3" :refer [S3Client GetObjectCommand]]))
(def cli (S3Client. #js{:region "eu-west-1"}))
(-> (.send cli (GetObjectCommand. #js{:Bucket "my-bucket" :Key "some-file.txt"}))
(.then
(fn [resp]
(js/Promise.
(fn [resolve reject]
(let [stream (.-Body resp)
arr #js[]]
(.on stream "data" (fn [chunk] (.push arr chunk)))
(.on stream "error" reject)
(.on stream "end" (fn [] (resolve (-> arr
js/Buffer.concat
(.toString "utf-8"))))))))))
(.then println)
(.catch println))
And if you want to go cool you can also use the node streams api to pipe the response to some other thing that supports streams. Like here I’m piping the body to stdout.
(ns s3
(:require ["@aws-sdk/client-s3" :refer [S3Client GetObjectCommand]]
["stream/promises" :as sp]))
(def cli (S3Client. #js{:region "eu-west-1"}))
(-> (.send cli (GetObjectCommand. #js{:Bucket "nbb-adapter-test" :Key "concave.json"}))
(.then (fn [resp] (sp/pipeline (.-Body resp) js/process.stdout))))
Pipeline can also have transforming steps. I think I wrote something that here a while ago
Yep here it was https://clojurians.slack.com/archives/C029PTWD3HR/p1660679566907229?thread_ts=1660670072.874819&cid=C029PTWD3HR
I don’t know what you’re going to do with the s3 object but if you for instance want to send it forward you can do that with very low memory footprint using streams.
just processing some data but those are great tricks - thanks @U6N4HSMFW