This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-04-15
Channels
- # announcements (3)
- # architecture (1)
- # babashka (52)
- # beginners (228)
- # calva (1)
- # chlorine-clover (31)
- # cider (9)
- # clj-kondo (16)
- # cljs-dev (25)
- # cljsrn (21)
- # clojure (116)
- # clojure-argentina (8)
- # clojure-europe (18)
- # clojure-france (17)
- # clojure-germany (1)
- # clojure-nl (5)
- # clojure-spec (49)
- # clojure-uk (63)
- # clojurescript (59)
- # community-development (14)
- # conjure (89)
- # core-matrix (1)
- # cursive (18)
- # data-science (1)
- # datomic (27)
- # exercism (4)
- # figwheel-main (5)
- # fulcro (38)
- # ghostwheel (8)
- # graalvm (5)
- # hoplon (2)
- # jobs-discuss (17)
- # juxt (1)
- # lambdaisland (5)
- # luminus (1)
- # lumo (9)
- # malli (7)
- # off-topic (32)
- # planck (24)
- # re-frame (14)
- # reagent (14)
- # reitit (14)
- # rum (23)
- # shadow-cljs (80)
- # spacemacs (2)
- # sql (6)
- # unrepl (1)
- # xtdb (2)
Well look at this. Native-image executables seem to respect -Xmx
!
$ bb -Xmx128m -XX:+PrintGC -XX:+PrintGCTimeStamps -XX:+VerboseGC -e '(while true (+ 1 2 3))'
[Heap policy parameters:
YoungGenerationSize: 13421770
MaximumHeapSize: 134217728
MinimumHeapSize: 26843540
AlignedChunkSize: 1048576
LargeArrayThreshold: 131072]
[[48 msec: GC: before epoch: 1 cause: CollectOnAllocation.Sometimes]
[60 msec: Incremental GC (CollectOnAllocation.Sometimes) 13259K->1019K, 0.0119090 secs]
[60 msec: GC: after epoch: 1 cause: CollectOnAllocation.Sometimes policy: by time: 50% in incremental collections type: incremental
collection time: 11909021 nanoSeconds]]
How can I download images (or other files) from web asynchronously directly to local file system, without storing in memory using babashka?
Will this script fulfill the requirements I mentioned?
@ahmed1hsn You can use babashka.curl with the :as :stream
option.
E.g.:
(io/copy
(:body (curl/get ""
{:as :stream}))
(io/file "icon.png"))
Is this feasible way to download 500 images at once? (by mapping through a list/vector of links)
java.lang.IllegalArgumentException: "No method in multimethod 'do-copy' for dispatch value: [clojure.lang.PersistentArrayMap http://java.io.File]" This error is given, when I try to run your code.
@ahmed1hsn did you try that exact code? what version of bb are you using?
Exact. babashka v0.0.78
Update. file downloads, but it's not opening in default image viewer in Ubuntu. Maybe it's corrupted.
hmm @UG1C3AD5Z could you try the above snippet as well maybe on linux?
(io/copy
(:body (curl/get ""
{:as :stream}))
(io/file "icon.png"))
I do have a unit test for this comparing the length of the original and downloaded oneThis script works perfect.
Oh, There was problem with args, It works now.
What are trade-offs between these two methods? (1. with babashka.curl 2. with io/input-stream and io/output-stream)
@ahmed1hsn With your method, I'm not sure if it handles redirects correctly
It seems both methods are directly saving data to file system, instead of buffering to memory?
input-stream would be useful if I want to do operations on the file, so in my case babashka.curl would be more appropriate.
Just a thought that popped into my head when my partner mentioned that she was learning powershell.
I kinda prefer a REPL and writing scripts over some custom shell DSL, but this project tries to replace shells with something Clojure-y: https://github.com/dundalek/closh
my talk was right after that one, very appropriate because I didn't have to explain why clojure is a better alternative to bash 🙂
before that talk there was @rahul080327 with his talk on bob (data driven docker / build tool using clojure). It was a nice trilogy
I wasn't fishing for compliments btw, just found it funny that there was an entire afternoon of ops/scripting things 🙂
Can't we accomplish everything Closh does with Babashka?
hmz, I can't get bb on Windows working anymore. I tried re-installing that redistributable again and again.
@U04V15CAJ Perhaps https://gitlab.com/tvaughan/windows-vm may be helpful
Nice, they now documented this for Windows: https://github.com/oracle/graal/issues/1852