This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-06-15
Channels
- # announcements (11)
- # aws (5)
- # babashka (69)
- # beginners (138)
- # calva (5)
- # cider (11)
- # cljsrn (3)
- # clojure (86)
- # clojure-europe (8)
- # clojure-italy (2)
- # clojure-nl (5)
- # clojure-norway (13)
- # clojure-uk (21)
- # clojurescript (13)
- # conjure (58)
- # core-typed (2)
- # data-science (2)
- # datomic (5)
- # degree9 (4)
- # emotion-cljs (4)
- # figwheel-main (8)
- # fulcro (29)
- # graalvm (70)
- # interop (2)
- # joker (1)
- # lein-figwheel (2)
- # leiningen (13)
- # liberator (4)
- # off-topic (21)
- # pedestal (2)
- # re-frame (10)
- # shadow-cljs (42)
- # spacemacs (29)
- # spire (1)
- # tools-deps (11)
- # vim (26)
- # xtdb (5)
I'm getting a bit tripped up working with a vector, applying a filter, and suddenly having a lazy seq. It makes me feel like I am misunderstanding how things should be used. Can I filter a vector without producing a seq? Can I turn my seq back into a vector?
on a more philosophical level, it can help to think of data belonging to functions, instead of functions being helpers for data - most clojure functions produce a specific data type, even if they are flexible about the type of their input
@radicalmatt Why do you need the vector? Could you do most stuff with sequences, in terms of map/filter/etc? Vectors are useful in some situations where you need specific performance guarantees, or you specifically need indexed access -- but trying to maintain vectorness all the time can be worse for performance, depending on what you're doing.
For example, it's worth noting that mapv
and filterv
are eager (based on reduce
) whereas map
and filter
are lazy.
An empty list has an easy literal too ()
🙂
Little confused on something, from what I understand for
loops return lazy seqs and only run when it's results are required to run some calculations but isn't that what I'm doing by passing each user to send-to-api
? I know doseq
is more idiomatic approach, just curious about what's going on internally.
(defn send-to-api [user]
(println "Sending to API:" user))
(let [users [{:name "Alysia"} {:name "Bob"}]]
(for [user users]
(send-to-api user))
(println "DONE")) ;; => prints just DONE
;; using doseq achieves the desired results
;; prints each user followed the the "DONE" statement.
(let [users [{:name "Alysia"} {:name "Bob"}]]
(doseq [user users]
(send-to-api user))
(println "DONE"))
so your for is "given this list of users, create a new sequence by passing each one to send-to-api"
Oh that makes sense.
Got it. Thank you.
hi 🙂 are the following equivalent, or is there some sneaky way in which they are not?
(empty? x)
and
(nil? (seq x))
@fappy In the REPL, you can see how empty?
is defined:
user=> (source empty?)
(defn empty?
"Returns true if coll has no items - same as (not (seq coll)).
Please use the idiom (seq x) rather than (not (empty? x))"
{:added "1.0"
:static true}
[coll] (not (seq coll)))
nil
user=>
And seq
either produces nil
or a non-empty sequence. So nil?
of nil
is true
and nil?
of any non-empty sequence will be false
.
And in the definition of empty?
, not
of nil
is true
, and not
of any non-empty sequence will be false
.
The for
expression is not the last one of the let
body, so its return value is never consumed by anything.
Hello Guys, I am trying to use reader conditionals for reading environment variables:
(def ^:dynamic *env*
(into {} #?(:clj (System/getenv))))
I can’t seem to find a way to do this for cljs block.is your cljs destined for the browser? nodejs? something else? multiple targets?
I am trying to build it for nodejs
looks like you can retrieve environment variables with https://nodejs.org/api/process.html#process_process_env in nodejs
potentially, something like:
(def ^:dynamic *env*
#?(:clj (into {} (System/getenv))
:cljs (js->clj (.-env js/process))))
Hi guys and girls, Does the latest `Clojurescript` make it easier to use `Storybook'??` How has anyone used `storybook` `clojurescript?`
They are not the same thing, recur
is for calling a function recursively, reduced
is to provide a reduced value in reduce
(e.g. stop the reducing)
both allow the same thing: doing something to each item in a collection, with the possibility returning early
Then I misunderstood your question
my personal rule is never use loop
to consume a single collection item by item in order - you can easily replace it with reduce
thanks to reduced for early exit
(that plus using a data structure as an accumulator when you need multiple values to update per step)
hey, can anyone share the link to Rich hickey's talk about semantic versioning, not changing meaning of apis, about software accretion etc..
There are transcripts of most of Rich Hickey's talks available in this Git repo, too, if you prefer reading, or want something text searchable later: https://github.com/matthiasn/talk-transcripts
Dumb question, if I use a library in my lein project and declare using clojure 1.10, but the source code of the library declares a dependency on clojure 1.3, how many versions of clojure exist in my JVM?
only one, version is determined based on dependency resolution algorithm. With leiningen you can check which lib resolves to which version - lein deps :tree
. for more options - lein deps -h
thank you
I think Maven (and thus Lein) uses the version first encountered while traversing the dependency tree. Deps CLI improves this resolution logic by always picking the latest if there is a clash. If you are not tied to Lein, you may consider moving to deps.edn based project management. One downside is, Lein has more features and more readymade stuff that you may have to do yourself in Deps CLI.
Could you help me the scheme lambda is “equivalent” with Clojure anonymous functions? (if somebody is here with scheme language knowledge)
I do not know if Scheme has anything like this, but Clojure's fn
can optionally be followed by a symbol, which gives a local name to the otherwise anonymous function, which can be useful for identifying it if it ever appears in a stack trace of an exception.
Clojure also has syntax like #(inc (* 2 %))
for declaring anonymous functions with a few less characters. That example is equivalent to (fn [x] (inc (* 2 x)))
@U0CMVHBL2 thank you very much the detailed answer!
How do I go about reading a large file into clojure for a batch process.
There are only about a dozen different ways, depending upon things like whether you want the whole thing in memory at once, what format the file is (text with no other structure, CSV, binary, etc.)
Clojure strings are Java strings, which require about 40 bytes of memory plus 1 byte per character, if they are all ASCII, or 2 bytes per character if there are non-ASCII characters in a line.
Optimally I'd like to pull in batches of let's say 10,000 lines at a time and push that to another part of the process asynchronously
(with-open [reader (io/reader "/home/wskinner/tmp/DX_****_20200422_214932.txt")]
(count (line-seq reader)))
give me OOM exception for a 1.4 million line file.You may need to specify a larger max heap when you start the JVM process on which Clojure runs, e.g. -Xmx2g
for 2 Gbytes
I'd like to keep this operating at or under 2 GB. It's going to eventually be running in resource constrained cluster.
If the file is completely ASCII characters, then 1 byte per character plus 40 bytes per line should be enough for that data
Have you explicitly specified max heap when you got OOM, and if so, what max heap did you try?
I haven't let me do that now
https://github.com/seancorfield/depstar/blob/master/src/hf/depstar/uberjar.clj#L135-L144
have a consume-file
function that takes a function f
and a chunk size. Call line-seq and then partition and call f
on each partition
Oh nice, thank you
@skinner89 just checking, but are you running under a debugger like intelliJ/cursive?
Thanks @U050ECB92 with reduce it ran in 760 msecs w/o the OOM
I'm running this in a cursive/line repl in intellij yes
not in debug mode though
The next part of this process is going to be pushing the chunks to a stage that inserts the lines into a sql database. As I understand channels will allow me to do that asynchronously while respecting backpressure so the process reading from the file will wait. Is that right?
This has probably been solved by some clojure ETL libraries 🙂
Hi, how do you convert a single character to a string in Clojure? I want \H
to become "H"
or basically (\H)
Follow up I just found https://www.grammarly.com/blog/engineering/building-etl-pipelines-with-clojure-and-transducers/ . Great read.
The article introduces this function for creating something reducible from a bufferedReader:
(defn lines-reducible [^BufferedReader rdr]
(reify clojure.lang.IReduceInit
(reduce [this f init]
(try
(loop [state init]
(if (reduced? state)
state
(if-let [line (.readLine rdr)]
(recur (f state line))
state)))
(finally (.close rdr))))))
Has anything more idiomatic been added to the language?in theory that could become part of the definition of line-seq
as a performance optimization
I would add something, not levy it on line-seq
but adding something would be good (didn't we already have this discussion like a year ago?)
there is some work around a function called iteration
https://clojure.atlassian.net/browse/CLJ-2555 @skinner89
is it possible to check if an object implements a method -foo
from protocol Foo
?
satisfies?
-- if it is declared to implement the protocol. You might also need to check the metadata on an object, if the protocol is declared to be extensible via metadata.
yeah, but I’m interested in particular method
neither of both) just curious is it possible or not
because it is possible to partially implement a Protocol
why is that?
because polymorphism is about not caring about the target, and having the dispatch automatically do the right thing
so what you recommend? in case if I need to call x-method on something that is out from my control?
(defprotocol Foo
(-foo [this])
(-bar [this]))
(-bar
(reify
Foo
(-foo [_] "!")))
anyway, this is what I’m interested in ^
-bar
is not implemented and calling to it throws
and because it is possible to partially implement a protocol I wonder do clojure have a way to check a methodfrom a public api perspective, no
if you want to get dirty, protocols are just maps and you could introspect those maps to answer this question
I already tried that, no luck ( how could i introspect an object that reify is returning?
you can't
Ok, thanks anyway)
or at least partially answer it
I've got a noob question. I'm trying to process a string which contains at
and it isn't being handled properly. How can I escape the at
so the string works properly?
(def song "Nothing at All")
(println song)
Output: Nothing
Desired Output: Nothing at All
have you actually run this code and had this problem? or is this just a reduced example you haven't run and seen the problem on?
this example is such a reduced case, that if you do see the problem when you run it, my guess is there is some kind of error/bug in whatever codepad like thing you are using. but if you don't actually see the error when running this reduced case, my guess is you haven't correctly reduced it. something else in all the other code that are running is causing the issue
Yeah, this was run and came out exactly like this. For that reason, I've concluded the issue was with the environment not my code.
it should work that way already
maybe you can share more of your actual code
My actual code isn't much more involved honestly, I'm just trying to test the validity of the input, and I ran into this issue: This outputs: [Bye Bye Love Nothing
(defn print2DStringArray [sArr2D]
(print sArr2D)
(println)
(println))
(def songs-2 [
"Bye Bye Love"
"Nothing at All"
"Money for Nothing"
"Love Me Do"
"Do You Feel Like We Do"
"Bye Bye Bye"
"Do You Believe in Magic"
"Bye Bye Baby"
"Baby Ride Easy"
"Easy Money"
"All Right Now"
])
(printStringArray songs-2)
i copied that and it should be print2DStringArray
make sure you aren't calling an older function that is still in your repl
but i get this when i call the 2D version:
user=> (print2DStringArray songs-2)
[Bye Bye Love Nothing at All Money for Nothing Love Me Do Do You Feel Like We Do Bye Bye Bye Do You Believe in Magic Bye Bye Baby Baby Ride Easy Easy Money All Right Now]
Sorry, I copied the wrong piece:
(defn printStringArray [sArr]
(print sArr)
(println)
(println))
I think it might be a bug in my environment, I just tried another one and it worked fine. Sorry to both you guys and thanks for your help.
oh. i was wondering if its possible for the jvm to exit before its flushed everything to the output.
I don't think it's that because it correctly prints other things later in the code (this is the middle of a larger section).
print
doesn't flush
println
does
flush
is an explicit function you can call too if needed
Hi, people. I want to delay the execution of a function (it is actually a simple TTL). I thought about using future
, but then I'd have to use something like (Thread/sleep some-time)
and, as far as I know, Thread/sleep
still occupies some resources while "waiting". What would be the best way to accomplish it? Thank you.
Delay would require me to deref it explicitly, I think. I would like to schedule a cleanup function to execute after a given amount of time. The function calling this "scheduler" sets the amount of time.
The "scheduler" would look like
(defn scheduler
[ttl]
(future
(Thread/sleep ttl)
(cleanup-fn)))
This is the behavior I want to achieve, but without the downside of Thread/sleep
I believe that ScheduledthreadPoolExecutor
will do the job. Thank you for your help!
yeah, delay is not a timed delay, just a form of laziness
Cool. This is exactly what I was looking for. Thank you.