This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-03-19
Channels
- # announcements (6)
- # aws (10)
- # beginners (73)
- # bristol-clojurians (2)
- # calva (9)
- # cider (25)
- # clj-kondo (7)
- # clojure (160)
- # clojure-dev (2)
- # clojure-europe (63)
- # clojure-italy (7)
- # clojure-nl (10)
- # clojure-uk (76)
- # clojuredesign-podcast (6)
- # clojurescript (63)
- # cursive (6)
- # data-science (3)
- # datomic (26)
- # duct (59)
- # emacs (1)
- # fulcro (12)
- # graalvm (17)
- # hoplon (23)
- # jobs-discuss (2)
- # kaocha (6)
- # meander (7)
- # off-topic (3)
- # pathom (2)
- # rdf (68)
- # re-frame (12)
- # reagent (20)
- # reitit (5)
- # ring (3)
- # ring-swagger (1)
- # shadow-cljs (14)
- # spacemacs (10)
- # sql (3)
- # tools-deps (30)
- # yada (9)
It doesn't seem to actually exclude anything tho'...
user=> (ns wibble (:refer-clojure :exclude [this-thing-does-not-exist]))
nil
wibble=>
So it refers in all of clojure.core
except the named symbols... and if those symbols don't exist, it gives no errors @U11BV7MTK
interesting. there's a defined all
in this file. it looks like tim just remembered something from core incorrectly. maybe confused every? with all or something like that
ah and there's an all
in core.logic. i was checking the docstrings of ns and others to see if exclude [all] was a thing. thanks for checking sean
My test case was actually
(ns stuff (:refer-clojure :exclude [this and that and a whole bunch of other stuff]))
🙂Hi! I have a doubt about lazy sequences...
Let's assume I have a fn, get-data
which returns a lazy sequence. Each element of the sequence is a vector of N (N~=1000) elements of a redis stream. I would like to process the stream one element at a time, so (apply concat (get-data))
would get me the full stream of events, one by one.
However, I'm concerned about the memory cost of doing this:
• First, I wouldn't like the whole seq to be realized in memory. Would something like this mean maintaining a reference to the sequence and thus having to store it fully in memory at some point?
(let [data (apply concat (get-data))]
(reduce (fn [] ...) ... data))
• Also, I don't know if concat will lazily get the elements only when needed, or will do some form of pre-fetching in blocks of 32, like some clojure functions do.concat
is lazy no doubt. Where I have a little doubt is apply
. I am not 100% sure, but I think apply
will pull all the chunks eagerly and then the things within each chunks will be pulled as and when needed, by concat
.
I'm still concerned about my first point, though.. Will storing a reference to the head of the lazy seq mean that at some point all the seq will be realized in memory? If so, can I avoid this?
Yes. But wait for someone else to confirm the behavior of apply
, as I am not fully confident this is the case.
If you hold onto the head of a lazy sequence, and realize all elements, it will pull everything in memory. You need to let go of the head to free memory. This is one of the known gotchas.
and how can I let go of the head? I'm assuming the code I posted above has this issue, right? :S
Yes, you cannot retain a reference to data
while simultaneously realizing it, I believe.
If you instead did (let [[h & r] (apply concat (get-data))] ...)
and the recur with only r
, it would work.
And avoid the apply
. I only use apply on a known small arglist.
Thinking about it, it does make sense. apply
passes a collection as arglist to another function. To be able to actually call the function, it needs to evaluate the arglist eagerly.
apply is not eager
chunking is not controlled by the consuming function, it's a property of individual lazy-seq implementations (seq over vector is chunked for example)
proof that apply isn't eager - this returns instantly
user=> (apply (fn [x & ys] x) (range))
0
@UP0Q30S10 in a let block surrounding a reduce like in your example, the clojure compiler is smart enough to delete the gc root of data
since it can prove nothing can use it, so that is safe
user=> (let [massive-data (iterate inc' 0)] (reduce (fn [x y] y) massive-data) ; this won't return for years, but it won't use up ram until an individual bignum is big enough to do so
@UJRDALZA5 these things are easy to test, please don't just speculate answers
@U051SS2EU Wasn't sure how to test the memory consumption part. Thanks for the clarification. For apply, I looked at the source and found a reference to applyTo but could not understand where that is coming from.
applyTo is a method on IFn
apply is not eager, your function might be - it depends on what it does with its arg list
haha my reduce (with an added paren that is missing in the paste above) has been running for hours now :D -- 100% CPU but no expansion of ram used according to htop
no, because clojure compiles let blocks in a way that removes all references to each binding at the last line where it's used
so the gc doesn't hold onto the binding, and it can be freed
of course if you pass the binding out of the block, it isn't cleared and can't be freed from within the block
if you want more info this is called "locals clearing" - some debuggers turn it off btw
@U051SS2EU thanks!! Not only I see it clearly now, but I finally understand why locals clearing is useful 😅
Hi everybody,
I am asking for help 🙂.
I am using https://github.com/weavejester/ring-oauth2 to use corporate IDP.
My application is SPA served on /webpage
. And launch_uri (initializes auth) is on /login/
.
My redirect URI is /login/callback
. I do not know what should be set in this route so there is nothing right now. Do I have to have some logic there?
When I recieve the code from IDP I am redirected to /login/callback
where I recieve state mismatch
error. But I can see that state
query string parameter is same as the first time I call IDP.
Can somebody point me to the right direction as I am hopeless right now?
The only hint I found is https://github.com/jupyterhub/jupyterhub/issues/1438
I am not using Auth0 but I am using NGINX with Let'sEncrypt certificate for providing HTTP content over HTTPS.
Maybe this can cause some issues?
Thank you!
Hi all, I'm getting a reflection warning from one of my programs and I've never tried to fix one of these before, and could use a hand. The warning is WARNING: Illegal reflective access by clojure.lang.InjectedInvoker/0x00000008401e8040 (file:/home/eamonn/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar) to method sun.security.x509.X509CertImpl.getSubjectDN()
and the function is pretty small (see snippet). From my vague understanding, I need to probably slip in a ^String somewhere, but I have no idea where.
The cert arg comes from the above function, if that helps.
I think Clojure doesn't know what cert
is, so it has to use reflection to get the getSubjectDN
function - it cannot resolve it directly.
Or rather, using the interface, with [^X509Certificate cert]
where X509Certificate
is in java.security.cert
.
Bingo, that did it. Thank you very much! I'm always shocked how quick it is to get an answer here. 🙂
https://clojure.org/guides/faq#illegal_access for some more info too
poll question:
when do you use if-not
over if
?
My organization had a use case yesterday: we wanted to thread a value
in a ->
only if value
was not an empty collection. So, we wrote this:
(some-> value
(#(if-not (empty? %) %))
...)
Would be interested to see if anyone else has a more clever way to cast values which don't meet some predicate to nil
To answer the OP question - I never use if-not
. I have no clue as to why it exists, and I cannot find anything about it and about when-not
.
honest question: why is (not (empty? x))
and anti-pattern?
thanks for the tips! you are correct on all fronts
also i tend to prefer if-not
(as its writers likely intended) when a simple condition is wrapped in a not
:
(defn ensure-protocol [s]
(if-not (str/starts-with? s "http")
(str "http://" s)
s))
(defn ensure-protocol [s]
(if (not (str/starts-with? s "http"))
(str "http://" s)
s))
the first reads easier to me
yes, @UGRJKK74Y, and that is actually the suggestion on the community style guide: https://guide.clojure.style/#if-not
@U2FRKM4TW’s point about when
is also in the style guide as well, I believe
yep, single form if
calls always feel weird to me
though i dont' always agree with the style guide (`seq` instead of (not (empty? ))
being the primary example)
I would be interested to see the reasoning behind that suggestion as well
I use if-not
or if
based upon which branch is the 'long' one. The first branch is the quick return and the else branch is the bulk of the logic... Keeping track of the else
while reading the code in a normal if
when there's a really big then
is annoying.
it IS in the source code FWIW: https://github.com/clojure/clojure/blob/clojure-1.10.1/src/clj/clojure/core.clj#L6208
i guess technically, by doing (not (empty? x))
you're actually doing: (not (not (seq x)))
so plain (seq x)
it's just saving steps
Yes, that's exactly why (not (empty? x))
is an anti-patter. IIRC it's in the empty?
's docstring.
And I always just use (if x then else)
instead of (if (not x) else then)
or (if-not x else then)
.
Regarding that particular style guide - it has some quite opinionated pieces, it has been discussed before.
nice, that makes perfect sense to me. I think that whether to use if
or if-not
, it just depends on how you would explain the test you are making in plain english to someone else.
I don't understand that argumentation at all. You don't write in Clojure as you write in English. Python tried that, and it failed miserably IMO with its grotesque 1 if cond else 2
.
readability is important in every language. if you can read (if (not ..))
easier than (if-not ..)
more power to ya
i'd also likely argue readability in clojure is even more important that other langues given how expressive the language can be.
I think that (if test else then)
does not read as well as (if-not test then else)
, but what qualifies as an else
branch or a then
branch is up for debate
Why would you prefer
(if-not (odd? x)
(println "X is even, I like it.")
(throw (ex-info "X is odd, I don't like odd things" {:x x})))
over
(if (odd? x)
(throw (ex-info "X is odd, I don't like odd things" {:x x}))
(println "X is even, I like it."))
If you think my example is bad, can you come up with a good one where if-not
is actually more preferable than just if
with swapped branches?I think that is a good enough example for my point. I think the if-not
version "suggests" that I intend to be working with even values/that even values are somehow more important to this part of the program. This is of course a qualitative notion, so I think that either style makes sense, but that I would just personally write it the first way
the then
branch often represents the path that you "want" to go down
generally i think it's the idea of writing your conditions for the things that you expect to happen. sometimes if
reads better, sometimes if-not
.
Ah, I should've just used println
in both branches. :D With throw
using if-not
makes even less sense though since you can replace it with
(when (odd? x)
(throw ...))
(println ...)
Just searched for if-not
usages throughout the whole classpath of one of my projects (excluding my own code, since I don't use if-not
at all). Among the first 50 or so usages when there are both then
and else
branches, about 80% seem to have more or less similar branches.It seems to always be personal preference, and I cannot guess any pattern behind it whatsoever. Which makes me dislike if-not
even more, because it adds yet another way or writing the exact same thing while not adding any objective advantages.
Yeah, I agree that there is no discernible semantic difference between the then
and else
branches in the vast majority of cases. So you should prefer if
to if-not
most of the time
{:shop/uuid #(safe-coercion uuid/as-uuid %)
:shop/config #(safe-coercion json/read-str %)
:shop/foo {:bar {:baz inc}}
{:shop/uuid "00000000-0000-0000-0000-000000000000"
:shop/config "{}"
:shop/foo {:bar {:baz 0}
:shop/nothing-to-do 1}
Do you have ready fn
to share which can do coercion on map considering also nested keys coercion?Just wondering. Which form do people prefer to write?
{:human/name "foo"
:human/age 99
:human/passion [:clojure]}
Or:
#:human{:name "foo"
:age 99
:passion [:clojure]}
Personally I prefer the first. I find it more consistent (mixing namespaces in a single map). And it's less misleading, it's immediately clear the keys are namespaced
It also makes it immediately obvious that all keywords within the map that don't have /
in them belong to the same ns.
Compare
#:bar {:a 1
:baz/b 2
:c 3}
to
{:bar/a 1
:baz/b 2
:bar/c 3}
That wouldn't work in EDN though, at least I don't think it has a merge reader or something
If you have to read EDN with your eyes or write EDN manually, then yeah, it's a bummer. But you can create custom readers - that's what Aero does.
If you read and write EDN with software, then why would you care about the #:bar {...}
syntax anyway?
A lot of the work I do is write EDN manually, using Integrant and Duct. I'm just wondering which form people prefer and why
2., no alignment
hey, is it possible to shell out to a command that won't take any command line arguments but instead prompts the user for input along the way?
clojure.java.shell/sh
seems to exit at the first prompt. can i connect the :in
/ :out
to a stream of some kind to interact with the subprocess?
thanks! a comment at the end of the source file had this snippet in it:
(sh "sed" "s/[aeiou]/oo/g" :in "hello there\n")
which seems to work when a single piece of input is needed :thumbsup:If I have a clojure function that has a parameter that requires some further specification, what's the correct clojuredoc way of doing it. for example if an argument some-fn
has to have certain paramenters, or an argumetn some-map
has to have keys of a certain form what's the general guideline. I know in e-lisp it generally is just specified via example. Is there a good example I can follow?
gen-class is a function with a long docstring with hand-written examples https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/gen-class
what do you mean by "clojuredoc" here?
sorry, documentation comments 🙂
I was roaming on clojuredocs just now
wires crossed
I have an interesting issue when trying to make a request with clj-http. I want to encode a query param with a space. Here's an example of the invocation:
(comment
(require '[clj-http.client :as client])
(client/get
""
{:query-params {:x "This is a test"}
:debug? true}))
When I inspect the dumped output, it shows the query string as :query-string "x=This+is+a+test"
and the final url as "http://localhost?x=This+is+a+test". Does anyone know if there's a simple way using clj-http to do url encoding such that the spaces are replaced with %20? I've tried adding :content-type x-www-form-urlencoded but that doesn't seem to have any effect.Perhaps this is relevant? https://github.com/dakrone/clj-http#optional-dependencies
Isn't :as the format of the response and :content-type is the format of the input?
I've added ring/ring-codec "1.1.2" to my project and am invoking:
(comment
(require '[clj-http.client :as client])
(require '[ring.util.codec])
(client/get
""
{:query-params {:x "This is a test"}
:as :x-www-form-urlencoded
:debug? true}))
Still encoding with the + signs.That's for body encoding. Let me try it and see.
Yeah, that's what I read, too. This seems odd - almost like it used to work.
No problem. I consider myself fairly experienced with using this library but I feel like I'm doing some newbie mistake or something here.
I'm looking for some advice on how to produce documentation of system behavior for external consumption. #testing is a little dead so I'm hoping for some more feedback from #clojure.
this might be a fun format https://github.com/cognitect-labs/transcriptor
I think what I'm really looking for is an easy way to generate a markdown file from clojure code. I've found some libraries for parsing markdown into hiccup or html but haven't yet found one for easily writing code that emits markdown. Is there such a thing?
randomly weird thing, but does (doc recur)
print the docstring twice for everyone else?
I see part of it printed twice in a local repl, but not on http://clojurescript.io. :thinking_face:
I am seeing that behavior on def
, throw
, try
, and all the functions/macros that are listed here https://github.com/clojure/clojure/blob/b6835b1341e008cc69b1e6d3fa28e4362ebe16b4/src/clj/clojure/repl.clj#L19
that was a bug in doc
I thought it was fixed, but seems not
definitely known
that is the ticket
or https://ask.clojure.org/index.php/737/clojure-repl-doc-repeats-string-output-special-forms-clojure if you want to vote
I've added it to my 1.10.2 tracking list
Awesome. Thanks @alexmiller
I think what I'm really looking for is an easy way to generate a markdown file from clojure code. I've found some libraries for parsing markdown into hiccup or html but haven't yet found one for easily writing code that emits markdown. Is there such a thing?
Does leiningen automatically also use Clojars to look for dependencies, or do I have to add it? My project keeps failing to find something from maven central that is supposed to be in clojars when I build
leiningen does use clojars by default
if you add extra repositories, it should merge them instead of replacing as well
you may find that if a dep is not found in any repository, it may report only the first or last it checked, which is easy to misinterpret
so it may also have checked clojars for example and not found it there either (if perhaps there is a typo)
@austin021 What is the dependency you're having problems with?
@vlaaad facepalm Thanks... The old version I was using had the name io.replikativ/replikativ, but they changed to io.replikativ/konserve at some point. I was trying to update just the version number and didn't notice it.