Fork me on GitHub
#clojure
<
2020-03-19
>
dpsutton04:03:47

i didn't know this was a thing: (:refer-clojure :exclude [all])

hindol04:03:57

That looks like a red button you should not press, 😀

seancorfield05:03:37

It doesn't seem to actually exclude anything tho'...

seancorfield05:03:10

user=> (ns wibble (:refer-clojure :exclude [this-thing-does-not-exist]))
nil
wibble=>

seancorfield05:03:04

So it refers in all of clojure.core except the named symbols... and if those symbols don't exist, it gives no errors @U11BV7MTK

dpsutton05:03:12

interesting. there's a defined all in this file. it looks like tim just remembered something from core incorrectly. maybe confused every? with all or something like that

dpsutton05:03:19

ah and there's an all in core.logic. i was checking the docstrings of ns and others to see if exclude [all] was a thing. thanks for checking sean

seancorfield05:03:18

My test case was actually

(ns stuff (:refer-clojure :exclude [this and that and a whole bunch of other stuff]))
🙂

Setzer2208:03:28

Hi! I have a doubt about lazy sequences... Let's assume I have a fn, get-data which returns a lazy sequence. Each element of the sequence is a vector of N (N~=1000) elements of a redis stream. I would like to process the stream one element at a time, so (apply concat (get-data)) would get me the full stream of events, one by one. However, I'm concerned about the memory cost of doing this: • First, I wouldn't like the whole seq to be realized in memory. Would something like this mean maintaining a reference to the sequence and thus having to store it fully in memory at some point?

(let [data (apply concat (get-data))]
  (reduce (fn [] ...) ... data))
• Also, I don't know if concat will lazily get the elements only when needed, or will do some form of pre-fetching in blocks of 32, like some clojure functions do.

hindol08:03:20

concat is lazy no doubt. Where I have a little doubt is apply. I am not 100% sure, but I think apply will pull all the chunks eagerly and then the things within each chunks will be pulled as and when needed, by concat.

hindol08:03:06

It is better to write your custom function that pulls chunks and then events lazily.

Setzer2208:03:37

ok, that makes sense!

Setzer2208:03:51

and shouldn't be too difficult to implement

Setzer2208:03:26

I'm still concerned about my first point, though.. Will storing a reference to the head of the lazy seq mean that at some point all the seq will be realized in memory? If so, can I avoid this?

hindol08:03:42

Yes. But wait for someone else to confirm the behavior of apply, as I am not fully confident this is the case.

hindol08:03:56

If you hold onto the head of a lazy sequence, and realize all elements, it will pull everything in memory. You need to let go of the head to free memory. This is one of the known gotchas.

Setzer2208:03:42

and how can I let go of the head? I'm assuming the code I posted above has this issue, right? :S

hindol08:03:31

Yes, you cannot retain a reference to data while simultaneously realizing it, I believe.

hindol08:03:49

If you instead did (let [[h & r] (apply concat (get-data))] ...) and the recur with only r, it would work.

Setzer2208:03:16

ok, thanks!

hindol09:03:02

And avoid the apply. I only use apply on a known small arglist. Thinking about it, it does make sense. apply passes a collection as arglist to another function. To be able to actually call the function, it needs to evaluate the arglist eagerly.

noisesmith16:03:34

apply is not eager

noisesmith16:03:08

chunking is not controlled by the consuming function, it's a property of individual lazy-seq implementations (seq over vector is chunked for example)

noisesmith16:03:34

proof that apply isn't eager - this returns instantly

user=> (apply (fn [x & ys] x) (range))
0

👍 1
noisesmith16:03:10

@UP0Q30S10 in a let block surrounding a reduce like in your example, the clojure compiler is smart enough to delete the gc root of data since it can prove nothing can use it, so that is safe

noisesmith16:03:45

user=> (let [massive-data (iterate inc' 0)] (reduce (fn [x y] y) massive-data) ; this won't return for years, but it won't use up ram until an individual bignum is big enough to do so

noisesmith16:03:47

@UJRDALZA5 these things are easy to test, please don't just speculate answers

hindol18:03:15

@U051SS2EU Wasn't sure how to test the memory consumption part. Thanks for the clarification. For apply, I looked at the source and found a reference to applyTo but could not understand where that is coming from.

noisesmith18:03:44

applyTo is a method on IFn

noisesmith18:03:09

apply is not eager, your function might be - it depends on what it does with its arg list

noisesmith18:03:41

haha my reduce (with an added paren that is missing in the paste above) has been running for hours now :D -- 100% CPU but no expansion of ram used according to htop

hindol19:03:22

That's a risky thing to test.

hindol19:03:03

So, the reduce example frees memory because reduce is implemented that way, right?

noisesmith20:03:09

no, because clojure compiles let blocks in a way that removes all references to each binding at the last line where it's used

noisesmith20:03:22

so the gc doesn't hold onto the binding, and it can be freed

noisesmith20:03:56

of course if you pass the binding out of the block, it isn't cleared and can't be freed from within the block

noisesmith20:03:26

if you want more info this is called "locals clearing" - some debuggers turn it off btw

Setzer2208:03:38

@U051SS2EU thanks!! Not only I see it clearly now, but I finally understand why locals clearing is useful 😅

Akiz08:03:11

Hi everybody, I am asking for help 🙂. I am using https://github.com/weavejester/ring-oauth2 to use corporate IDP. My application is SPA served on /webpage. And launch_uri (initializes auth) is on /login/. My redirect URI is /login/callback. I do not know what should be set in this route so there is nothing right now. Do I have to have some logic there? When I recieve the code from IDP I am redirected to /login/callback where I recieve state mismatch error. But I can see that state query string parameter is same as the first time I call IDP. Can somebody point me to the right direction as I am hopeless right now? The only hint I found is https://github.com/jupyterhub/jupyterhub/issues/1438 I am not using Auth0 but I am using NGINX with Let'sEncrypt certificate for providing HTTP content over HTTPS. Maybe this can cause some issues? Thank you!

leontalbot12:03:45

poll question: when do you use if-not over if?

Mitch12:03:23

My organization had a use case yesterday: we wanted to thread a value in a -> only if value was not an empty collection. So, we wrote this:

(some-> value
        (#(if-not (empty? %) %))
        ...)
Would be interested to see if anyone else has a more clever way to cast values which don't meet some predicate to nil

p-himik13:03:46

You can replace #(if-not (empty? %) %) with just not-empty.

p-himik13:03:21

Also, I prefer using when instead of if when there's no "else" block.

p-himik13:03:07

Also #2: (not (empty? x)) is an anti-pattern. Use (seq x).

p-himik13:03:05

To answer the OP question - I never use if-not. I have no clue as to why it exists, and I cannot find anything about it and about when-not.

Darin Douglass13:03:57

honest question: why is (not (empty? x)) and anti-pattern?

Mitch13:03:03

thanks for the tips! you are correct on all fronts

Darin Douglass13:03:22

also i tend to prefer if-not (as its writers likely intended) when a simple condition is wrapped in a not:

(defn ensure-protocol [s]
  (if-not (str/starts-with? s "http")
    (str "http://" s)
    s))

(defn ensure-protocol [s]
  (if (not (str/starts-with? s "http"))
    (str "http://" s)
    s))

👍 2
Darin Douglass13:03:31

the first reads easier to me

Mitch13:03:14

yes, @UGRJKK74Y, and that is actually the suggestion on the community style guide: https://guide.clojure.style/#if-not

Mitch13:03:03

@U2FRKM4TW’s point about when is also in the style guide as well, I believe

Darin Douglass13:03:42

yep, single form if calls always feel weird to me

Darin Douglass13:03:12

though i dont' always agree with the style guide (`seq` instead of (not (empty? )) being the primary example)

Mitch13:03:29

I would be interested to see the reasoning behind that suggestion as well

cursork13:03:35

I use if-not or if based upon which branch is the 'long' one. The first branch is the quick return and the else branch is the bulk of the logic... Keeping track of the else while reading the code in a normal if when there's a really big then is annoying.

Darin Douglass13:03:54

i guess technically, by doing (not (empty? x)) you're actually doing: (not (not (seq x)))

Darin Douglass13:03:14

so plain (seq x) it's just saving steps

p-himik13:03:42

Yes, that's exactly why (not (empty? x)) is an anti-patter. IIRC it's in the empty?'s docstring.

p-himik13:03:23

And I always just use (if x then else) instead of (if (not x) else then) or (if-not x else then).

👍 1
p-himik13:03:11

Regarding that particular style guide - it has some quite opinionated pieces, it has been discussed before.

Mitch13:03:56

nice, that makes perfect sense to me. I think that whether to use if or if-not , it just depends on how you would explain the test you are making in plain english to someone else.

👍 1
p-himik13:03:23

I don't understand that argumentation at all. You don't write in Clojure as you write in English. Python tried that, and it failed miserably IMO with its grotesque 1 if cond else 2.

Darin Douglass13:03:18

readability is important in every language. if you can read (if (not ..)) easier than (if-not ..) more power to ya

Darin Douglass13:03:43

i'd also likely argue readability in clojure is even more important that other langues given how expressive the language can be.

Mitch13:03:44

I think that (if test else then) does not read as well as (if-not test then else) , but what qualifies as an else branch or a then branch is up for debate

p-himik13:03:54

Why would you prefer

(if-not (odd? x)
  (println "X is even, I like it.")
  (throw (ex-info "X is odd, I don't like odd things" {:x x})))
over
(if (odd? x)
  (throw (ex-info "X is odd, I don't like odd things" {:x x}))
  (println "X is even, I like it."))
If you think my example is bad, can you come up with a good one where if-not is actually more preferable than just if with swapped branches?

Mitch13:03:52

I think that is a good enough example for my point. I think the if-not version "suggests" that I intend to be working with even values/that even values are somehow more important to this part of the program. This is of course a qualitative notion, so I think that either style makes sense, but that I would just personally write it the first way

Mitch13:03:37

the then branch often represents the path that you "want" to go down

Darin Douglass14:03:55

generally i think it's the idea of writing your conditions for the things that you expect to happen. sometimes if reads better, sometimes if-not.

p-himik14:03:53

Ah, I should've just used println in both branches. :D With throw using if-not makes even less sense though since you can replace it with

(when (odd? x)
  (throw ...))
(println ...)
Just searched for if-not usages throughout the whole classpath of one of my projects (excluding my own code, since I don't use if-not at all). Among the first 50 or so usages when there are both then and else branches, about 80% seem to have more or less similar branches.

👍 1
p-himik14:03:20

It seems to always be personal preference, and I cannot guess any pattern behind it whatsoever. Which makes me dislike if-not even more, because it adds yet another way or writing the exact same thing while not adding any objective advantages.

Mitch14:03:49

Yeah, I agree that there is no discernible semantic difference between the then and else branches in the vast majority of cases. So you should prefer if to if-not most of the time

kwladyka12:03:02

{:shop/uuid #(safe-coercion uuid/as-uuid %)
   :shop/config #(safe-coercion json/read-str %)
   :shop/foo {:bar {:baz inc}}

  {:shop/uuid "00000000-0000-0000-0000-000000000000"
   :shop/config "{}"
   :shop/foo {:bar {:baz 0}
   :shop/nothing-to-do 1}
Do you have ready fn to share which can do coercion on map considering also nested keys coercion?

kwladyka14:03:12

Will it work with string -> uuid and string -> json ?

souenzzo14:03:16

(sc/def ::my-custom-attr my-str->json-fn)

kwladyka14:03:56

Thanks, I will try this!

kwladyka12:03:34

hmm maybe I can make this easier using [:shop/foo :bar :baz] instead

kwladyka12:03:10

sounds good 🙂

samoleary14:03:43

hey, is it possible to shell out to a command that won't take any command line arguments but instead prompts the user for input along the way? clojure.java.shell/sh seems to exit at the first prompt. can i connect the :in / :out to a stream of some kind to interact with the subprocess?

souenzzo14:03:58

You may end up with java.lang.Process take a look at (source sh/sh)

👍 1
samoleary15:03:54

thanks! a comment at the end of the source file had this snippet in it:

(sh "sed" "s/[aeiou]/oo/g" :in "hello there\n")
which seems to work when a single piece of input is needed :thumbsup:

theeternalpulse17:03:34

If I have a clojure function that has a parameter that requires some further specification, what's the correct clojuredoc way of doing it. for example if an argument some-fn has to have certain paramenters, or an argumetn some-map has to have keys of a certain form what's the general guideline. I know in e-lisp it generally is just specified via example. Is there a good example I can follow?

bfabry17:03:31

use clojure.spec

bfabry17:03:39

the spec will be included in the docstring

bfabry17:03:04

or if that's too formal/annoying, then just put it in the docstring

bfabry17:03:33

gen-class is a function with a long docstring with hand-written examples https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/gen-class

noisesmith17:03:30

what do you mean by "clojuredoc" here?

theeternalpulse17:03:59

sorry, documentation comments 🙂

theeternalpulse17:03:04

I was roaming on clojuredocs just now

markbastian18:03:29

I have an interesting issue when trying to make a request with clj-http. I want to encode a query param with a space. Here's an example of the invocation:

(comment
  (require '[clj-http.client :as client])
  (client/get
    ""
    {:query-params {:x "This is a test"}
     :debug?       true}))
When I inspect the dumped output, it shows the query string as :query-string "x=This+is+a+test" and the final url as "http://localhost?x=This+is+a+test". Does anyone know if there's a simple way using clj-http to do url encoding such that the spaces are replaced with %20? I've tried adding :content-type x-www-form-urlencoded but that doesn't seem to have any effect.

bfabry18:03:36

it is, you want ring-codec and specify

:as :x-www-form-urlencoded

markbastian18:03:41

Isn't :as the format of the response and :content-type is the format of the input?

markbastian18:03:06

I've added ring/ring-codec "1.1.2" to my project and am invoking:

(comment
  (require '[clj-http.client :as client])
  (require '[ring.util.codec])
  (client/get
    ""
    {:query-params {:x "This is a test"}
     :as           :x-www-form-urlencoded
     :debug?       true}))
Still encoding with the + signs.

bfabry18:03:01

oh actually it should be urlencoded to begin with according to the docs

markbastian18:03:04

That's for body encoding. Let me try it and see.

markbastian18:03:27

Yeah, that's what I read, too. This seems odd - almost like it used to work.

bfabry18:03:26

yeah this is past my knowledge of clj-http sorry 😅

markbastian18:03:08

No problem. I consider myself fairly experienced with using this library but I feel like I'm doing some newbie mistake or something here.

Ben Grabow18:03:42

I'm looking for some advice on how to produce documentation of system behavior for external consumption. #testing is a little dead so I'm hoping for some more feedback from #clojure.

Ben Grabow21:03:48

I think what I'm really looking for is an easy way to generate a markdown file from clojure code. I've found some libraries for parsing markdown into hiccup or html but haven't yet found one for easily writing code that emits markdown. Is there such a thing?

dpsutton20:03:02

randomly weird thing, but does (doc recur) print the docstring twice for everyone else?

1
jumpnbrownweasel20:03:23

Seems like an inside joke.

😄 2
Eddie20:03:17

I see part of it printed twice in a local repl, but not on http://clojurescript.io. :thinking_face:

dpsutton20:03:32

not on 1.8 but 1.9 and above it prints twice for me

Eddie20:03:44

I am seeing that behavior on def, throw, try, and all the functions/macros that are listed here https://github.com/clojure/clojure/blob/b6835b1341e008cc69b1e6d3fa28e4362ebe16b4/src/clj/clojure/repl.clj#L19

alexmiller20:03:52

that was a bug in doc

dpsutton20:03:15

Ah. Already known then?

dpsutton20:03:28

Based on tense perhaps fixed?

alexmiller21:03:16

I thought it was fixed, but seems not

alexmiller21:03:18

definitely known

alexmiller21:03:28

I should track that down for 1.10.2

parens 1
dpsutton21:03:17

If you’re doing other stuff happy to make a jira ticket

alexmiller21:03:05

that is the ticket

alexmiller21:03:17

I've added it to my 1.10.2 tracking list

λustin f(n)22:03:05

Does leiningen automatically also use Clojars to look for dependencies, or do I have to add it? My project keeps failing to find something from maven central that is supposed to be in clojars when I build

dpsutton22:03:21

what is it failing to find?

noisesmith22:03:33

leiningen does use clojars by default

noisesmith22:03:50

if you add extra repositories, it should merge them instead of replacing as well

alexmiller22:03:11

you may find that if a dep is not found in any repository, it may report only the first or last it checked, which is easy to misinterpret

1
alexmiller22:03:52

so it may also have checked clojars for example and not found it there either (if perhaps there is a typo)

seancorfield22:03:04

@austin021 What is the dependency you're having problems with?

vlaaad22:03:27

could it be a typo in dependency coordinates?

λustin f(n)22:03:16

@vlaaad facepalm Thanks... The old version I was using had the name io.replikativ/replikativ, but they changed to io.replikativ/konserve at some point. I was trying to update just the version number and didn't notice it.

yuhan22:03:36

Is it valid to use #^:foo as metadata? I can't find any reference to this syntax or the difference from regular ^:foo

hiredman22:03:29

it is very old syntax, '^:foo' is currently prefered