Fork me on GitHub
#beginners
<
2022-07-29
>
popeye05:07:46

I have a 4 variables `a b c d` how to produce combination as below in Clojure, any inbuilt function ?

``````a a
a b
a c
a d
b a
b b
b c
b d
c a
c b
c c
c d
d a
d b
d c
d d``````

R.A. Porter05:07:49

This will get you the Cartesian product, but I'm sure there's a tighter solution.

``````(let [l '(:a :b :c :d)]
(for [x l
y l]
[x y]))``````

š 1
ahungry05:07:40

https://github.com/clojure/math.combinatorics/ is as close to a built in as there is I think

popeye05:07:16

Thank you, will take a look

popeye09:07:56

I have a list `([:a :b] [:c :d] [:e :f])` How to check if `[:a :b]` is present in the sequence , `contains?` did not worked for me

jumar09:07:53

Make a set

``````(contains? (set '([:a :b] [:c :d] [:e :f])) [:a :b])
;;=> true``````
or use `some`
``````;; this works in a linear time
(some #(= % [:a :b]) '([:a :b] [:c :d] [:e :f]) )``````
List is a numerically-indexed collection (see docstring for `contains?`) so it doesn't work as expected (it throws exception on a list) It can be used with vectors, but it works differently:
``````(contains? [[:a :b] [:c :d] [:e :f]] [:a :b])
;;=> false

(contains? [[:a :b] [:c :d] [:e :f]] 0)
;;=> true``````

š 2
Martin PÅÆda09:07:35

One more:

``````(.contains [[:a :b] [:c :d] [:e :f]] [:a :b])
=> true``````

popeye09:07:41

@U01RL1YV4P7 is this java method?

jumar09:07:51

Oh, yeah, java interop. Good point š

sheluchin12:07:16

Is there a better way to write this?

``````(remove (every-pred #(nil? (:x %))
#(nil? (:y %)))
coll) ``````

Martin PÅÆda12:07:27

Maybe this one:

``````(remove #(every? nil? ((juxt :x :y) %))
coll)``````

1
teodorlu14:07:45

You want to remove all items where both `:x` and `:y` are nil? Should be equivalent for filtering away nil xs, then filtering away nil ys.

``````(->> [{:x 1}
{:hello :there}
{:y "abc"}
{:x 1 :y 2}]
(filter :x)
(filter :y))

;; => ({:x 1, :y 2})``````
Note - here I filter away stuff like `:x false` too.

1
Martin PÅÆda16:07:01

@U3X7174KS I think your solution doesn't work correctly:

``````(->> [{:x 1}
{:hello :there}
{:y "abc"}
{:x 1 :y 2}]
(remove (every-pred #(nil? (:x %))
#(nil? (:y %)))))

=> ({:x 1} {:y "abc"} {:x 1, :y 2})

(->> [{:x 1}
{:hello :there}
{:y "abc"}
{:x 1 :y 2}]
(remove #(every? nil? ((juxt :x :y) %))))

=> ({:x 1} {:y "abc"} {:x 1, :y 2})

(->> [{:x 1}
{:hello :there}
{:y "abc"}
{:x 1 :y 2}]
(filter :x)
(filter :y))
=> ({:x 1, :y 2})``````

if you're doing simple validation (filter #(and (:x %) (:y %)) ...)

2
Martin PÅÆda16:07:29

@U050ECB92 I'm not sure what is OP's use case, but your code behaves differently:

``````(->> [{:x 1}
{:hello :there}
{:y "abc"}
{:x 1 :y 2}
{:x false :y false}]
(remove (every-pred #(nil? (:x %))
#(nil? (:y %)))))

=> ({:x 1} {:y "abc"} {:x 1, :y 2} {:x false, :y false})

(->> [{:x 1}
{:hello :there}
{:y "abc"}
{:x 1 :y 2}
{:x false :y false}]
(filter #(and (:x %) (:y %))))

=> ({:x 1, :y 2})``````

sheluchin17:07:02

My use case is a collection of maps and I want to remove all the ones that have neither `:x` nor `:y` values. Sometimes the value is a `nil` (rather than just the absence of the key), and I want to remove those as well.

Chase15:07:01

Anyone want to be nerd sniped and see if they can come up with an efficient "simple" solution to the problem presented here: https://benhoyt.com/writings/count-words/ ?

Chase15:07:57

I've only been able to get my simple solution down to like 8.5 seconds which makes it one of the slowest solutions in the whole bunch. I used `slurp` which was technically against the rules as you are supposed to read in the data (make sure you use the 10x text, not just the text which I see many accidentally do in the HN discussions) line by line. That's where I get confused with having to do that and keep the running frequencies (I tried using a reduce function with the whole `line-seq rdr` thing but keep coming up short).

Chase15:07:08

At this point, I'm just curious to see what other folks come up with and will probably ask you to guide me through your thought process. I would love to see a solution with transducers because I've been trying to wrap my head around those.

rolt16:07:21

for the buffered input and line-seq:

``````(defn word-freqs [text]
(with-open [rdr ( text)]
(->> (line-seq rdr)
(mapcat #(clojure.string/split % #"\W+"))
(map #(clojure.string/lower-case %))
(frequencies)
(sort-by val >))))``````

R.A. Porter16:07:54

This is less elegant, but also works with the buffered stream: https://gist.github.com/coyotesqrl/c9e57fee6cf61cafd7128648e83123f7

Chase16:07:00

`mapcat` ! One tiny little tweak changes everything. I could not figure that out from the error messages I was getting.

Chase16:07:13

I'm getting about 5.6 seconds on that one. Still quite a bit slower than the python, ruby, lua, etc solutions which I still find surprising but much faster already than what I had.

Chase16:07:31

I also notice I get a huge amount of variance when running on my machine. My own solution will be 8.5 seconds most times but sometimes over 20 seconds.

Chase16:07:19

The variance seems to be bigger when actually printing out the results (part of the requirement) too but I'm still learning how to use profilers to inspect it all.

Chase16:07:12

@U01GXCWSRMW Thanks for providing your solution. I'm not seeing it being more performant but it really helps me see how I could keep that running tally of the frequencies going line by line.

Chase16:07:33

Yeah in running it multiple times actually printing out the values I'm not seeing much difference between using the `line-seq` method and using my original `slurp` method. I was curious if there would be much of a difference there.

R.A. Porter16:07:40

Yeah. Mine's creating too much garbage - a new map for each line. Good place for a transient, but then we'd need to create our own `merge-with!`. Or drop down to Java map manipulation directly, but that feels like cheating.

rolt17:07:12

i'm surprised that it's that much slower than python, is the python string split based on the regex or a simple space ?

rolt17:07:24

for the transducer part, unfortunately `frequencies` does not have it, but if you simply augment frequencies with a transducing arity you may get better performances

rolt17:07:46

``````(defn frequencies2
[xform coll]
(persistent!
(transduce xform
(completing (fn [counts x]
(assoc! counts x (inc (get counts x 0)))))
(transient {}) coll)))

(defn word-freqs [text]
(with-open [rdr ( text)]
(->> (frequencies2 (comp (mapcat #(clojure.string/split % #" "))
(map #(clojure.string/lower-case %)))
(line-seq rdr))
(sort-by val >))))``````

rolt17:07:57

for printing, maybe the IDE is slowing it down, you could print out to a file

Chase17:07:26

Yeah I did change to just splitting on a single space (well a single space and `\n` because I was seeing different behavior without that). I didn't notice much difference. I am just running in the terminal instead of the editor. It cuts down on some of the variance but I'm still surprised I see so much. My "simple" Rust solution I wrote does consistently in about 2 seconds or under. I knew clojure would be much slower than that but not slower than things like ruby.

rolt17:07:21

yes but if the requirement is only splitting on space and newline, then you can drop the regex and there is most likely a much faster solution (nothing comes to mind in idiomatic clojure though)

Chase18:07:15

Haskell was also way slower than everything else. It might just be lazy, functional languages aren't the best fit for this specific problem.

rolt18:07:16

i tried with a java scanner to avoid the creation of unnecessary datastructures but performances were worse š

Chase18:07:06

I warned ya it was a nerd snipe. hahaha

Chase18:07:10

Java, itself, does great on it.

ahungry23:07:15

Reminds me of this: https://wiki.haskell.org/Why_Haskell_matters#The_speed_of_Haskell retort, and why these types of language 'benchmarks' are often a waste of time

Chase23:07:40

Oh definitely, I see the merit in that and can agree these things are mostly silly unless maybe used as an opportunity to learn some low hanging fruit optimizations in the language you are using. But even knowing that, there is still something in human nature that draws you into these things. And even if it's silly, with the speed of modern computers it just physically feels slow watching something take over 10 seconds when you see it complete in under 2 seconds in almost every other implementation.

ahungry00:07:30

haha very true (and I myself am also a sucker for these silly wastes of time š )

jumar11:08:26

I haven't tried to implement this problem yet, just btw: I found this video talking about a similar problem: https://youtu.be/R_yX0XjdSBY?t=89 They seem to be optimizing it quite a bit.

Chris McCormick17:07:51

Is there a small library for working with `vec` + a position pointer for doing things like: ā¢ Insert a new item at the position pointer and update it to point at the new item. ā¢ Remove the item at position pointer and move the pointer to the previous item. ā¢ Move the position pointer forward or backward in the vec. ā¢ Given an item `X` point the position pointer at the first occurence. ā¢ Move the item at position pointer forward or backward in the list (changing order). I saw `clojure.zip` but it doesn't simply address all of these issues as far as I can tell. I find myself repeatedly re-writing the functions to do the above for many projects.

Martin PÅÆda17:07:09

Can you write some use cases/ problems for functions like this? (or add a link to some Github repo where you needed this functionality)

hiredman17:07:50

inserting a new item in the middle of a vector is not very efficient, it can't actually be done with the built in vectors without completely rebuilding the vector.

Chris McCormick18:07:05

@U01RL1YV4P7 in general I have a `vec` of datastructures representing some thing (like a file, game entity, sound effect, etc.) and I want to keep track of the thing the user has selected last and do things like those I mentioned above `[{:id 123 :name "thing" :property 42} {:id 456 :name "other" :property 1}...]`.

hiredman18:07:52

I would use a map, or multiple maps

Chris McCormick18:07:13

@U0NCTKEV8 I don't care about efficiency. These are vectors of less than 100 items modified through user interaction. Rebuilding the vector is fine, and that's how I generally do it already. I am using ClojureScript, not Clojure. I know how to find the position of and item `x` already, I just want a library that does all of these things for me so I don't have to re-implement it every time.

Chris McCormick18:07:05

I don't use a map because I care about order.

Chris McCormick18:07:35

I'm looking for a library which makes it simple to perform the operations I listed above on a `vec`.

Ben Sless18:07:31

hiredman18:07:50

this kind of thing comes up with people writing TODO apps a fair bit

hiredman18:07:02

the thing with vectors is the order is implied by the vector structure, and vectors are less flexible, but maps don't have an implied order, but are more flexible

hiredman18:07:54

there is no reason you can't have map with an explicit order defined in some way, either each entry have a number (this can make inserting in the middle tricky) or some kind of before/after relation

hiredman18:07:10

I understand the desire for a library that makes your exact approach less burdensome, but I would be surprised to come across a well supported one, because the general advice is to do this a different way

Chris McCormick18:07:01

I've tried to do it with a sorted map before but maybe I need to revisit that. Thanks for your input.

Chris McCormick18:07:58

> the general advice is to do this a different way What way? What about for your example of a TODO list where items can be reordered?

hiredman18:07:37

for example a todo list would be a map (maybe multiple maps depending) of id to item, and each item would have a key :order which is a number

hiredman18:07:49

the initial item in the list gets an :order of 1

Chris McCormick18:07:29

I guess removing an item would require a re-indexing.

hiredman18:07:29

and when you put an item at the top of the list, it gets an order of half whatever the previous top item's order was

hiredman18:07:03

when you put an item at the bottom it gets order of twice whatever was previously the bottom

hiredman18:07:31

and when you insert an item between two items, you give it an order halfway between their orders

hiredman18:07:05

you might want to periodically re-normalize the order to avoid your floating point numbers getting out of hand, but they might be fine for a good long while (in clojure there is a ratio type, which would be great for this, but harder to store in say a database if you would want that)

hiredman18:07:20

when it comes time to render the list in order, you sort-by :order

Chris McCormick19:07:06

Interesting thanks. This sounds quite a bit more complicated than using `vec`'s implicit order and just manipulating it as I have been.

hiredman19:07:53

alternatively you do some kind of linked list (every item has a :next which has the id of the next item, or both a :next and :previous) which might be faster, if you keep a note of whatever the head's id is, and generate the list in order by walking the links (might be faster than sorting)

Chris McCormick19:07:14

Yeah a linked list is interesting. :thinking_face:

Chris McCormick19:07:57

I have to say the existence of `clojure.zip/vector-zip` tells me that people do want to do stuff like this with `vec` . The only problem is that `clojure.zip` doesn't cover all of the use-cases I have. Likewise Stackoverflow questions like this: https://stackoverflow.com/questions/4830900/how-do-i-find-the-index-of-an-item-in-a-vector Maybe I just need to write a small library that does what I want so I don't keep re-implementing it. :thinking_face:

loganrios18:07:50

Iām currently trying to write my first REST API with Clojure, but am having trouble figuring out user authentication. It seems like web services are a common use-case for the language, but the only resources I can find use proprietary services (e.g. Auth0) or `buddy-auth`, which appears to be phasing itself out based on the project homepage. Is there a better, more common way to do HTTP auth in Clojure that Iām missing? Or does everybody just use proprietary services and not worry about it?

jumar18:07:35

Many people still use buddy or even friend

Daniel Craig19:07:36

I think a lot of people use Cognito, although that's also proprietary

jumar19:07:51

I mean, we used Auth0 before for integrating with GitHub Oauth but we were also using buddy to manage cookie-based sessions

loganrios20:07:36

Oh neat, I hadnāt even heard of `friend`. Iāll look a little deeper into Auth0 and Cognito and see if I can find something that fits my shoestring budget, too.

skylize20:07:04

Unlike in, say JavaScript, a library in Clojure does not become immediately useless if doesn't get daily updates to keep pace with v8. And popular mature libraries are commonly passed on to new owners after the creator backs out. Buddy is still in use by Luminus, which is well maintained. And the author (@U050CBXUZ) has already shown some willingness to take over stale projects if necessary.

yogthos21:07:03

I tend to use Buddy, so I would potentially be open to taking it over if it was being phased out. Itās a fairly small and simply library, so I definitely see why it doesnāt need a lot of updates.

skylize22:07:07

> if it was being phased out @U050CBXUZ I'm pretty sure the claim of being phased out refers to this statement on the `buddy-auth` readme > NOTE: this project is in maintencance mode, and looking for a new maintainer. which does not appear on any of the other Buddy modules. There are only 20 open issues across the entire suite, but you will find a PR on `buddy-hashers` written in April 2020 with no conflicts, that (even after a new offer for amendment this January) has still not even received comment as to whether it might be considered for merge. ... So š¤·:skin-tone-3:

skylize22:07:13

... unless @U03DKTAF52P Are you talking about the deprecation warning on https://github.com/funcool/buddy ? If so, read closer. Buddy used to be a single lib, that was split into 4. Links to all 4 appear on that page.

loganrios23:07:36

@U90R0EPHA Yeah, sorry, shouldāve been more specificāI did mean the āmaintenance modeā note on `buddy-auth`. And I appreciate the tip about those libraries still being fine to useāIām coming from a React Native background so Iāve got an irrational fear of breaking changes š

zakkor20:07:57

How can I put a newline in a string and have it work in the REPL? ('m using calva

hiredman20:07:51

what the repl attempts to print out is a serialized representation of the string you give it that would work as a string literal in a clojure program

hiredman20:07:15

if you don't want it printed like that, you need to print it some other way

hiredman20:07:54

pr, prn, etc are the family of printing functions that attempt to print things(not just strings) out in a way that can be read via the clojure reader

hiredman20:07:25

print, println, printf are similar functions but are somewhat more literal about what they print

hiredman20:07:10

``````user=> (def s "\"\n\"")
#'user/s
user=> s
"\"\n\""
user=> (prn s)
"\"\n\""
nil
user=> (println s)
"
"
nil
user=>``````

š 2