This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-08-12
Channels
- # announcements (5)
- # babashka (1)
- # beginners (193)
- # calva (79)
- # cider (18)
- # clara (4)
- # clojure (38)
- # clojure-europe (12)
- # clojure-france (8)
- # clojure-nl (12)
- # clojure-sweden (1)
- # clojure-uk (50)
- # clojurescript (37)
- # conjure (30)
- # cursive (3)
- # data-science (2)
- # datalog (7)
- # datomic (12)
- # events (2)
- # expound (3)
- # figwheel-main (1)
- # fulcro (45)
- # graalvm (1)
- # jobs (1)
- # jobs-discuss (11)
- # luminus (1)
- # malli (5)
- # off-topic (32)
- # reagent (6)
- # reitit (32)
- # shadow-cljs (25)
- # spacemacs (2)
- # sql (22)
- # vim (6)
hi - i’m trying to write a run-tests
function in my user.clj
- but i don’t know how to get the regex to find the same tests as lein test
(require '[clojure.test :as t])
(t/run-all-tests #"_test*")
If I use #"*_test*"
, i get
Dangling meta character '*' near index 0
*_test*
^
I guess this is a regex question, basically - how do i build a regex that mimics what lein test
does?Look into tools.namespace - it has functions to do what you want
Have you tried #”.*_test.*”
?
user=> (t/run-all-tests #".*_test*")
Testing user
Ran 0 tests containing 0 assertions.
0 failures, 0 errors.
{:test 0, :pass 0, :fail 0, :error 0, :type :summary}
i’m not sure what that regex is supposed to match, actually - there are namespaces in my project like x.y.z_test
and x.y.z-test
, but switching in the dash in my regex still gives me the above output, just Testing user
Do you need the regex or can you just call the 0-arity run-all-tests?
huh - that prints out a bunch of namespaces, including things like clojure.core
and clojure.walk
(which i use in my project)
but still
Ran 0 tests containing 0 assertions.
0 failures, 0 errors.
{:test 0, :pass 0, :fail 0, :error 0, :type :summary}
Do you have your test ns loaded? run-all-tests just filters on every namespace loaded
is there an overarching namespace i can load? can i somehow load myproject.test.*
or something?
You could do a file-seq
over your test directory and load every file that matches
I recently wrote some code that might be similar:
(require '[clojure.tools.namespace.find :as find])
(require '[clojure.java.classpath :as classpath])
(defn require-clojure.*-namespaces []
(let [nss (->> (find/find-namespaces (classpath/classpath))
(filter #(.startsWith (name %) "clojure.")))]
(try
(doseq [ns nss]
(require ns))
(catch Exception e
(println e)))))
you'll need `[org.clojure/tools.namespace "1.1.0-SNAPSHOT"]` [org.clojure/tools.namespace "1.0.0"]
as dependency
oh, sorry. it's available on sonatype:
:repositories [["sonatype-oss-public"
" "]]
I’d be incredibly surprised if that does not exist on clojars given it’s under the clojure org
It looks like dev snapshot versions are on sonatype https://github.com/clojure/tools.namespace#development-snapshots
ya - Could not find artifact org.clojure:tools.namespace:jar:1.1.0-SNAPSHOT in clojars (
, but it worked with that :repositories
bit
Not on clojars
Why are you using a snapshot at all?
Just use 1.0.0
I'm pretty sure I just copy and pasted the wrong snippet when I was in a hurry 😬
yeahhhhh buddy - thanks @ddouglass @smith.adriane @alexmiller
So I have some basic multimethod code like this:
;; Dispatch on the type of each argument
(defn selection-dispatch [& args] (mapv type args))
(defmulti selection #'selection-dispatch)
(defmethod selection
[PersistentVector PersistentVector js/Boolean]
[[start-paragraph start-offset] [end-paragraph end-offset] backwards?]
{:start {:paragraph start-paragraph
:offset start-offset}
:end {:paragraph end-paragraph
:offset end-offset}
:backwards backwards?})
;; Example:
(selection [:p1 1] [:p1 10] false)
(Ignore the actual content and the fact that there's no other methods defined yet, this is as much an exercise as anything 🙂).
It works fine, but if I want to make my code a little briefer by redefining my dispatch-fn with partial
, like this:
(def selection-dispatch (partial mapv type))
and then try to run my test code again, I get Error: false is not ISeqable
. What's happening here? Isn't the partial equivalent to the function written out?Your original function took the args as varargs, your new one takes a collection
So, not the same
Ah, I understand. For some reason I had it in my head that what was being passed to the dispatch function was the list of arguments as a single vector (rather than one at a time)
Makes perfect sense now though, thanks for clearing that up
I've been confused and fearful for a while about certain idiom w.r.t. backquote. I've posed the https://clojureverse.org/t/backquote-tilde-in-and-outside-of-macro-definitions/6388?u=jim_newton.
question about destructuring: I see https://clojure.org/guides/destructuring how I can both destructure a sequence with & rest :as all
. But it is not clear to me how both bind an destructure other sequences. For example
(let [[[a b] & more] some-data]
...)
I'd like to treat the first element of some-data
as a pair whose content I'd like to bind respectively to a
and b
but I'd also like to bind the pair [a b]
to ab-pair
. Is there a syntax for this. Of course I can include a second line of the let
as follows
(let [[[a b] & more] some-data
ab-pair [a b]]
...)
or
(let [[ab-pair & more] some-data
[a b] ab-pair]
...
)
But is there a syntax for binding a
, b
, and ab-pair
at the same time?a note about nested / combined destructures: destructuring is compact in code but not efficient to calculate, and multiple destructure blocks usually increase readability while not impacting the cost of the generated code also, more than once I've had to replace a destructure in a hot loop with explicit assignments, they generate a lot of code
I don't think you can do it in one expression. I'd be tempted to inline [a b]
in the body of your let statement without giving it a name. Here's Clojure: The Essential Reference's take on what you can do with a vector in a destructure call:
[bind1 .. <& bindN> <:as sym>]
yes, inlining [a b]
is not bad for this simple case. I intentionally made the example smaller for the question. But in general [a b]
may be replaced with something much more complicated. Scala has an idiom for this
data match {
case ab@(a,b)::_ => .. // now a, b, and ab are bound after destructuring
}
I hope I have the syntax correct.Would that problem be better served by using a map as the argument instead of implicit names using indices?
Not sure what you mean.
I've been going through 4clojure exercises and came across happy numbers http://www.4clojure.com/problem/86 I feel like I'm quite close to the solution, but instead of returning true at the easiest test case when I pass 7, my method starts an endless loop.
(defn happy? [number]
(loop [ number number
acc []]
( let [digits (num->digits number)
squared-sum (reduce + (map (fn [number] (* number number)) digits))
happy-number (= squared-sum 1)
this-squared-sum-appeared-before (and (not (empty? acc)) (contains? acc squared-sum))]
(case [happy-number this-squared-sum-appeared-before]
[true _] true
[_ true] false
(do (println squared-sum acc)
(recur squared-sum (conj acc squared-sum)))))))
I am trying to understand why the case conditional does not match and return true when the squared sum of digits is 1. Any ideas?I recommend using an editor with a debugger and stepping through the code expression by expression. That's how I solved issues previously with big loop recur code. https://practicalli.github.io/spacemacs/debug-clojure/cider-debug.html Jump to 19.05 in the video to see debugging in action
@U2J6U9D1R I believe that your implementation would work if you used core.match/match instead of case, which doesn't support the pattern matching style you're trying to employ.
alternatively, something like
(cond
happy-number ;;foo
this-squared-sum-appeared-before ;;bar
:else recur)
would work in this case because you're only trying to match against truthinessI think you just can't use _
as a wildcard in the case expression like that
(case [true :something]
[true _] :success
[_ true] :fail
:fallback)
;; => :fallback
(case [true '_]
[true _] :success
[_ true] :fail
:fallback)
;; => :success
yes, you're totally right!
thank you!
Can anyone point me to a clear example of how parameters are passed through an application using ring and compojure? As a simple example, I have layout function in my handlers that would take a map of params as an argument. I want to set the title of each page in the handlers that render the content for each page. Subsequently I'll want to pass queries into those handlers for each page. TIA
Would Sean's example app server the purpose? https://github.com/seancorfield/usermanager-example/blob/develop/src/usermanager/controllers/user.clj#L53
I've been studying that, and almost understand, but not quite. I'm not sure where the [req] param passed through the application is originating from. I have to stare at the code more, but does compojure / ring generate that with each request automatically?
Looking at ring's and compojure's docs might help - one useful piece is https://github.com/ring-clojure/ring/wiki/Parameters
The issue I'm having with Sean's example is that there are parts of the code I don't understand, particularly the machinery in main.clj. I could just copy it and get something running, but I'd prefer to understand for myself how an app is built up.
I’d also recommend some of the courses on purely functional by @U050P0ACR - not sure if he has up to date web app course but there was a useful one a few years ago
another records/protocol question lol: i have a bunch of records and a protocol in a namespace A, and then i import the records explicitly and require :refer :all
the protocol and record constructor functions into namespace B. in namespace B, I use extend-type
to implement the protocol for the records. When I run lein repl
and lein test
, this works great. and when I run lein uberjar
on my development machine, it works. but when I run lein uberjar
on the server/target machine, it fails with: Caused by: java.lang.IllegalArgumentException: No implementation of method: :cost-name of protocol: #'game.cost-interfaces/CostFns found for class: game.cost_interfaces.Click
is there any way to debug why it might be failing only when running uberjar
?
i tried changing the extend-type
calls to be defrecord
calls, removing the import and definition of the records in the other namespace, and that works as well
kind of complicated, so my apologies if you try to understand it
Anyone have a nice solution for pretty printing JSON? I'd like to include some json in my documentation (source code) but without escaping quotes for example.
https://github.com/dakrone/cheshire has pretty printing
Yeah I'm using it now, but maybe I'm confused. I'm doing this:
(json/generate-string {:message-type :discovery
:device-id (uuid)
:device-type :example
}
{:pretty true})
And getting the following REPL output (using Cursive repl):
"{
\"message-type\" : \"discovery\",
\"device-id\" : \"bafcaef0-2b66-49f1-bcb1-c66935ceb2ba\",
\"device-type\" : \"example\"
}"
I'd like my example to look like this in my docs later:
{
"message-type" : "discovery",
"device-id" : "bafcaef0-2b66-49f1-bcb1-c66935ceb2ba",
"device-type" : "example"
}
try writing (println (json/generate-string ...))
cuz it looks like generate-string
is returning a string object, which is getting "printed" as a string, vs getting printed
Is there a "non-strict" equality func in clojure.core
eg. (equal-enough-for-me "3" 3) ;;returns true
?
Or should I just be happy with (= (str 3) "3")
?
clojure makes you be explicit about what you want. too many degrees of freedom in "non-strict"
Fair enough. But I still think such non-strict-eq
might be useful sometimes.
hi, curious about clojure and memory,
I have a large data set (`sequence of strings`) which i'd like to have a copy of - same data just separated to starts-with-chars-buckets
(to later search on), while still maintaining a copy of the original.... a clojure map would do:
A-> #{"A1", "A2", ..... "An"}
B-> .... etc
but assuming this would copy the original data once more (or not? does the JVM optimizes it?), i'd like to have some "pointers" pointing to each "allocation",
so in a C++ world i'd allocate data (strings) once and point to them, each Character would have an array of pointers (or something else...)
1. does the strings are copied once more? or the JVM "takes care of that" what would be a clojure way to not copy the data once more?
2. does clojure has any "pointers" way of thinking? heap? stack? should I care about that? what if my original data is huge?
3. what would be a good source to look on? I read abit on atoms/refs/agents but not sure ?
thanks 🙂
All of your data is immutable; you don’t need a copy
(def data-set ["A1" "A2" "B1" "C1"])
(def bucketed {:\A #{"A1" "A2"} :\B #{"B1"}....)
so my bucketed-version would "copy" the "memory-allocations" ? (not sure how this works under the hood)
Are you making these independently or making one from the other?
I create buckets from existing data-set (huge)
right now it's all in-memory,
In that case, buckets will be using a heap reference to the same string in data-set
(let [;; 'dataset' is someting like ["A1" "A2" "B1" "B2"]
dataset (load-dataset)
;; 'bucketed' is something like {\A ["A1" "A2"] ...}
bucketed (group-by first dataset)]
...
If some of the strings are duplicates, you can look into explicit string interning, or some garbage collectors do this automatically in the heap
so the group-by version would not "copy" anything?
awesome,
makes sense, didn't thought on group-by 🙂 just reduced and throwed it all to sets heh
Java stores objects on the heap and passes the reference
and what happens in a case like (def a "hello") (def b "hello")
how should I think on a,b
You will only get new string instances if you call String constructor or use a literal “abc”
In that case you made two literal strings
So there are two
got it,
The gc may dedupe them internally or you can explicitly intern with (.intern “hello”)
really? so hard to keep up these days :)
I generally wouldn’t recommend the latter unless you really understand what’s happening and why it would matter for you
Actually I think literal strings are automatically interned
i'm more curious how can I "know" what's going on? so group-by gives the feeling that stuff are not copied,
so in java world I can think on the stack/heap,
but with clojure?
@aviv I think that the awnser is "you don't need to think about it" All operations in clojure are "immutable" In general they only create a "new index" without copy anything
clojure is using everything you know from java, so keep thinking that way :) there are no wrappers around anything - Clojure strings are Java strings, etc
the place where it diverges is that Clojure collections are immutable and "changing" a Clojure collection creates a new collection that typically shares most of the data with the prior collection
certainly the values, and often much of the internal structure as well
awesome, thanks for you both
Quick question, if one was starting a project to build an API, would you recommend using something like Pedestal or a plumbing together a slew of libraries like Ring+Compojure?
https://github.com/metosin/compojure-api is a practical starting point. It has enough libraries and docs to cover building functional API's with data schema and living API document and test tools using swagger (openapi). compojure-api has a relatively small learning curve.
it's my go-to these days, esp combined with malli to check the inputs conform to specs
Hello all, just a quick question is there a way to do this without flatten
?
(merge-with
(fn [a b] (flatten (list a b)))
{:a 1} {:a 2} {:a 3}) => {:a (1 2 3)}
not without somehting ugly using coll?
, or an extra step to create {:a ()}
first
do you care about list/vector and order?
you could just use conj
then?
I guess you're going to need a special case regardless
I tried but I can not conj
two longs or can I?
class java.lang.Long cannot be cast to class clojure.lang.IPersistentCollectio
yeah, that's the special case
@lukasmoench1113 also if the first input was {:a 1 :b 2}
should we get {:a (1 2 3) :b (2)}
?
one approach: (reduce (fn [m [k v]] (update m k (fnil conj []) v)) [{:a 1} ...])
ty i tried something similar but it wasn't that sophisticated and didn't work out that way
fixed
user=> (reduce (fn [m el] (reduce (fn [m [k v]] (update m k (fnil conj []) v)) m el)) {} [{:a 1} {:a 2} {:a 3}])
{:a [1 2 3]}
I have a bad habit of just typing in one liners, here's the formatted version
user=> (reduce (fn [m el]
(reduce (fn [m [k v]]
(update m k (fnil conj []) v)
m el))
{} [{:a 1} {:a 2} {:a 3}])
{:a [1 2 3]
fixed
user=> (reduce (fn [m el] (reduce (fn [m [k v]] (update m k (fnil conj []) v)) m el)) {} [{:a 1} {:a 2} {:a 3}])
{:a [1 2 3]}
what about just using group-by
instead?
is there other stuff in these maps?
Tbh to find my solution took me pretty long:joy: I hope one day I'm as good as your are
I've written this function a number of times, it's just small enough that it never ends up in a lib
it's a few very common idioms concatenated together
ye i find it especially hard to write reducers, my version did only join the last keys in the map ...
to cheer up @hiredman
user=> (reduce (fn [m [k v]]
(update m k (fnil conj []) v))
{} (concat {:a 1} {:a 2} {:a 3}))
{:a [1 2 3]}
it's less readable for one
"unnecessary multiplication of entities"
the second reduce was really a hidden concat, which is silly
you can view reduce as final operation to make something concrete (this is explicit in transducers), so when you see reduce used twice like that in my mind it sort of implies a seam
you can write a transducer that just kind of does sort of thing, I don't know of any off the shelf libraries
but something that takes a reducing operation, and runs it per value of each keyin a collection of maps instead of per map is something I've seen called faceting
(defn facet [f]
(fn
([]
(f))
([accum]
(f accum))
([accum value]
(reduce
(fn [accum [k v]]
(assoc accum k (f (get accum k) v)))
accum
value))))
(transduce
facet
(fnil conj [])
{}
[{:a 1} {:a 2} {:a 3}])
and facet is neat because you can pass in other reducing functions
(transduce
facet
(fnil + 0)
{}
[{:a 1} {:a 2} {:a 3}])
facet sort of splits the two reduces at the seam, facet itself is very much the inner reduce and the outter reduce is replaced by whatever reducing context you use the transducer in
https://github.com/aphyr/tesser is where I got that name
public static <T, K, A, D> Collector<T,?,Map<K,D>> groupingBy(Function<? super T,? extends K> classifier,
Collector<? super T,A,D> downstream)
returns a collector that contains a "downstream" collector, where collector == reducing fn
I was wondering if I could pick your brain about a personal game project that I alluded to here yesterday. It’s the 4x4 grid of the keys :a : b :c :d
by :w :x :y :z
. So, I now have a vector of 16 “tiles”, each with 2 of these unique key pairs. So far, so good.
I wanted to see how many possibilities exist of how we can shuffle the tiles, and came up with about 21 TRILLION. 2.09 x 10 ^ 23. That’s a hella-lotta possibilities. (16 Factorial) “Sure,” I said, “Let’s throw them into a defined sequence and do some research on it.”
(def tperm (combo/permutations tiles))
(nth tperm 0)
=> ([:a :w]
[:a :x]
[:a :y]
[:a :z] ; ...(and a bunch more)…
[:d :z]
However, in the game, I don’t want the full list. I want to exclude the possibilities that have 3 in a row or column. So, exclude it if there’s 3 :x’s in a row. Exclude it if there are 3 :b’s in a column. In other words, I want to exclude the above “nth tperm 0" because it has at least 3 of :a in the first row. (A row is defined as (partition 4 tiles) and an example column would be positions %1, %5, %9, %13.
I have set up a checking function to find a match of any key in any row or column, and used the remove
function on that long sequence.
(def tpermf (remove any-rc-match? tperm)) ; removes any 3-of-a-kind row/column match
=> #'game.core/tpermf
(first tpermf)
OutOfMemoryError GC overhead limit exceeded clojure.core/take (core.clj:2752)
So, I got this Out Of Memory error after several minutes and my laptop heating up to temperatures able to cook dinner. :thinking_face: :rolling_on_the_floor_laughing: Clearly I need to rethink things, and I don’t know if my initial approach is even doable. I’d prefer to work with this subset of the larger permutation. I’d like to maybe be able to count how big that subset is. But if that means sorting through literally trillions of possibilities, maybe it’s not possible.
Thoughts on how to not cook my laptop? Thanks!I'm having flashbacks to my college combinatorics class. it should be possible to calculate the size of the set your interested in. most of these problems take the form of total - (number of permutations to discard)
you've already calculated the total number of permutations. you just need to calculate the number of permutations of grids with 3 in a row
I'm not sure what you mean by: > It’s the 4x4 grid of the keys `:a : b :c :d` by `:w :x :y :z`. can you provide an example grid?
https://github.com/clojure/math.combinatorics provides some nice tools to generate lazy sequences of combinations
with that you could skim through without ever realizing more than one at a time in memory
assuming you are careful not to hold the head of the sequence
which would probably also help your previous code
tperm
is holding the head there
Hi @alexmiller - The tiles (actually 4 colors and 4 psychic Zener symbols) look like this:
[[:purple :plus]
[:purple :circle]
[:purple :star]
[:purple :bacon]
[:blue :plus]
[:blue :circle]
[:blue :star]
[:blue :bacon]
[:green :plus]
[:green :circle]
[:green :star]
[:green :bacon]
[:orange :plus]
[:orange :circle]
[:orange :star]
[:orange :bacon]]
I’m using Math/combinatorics aliased as “combo” in the definition:
(def tperm (combo/permutations tiles))
then you're almost there - just stop holding the head :)
@alexmiller What does it mean to “hold the head”?
<- Dumb guy in #beginners channel. 😄
no worries! lazy sequences can be traversed from beginning to end and the early parts of the sequence will be garbage collected behind you as you traverse UNLESS someone has a reference to the first thing in the sequence
here your tperm is holding a strong reference to the head of the lazy sequence. as you traverse in tpermf, the entire sequence will be held in memory, eventually filling it up and causing an OOME
(remove any-rc-match? (combo/permutations tiles))
does not hold the head - that's the lazy sequence of filtered combinations. depending what you want to do, you might do further sequence ops on it, then take a few of them at the end, or loop/recur through them, etcSo, can I take the lazy sequence (combo/permutations tiles)
and use a threading macro to thread through several transformations, and then hand it to the def
?
threading macros are purely syntactic rearrangement - once the macro is expanded, they look exactly like my code above
you can even use let or defns presuming you are somewhat careful about how you use the head of the lazy seq - the Clojure compiler will aggressively clear local references to minimize this issue
@alexmiller One note on this. I assume filter
or remove
has to do its function in front to back order. In my case, the permutations
function shuffles from the bottom up, and I have a 3-of-a-kind in positions %1
, %2
, and %3
, making at least the first 13-factorial items I’m removing. That means I’m going through at least 6 Billion items before I get my first item that I want to keep.
That’s one reason why it seemed like it was locking up: It was saying “nope” on the first 6 billion items! :rolling_on_the_floor_laughing:
Either way, I’m able to get what I need with a simple shuffle
function, and then testing if a legal setup without a 3-of-a-kind. MUCH simpler and takes a fraction of a second. I don’t need the whole set! I just need one to play with.