Fork me on GitHub
#clojure
<
2022-08-18
>
Aron08:08:42

I have a vector of integers called ranks and I would like to return a hash-map where the key is the integer and the result is the number of occurrences of that integer in the input vector, how to do nicer than this? (zipmap ranks (map (fn [rank] (count (filter #(= rank %) ranks))) ranks))

Aron08:08:08

lol, of course, thank you:)

amithgeorge14:08:26

Purely as an exercise for an alternative implementation

(reduce (fn update-counts [acc item] 
          (update acc item (fnil inc 0))) 
        {} 
        coll)

amithgeorge14:08:40

Oh, I just checked how frequencies is actually implemented, and my "alternative implementation" is pretty much how frequencies is implemented. They use transients for the perf.

hanDerPeder11:08:27

could use some help with a macro 🧵

hanDerPeder11:08:37

I want to use the is macro in a threading context

(deftest foo1
  (-> {}
      (assoc :foo 10)
      ((fn [m]
         (is (= 10 (:foo m)))
         m))))

;; The above works but is kinda ugly


(defn check [m f expected]
  (is (= expected (f m)))
  m)

(deftest foo2
  (-> {}
      (assoc :foo 10)
      (check :foo 11)))

;; This works, but with a not very helpful error message
;; FAIL in (foo2) (NO_SOURCE_FILE:39)
;; expected: (= expected (f m))
;;   actual: (not (= 11 10))

;; check needs to be a macro

(defmacro checkm [m f expected]
  `((fn []
      (is (= ~expected (~f ~m)))
      ~m)))

(deftest foo3
  (-> {}
      (assoc :foo 10)
      (checkm :foo 11)))

;;Seems to work, but symbols are ofcourse expanded with namespace
;; FAIL in (foo3) (NO_SOURCE_FILE:63)
;; expected: (clojure.core/= 11 (:foo (assoc {} :foo 10)))
;;   actual: (not (clojure.core/= 11 10))

hanDerPeder11:08:28

I feel there should be an existing solution to this problem but I haven’t found one. Any ideas, or improvements I could make to checkm?

jpmonettas12:08:48

maybe like this :

(defmacro checkm [m f expected]
  `((fn []
      (is (~'= ~expected (~f ~m)))
      ~m)))

jpmonettas12:08:12

not thinking about the problem, just replaced = by ~'= so it doesn't expand with the namespace

hanDerPeder12:08:56

seems there are more problems with this:

(deftest foo4
  (-> {}
      (assoc :foo 10)
      (checkm :foo 10)
      (assoc :bar 10)
      (checkm :bar 10)
      (assoc :baz 10)
      (checkm :baz 11)
      ))
;;FAIL in (foo4) (NO_SOURCE_FILE:40)
expected: (clojure.core/= 11 (:baz (assoc (checkm (assoc (checkm (assoc {} :foo 10) :foo 10) :bar 10) :bar 10) :baz 10)))
  actual: (not (clojure.core/= 11 10))

Ran 2 tests containing 10 assertions.
so each test case receives the unevaluated expanded threading form

Ed12:08:59

how about

(deftest foo3
  (-> {}
      (assoc :foo 10)
      (= {:foo 11})
      is))
?

hanDerPeder12:08:46

that wont work if you have more checks, right?

(-> start-value
    (transform 1)
    (check)
    (transform 2)
    (check))

Ed12:08:50

yup ... fair point ... apologies for not actually reading your actual question properly facepalm

Ed12:08:56

(deftest foo3
  (-> {}
      (assoc :foo 10)
      (doto (-> (= {:foo 11}) is))
      (update :foo inc)
      (doto (-> (= {:foo 12}) is))))
works ... but i can't say it's any more readable than your original

Ed12:08:24

if you really want a macro, this might work?

(defmacro is= [x y]
  `(let [x# ~x]
     (is (= x# ~y))
     x#))

(deftest foo4
  (-> {}
      (assoc :foo 10)
      (is= {:foo 11})
      (update :foo inc)
      (is= {:foo 12})))

hanDerPeder12:08:13

don’t think this solves the problem of multiple evaluations, but I need to brush up on my macro knowledge

Ed12:08:38

in your checkm you're evaling m twice which will often cause problems in macros. to get around this, you can use let and x# to generate a symbol to store the results of evaling x only once.

hanDerPeder12:08:49

good point! thanks. I’m now at this:

(defmacro checkm [m f expected]
  `(let [m# ~m]
     (is (= ~expected (~f m#)))
     m#))

(deftest foo4
  (-> {}
      (assoc :foo 10)
      (checkm :foo 10)
      (assoc :bar 10)
      (checkm :bar 10)
      (assoc :baz 10)
      (checkm :baz 11)
      ))
FAIL in (foo4) (NO_SOURCE_FILE:45)
expected: (clojure.core/= 11 (:baz m__96399__auto__))
  actual: (not (clojure.core/= 11 10))
This is usable, but not sure how to get the expected line to output the value of m__96399__auto__. ~m# isn’t possible. Makes sense since there is no symbol m# defined.

Ed12:08:11

how about

(defmacro checkm [m f expected]
  `(let [m# ~m]
     (is (= ~expected (~f m#)) m#)
     m#))

(deftest foo4
  (-> {}
      (assoc :foo 10)
      (checkm :foo 10)
      (assoc :bar 10)
      (checkm :bar 10)
      (assoc :baz 10)
      (checkm :baz 11)))
?

hanDerPeder12:08:48

FAIL in (foo4) (NO_SOURCE_FILE:45)
input: {:foo 10, :bar 10, :baz 10}
expected: (clojure.core/= 11 (:baz m__96643__auto__))
  actual: (not (clojure.core/= 11 10))
This is acceptable, thanks for the assistance @U0P0TMEFJ!

Brett Rowberry13:08:27

If it weird that docstrings can't be added to defmethod ? Is that on the roadmap to fix?

Alex Miller (Clojure team)13:08:21

No, and no. The defmulti defines the function.

Brett Rowberry13:08:04

Thanks for the explanation!

Brett Rowberry13:08:22

I guess the reason it felt like an omission to me is that Integrant defines the defmulti . As a user, I add defmethods for my components.

Alex Miller (Clojure team)13:08:24

there is only one function, and it's docstring is defined by the defmulti - the defmethods are impls of that and there really is no place to hang a docstring (plus they are dynamically added on load)

👍 1
Ferdinand Beyer14:08:24

@U021RHDFFHN — you can think about it like this: Where would you expect to read the docstring? In API docs, IDEs, etc., you would call ig/init-key, and see its docstring, as defined with defmulti. It documents the contract. The only way to see documentation for a defmethod is in the code itself, here you could just use a comment as necessary.

👍 1
vemv18:08:45

Anyone regularly uses a solution such that (get m k) fails if all keys in m are of a different type of that of k ? e.g. m is a map of strings, so calling (get m :foo) is very likely a bug (due to a bad refactoring etc)

vemv18:08:01

Like anyone else, I can hack something, but I'm specifically asking if someone has had a long-time favorite solution.

Ferdinand Beyer18:08:36

I tend to not do type checking. Instead, I’d resort to spec or malli. Both of these also support instrumentation, where you can define what your functions expect and automatically throw assertion errors should you call the function with invalid parameters.

Ferdinand Beyer18:08:44

What you are describing sound more a safeguard against programmer mistakes. You should validate data from external sources anyway — again I’d recommend malli for this 🙂

Alex Miller (Clojure team)18:08:32

the main case where this happens is with sorted maps, which have a comparator that makes type assumptions. should not happen with regular maps.

Alex Miller (Clojure team)18:08:56

Clojure follows Java in throwing in this case

vemv18:08:19

With help of the duckie I got this:

(fn [table]
  {:pre [(check! [:map-of :string :any] config-table
                 :string table)]}
  (-> config-table
      (get table)
      :foo))
...which probably will do the job, however it illustrates why @U031CHTGX1T's answer doesn't quite address my question Here the check is at the fn boundary, but I want it at the get boundary. In this example it might look the same, but if the get happened to be far away from any function boundary, it would not be nearly as easy. I think get, get-in etc could get some official love from spec or malli. They're functions where any type-like facility meets high dynamism so they're hard to get right

Ferdinand Beyer18:08:33

Not sure if I follow. get does not care about the structure of its map. It is totally OK to give it keys of different types. What you want is that your own function is guarded against being called with an invalid table argument, so it’s your function that should check or be spec’ed

💯 1
Alex Miller (Clojure team)18:08:49

user=> (def sm (sorted-map 1 1 2 2 3 3))
#'user/sm
user=> (get sm :a)
Execution error (ClassCastException) at user/eval1 (REPL:1).
class java.lang.Long cannot be cast to class clojure.lang.Keyword

👀 1
vemv18:08:35

> It is totally OK to give it keys of different types. It is technically OK (and a poster child of rich's ideas) but spec and malli both have map-of which shows how it's very often pragmatic to keep maps homogeneous. Anyway I'd want to abort any possible discussion before it happens - we all have slightly different ideals about types. (If anyone has a specific answer for my get question, that's welcome!)

Ben Sless19:08:13

Does it have to be get, or is it okay to provide a protocol, for example?

vemv19:08:11

I'd be interested in a generic mechanism. A protocol might be ok if it's not domain-specific

Ben Sless19:08:44

use a protocol, extend-via-metadata, stick a validator on an instance

1
Alex Miller (Clojure team)19:08:32

I feel like this whole idea is unnecessary

4
vemv19:08:50

Well, much of the point of gradual typing is to spec things as much (or as little) as one deems necessary.

Ben Sless19:08:52

you want gradual typing at runtime?

vemv19:08:50

I use the terms "types", "gradual typing" etc liberally

vemv19:08:33

> use a protocol, extend-via-metadata, stick a validator on an instance I can only approximately imagine this - care to share a snippet?

Ben Sless20:08:54

(defprotocol IFoo
  :extend-via-metadata true
  (-get [m k]))

(-get (with-meta {:a 1} {'user/-get (fn [m k] (assert (keyword? k)) (get m k))}) 1)

vemv20:08:55

Could be. I also thought in the meantime of a function that transformed a schema into a get ter

(def Config [:map-of [FancyString :map]])

(def get (schema->getter Config))

(get m "foo") ;; fails if "foo" it's not a FancyString
the cool part being that schema->getter is completely generic you could go as far as linting that you always use generated get s (if one is into castle building and all that)

john21:08:45

If you limit yourself to primitives, you might also get the benefit of a more efficient thing

john21:08:29

with an extra deftype

didibus23:08:42

Nope, but if I cared, I would use malli or Clojure spec and validate the map against the spec in key places.

didibus23:08:18

I don't understand why you'd need to validate this everytime you call get? Seems unnecessary? Either validate it after you've inserted into the map, or like in some key places where the map is received, why on each get? On the get you might want to just validate that the key exists, but not that like all other keys are of the expected type, that seems unnecessary and overengineered

john23:08:43

enforcing string on the get is probably faster than actually performing the get, I bet

didibus01:08:30

Well, if I understood it's enforcing that every key in the map contains a string no? Not just the one from the get? Maybe I misunderstood

john01:08:39

I just read it as a request to enforce the get but yeah I agree, for whatever purpose you'd probably want to enforce on the insertion side

john01:08:45

If you're not making a new type though, and instead just dressing up the insertion on a plain map, you might as well make a custom get too, to not waste lookups on keys that aren't strings

john01:08:33

Ordered map and specs/malli are probably just as easy though

didibus02:08:40

Ya, I guess you could use meta to tag a Map<string, any> kind of "type" and have a get that validates you're giving it a key of the right type and getting back a value of the right type. If that's all you want to enforce.

didibus02:08:43

Maybe, you just tag the map with it's spec? And then have the get validate the spec on read.

didibus02:08:49

It kinds of break the conceptual thought behind spec though. In that, normally with standard "types" things have types, the type of the thing is part of the thing. But with spec, it's more that at a certain time you ascertain that the thing is now in the form you expect at this time. It's a different mindset. You don't go, what type is this map, you go, is this map what I want it to be at this moment.

didibus02:08:34

Practically it means:

(get m k ::spec)
;; Get the value at key k from m, and make sure it's valid to our spec.
Whereas with types it would be more like
(get m k)
;; Get the value at key k from m, and make sure k makes sense for the type of m.
The difference is about what owns the "type". Is it the code calling get that is in charge of defining the type of what it wants. Or is it the code that creates m that is in charge of defining the type that the code calling get is allowed to use?

didibus02:08:48

That's one point I understood from the Maybe Not? talk. In the classical "type" style, you either begin to overload your types with a bunch of optional things, that will be mandatory but only in certain places/times so you have to make everything optional eventually. Or you have to create a bunch of iterations of the type, NotRegisteredCustomer, RegisteredCustomer, RegisteredNotVerifiedCustomer, etc. Which gets you closer to spec, but also couples it a bit with the structure and more verbose, and doing combinations become hard. With spec you could do, here I need a map who has the key/values that a registered customer should have and those that a verified customer should have.

didibus02:08:45

Okay, I'm taking a tangent here, I'll stop haha.