Fork me on GitHub
#beginners
<
2022-12-17
>
sheluchin12:12:38

What is the right way to write a unit test for a private function? Is there something different I have to do vs a regular function? Is testing a private function even something I should be doing?

valtteri13:12:46

It’s quite common to use public/private functions to distinguish between namespaces “public api” and the implementation details. In unit tests you can resolve the private vars like this

user> (defn- my-private-fn [] "foo")
#'user/my-private-fn
user> (in-ns 'other-ns)
#namespace[other-ns]
other-ns> (user/my-private-fn)
Syntax error (IllegalStateException) compiling user/my-private-fn at (webapp:localhost:57463(clj)*:5:11).
var: #'user/my-private-fn is not public
other-ns> (#'user/my-private-fn)
"foo"

valtteri13:12:09

First there’s an example how it fails if you try to call the function in a “normal way” but if you use #' reader macro, then it’s same as doing ((var user/my-private-fn))

valtteri13:12:29

And it will resolve the var where the function is bound to and call it.

valtteri13:12:50

So you would write unit-tests the way you would normally do but when calling private functions you need to do that little trick to access them.

sheluchin13:12:00

That is how I have been trying to do it, but with some mocks involved and using a testing library. I guess my error must be somewhere else then. I have a test like this:

(let [env {::db/postgresql {:datasource (fix/ds)}}]
   (behavior
    "A generic private function does the formatting and calling"
    (when-mocking
     (sut/format-stmt stmt) => (do (assertions
                                    "Arguments"
                                    stmt => :honeysql-data)
                                   :formatted-raw-sql-str)
     (assertions
      "foo"
      (#'sut/-execute! :_ :_ :honeysql-data :_) => :TODO))
and my result is:
expected: (= 1 (#'sut/-execute! :_ :_ :honeysql-data :_))
  actual: clojure.lang.ArityException: Wrong number of args (3) passed to: :-

valtteri13:12:40

Well, the error message tells that something ends up calling keyword :- with 3 args. 🙂 Nothing to do with the private var thing.

practicalli-johnny13:12:49

Personally I prefer write tests only for public functions, as they are the public interface to the namespace. Private functions are tested by proxy, as the public functions will call one or more of the private functions when being tested. I will typically validate private function behaviour by calling them in the REPL as I am designing those functions. Then convert any useful call data into a unit test assertion for the respective public function(s). Writing a unit test to everything adds to the maintenance responsibility of the code base, so I would only add a unit test for private functions if there was real value in doing so. There are lots of techniques you can use to do this, but they all add code to maintain.

👍 1
valtteri13:12:07

Good point, it’s always good to estimate how much each test adds value vs. the effort!

👍 1
sheluchin13:12:45

Thank you for weighing in, @U05254DQM. It's a good point and choosing what to test is always a judgement call. I'm used to doing lots of testing in Python, where 100% code coverage is often seen as the standard. In Clojure, that seems to be very far from the case. Testing in the REPL as I write the code seems adequate in so many cases, but the other side of the "tests add maintenance" argument is that having more tests gives you certainty and leverage.

sheluchin13:12:41

@U6N4HSMFW when is it necessary to use @#'sut/my-prv-fn rather than #'sut/my-prv-fn?

Ben Sless13:12:04

Alternatively, I don't believe in private functions. I may just put them in a different namespace

1
👍 1
valtteri13:12:27

Good question @UPWHQK562! Deref (`@` ) will pull out the actual value from the var. #' just resolves the var. A var happens to also be something you can “call” and that’s why function calls work without deref. So basically you need to deref when you need the actual value the var points to and not the var itself.

user> (def ^:private my-private-num 13)
#'user/my-private-num
user> (in-ns 'other-ns)
#namespace[other-ns]
other-ns> #'user/my-private-num
#'user/my-private-num
other-ns> @#'user/my-private-num
13
other-ns> 
☝️ this illustrates the difference better

🙌 1
valtteri13:12:15

Also..

user> (type #'my-private-num)
clojure.lang.Var
user> (type @#'my-private-num)
java.lang.Long
user> (type #'my-private-fn)
clojure.lang.Var
user> (ifn? #'my-private-fn)
true
user> (ifn? @#'my-private-fn)
true

valtteri13:12:25

..and

user> (type @#'my-private-fn)
user$my_private_fn

sheluchin15:12:49

Thanks very much.

seancorfield19:12:36

I'll echo @UK0810AQ2’s point: at work we use Polylith so the "interface" is all public and that's the API that is callable by other code and the "impl" is also usually all public because it is only ever referenced from the "interface". We do still have some private functions, but it's mostly a holdover from before we moved to Polylith, to be honest.

Clojure build/config 22 files 406 total loc,
    174 deps.edn files, 3227 loc.
Clojure source 546 files 107890 total loc,
    4772 fns, 1105 of which are private,
    640 vars, 44 macros, 103 atoms,
    86 agents, 22 protocols, 65 records,
    820 specs, 25 function specs.
Clojure tests 550 files 26265 total loc,
    5 specs, 1 function specs.
That's an overview of our codebase.

sheluchin19:12:55

I haven't started looking into Polylith yet, so I don't understand what it contributes to the context. What is it helpful about avoiding the encapsulation features of the language? What are the benefits of doing the "private" stuff in its own NS rather than just using the public/private functionality that the language offers?

seancorfield19:12:56

Private is an illusion -- it's informative at best -- because you can easily call private functions and get at private Vars.

practicalli-johnny20:12:32

I concur. Private was never that private even in OO languages like Java. Using Private in Clojure adds little value to the overall design. It maybe useful for some static tooling, but I would rather organise my code in other ways. If I have a number utility or helper functions, I tend move them into a separate namespace, making it easier to understand the public interface of a namespace So the util namespace acts as a way to separate functions that aren't the public API without having to add extra metadata on those until functions (and they can have unit test if really necessary without any code tricks)

sheluchin20:12:34

I understand that it's possible to work around it, but it seems to me like the added constraint and necessity to get around it can be useful in communicating intent. Having to use tricks to test stuff is definitely a down side, though. So are the impl.* namespaces usually just a form of what you guys are talking about? Collections of "private" functions without relying on metadata to communicate the encapsulation intent?

Ben Sless20:12:49

The thing is, do we want encapsulation? Why would we even encapsulate state with behavior? You might want to hide implementation detail and communicate to the user "don't rely on this", for which the impl prefix works just fine

Vincent Olesen15:12:24

Is there an idiomatic way to go from

[
  {:valve "AA", :flow 0, :tunnels ["DD" "II" "BB"]}
  {:valve "BB", :flow 13, :tunnels ["CC" "AA"]}
  ...
]
to
{ "AA" {:flow 0, :tunnels ["DD" "II" "BB"]}
  "BB" {:flow 13, :tunnels ["CC" "AA"]} ...}

Alex Miller (Clojure team)15:12:18

(update-keys :valve the-map) ?

nbardiuk15:12:55

I would probably map into pairs of key, value and collect them into a hash map

(->> [{:valve "AA", :flow 0, :tunnels ["DD" "II" "BB"]}
      {:valve "BB", :flow 13, :tunnels ["CC" "AA"]}]
     (map (juxt :valve #(dissoc % :valve)))
     (into {}))

Alex Miller (Clojure team)19:12:24

That is both more complicated and slower than update-keys

nbardiuk19:12:58

update-keys is good when input is a map, we are missing one step to transform vector into a map

Alex Miller (Clojure team)20:12:29

oh, I didn't even notice that - between my eyes and a phone, could not tell :) but in that case, most direct would be a reduce

Alex Miller (Clojure team)20:12:48

or into with a transducer

James Pratt09:12:04

(reduce (fn [m-out {valve :valve :as m-in}] 
          (assoc m-out valve (dissoc m-in :valve)))
        {}
        [{:valve "AA", :flow 0, :tunnels ["DD" "II" "BB"]}
         {:valve "BB", :flow 13, :tunnels ["CC" "AA"]}]
        )

(into {}
      (map (fn [{valve :valve :as m}] (hash-map valve (dissoc m :valve))))
      [{:valve "AA", :flow 0, :tunnels ["DD" "II" "BB"]}
          {:valve "BB", :flow 13, :tunnels ["CC" "AA"]}])

James Pratt09:12:59

Thought I'd have a go at this over my morning coffee.

phill18:12:32

Also consider whether it is necessary to "remove" :valve. It's less effort to put a new map around those existing 3-key maps than to also create new 2-key maps.