Fork me on GitHub
#clojure-uk
<
2020-02-05
>
practicalli-johnny00:02:50

definining things as private cam be worked around, as many have done when writing tests. My preferred approach is to have a naming scheme to denote names that should only be used within the current namespace. A subtle naming is to simply add - at the end of the name. This also shows up in autocomplete popups (as it's part of the name). I also organise my code in sections using comment blocks, e.g system, helper functions, domain model, public API functions. An additional benefit of this is it makes separating code into different namespaces a trivial thing, as you have already demarcated sections of your code. It's easy to visually see when a section has evolved enough to merit its own namespce.

16
dharrigan06:02:06

That's good advice John šŸ™‚

guy07:02:08

Morning!

guy08:02:04

Thats great news

dominicm08:02:42

Totally think someone should download those videos and archive them somewhere

dotemacs16:02:10

My first thought as soon as I read the tweet above!

guy08:02:12

mmm could potentially be some IP issues with that tho?

guy08:02:21

unless you mean they do it sneakily haha

dominicm08:02:43

if necessary ĀÆ\(惄)/ĀÆ

dominicm08:02:10

It would be nice to have them available independently of whether they're bankrupt

thomas08:02:25

morning and good to hear that Skills Matter is back.

maleghast09:02:43

Morning All - what lovely / fantastic news about Skillsmatter!

wotbrew10:02:54

Forgive the xpost but just letting you know Riverford is hiring šŸ™‚ see #jobs !

maleghast10:02:02

@danstone - That looks like a fabulous job; in another life I'd have been all over it, but I live in Scotland and that's not going to change, and I do actually like my current job very much too, so that's not going to change either, but still, wanted to say "nice one" šŸ™‚

āž• 4
Ben Hammond10:02:44

haha > Farm To Fork with the Clojure Stack

šŸ„• 20
jasonbell11:02:35

Morning

šŸ‘‹ 8
Aleksander11:02:53

<HTTPS://skillsmatter.com|HTTPS://skillsmatter.com>

šŸ‘ 24
Aleksander11:02:20

It's coming back!

Ben Hammond12:02:43

> There were moments of gold and there were flashes of light > There were things I'd never do again > But then they'd always seemed right Oops. šŸ˜Š Word Association (Football)

Wes Hall12:02:07

@dharrigan Yeah, I would add that I have also used @jr0cket's naming scheme approach in the past too. You can usually tell a Haskell guy because they add ' to the end of a lot of names and say "prime" a lot šŸ˜‰. I've also done the thing where I have marked things as private with (defn-) or equivalent and hacked around this in the tests with reader macros but I always feel like I am essentially documenting how to bypass the privacy modifier in the tests, and can remember at least one instance where somebody saw this "solution" in the test and concluded it was fair game to use it in the code proper, so I went cold on it as a technique, but YMMV.

dharrigan12:02:15

I started to play with both approaches - one by adding metadata to the functions to say private (as this seems to be the preferred way and not using (defn- and then by adding a - to the end of the functions. In the end, I think I agree more with your approach. Every function should stand alone and be testable, thus with some good docstring for public functions, then no need for privacy šŸ˜‰

seancorfield18:02:56

@dharrigan Where did you see this preference expressed? > adding metadata to the functions to say private (as this seems to be the preferred way and not using (`defn-` I've seen comments that, given a do-over for Clojure, defn- would almost certainly never have been added, but I've not seen anyone discourage its use in favor of ^:private

dharrigan19:02:23

You're right, I don't recall it being explicitly stated, but I inferred it.

dominicm19:02:29

Using the metadata is for alternative universe theorists.

dominicm19:02:54

If you behave as if you're in the alternative universe, the barriers fall and you move into that universe

dharrigan19:02:54

There is this from Alex:

dharrigan19:02:15

So I guess he likes it still šŸ™‚

seancorfield19:02:43

The context of those discussions is usually around why Clojure (core) doesn't have equivalents to defn- for def and defmacro -- and that's when you'll hear "If we did Clojure again, we wouldn't include defn-"

seancorfield19:02:03

Oh, and there it is from Alex in that very thread: "We will not be adding any other "-" forms to core. I think perhaps if we could do it over, defn- would not be added either, but we're not going to take it away."

seancorfield19:02:42

Oh... and there's me in the Google thread linked from that issue, way back in 2012, saying "It seemed to be consensus that defn- was probably not a good idea but was too common to remove..." šŸ™‚

dharrigan19:02:19

You've been around a bit šŸ™‚

šŸ˜„ 4
Wes Hall12:02:47

It works for me. I don't know that I would have it as an "absolute, don't break under any conditions!", rule because those tend to be bad anyway. I can probably conceive of a situation where I want to a) factor out some code, b) am intended to release said code onto an unsuspecting team and/or public and c) really don't want to have to maintain this function when somebody uses it directly when they shouldn't... I just think I would try every other option before allowing these circumstances to occur. I suppose my real rule is, "private functions should be a last resort, rather than a standard practice".

guy13:02:16

I liked Johns idea above with the code comments tbh

guy13:02:29

So you keep it all as defn but then just comment it out

guy13:02:43

It still leaves it up to the user and means my tests can still work normally haha

Wes Hall14:02:37

@guy Not sure what you mean by "comment it out". I don't think that was the suggestion, just to use comments in your code to say which parts are private / public (and perhaps a naming convention). The notion of only using docstrings for the parts of the code you consider public is definitely fine in principal, but I have to say that I don't docstring all of my functions. There are times when I am not sure I could improve upon the function name for documentation purposes... but if the rule is, "docstrings = public", then this door closes. Maybe not so bad, but honestly... `(defn get-customer "Gets a customer" ...` Just serves to irritate me mostly šŸ™‚

practicalli-johnny13:02:47

@dharrigan I am not subscribed to the idea that every function must be unit tested driectly, especially if this slows down the feedback you get from tests (or stops the team running tests so often). Every function should be tested of course and any public API to your code should be tested rigorously (ideally generative testing). Private functions are often very small and generic helper functions, used by many other functions. Those helper functions are therefore tested extensively by the unit and integration tests of the public functions. If I did write direct unit tests for all functions, I would invest more in the design of those that test public functions and less in those tests for helper functions. After all unit tests require maintenance. However, I do think it is a lot less maintenance in Clojure to write unit tests for absolutely every function. Many languages struggle with the amount of unit tests over time. Following the typical modular approach in Clojure design and the more terse nature of the code, excessive amounts of tests. Organising your tests into categories is very effective, so only subsections of tests are run. This is especially so when you have integration and performance tests and the larger the codebase you have.

guy13:02:40

How do you balance out generative testing with time taken by the tests?

guy13:02:57

I've always found they add a signifcant overhead time wise (couple mins) vs unit tests

guy13:02:09

Or do you just not run them that often?

guy13:02:05

ah just reread what you said

šŸ˜ 4
practicalli-johnny13:02:08

If you can group together generative tests, then you can get your CI to run then and just run unit tests as a dev profile. There is plenty of opportunity to run generative tests locally, eg. just before a meeting, comfort break, getting a glass of water, lunch, etc.

šŸ‘ 4
guy13:02:21

ok cheers

Samuel13:02:59

Afternoon folks

Samuel14:02:19

Could someone point me in the right direction for embedding clojure in a C# application? I'd like to walk some datastructures as the application runs, and I can't bring myself to do it in csharp

Samuel14:02:43

cool, this is where I'm looking at the moment -- I'll make a note to update these docs if I run into any useful tricks

Samuel14:02:05

I'm deep in a .net 'framework' (win binaries only) application, but this is quite the feat

flefik14:02:00

thatā€™s one of the benefits of a build tool like Bazel (discussed earlier here this week). You never really get to the point where tests take too long, because tests are only ever run if something that it depends on changes. Dependency graphs in source code is generally quite sparse, so itā€™s actually unnecessary to waste clock cycles re-running all tests in CI if the results provably could not have changed due to your code.

Wes Hall14:02:29

@guy Generative tests for functions are a conundrum in themselves. Had an interesting conversation with @folcon about this (which I guess I am now opening up to the floor here), but using something like spec to do generative testing of functions always seems to lead to this wider question of how broadly or specifically you want to fn spec a function. If you have a function that sums two integers, do you want to function spec that it takes two ints and returns an int? Do you want to fn spec that it takes two ints and returns and int that is >= either of the arguments? (let's assume positives for argument sake ;)). Do you want to fn spec that the int returned is the sum of the arguments? If you do the latter thing (which, I tend to find is the natural inclination, particularly if you want your generative tests to be exhaustive, without having to write too many manual tests), then the result is you end up implementing your function twice. Once in the function body, and once again in the spec šŸ™‚. This, I find, is something of a hard problem, because it's quite subjective and difficult to communicate. If you rely largely on generative testing, your "coverage" is a factor of the specifity of your function specs... This, I feel, is not a great position to be in. I've never had a super clear solution to this problem.

šŸ‘ 4
Wes Hall14:02:58

If the goal of generative tests for your particular project is simply to ensure that functions never throw exceptions or return types that you don't expect, that is probably a good rule... but you need to find a nice clear space of wall to write this on in big letters šŸ™‚

rickmoynihan15:02:14

@wesley.hall: It can certainly be hard; but you should find properties that are worth testing or prove an invariant and only add them if they add value to finding bugs. You typically shouldnā€™t ever reimplement the function in the :fn spec; the only exception to that is if you have a test oracle alreadyā€¦ e.g. an existing 3rd party implementation, or a simpler algorithm that is easier to show correctness on; e.g. using bubble-sort as a test oracle for timsort, or javaā€™s timsort as a test oracle for your sort algorithm. Iā€™ve been down this path though; and I think the motivation for persuing it to absurdity comes from wanting spec to generate you :ret values that are realistic enough to use elsewhere. I found the way out of this bind is to keep the specā€™s relatively coarse; and just write :gen eratorā€™s that generate more meaningful values. You can combine generators together to cover the space more easily too.

flefik15:02:15

i live and die by the nil, 0, 1, n rule in testing. Iā€™ll try to test each of these cases by examples: Does (foo nil), (foo []), (foo [bar]) and (foo [bar baz]) return the correct values? For functions with multiple arguments I try to test each pair (http://www.pairwise.org/)

rickmoynihan16:02:10

šŸ’Æ Iā€™ve been interested in pairwise testing for a whileā€¦ Iā€™m particularly curious about the intersection between it and generative testing. i.e. model each ā€œnotable valueā€ as a test generator, then pairwise test across the models. For simple boolean arguments the model is just #{true false} in which case each value contributes to the pairing; but more complex ones with large or infinite value-spaces would be modelled into notable models; which are then generated. I havenā€™t thought about it too deeply; so no idea if itā€™s a good idea or not but I think it might be nice to guide t/check to at least generate all pairsā€¦ then when spending more effort spend it wisely investigating interactions between pairings in your models

dominicm15:02:56

I have heard of a machine configured to run generative tests 24h a day

Wes Hall15:02:56

@rickmoynihan Indeed. This is generally how I approach it too, but phrases like, "relatively coarse" can be a bit difficult to reason about. Even outside of the scope of generative testing. Simply using function specs to document function behaviour this problem can appear. There is the old adage about, "the code is the documentation". Naturally then, if I am trying to document the behaviour of the function in a spec, and the code is the documentation, then I will tend towards placing the code in the spec šŸ˜‰. This obviously feels redundant. So you are left with the problem of abstraction. The correct documentation for the function behaviour lies somewhere between "Takes some stuff and returns some stuff", and copy pasting the function body and the correct level of abstraction is not always obvious in my experience.

folcon15:02:00

I tend to go for the middle, ie, a + b = c, c >= a | b and find alternative properties... if you and specify a few that covers a lot...

folcon15:02:35

That ends up providing the abstraction in question while not "overfitting"

Wes Hall15:02:52

Yes indeed. This I think also informs another important principal which is that generative testing should augment rather than replace traditional written tests. This is a hard pill to swallow because the notion that "I can generate tests, so I don't have to write so many = YAY!" is a hard utopia to let go of šŸ˜‰. If there is more than one possible function that has the properties that your generative test checks for, then you still need to roll up those sleeves šŸ™‚

folcon16:02:46

I sort of expect that?

rickmoynihan16:02:09

you can in theory just bake your example based values into the generators; then you can get the best of both worldsā€¦. non-determinism aside šŸ˜

rickmoynihan16:02:34

in practice I think youā€™re best with both

folcon16:02:31

I think examples, static or with minor generation covers your ā€œhappy pathā€, generators help you flush out everything elseā€¦

folcon16:02:19

I mean fundamentally theyā€™re only as good as your ability to reason out classes of error, but once youā€™re thinking on that level, you can eliminate a lot of errorsā€¦

rickmoynihan16:02:27

@wesley.hall: I find the real problem is that choosing between coarseness vs specificity depends on who is asking. And also crucially on what you are using spec for. Is it a function contract? Or is it for testing? I find specā€™s complected nature here makes it ambiguous as to what concern any particular spec was written for.

guy16:02:53

Have any of you folks tried spec2 yet?

guy16:02:03

ive been meaning to try it out but just never have the time sadly

Wes Hall16:02:31

@rickmoynihan 100% on, "depends on the purpose". This of course is sometimes problem with the, "one thing does many things", design. Right now I tend to use spec mostly for validation and documentation than generative testing.

Wes Hall16:02:10

@guy Played with spec2. I have this kind of feeling about it that it seems that the problems have been identified but not absolutely sure about the solutions yet. I'm probably waiting for spec3 šŸ™‚

šŸ‘ 4
rickmoynihan16:02:43

yes Iā€™ve played with it. I really like the schema/select stuff; thatā€™s a big improvementā€¦ though I discovered a pattern to simulate in spec1.

šŸ‘ 4
rickmoynihan16:02:24

also I ran into a few bugs in itā€¦ they might well have been fixed now though

rickmoynihan16:02:37

but it was pretty usable

Wes Hall17:02:38

Yeah, I like the select stuff too. I still have this niggling feeling that it is an intermediate step to the "right" solution. I mostly use spec1 still because 2 doesn't quite generate the šŸ’”for me yet. At least not quite enough to switch.

Wes Hall17:02:58

I am sure Rich is in his hammock as we speak šŸ˜‰

dominicm17:02:14

Just discovered that test-var doesn't run fixtures! That was a head scratcher. test-vars does of course...

seancorfield18:02:44

@dominicm Yeah, the various test runners in clojure.test are a bit odd. (test-fn) is like (test-var #'test-fn) but then all the others run fixtures. The difference between test-ns and test-all-vars is also a subtle one.

seancorfield18:02:23

I mention it a bit in the Expectations docs (for the clojure.test-compatible version) but didn't dig too deep there.

seancorfield18:02:26

@guy I created a branch at work of our monorepo and managed to get it all running with Spec2 -- just making the changes to specs necessary to get things working but still using s/keys. That was a lot of effort and as Spec2 evolved, it kept breaking. Some breakages were bugs in Spec2, some were deliberate changes. In the end, I gave up tracking it because it was evolving too merge (to be fair, Alex has been extremely clear that it is not yet stable for production usage).

šŸ‘ 4
guy20:02:10

Thanks for the insight šŸ™‡

seancorfield18:02:11

(we've been heavy users of Spec1 ever since it was introduced -- in an alpha of 1.9, I believe?)

seancorfield18:02:33

Alex says that Spec2 may drop s/keys and may have a completely different solution for function specs so there may not be a migration path from Spec1 to Spec2 (although Alex has also hinted there may be an automated migration tool at some point... which... šŸ‘€ ). I think the schema/`select` approach is a huge improvement over keys, but I think our future adoption of Spec2 will be very incremental and we'll continue to rely on Spec1 for a long time...

seancorfield18:02:56

@dharrigan Where did you see this preference expressed? > adding metadata to the functions to say private (as this seems to be the preferred way and not using (`defn-` I've seen comments that, given a do-over for Clojure, defn- would almost certainly never have been added, but I've not seen anyone discourage its use in favor of ^:private