Fork me on GitHub
#clojure
<
2022-10-31
>
skylize01:10:05

Is this a bug or a mistake in the docs? > Sequential destructuring represents a sequential data structure as a Clojure vector within a let binding. > > This type of destructuring can be used on any kind of data structure that can be traversed in linear time, including lists, vectors, and anything that can produce a sequence. > https://clojure.org/guides/destructuring

(let [m {:foo :bar}
      [a] m]
  a)
=> Execution error (UnsupportedOperationException)

skylize01:10:43

(Works fine if I call seq manually.)

(let [m (seq {:foo :bar}) [a] m] a)
=> [:foo :bar]
(let [m {:foo :bar}) [a] (seq m)] a)
=> [:foo :bar]

Alex Miller (Clojure team)01:10:24

that is a bug in the docs and it should say "anything sequential"

Alex Miller (Clojure team)01:10:59

really it is anything that supports nth

Alex Miller (Clojure team)01:10:43

the reference docs are at https://clojure.org/reference/special_forms#sequential-destructuring and say more precisely "Vector bindingform_s sequentially bind values in collections like vectors, lists, seqs, strings, arrays, and anything that supports nth."

šŸ‘ 2
Alex Miller (Clojure team)01:10:26

I've pushed a fix to the guide page to sync that up, will be fixed in a few minutes

šŸ™Œ 4
Lior Neria08:10:49

Hi šŸ‘‹, I need to implement a function that iteratively goes through all the folders that are under resources and reads all the json files that are in each subfolder and unites them into one file - so that each subfolder will have one json file. I was able to implement the function in the meantime only with a reference to the specific path and to combine two Json files only. Is anyone knows how I can run iteratively on all the subfolders? Thank you :)

Martynas Maciulevičius09:10:40

What I noticed is that when I use throw in my code I tend to pass debug-info parameters that are too specific for the code but I still want to have this context when the error happens. Example (this is deliberately abstract code):

(defn check-number! [a b debug-info]
  (if (<= a b)
    :ok
    (throw (ex-info "Omg! Radically illegal input!" {:info debug-info}))))

(defn validate-age! [age max-age]
  (check-number! age max-age "age is not valid"))
Alternatively I could return the results to the outer function to decide what to do with them but then I'd lose stacktrace information. But then I'd be using more tuples all over the place. :thinking_face:

magnars09:10:09

Maybe you could throw without the debug-info in check-number! , catch the exception outside, and re-throw with (ex-info "..." debug-info inner-exception) ?

p-himik09:10:08

I usually only log values that are available only at run time. For everything else, a stack trace is almost always enough. In the OP, "age is not valid" is not really needed unless it becomes a user-facing string at some point - you can figure that out yourself by the fact that validate-age! is in the stack trace.

Martynas Maciulevičius09:10:32

Probably you're right. It's not that I want to have very many of these debug-info things. I simply noticed that I did it more than once. In my case I report this data through the API so that devs that are there could look into it when they use the API by hand. This is always internal use. So in this case the string is actually some kind of nested data. If I'll wrap it in catch-catch-catch then the best thing to do is to wrap exceptions into ex-info further and further (the Java way). But this further wraps the errors and make it badly readable by hand and also by machine if they'll want to log them themselves. I didn't look for a solution because it's the matter of taste. I just wanted to share as it may be an interesting thing to think about.

dpsutton13:10:27

you could look at making a dynamic binding and a helper function that would check if any useful bindings are there? (binding [info {ā€¦}] ā€¦) and when you throw (throw "bad stuff" (helper-fn {:additional :context}) and the helper-fn would check for stuff in your binding[s]

šŸ‘ 1
jmckitrick13:10:25

Is there any useful way to include doc strings in each of the methods of a multi-method? And can IDEs use them?

jpmonettas13:10:42

I don't think it makes sense to have different doc strings, since they are the same fn (from a callers perspective)

eskos13:10:27

defmethod indeed does not support dosctrings at all; the logical spot for docstring in fact is used for https://clojuredocs.org/clojure.core/defmethod#example-542692c7c026201cdc3269cd for eg. stacktraces.

šŸ™Œ 2
vemv14:10:12

never had noticed that! @U8SFC8HLP

āž• 2
vemv14:10:03

if you read the defmethod source, you'll notice it boils down to a simple .addMethod call. So you could (.add-method x my-defn) where my-defn has a docstring, as desired. This also has the advantage of being an explicit side-effect (unlike defmethod) which is nicely placeable in a Component or such.

jmckitrick14:10:31

Well, in my case, I have a ā€˜pre-validateā€™ multimethod then a ā€˜post-validate-actionā€™ multimethod. Depending on the dispatch value, the action taken is different, and Iā€™d like that to be in a docstring rather than just inline comments.

eskos14:10:28

If you long for more CLOS like multimethods, thereā€™s https://github.com/camsaul/methodical. It has support for auxiliary before/after/around methods. It also does more with dosctrings in general.

jmckitrick14:10:07

Yeah, I miss CLOS. But not enough to bring in another library just yet.

didibus17:10:58

Only the defmulti can have doc-string, so you have to put all the docs there.

Sam Ritchie19:10:09

Another use case for this is overloading some mathematical method like ā€œ+ā€ as ā€œassociative combineā€. It would be great for documenting what + means for each data type

Sam Ritchie19:10:35

But I see why itā€™s not supported, it should be an extra metadata repository

didibus20:10:47

I think it could have a feature that doc on the method like appends itself to the doc-string of the multi. But otherwise, you wouldn't really be able to lookup the doc for a specific method, when you lookup doc all you have is the Var.

Sam Ritchie20:10:43

right, or for specific cases you could browse a dispatch table; what I would find REALLY helpful is ā€œjump to the defmethod that was triggered for these inputsā€

didibus20:10:23

You can already kind of do it yourself with an extra step:

(defmulti add
  "Adds two things of various types together."
  :type)
(defmethod add :number
  ([{:keys [a b]}] (+ a b)))
(alter-meta! #'add update :doc
             #(str % \newline
                   "When type is number: adds :a and :b together as done by +"))
(doc add)
-------------------------
user/add
  Adds two things of various types together.
When type is number: adds :a and :b together as done by +
Something like that. Could make a nicer util out of that alter-meta! with better formating, and you've got what you want.

šŸ‘ 1
jpmonettas20:10:12

I guess it depends on your use case but the multimethod system is also dynamic, you can add or remove methods on the fly

Sam Ritchie20:10:48

@U01D37REZHP did some nice work on a Sussman inspired generic dispatch system which is very explorable

Sam Ritchie20:10:12

Iā€™ll probably take it into #sicmutils at some point!

didibus20:10:52

@U017QJZ9M7W You mean like you'd like your IDE to have that feature? Like a go to definition that prompts for an input?

didibus20:10:15

Ya, I admit, more IDE support would be cool, or even just a got-to that shows a list of all defmethods and let you choose. Not sure if Cider has any of that.

ā¤ļø 1
David G23:10:39

Hi, I wanted to get peoplesā€™ opinion to try and settle some other discussion Iā€™ve had someplace else around using with-redef or more precisely on (A) stubbing out methods vs. (B) having functions that act on the input data and then only changing the input data. What is the usual best approach?

plus_one 1
David G23:10:55

I think (B) is the better approach because: 1. It promotes more blackbox testing/approach, i.e. more implementation agnostic 2. More readable because you need to worry about the input data and not the implementation (kinda similar to #1) What are your thoughts on this topic? Are there any resources that go into this topic that you know of (other than ā€œitā€™s the best practiceā€)?

timothypratley01:11:32

Hi D G, My thoughts are that both are supported and there isn't an established "best practice" in Clojure or the Lisp community at large. One can broadly categorize a preference for passing all information through a call chain vs dynamic bindings, but I think it's hard to say always prefer one over the other because: A) dynamic bindings exist and in some situations are the only way to achieve a desired outcome B) passing all information can be a little tedious FWIW I do think it is best to avoid dynamic binding when that is under your control. But there are situations where it is convenient, or required by upstream libraries. The case that dynamic vars are bad: - Why not make the interface explicit instead of hidden? - Relying on the existence of globals is suspicious - Functions that use a dynamic binding have a hidden dependency/interface which is poorly defined Where are Dynamic vars used? - *out* *err* - Probably a good call for convenience of printing things - Have reasonable way to circumvent with-out-str - *db* but I'd rather pass it as an argument - *error* functions https://youtu.be/zp0OEDcAro0?t=1360 - Mocking, monkey patching Mocking and monkey patching deserve a little more attention. In Clojure we always have the option to use with-redefs or to modify the global environment in such a way as to replace things that were not declared as intentionally being dynamic. This is convenient when for example an upstream library has a bug that you can't easily avoid and don't want to wait for a patch/release cycle. You can just redefine some behavior globally and away you go. It's convenient for testing for mocking out parts of the system that aren't relevant to the test. There is a tiny gotcha here... if you run tests in parallel that use with-redefs instead of bindings you might be surprised to discover that they can interfere with each other because with-redefs changes global root bindings. It's easy enough to solve this by mutating globals to be dynamic if parallel tests are important, but I think most people just do serial testing. TLDR I do think it's best to go with B as much as it is possible/convenient to do so, but A is a nice card to have when you need it šŸ™‚

didibus02:11:09

(b) is favoured in general in Clojure I believe, that is, have pure functions that only use pure functions, and when you tests those, you only need test input -> output and don't need to stub the functions used by the function as it's pure all the way down. (a) is needed when a function isn't pure and uses impure functions.

David G03:11:09

Thank you both for your 2 cents. I agree with try B before A, B just sounds the more "functional programming" way to me. One way to enable with-redef-less mocking would be to use protocols & records, where one would pass a mock record instead of the real one (not sure what's this pattern is called). Would you say the tediousness in passing clients as arguments is implementing this setup/framework and also creating mocked clients? Or does it usually come to down to something else being tedious? (Talking in the context of a simple example of having some DB client you're trying to mock

David G04:11:44

+ In the long term, is it true to say in most cases that stubbing out functions would cause more work to be done as the implementations take different forms rather than investing in organizing the code/tests frameworks to support "blackbox"/input->output testing?

timothypratley04:11:27

šŸ‘ Regarding "tedious" I would say that (a) it can lead to a feeling that now all these resources are getting handed down multiple layers explicitly is repetitive and (b) it can be annoying to have to specify things that should have sensible defaults... but both of these are straw arguments IMO because Clojure has excellent support for passing a single map argument that can package up resources and extensions, and for chaining calls over things like that. Said another way; I personally don't find it more tedious to pass stuff explicitly, but I think that dynamic globals are often touted as reducing those (as evidenced by many libraries choosing that path). My view is that I'd rather have a passed environment than a bound environment. Having said that there really isn't any difference and it's easy to make use of either existing pattern where the implementation already exist. IMO creating mocks/setup/framework is the same amount of work either way. It should be a goal to separate side-effects from pure functions, but I think the conversation necessarily goes beyond pure functions to functions that are doing side-effects, but where there are different side-effects in testing than in production. Regarding the long term stubbing vs input-output, I don't see much practical difference (I don't have any quantitative or qualitative assessment). An orthogonal concern is the effectiveness of integration testing vs unit testing, but I think it's a good thing to bring up as relevant to separating side-effect code from pure code. I personally believe that integration tests tend to be more effective long term, but that unit tests are helpful during development as a workflow thing (I'd rather press my re-run test key than my send to repl key). I do think it is important to try to minimize the footprint of side-effect code which will need to receive an environment (passed or bound). And further that it makes it very clear then the type of testing that will be effective in each scenario (unit tests for pure functions, integration tests for environment dependent functions).

David G05:11:43

Thanks for your input on this!

didibus16:11:09

I've found that with-redef works better than injection. When you inject, you have to mock the interface of the thing injected, but you only want to test the function. The thing injected could be used by functions further down that the function calls another that then uses the injected thing. Also mocking that thing you inject can be trickier, depending what shape it takes. If you with-redefs the functions your function under test calls, the scope of mocking is focused on that function alone, you don't really need to know anything of what is injected or what functions down below do in order to mock or test it. I've found this to be both easier and so a lot faster to write your test, but also more reliable, the test is less likely to break, so for example if a function down below suddenly decides to use one more function from the injected component your test doesn't break because you hadn't mocked that.

didibus16:11:39

So normally I have 5 levels of testing, the higher levels take more effort and so I don't have as many of them: 1. Unit test pure functions directly, no mock or stubbing, call the function with test input and assert output is correct. Sometimes also do generative property testing on those. 2. Unit test impure functions, look at the functions that function calls, and for each one that it calls that is impure, do a with-redefto mock or stub it. The ones it calls that are pure leave them be. If it calls a Java interop method that cannot be with-redef wrap that function in a Clojure function that does nothing else but proxy the call to Java and with-redefthat instead. If the function it calls is part of a Java instance class, have that be injected into the function and provide a Java mock using Mockito or use proxy to a similar effect. 3. Functional test impure functions, that means don't mock the impure things they call, instead try to mock the effect-full environment. This would be for example providing an in-memory version of the DB instead, or swapping to a mocked Email client that does nothing when called. This lets you test the integration of units together internally, but does not test integration with outside components, because it does not actually call the real DB for example or the mail server. 4. White box integ test impure functions. This is a test that will connect to a fully running development environment, so a real database is called, a real email server, and all that. It is white box because it can be done on inner functions, it doesn't have to exclusively use public APIs/user interface of my application, though it also can. And it can assert the correct behavior by having special access to the environment as well, it can inspect the DB directly for example to check if the change was done correctly. 5. Black box integ test impure APIs/user interface. This test will also use a fully working devo environment with a real DB and all that. But unlike the White box one, this is black box, it means that you can only test through the public APIs/UIs the application exposes, and cannot access anything that can't be accessed by a normal client/user. Basically it is a test that replicate what a user would do and how a user would assert things are working, but automated. So it calls an API and then check the response is correct, but also maybe call another API after and see that the second API responds with what you'd expect giving the call to the first API you made, etc.

didibus16:11:12

If you look at this, "level 1" and "level 4" and "level 5" are the only levels that don't mock or stub anything. A test that does not mock or stub is way better, it's basically as close to prod as you get, testing everything. Those offer you the best coverage. But because "level 4" and "level 5" are really complex to setup, write the test and maintain the test, you're not going to be able to have a lot of those if any at all depending on how much time and effort you've got. But "level 1" is both the cheapest and easiest test to write and maintain, but also gives you perfect coverage like the "level 4" and "level 5" ones!!! So if you can write your code so most of your code falls in the "level 1" test group, imagine the benefit!

David G06:11:02

I think in general I agree with the test level and the difficulty as the high up you go in the system test. I think that code that accommodates/facilitates testing by injection would need to have a high degree of separation of concerns, although the same pattern would also help stubbing out be easier. Would you agree that stubbing out would preferably be eliminated when itā€™s convenient? It just sounds to me the lower you stub in the call stack of whatever thing youā€™re trying to test, then the bigger the likelihood that the test would break due to an implementation change. Iā€™m coming with the assumption that 1. contracts/protocols probably usually donā€™t change as frequent as implementations do, and 2. injecting can help the reader with abstracting implementation details/making the tests more focused (with the assumption that thereā€™s a test framework to help with that). Plus thereā€™s a risk of stubbing out the wrong methods and actual things happen in the prod env

David G06:11:35

I certainly don't think there's a single way of doing things, I'm just trying to hear some of the arguments for some of the points you guys mentioned, like I said most of the time it depends on quite a few factors

didibus16:11:14

I think if you have impure functions, you have to inject dependencies. And then you can pick if mocking the dependency it uses, or mocking the impure functions it calls with with-redef is easier. Both are valid, but I find you want to mock close to the function ideally, so if the function uses the dependency directly you could mock that, but if it passes it down and eventually something else uses it, I would prefer to mock the function it calls and passes the dependency too.

šŸ‘ 1