Fork me on GitHub
#clojure
<
2023-06-01
>
Skyb0rg02:06:34

I'm wondering how to move the location that Leiningen uses to cache dependencies, mainly to clean up my home directory. I asked https://www.reddit.com/r/Clojure/comments/13vrrl0/leiningen_moving_the_maven_local_repository/, but I'm pretty sure this is currently impossible. But I'm also looking for a side-project, and if there's any info on what package actually provides ~/.m2/repository to Leiningen so I can see if that's a feature I can add.

Bob B02:06:34

based on <https://codeberg.org/leiningen/leiningen/src/commit/64e02a842e7bb50edc9b8b35de1e2ef1fac090dd/sample.project.clj#L144>, I think :local-repo allows for overriding the repo location - it's commented as a relative path, so I'm not 100% sure what would happen if an absolute path were provided (in profiles.clj or something), but it might not hurt to give it a shot (and if you use maven at all, you might want to update settings.xml)

Skyb0rg02:06:19

That option does work! However that option doesn't change all the created directories, only some of them. I outlined the details in my Reddit post but Leinengen still creates and populates ~/.m2/repository, even with :local-repo set in profiles.clj.

hiredman02:06:02

~/.m2 isn't lein specific, it is a maven thing, and anyone that wants to work with maven also uses it

Skyb0rg02:06:19

That isn't true, as Maven allows for the configuration of the localRepository , and does not use ~/.m2 if you change it. Leiningen does not respect Maven's settings.

hiredman02:06:31

Lein almost certainly does, because lein uses maven libraries for fetching deps, etc

Skyb0rg02:06:28

I've modified my ~/.m2/settings.xml file to move the localRepository somewhere else, however Leiningen still dumps its downloads into ~/.m2/repository. I want Leiningen to not do that; Maven is perfectly fine doing so.

practicalli-johnny04:06:08

There is a Leiningen issue from 2016 to support the Free Desktop XDG standard, which would place Leiningen config in $XDG_CONFIG_HOME. Ideally it should also use $XDG_CACHE_HOME/maven to save the library dependencies to. https://github.com/technomancy/leiningen/issues/2087

practicalli-johnny04:06:52

The Clojure CLI can use:

:mvn/local-repo "/home/practicalli/.cache/maven/repository"
For Leiningen, I use a symbolic link and manually move all the files to the $HOME/.cache directory
mv $HOME/.m2/repository $HOME/.cache/maven/repository 
ln -s $HOME/.cache/maven/repository $HOME/.m2/repository
Ideally. The contents of $HOME/.m2 should be in $XDG_CONFIG_HOME/maven Obviously this doesn't completely address the original goal Most of the Clojure tooling does support the XDG standard though https://practical.li/blog/posts/adopt-FreeDesktop.org-XDG-standard-for-configuration-files/

Skyb0rg16:06:38

Thanks for the link. After looking into it, I think the underlying issue comes from pomegranate, the Maven API for Clojure that's used by Leiningen. I'm looking into submitting a pull request there, thanks!

👍 2
Paavo Pokkinen06:06:17

Which is more idiomatic Clojure: use exceptions, or return map with :error key?

🔖 2
🔥 4
Paavo Pokkinen06:06:08

Seen eg. clj-http using exceptions, but then also seen code like this in library:

(defn create-page
  "Creates a new page in Notion.
  Takes the authentication token and the new page's content/body as parameters"
  [token body]
  (if (validate-page-creation body)
    (let [response (post-page token (json/encode body))]
      (if (:error response)
        response
        (json/parse-string (:body response) true)))
    {:error "page-body doesn't match the page spec"}))
Which means I should check for :error myself

igrishaev06:06:55

It's better to throw an exception. Otherwise, everyone who uses your code should check the result with if/else and throw an exception anyway

daveliepmann07:06:08

Both are useful in different contexts, and can coexist

Hendrik07:06:49

Do you have some examples, in which context you use use :error key and in which exceptions?

igrishaev07:06:26

some HTTP clients accept an option usually called :throw-exception? to decide: either return an error response or throw it

igrishaev07:06:24

Try to imagine how the code looks like in case you got an error. Most likely you'll need if/else check

igrishaev07:06:30

Such kind of checks are usually noisy and consume time and attention. Moreover, you cannot recover from that error. If you failed to create a page, it's over

igrishaev07:06:59

Also, what if you call create-page in let? it would be quite inconvenient to check the result in bindings

henrik07:06:09

Useful when you need errors moving over the wire etc. If you use Cognitect’s AWS API, you’ll encounter these.

daveliepmann07:06:32

I don't have a clear, explicit decision tree for returning data versus throwing, but in general it's sometimes nice to return data and act on it, and sometimes necessary to stop work and throw. The traditional advice is to avoid using exceptions as control flow. Sometimes an exception can feel like a GOTO and sometimes it's the right way to give up on a situation so some other part of the system can clean up.

igrishaev07:06:29

Btw, we had cognitect/anomalies in a vast Clojure project and then got rid of them. They were quite inconvenient: you never know wether you succeeded or got an error map

valerauko07:06:56

I'd go with returning errors instead of raising exceptions. Other than having to try-catch the exception anyway (so might as well check for an error in the return value), exceptions are very expensive to throw so I avoid them whenever possible. I only use exceptions when it's some should-be-impossible error case when it's "okay" for the app to go down burning.

igrishaev07:06:26

we even had bugs when a developer would not check the result and proceed with the happy path

henrik07:06:43

We use them, for the above reason. In our case, we made a predicate for detecting them. For example, we might return entities. When there’s a problem, we’ll still return an entity (because it contains context), but add the anomaly keys. But not exclusively, sometimes an exception is fine and an anomaly is just convoluted.

2
valerauko07:06:08

if your developer fails to check for the anomaly they'll just as well forget to try-catch

henrik07:06:30

Yeah, same difference.

igrishaev07:06:43

that's right, but in this case we'll get a sentry alert with all the details

henrik07:06:27

I’m sending anomalies to Sentry as well.

igrishaev07:06:04

also, all web apps usually have a top-level middleware that captures all the exceptions and logs them

2
daveliepmann07:06:05

Henrik's mention of returning entities reminds me of a case where I preferred returning data: a pipeline which processes a bunch of items. If item 521 in a seq of 3000 items fails, I don't want to stop the presses. Just create an error map and return it so I can handle it in a later step as data.

henrik07:06:58

Yeah, that’s the kind of scenario where they are good. Sometimes you want to allow them to flow through the system and catch them at the edges.

henrik07:06:47

Or in our case, a client connects to the server and starts a workflow that is essentially stateful at both ends. A step by step process where a state machine on each side is updated. We use anomalies in this case to allow retries, or going down a different path etc. without disrupting the ongoing context.

agigao08:06:01

My approach is to throw ex-info:

(throw
  (ex-info "message" 
           {:description "...}))

👍 4
vemv08:06:54

If you're heavy on spec/malli instrumentation or similar approaches, :error makes sense because it's generally not possible to encode exceptions in those "type systems" (I know they aren't, d@m) Similarly, you cannot generatively-test exceptions, while you can spec/gen data, function calls, etc when :error is just data, like everything else. It can be interesting for simulating distributed systems (e.g. what happens on timeouts? Without actually waiting for a timeout) If you're not making any such investment in Spec/etc, you're better off with exceptions, simply because it's what everyone is doing in the JVM and Clojure. If/when performance matters, skip stacktrace generation, see e.g. https://github.com/fmnoise/flow/tree/4.2.1#isnt-using-exceptions-costly

Hendrik08:06:34

Using :error with generative testing seems like an interesting approach 🙂 I didn’t think of this use case, yet

practicalli-johnny08:06:33

returning a map with details feels more Clojurey that using and exception to me, certainly in general terms. Most of the functions I use are returning a hash-map or some kind of data structure that is easy to parse.

Ernesto Garcia08:06:19

I wouldn't consider returning an {:error ..} map idiomatic. Throwing an exception is, as well as returning nil .

Ernesto Garcia08:06:57

Also throwing exception-infos.

Paavo Pokkinen08:06:01

I didn’t expect such lengthly discussion to raise about this. Looks like there’s lot of different preferences here! 😁

🙌 2
Ernesto Garcia08:06:46

Yes, though you didn't actually ask for preferences, you were asking about idiomatic use, weren't you?

Paavo Pokkinen08:06:07

Reason why I asked was that I actually did have a bug in code, as I expected exception if something fails in a library, but it returned a error map instead.

igrishaev09:06:16

Funny: I just had a call with my teammate. He's lost an hour trying to dig an error because the initial exception was not thrown but silently logged, and the logging system didn't print it into the console. It was an outdated version of Migratus.

mpenet11:06:23

exoscale/ex is a decent mix of both approaches. It also enables use of exception hierarchies with ex-infos

mpenet11:06:28

Problem with just using maps is that you ll have to check 2 formats very often (eg when any interop is involved)

mpenet11:06:40

exo/ex also enables some nice tricks: you can reuse its hierarchy with multimethods and react on classes of problems very simply as a result

mpenet11:06:19

clojure hierarchies are an underrated gem of the language imho

henrik11:06:45

@U050SC7SV Looks nice, but I’d highly suggest sticking with the Cognitect keywords rather than making your own set.

mpenet11:06:33

we use these internally. But you're free to use whatever

mpenet11:06:51

the lib doesn't force you to use these in particular

👍 2
potetm12:06:04

Yeah @U2HBNQQBE has the right answer here. Throw ex-info.

skylize13:06:14

> It's better to throw an exception. ... - @U1WAUKQ3E > The word "better" here is extremely opinionated. It might be "better for" a particular use case or coding style. But (as already hinted at by the discussion in this thread) it is definitely not universally "better". There are trade-offs to passing errors through your app's control flow vs bypassing that flow. But even ignoring those trade-offs, if you take a pure and referentially transparent function and stick a throw in it (or any of its dependencies), then you get a function which sometimes has predictable return values but sometimes blows up instead. > ... Otherwise, everyone who uses your code should check the result with if/else ... > The consumer only needs to if/`else` for the error if they care whether the value might be a failure state rather than a success. If not, they can simply pass it on to the next potential consumer unchanged. If they do care, then if/`else` is no more work than catching, though it is more work than not catching. There are certainly cases where consumers of pass-by-value errors need to explicitly check for those errors, when a nonexistent catch could silently imply that check. This might seem like boilerplate, but being explicit is generally considered idiomatic in Clojure. > ... and throw an exception anyway > This is just nonsense. Nothing about OP's question indicated that whatever error is under discussion must absolutely be propagated all the way to the root of the program. Only critical app errors need to bubble all the way up. Many (maybe most?) "errors" only interfere with some particular local action, rather than indicating the need to shut down the whole system. These can and should be dealt with only locally. Failing to handle them locally still usually does not indicate desirability of automatically killing everything else. Throwing non-critical errors subjects your entire app to the risk of crashing on inconsequential local failures if you forget to handle them.

2
igrishaev13:06:44

The function from the initial comment creates a Notion page via API, it's not pure by default. Dozens of exceptions might appear during the HTTP/TCP interaction. It has nothing in common with purity and ref. transparency

igrishaev13:06:16

As I mentioned above, we had plenty of issues when a developer would't check if the result is an anomaly map or a regular data => weird bugs.

igrishaev13:06:20

Silencing exceptions occurred during IO (database, HTTP, etc) is a terrible idea, really. I've had too much of that

henrik13:06:40

We’ve wrapped our API calls in what essentially amounts to service endpoints (though we’re not doing microservices). Anything wonky happening will be logged, and you will be screamed at by the app, but then we return an anomaly anyway. This is a consequence of dealing with pipelines, though. The consumer of the error often sits asynchronously waiting for a response (whether it’s the end client, or a service in the middle). The bit of code that can deal with the error is rarely in a direct call stack. I guess this contributes to the leaning towards anomalies. Though locally, exceptions do happen. It’s just that they are caught, logged, converted to data, and passed on.

igrishaev13:06:54

Here is an example with notion:

(let [{:as page1 :keys [id]}
      (create-notion-page {:name "foo"})
      
      page2
      (create-notion-page {:name "bar" :parent id})]

  ...)
which creates the second page referencing the first one. If create-notion-page returns an error map, that would lead to weird bugs

igrishaev13:06:19

rewriting it with if/else would take more code, and later on, another developer will mess up the same

igrishaev13:06:59

briefly: with exceptions, you have two outcomes. With an error map, you have three which complicates everything.

henrik14:06:43

I’d agree so far as you might as well go with exceptions until you have a reason to do otherwise.

skylize14:06:46

> The function from the initial comment ... has nothing in common with purity and ref. transparency > My mention of purity and referential transparency was not any attempt to describe all possible error situations, just to clearly and simply lay out a case where throwing is definitively not better. As to how much that example represents the original question, which was simply > Which is more idiomatic Clojure: use exceptions, or return map with :error key? > , the additional code example appears to be just a randomly chosen example from a public project that happens to use an error value instead of throwing. > we had plenty of issues when a developer would't check if the result is an anomaly map or a regular data => weird bugs. > Sounds frustrating. I can't speak to that. Perhaps this might be a situation where "better" does apply, even if it is definitely not universal. > Silencing exceptions occurred during IO (database, HTTP, etc) is a terrible idea, really. > Seems like a fair concern. "Throwing is better for database APIs" sounds like a reasonable (though still maybe debatable) claim to me.

skylize14:06:01

(let [{:as page1 :keys [id]}
       (create-notion-page {:name "foo"})
       
       page2
       (create-notion-page {:name "bar" :parent id})]
 
   ...)
This would probably call for a create-notion-page function which takes a map and handles the possibility of an error in that map, instead of just dropping the raw map into the let binding.

jamesleonis16:06:49

Man I have Opinions about exceptions vs return codes, but I want to dig more into the original question. I think there are three different contexts where I prefer three different error handling; nil, throw, and {::error ...} • For regular functions, 90% of the time I'm writing something to transform data. I generally follow the Clojure Standard Library conventions. Either the transform works, or I return nil. This keeps the function consistent with how other parts of Clojure work and leverages their affordances. • When its a bad day and/or I really dun goof'd, throw can nicely engage with other error mechanisms that inform the user that there was a problem on our end, and I add a ton of error context for the programmer (aka, myself) so the problem can be addressed. This is when the ex-* functions really help. • Every now and again, I need to have a Job or Process that is I/O heavy, requires 3rd party resources, complex retry logic, complex user reporting logic, etc. Left on their own, these can turn into huge let-heavy functions that are fragile. Here is where I bring in a Error Monad pattern that makes heavy use of {::error ...} maps. All of the different processes get their own functions that operate on a meta-context map where I can store intermediate results, original inputs, and other metadata. I use namespaced keywords liberally since they add even more context for very little typing. The Error machinery handles the flow control, wrapping up the intermediate results, and handling {::error ...} maps into something consumable for the rest of the program, usually one of the other types above. If you squint, some-> is a rough and ready Error Monad built into Clojure's standard library.

4
seancorfield17:06:24

My rule of thumb for this is: • use exceptions for unexpected failures when you (the function) can't handle the failure • use error return values (`nil` or map with :error or whatever) for expected failures when you (the function) have documented that such failures produce such error return values Exceptions propagate by default so think "maybe someone higher up the call try will know what to do about this?" Errors are (should be) local to the call site, so think "I failed, my caller should be able to handle this"

daveliepmann17:06:57

For those folks who don't mind a function returning data on failure, here are some macros as one example of the monad-style handling James mentioned: https://gist.github.com/daveliepmann/fc2d9f93d51e6bdba0e7e28b1a2a6b26

nice 2
pavlosmelissinos17:06:20

> Exceptions propagate by default so think "maybe someone higher up the call try will know what to do about this?" Middleware (well, interceptors, not sure if traditional ring middleware work like this) kind of solve the "unexpected case" problem because you have layers of executions and different logic when you go down vs up the stack. I'm toying with the idea to combine this with https://blog.logrocket.com/what-is-railway-oriented-programming/, where you'd check your input against something like cognitect anomalies at the beginning of each layer and decide whether to process the input or forward it as is to the next layer, etc.

Joshua Suskalo14:06:36

These days I recommend a combination of railway-oriented programming with fmnoise/flow for "data errors" or "domain errors", and conditions+restarts for IO errors. At the moment I use my own implementation of conditions+restarts, farolero, since it's the most like Common Lisp's implementation, but any implementation that properly supports restarts would do. This allows you to remove noise as you only deal with errors where they matter, either in the pipelining of your domain, or at the level of the stack that actually cares about error handling and response, while not losing the flexibility you get with errors as values in the IO circumstance, and generally getting some more power besides.

Joshua Suskalo14:06:28

Generally I agree with the sentiment @U04V70XH6 mentioned about when to use exceptions vs error values, but I just substitute conditions for exceptions to give me more options for recovery.

lread16:06:42

I'm curious as to what Stuart Halloway wrote in "Clojure’s Exceptional Handling of Exceptions" in https://pragprog.com/titles/ppanth/functional-programming-a-pragpub-anthology/, there seems to be an https://medium.com/pragmatic-programmers/chapter-20-clojures-exceptional-handling-of-exceptions-1a416b9724c0... maybe I should just buy the book.

seancorfield16:06:25

I suspect there are other, more recent incarnations of something similar... ISTR a Conj talk about "conditions" at some early point...

seancorfield16:06:43

This is the design doc (and "spreadsheet") for Clojure's exception handling (that led to ex-info in the end): https://archive.clojure.org/design-wiki/display/design/Error%2BHandling.html

seancorfield16:06:26

(I just bought the e-book -- it looks interesting enough beyond just Stu's chapter)

👍 2
valerauko02:06:37

That's a really interesting historical document @U04V70XH6. Do you happen to know what happened to the "action at point of exception" bit? The table says the "modest proposal" solves it, but afaik we have no restarts in Clojure?

seancorfield04:06:09

I've no idea, I'm afraid...

🐼 2
jeeger08:06:10

I'm currently hacking on cider-nrepl, and I'm wondering how I can make lein repl use my local hacked version. I've tried using a new version (0.30.1), but leiningen doesn't seem to want to use my plugin under ~/.m2/repository/cider/cider-nrepl/0.30.1.

jeeger09:06:20

I did a PROJECT_VERSION=0.30.1 make install in the cider-nrepl repository, I don't know what I'm missing.

jpmonettas09:06:56

I would modify my cider-nrepl/project.clj version to something like 0.30.1-SNAPSHOT and then do a make install , then make sure the version got into ~/.m2/repository/cider/cider-nrepl/` If that is correct, on your project you can check with lein deps :tree to see if it is using your snapshot version. That should do it

jeeger14:06:02

Yeah, that's what I did, and it doesn't work. I've also tried reproducing it with a plugin of my own (just lein new plugin testplugin , and that doesn't work either. I must be missing something

jeeger14:06:09

Ah, got it working with my test plugin. Now to retry with cider-nrepl.

jeeger14:06:03

Ah, I think I got it, the project I was testing it with used a :local-repo in project.clj.