Fork me on GitHub
#tools-deps
<
2021-07-21
>
markaddleman03:07:53

Is there a magic incantation to get clj -X... 's :exec-fn to return a status code to the shell? I want to use clj -X... to execute my regression suite before a deploy but I need the bash script to check that the tests pass

didibus03:07:00

It returns whatever the program you call with -X returns I believe. So it depends on what it is you are calling with it

markaddleman03:07:39

That's what I thought but it's not working. I must be doing something wrong. Thanks for the confirmation

didibus03:07:55

Are you doing (System/exit 1) ?

didibus03:07:30

I might be wrong, I really just assumed so, maybe there's something weird with -X wrapping things

markaddleman03:07:27

I'm not doing (System/exit ...) The function is just returning an integer

didibus03:07:19

If you return an integer, I think that just goes to output. I think you need System/exit to set the process status code

markaddleman03:07:44

That makes sense. Thx

didibus03:07:27

Actually just returning an integer just has it go in the void lol. You need to print to have it go to the output. And (System/exit 0) or (System/exit 1) to set the status

didibus03:07:20

I just tried

seancorfield04:07:23

The idea is that a "failed task" should generally throw an exception (an ex-info for the best handling) and the CLI exec stuff will handle it.

seancorfield04:07:19

Right now the return value is not used. It was when you could chain -X functions (and I'm sure it will be again in the future -- like the -> macro).

seancorfield04:07:57

All my -X functions return a hash map -- usually the hash map they were passed in, unless they have something to add it to. That makes them chainable/threadable. And they throw an exception if they "fail".

markaddleman13:07:12

Throwing an exception did the trick. Thanks!

2
Alex Miller (Clojure team)04:07:07

-X/-T should either return 0 for success or 1 if an exception is thrown out of the function

Alex Miller (Clojure team)04:07:46

so depends on the test runner function to some degree

seancorfield04:07:08

Right, but what was missing (in that thread) was understanding that what the invoked function returns does not affect the status -- I commented that you either throw an exception for failure or you just return data for success.

didibus04:07:13

Would you say an explicit call to System/exit is bad practice inside a function meant to be called with -X or -T ?

seancorfield04:07:48

Such functions should either return data or throw exceptions. In a world where functions can be chained (threaded), those functions should probably return the data they were passed as an argument, possibly with additional keys (or perhaps they should remove some of the keys they were given).

seancorfield04:07:21

(I think it's a bit odd to see some of the build functions just returning nil right now but maybe when we get threading back those functions will either get updated to return their input data, or functions that return nil will cause the threading code to pass the original input data along to the next function?)

Alex Miller (Clojure team)05:07:29

a build is a thing you write for your project. if the build functions aren't parameterized, there's no reason to do so

seancorfield05:07:00

Sure, the internal functions, but the entry points, which all get passed arguments... I guess it'll depend on how much folks would use threading/chaining -- assuming we get that feature back (I see :fns in the latest clojure.run.exec so it looks like we'll get it back?).

Alex Miller (Clojure team)06:07:37

the entry points in most of the build scripts I've written are not parameterized

seancorfield06:07:58

Right, but they still have to accept a single argument? Because that's the hash map of stuff coming from the command line...

didibus05:07:18

Makes sense, System/exit would kill the chain even on success otherwise

didibus05:07:52

It looks like -T fails when there is no build.clj file and using ns-defaults, but it works if using fully qualified on the command line:

{:paths ["src"]
 :aliases
 {:foo {:ns-default foo}}}
(ns foo)

(defn bar
  [params]
  (println "Foo Bar")
  params)
> clojure -T:foo bar
Execution error (FileNotFoundException) at clojure.run.exec/requiring-resolve' (exec.clj:34).
Could not locate build__init.class, build.clj or build.cljc on classpath.
clojure -T:foo foo/bar
Foo Bar

seancorfield06:07:55

That makes sense to me. In the first case you are asking it to resolve build/bar and it can't because no build ns file exists.

seancorfield06:07:29

In the second case you are asking it to resolve foo/bar and it loads a foo ns file and resolves bar within it.

seancorfield06:07:57

It's not that "there is no build.clj file" so much as you are asking for a ns that doesn't exist -- it's just the same as doing clojure -M -m no-such and getting

(! 592)-> clojure -M -m no-such
Execution error (FileNotFoundException) at clojure.main/main (main.java:40).
Could not locate no_such__init.class, no_such.clj or no_such.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.

Alex Miller (Clojure team)06:07:53

where is build even coming from?

didibus19:07:10

That's what I'm wondering too. The only thing is I did have it as ns-default build at first and then changed it, could it be some kind of caching issue?

didibus21:07:51

-Sforce did the trick

tianshu06:07:13

Hi, I want to use tools.build with this

io.github.clojure/tools.build {:git/tag "v0.1.4" :git/sha "169fef9"}
But tools.deps says Error building classpath. Library io.github.clojure/tools.build has missing :sha in coordinate. My CLI version is Clojure CLI version 1.10.3.855 . Is there something I missed?

Alex Miller (Clojure team)06:07:08

that's the stable release - the new git dep coords are only in the latest prerelease

tianshu06:07:44

Got it, thanks!

tianshu06:07:17

Where can I find the latest CLI source code, is it part of Clojure project?

tianshu06:07:35

I found this. linux-install-1.10.3.920.sh

seancorfield06:07:21

Oh, I thought he had :ns-default build but maybe not?

seancorfield06:07:50

@doglooksgood You need the prerelease version for that format of coordinate

tianshu06:07:06

A prerelease version of tools.deps?

dpsutton06:07:10

what's the best way to get a repl up with build.clj on the classpath? is it just clj -A:build and then eval the ns form? Load-file? Or is there a way to essentially get "." on the classpath with -T but not invoke a build function. Pretty simple problem to solve just making sure i didn't overlook something obvious

seancorfield06:07:12

@doglooksgood The CLI. You need 1.10.3.912 at least.

tianshu06:07:24

Okay, thanks!

seancorfield06:07:38

@dpsutton what we do at work is clj -M:build -i build.clj -r

seancorfield06:07:03

Then build is already loaded and you can just call (build/some-fn {})

👍 3
Alex Miller (Clojure team)17:07:28

Clojure CLI prerelease 1.10.3.929 available • TDEPS-189 - Port -T changes to Windows scripts • Script cleanups in bash

Alex Miller (Clojure team)17:07:43

Nothing big there, but if Windows users want to kick the tires, go for it

seancorfield17:07:13

Is there anything non-Windows users might need to test in this update?

Alex Miller (Clojure team)17:07:07

nothing specific, just cleaned up some dead code mostly in the bash

2
seancorfield19:07:56

@alexmiller With tools.build.api/copy-dir, the :replace hash map would be stuff like {"{{version}}" "1.2.3", "{{username}}" "seanc"} -- a hash map with strings for keys and values?

seancorfield19:07:26

I realized that with a small wrapper around tools.build I can create a really nice, stripped down version of clj-new 🙂

seancorfield19:07:48

(and that answer just confirms what I was thinking about that)

didibus21:07:06

It seems that copy-file will overwrite a directory as a file, which I mean maybe is a cool feature, but I feel its unintuitive behavior, I've never used a linux command that behaved like this: e.g.:

;; output was a directory, but it will now become the tempo.clj file after calling this copy-file
clojure -T:build-api copy-file :src '"./src/tempo.clj"' :target '"./output"'

Alex Miller (Clojure team)21:07:55

You expected it to be in target?

didibus21:07:00

Ya, my expectation was if you don't specify the filename in :target it uses the same filename as the source file. But I'd be okay with just failing saying output is a folder. Overwriting the folder to be the file seems dangerous

didibus21:07:03

cp in linux does the former

didibus21:07:39

At least it seems copy-file fails if the directory is not empty

didibus21:07:50

Also weird is if do:

;; src is a directory with files in them, output is an empty directory
clojure -T:build-api copy-file :src '"./src"' :target '"./output"'
Then copy-file succeeds, but nothing seemingly happens. Like output is still an empty directory, and src is stil a directory with files in it.

Alex Miller (Clojure team)22:07:41

as the docstring says, "Copy one file from source to target, creating target dirs if needed." perhaps you are looking for copy-dir?

didibus00:07:41

None of the behavior I describe do this though?

didibus00:07:31

The latter one doesn't copy or create anything, the former overwrites an existing folder

didibus00:07:40

The latter I tested just to see what happens in that edge case. I think I'd expect a failure in that case, not a silent do nothing but succeed. Though well at least it's not destructive. But the former feel wrong, to overwrite a directory with a file? At least the doc-string I think should be explicit of it. Cause it's a very easy fat-finger incident I think

didibus00:07:33

Anyway, just reporting on the behavior in case you weren't aware of those edge cases. If its by-design that's fine too.

seancorfield16:07:10

@U0K064KQV While I too am a bit surprised that copy-file overwrites an (empty) directory, I'm also sympathetic to the fact that it does it because a) it's called copy file, b) it has :src and :target args not :target-dir. I think my preference would to add an :overwrite arg, that defaults to true and could be set to false so it would throw an exception if :target exists in any form.

seancorfield16:07:45

The default behavior would still be to silently overwrite :target, regardless of what it is, but if you want to be paranoid, you could use :overwrite false, and you could decide whether to delete the :target and re-copy or whatever.

seancorfield16:07:35

On the same subject, I'd also like to see :recurse false and :throw-on-missing true as options on delete -- but all of these are just "nice to have" knobs and dials as far as I'm concerned: the core functionality does what is needed for build programs -- and you have and interop if you need more.

didibus21:07:58

What's annoying with the copy over the directory, is this: copy-file :src '"./src/foo.cls :target '"./output"' Now output has become a file. Someone checks and says, oh that's not what I wanted. So they try: copy-file :src '"./src/foo.clj :target '"./output/foo.clj"' And now it throws with an error saying output is not a directory. So it's really inconsistent you see. It doesn't have well defined semantics. So everything seems to be undefined behavior. I'd be okay if it always overwrote everything. If it always overwrote empty things. If it always overwrote the file but not directories. Or if it never overwrote anything. Now as it stands, you're kind of forced to just experiment and learn all the undefined behaviors, which I assume would be different on different OS.

seancorfield22:07:09

"So everything seems to be undefined behavior." -- that's a bit... melodramatic... 🙂

seancorfield22:07:52

"I assume would be different on different OS" -- given that this is just Java I/O behavior, I'm not sure why you would assume that?

seancorfield22:07:59

Under the hood, copy-file is just this Java call: https://github.com/clojure/tools.build/blob/master/src/main/clojure/clojure/tools/build/util/file.clj#L59 -- so this is the semantics of Java's standard library itself.

seancorfield22:07:55

Bottom line: t.b.api/copy-file does exactly what Java's Files/copy command does, with the sole addition of calling (.mkdirs target-file) to auto-create any missing directories in the :target path.

didibus00:07:26

I was thinking might be different because it exposed a: > sun.nio.fs.UnixCopyFile/copy exception And the name of that made me think it could be OS specific. Seems from the javadoc that file attributes are OS specific and undefined how they will copy, but the rest should be same.

didibus00:07:05

Meh, ok, I guess this is just how nio copy works with REPLACE_EXISTING option, java is dumb 😛

seancorfield01:07:50

"java is dumb" good, "clojure is dumb" bad :rolling_on_the_floor_laughing: