Fork me on GitHub
#clojure
<
2023-09-21
>
kwladyka11:09:01

hey, which formatter do you recommend? I joined to new project where team use zprint, but I found a few issues about this. First of all my system have already zprint command which is the tool for memory and this make a conflict. Project has additional bash scripts to use zprint, because there is no way to run zprint on project, but need to be run on each file separately. This makes issues and blow up everything in my system. Custom bash scripts doesn’t work in my system as intended. I tried with babashka, but it blows my files in the same way. All of them after formatting are empty. Before I was using cljfmt and it was working. It was easy to use with one command for whole project and did the job. I am a little lazy to do my own deep research for all available tools to introduce another formatter to the team, but I don’t like zprint so far, because of reasons which I mentioned. Maybe there is another way to use it, but I prefer tools which don’t make troubles. What do you recommend and why?

borkdude11:09:45

cljfmt's fine, you can even use that from babashka:

bb -Sdeps '{:deps {dev.weavejester/cljfmt  {:mvn/version "0.11.0"}}}' -m cljfmt.main fix src

borkdude11:09:16

zprint's also fine, I don't understand why it's causing a problem for you, also works under bb nowadays.

borkdude11:09:40

clojure-lsp also uses cljfmt, you can invoke their command line tool as well: clojure-lsp format I believe

kwladyka11:09:41

Is a way to run zprint like clj -Tcljfmt fix?

kwladyka11:09:13

From what I saw it has to be run on each file separately, so there is custom bash script which find all clj files in repository.

kwladyka11:09:54

This need extra bash script which is first cons. Second cons is, it makes files empty in my system.

kwladyka11:09:33

clj -Tcljfmt fix winning so far for me. No need for custom bash scripts, no risk for conflicts, just work.

👍 1
borkdude11:09:31

I looked at zprint's docs and I don't see an option to format a directory recursively. If you're still interested, your best bet is probably to raise an issue at their github repo

borkdude11:09:33

they have a github discussions thing as well with the possibility of q&a: https://github.com/kkinnear/zprint/discussions

vemv11:09:07

zprint is a vanilla JVM library so you can also invoke it however you please within your usual JVM process

borkdude11:09:20

I think OP wants to have a convenient way to format a whole directory without writing any other boilerplate, which makes sense imo

👍 2
kwladyka11:09:20

To summary this is a choice between cljfmt vs zprint. Anything else worth to check?

vemv11:09:33

cljstyle

kwladyka11:09:49

So then the choice is easy. Thank you for your input.

vemv11:09:54

I still know of projects using it. But it's a cljfmt fork, honestly the less lisp curse the better. Maybe it served its purpose in making cljfmt more graalvm-friendly.

borkdude12:09:26

I'm using cljfmt from clojure-lsp whenever I hit a keystroke in emacs that does formatting. No complaints here.

Noah Bogart12:09:19

We have a script to format our directory with zprint and one to check only files that have changed from master.

Noah Bogart12:09:31

If you have naming conflicts with an existing zprint, you can put the clj one earlier in your PATH so it’s picked up first

kwladyka12:09:38

unless other program use zprint and blow up :)

Noah Bogart12:09:00

Surely that would never happen 😉

Noah Bogart12:09:09

I’ll post our check-formatting.sh when I get to work, it’s pretty simple

Noah Bogart12:09:31

If you put the script in your path and change it to call zprint under a different name (zprintl on linux for example), you can at least run the script without pain

kwladyka12:09:34

I can’t because other developers use zprint name.

kwladyka12:09:01

you see why I prefer cljfmt. It just work as it is. It is simple and easy.

kwladyka12:09:06

But @U45T93RA6 has a good point.`zprint` is a library while cljfmt is a tool. Each has it’s own purpose and pros / cons. I think with this words we can close this thread.

1
practicalli-johnny14:09:41

https://practical.li/clojure/clojure-cli/clojure-style/ as a local format tool and within the continuous integration workflow, it works very well. Cljstyle has also been a part of the setup-clojure GitHub actions and was added to Megalinter version 7 (the latest major release). I find cljstyle easier to understand it's configuration than cljfmt and especially zprint config. I also like the cljstyle report output, very easy to understand. Cljstyle formats to the Clojure Style guide and cljfmt, so no conflict

practicalli-johnny14:09:50

I am surprised the team you have joined are not giving more help, especially if zprint is the mandated format tool of choice. Hopefully the team has configured the CI workflow to report in PRs if there are formatting issues, so in theory all that would be required is using your preferred format tool that formats to the same rules. Zprint has a https://github.com/kkinnear/zprint/blob/main/doc/reference.md#community (although I have not tried this myself)

henrik11:09:55

Is anyone using https://github.com/cognitect-labs/aws-api? We’re doing a

(aws/invoke s3
  {:op      :PutObject
   :request …})
and it neither fails nor succeeds on our staging machine. It just hogs memory and a thread forever. I’d like it to work, of course, but before that, I’d like it to at least terminate. Does anyone know of a good way to kill the request if it goes on too long?

p-himik11:09:06

In case you're using a service for staging other than AWS S3, maybe that service is doing something bad. Is there a chance the object you're uploading is huge? Or, if it's a reader, that it hangs?

henrik11:09:21

Staging is configured like the other environments, with credentials stored in a secrets manager that ends up being env variables for the running system. It’s an image of a megabyte and change in the form of a byte array. I’ve isolated to literally the aws/invoke. Everything before is OK, and it never goes beyond this point. It’s replicable using the REPL and a byte array of the string “hello”.

p-himik12:09:07

A lot of people use that library, including me, so if some bug as seemingly trivial as this one existed in the library itself, it would've been noticed quickly. So either the bug is not trivial at all or there is still something in your app to blame, somehow. Would you be able to create an MRE?

henrik12:09:26

Possibly, I’ll try. It might very well depend on some interaction with something environmental though, or we’d see this across all machines.

ghadi12:09:33

are you uploading a large file?

ghadi12:09:49

because the client (at this moment) does not do streaming

pesterhazy13:09:39

at the risk of stating the obvious, the AWS S3 java sdk is more battle-tested – you're unlikely to run into issues like this with that sdk, and if you do there are millions of people on the internet screaming about it

p-himik14:09:56

Personally, I find it much easier to just fix whatever issues I might find with aws-api than to figure out how to make AWS S3 SDK do the thing I want it to do. :) I switched to the former right after spending a few hours figuring out why AWS S3 SDK was misbehaving with my configuration.

practicalli-johnny15:09:51

Was the Clojure REPL started after setting up the credentials? If credentials are changed then the JVM (and therefore the REPL) will need to stop/start to pick up changes to the OS environment Does listing buckets work?

;; Define a client for the AWS S3 service
(def data-bucket (aws/client {:api :s3}))
;; Validate requests to the S3 service to report incorrect arguments (especially useful when starting out with the library). Underlying this is a Clojure specification derived from the AWS specifications.
(aws/validate-requests data-bucket true)
Credentials are required for this request, so its a useful test to ensure the AWS CLI is configured correctly.
(aws/invoke data-bucket {:op :ListBuckets})

practicalli-johnny15:09:47

I used a https://practical.li/blog/posts/access-aws-with-clojure-from-scratch/, before discovering I should have setup an account via AWS IAM Identity Center and have that as a Single Sign-On (SSO) service. Once AWS was setup, then Cognitect aws-api is relatively simple to use

henrik17:09:33

Thanks for all the input! Switching to the Amazon Java SDK solved our problems. We traced this to some weird interaction between Cognitect’s lib and another library, which we had included. Probably conflicting Jetty versions (9 vs 11). Reading up on the potential solutions to these conflicts, going with the official SDK instead seemed like the smaller effort. The official SDK isn’t that bad!

ghadi17:09:22

@U06B8J0AJ would be amazing if you can file an issue with your information, we can improve aws-api

César Olea17:09:57

@U05254DQM are you planning on following up on your AWS accounts article with the SSO setup? I’d be very interested. Using STS with aws-sdk works, but requires you messing with credential providers and a lot of Java interop.

practicalli-johnny17:09:57

@U02DNF3TW3E yes, I'll write up the IAM identity server setup as it took me many AWS pages to understand it (and if it was 'free'). Although I might add to the Account section of https://practical.li/amazon-web-services/account/ rather than another very long blog article

César Olea17:09:34

Either option sounds great. Looking forward!

Ingy döt Net19:09:01

I did something like this and it worked

(ns foo)
...
(in-ns 'clojure.test)
(defn something ...)
(in-ns foo)
...
but it seems clumsy. Is there a better way?

Ingy döt Net19:09:02

Basically I needed a slightly altered version of a clojure.test function and I didn't want to have to fully qualify every symbol. @U2FRKM4TW I considered intern first but then I'd have to fq. This was much easier and trustworthy imho.

Ingy döt Net19:09:12

I was hoping there was something like a let-ns 🙂

p-himik19:09:47

intern is a function, not a macro, so you can e.g.

(let [n 'clojure.test]
  (intern n 'x 1)
  (intern n 'y 2))

Ingy döt Net19:09:41

My function was a variant of https://github.com/clojure/clojure/blob/clojure-1.11.1/src/clj/clojure/test.clj#L797C1-L811 so I was left with the choice of • interning but needing to fully qualify the calls to other clojure.test fns • temporarily switching the current ns to clojure.test

Ingy döt Net19:09:11

I can see the value in both approaches.

p-himik19:09:18

Ah, gotcha. Yeah, your approach is reasonable. And you can write a let-ns macro quite easily but I'm pretty sure there's nothing built-in.

p-himik19:09:40

Oh, wait - you should be able to

(binding [*ns* (find-ns 'clojure.test)]
  (defn ...))

1
😲 1
Ingy döt Net19:09:25

OK thanks for the validation. I din't really like having to copy this code, but it was the easiest way to integrate my test module with clojure.test.

Ingy döt Net19:09:31

Oh that's neat

p-himik19:09:32

FWIW, if you're writing a library that other people would use, I'd recommend against interning anything into other namespaces.

Ingy döt Net19:09:07

It could become a generic lib later. Basically I have a way to define a bunch of data points in a yaml file. Then the test files can load the yaml file and generate the tests appropriate for that test file. To run the tests I eval calls like:

(test/run) ; run all tests for that test file namespace
(test/run :all) ; run for all test namespaces
(test/run :verbose)  ; print test # and test name for every test run
(test/run 5 2 7)  ; run tests #5 #2 #7 in that order
it all works in the repl like clojure.test stuff but the coding of data driven tests is very minimal and clean

Ingy döt Net19:09:28

I probably don't need to intern anything into clojure.test actually.

Ingy döt Net19:09:20

but it was nice and quick to be able to do this to see if it was gonna work at all

Ingy döt Net19:09:23

@U2FRKM4TW the thing you suggested didn't work:

(binding [*ns* (find-ns 'clojure.test)]
  (defn ...))

Ingy döt Net19:09:54

All the unqualified symbols complained

Ingy döt Net19:09:43

including the earmuff ones

p-himik19:09:53

Mm, right. Unfortunate.

Ingy döt Net20:09:21

Ok I rewrote it to just be a function in my own ns

(defn run-test-vars [v]
  (binding [test/*report-counters* (ref test/*initial-report-counters*)]
    (let [ns-obj (-> (first v) meta :ns)
          summary (do
                    (test/do-report {:type :begin-test-ns
                                     :ns   ns-obj})
                    (test/test-vars v)
                    (test/do-report {:type :end-test-ns
                                     :ns   ns-obj})
                    (assoc @test/*report-counters* :type :summary))]
      (test/do-report summary)
      summary)))
and fq'd the symbols to public clojure.test things. This is much better, but glad I learned something for when I want to try monkey-patching external libs 😄

p-himik20:09:34

If you really don't like specifying namespaces or their aliases, you can use :use. Of course, it's not generally recommended and its scope is the whole namespace.

Ingy döt Net20:09:06

Good point. Seems too much here but I follow you. I've read that use and :use are discouraged if not deprecated. I only use it for loading development libraries atm.