Fork me on GitHub
#babashka
<
2021-05-18
>
grzm00:05:35

I’m working with the pod.babashka.aws pod and it’s very noisy with logging statements (via clojure.tools.logging). Is there a way to silence these logs? If this were straight up clojure, I’d do something along the lines of -Dclojure.tools.logging.factory=clojure.tools.logging.impl/disabled-logger-factory. I see I can set Java system properties that are read by babashka using -D, but I suspect they’re not being passed to the pod? The more likely case is I’m just Doing it Wrong™.

borkdude08:05:08

@grzm There is a workaround for this. You can start the pod using the filename instead of the qualified symbol and then pass "-Dclojure.tools.logging.factory=clojure.tools.logging.impl/disabled-logger-factory" as an argument to it

grzm14:05:19

Cheers! There might be some issues with the using the disabled-logger-factory this way in particular (see https://clojurians.slack.com/archives/C03S1KBA2/p1621346387005200), but the general idea likely still holds.

borkdude08:05:45

I think we could support this better by allowing to pass :command-line-args or so via the options

borkdude08:05:21

I'm not sure if this even works, since the command line property may already have been read at compile time (top level) so it would be good to try this out first.

andrea.crotti12:05:07

correct me if this thing already exists, but I think it would be nice to have somewhere in the docs a link to a comparison/translation between "standard" bash scripts and the equivalent Babashka scripts. Would be maybe a good way to onboard people who know bash but don't really know clojure much, if they can see how to do things they already know

👍 2
1
andrea.crotti13:05:48

not sure what's the best way to do that, maybe just a cookbook style, with two columns for each example and an explanation before/after the two implementations

borkdude13:05:19

There is currently an experimental setup possible for getting clojure-lsp support in babashka scripts: https://github.com/babashka/babashka/issues/733#issue-811939787 Feel free to give it a try and suggest improvements.

🎉 3
borkdude13:05:27

@UKFSJSM38 I wonder how clojure-lsp behaves if you have both a deps.edn and a bb.edn in the same project, which is pretty common

ericdallo13:05:19

it will the one specified from user I think

borkdude13:05:41

but what if you want lsp support in both src/myclojure.clj and script/mybb.clj ?

ericdallo13:05:28

Oh, it has the capability to support both but I found a bug where it uses only the specified by users instead of merging with the default

borkdude13:05:43

which might make sense in some cases

ericdallo13:05:45

so user would need to have two project-specs until this bug is fixed

ericdallo13:05:20

ATM it is (or (get settings :project-specs) default-project-specs)

borkdude13:05:42

I think that makes sense, it gives you the power to override lsp

borkdude13:05:11

but now lsp will use the bb classpath for clojure sources too?

borkdude13:05:34

which may not be a big issue

ericdallo13:05:52

no, it will use only if finds the path configured by project-path

ericdallo13:05:59

in this case bb.edn

borkdude13:05:31

yeah ok, but if I added one for deps.edn as well

borkdude13:05:45

then the classpaths would be concatenated?

borkdude13:05:20

that's probably fine, unless people use different versions of libraries in their clojure projects than what babashka uses

ericdallo13:05:24

it will use the same classpath for both script and src folder if that's what you are asking

borkdude13:05:37

and this is why the script may be a better solution since it gives you the flexibility to change something

ericdallo13:05:07

Hum, I still think it's too much manual work but it's a good start 😅

ericdallo13:05:25

what users could do that makes sense is have a project root different for the scripts

borkdude13:05:27

> it will use the same classpath for both script and src folder if that's what you are asking yes, so it will use the babashka.fs library from bb itself to give LSP support in a Clojure project too, which may use an entirely different babashka.fs version

👍 1
ericdallo13:05:06

users could have different project-roots on lsp-mode one for scripts and another for the src

ericdallo13:05:19

and different .lsp/config.edn with different project-specs

ericdallo13:05:33

but yeah, it looks too hard to configure IMO for the end user

borkdude13:05:56

probably in practice it should be fine

👍 1
borkdude13:05:07

let's first see what users come across

borkdude13:05:22

Btw. babashka also has an nREPL server which gives auto-completion for built-in libraries. Can this be used by LSP somehow

ericdallo13:05:32

No, LSP don't uses/connect to any repl

ericdallo13:05:54

this is a frequent question that people think that LSP should use the repl like cider or something, but this is not part of the LSP spec

borkdude13:05:59

that clojure-lsp uses clj-kondo behind the scenes is also not relevant to the LSP spec. similarly it could use an nREPL server to get extra info from

borkdude13:05:42

but people might as well use CIDER with the bb nrepl-server then, which people already do

ericdallo13:05:29

Yeah, it could be added but there are already libs that do that very well, not sure it worth, but it's and idea that could be explored

borkdude13:05:02

@UKFSJSM38 Like: clojure-lsp could offer some hook where people can insert a script that fetches the necessary information for clojure-lsp to do something. e.g. it could be a babashka script that spits out some EDN. where it gets this EDN from is up to the user.

borkdude13:05:26

One such script could just make an nREPL request and clojure-lsp can be completely agnostic about this

ericdallo13:05:19

yeah, looks cool, but is not clear to me when clojure-lsp would do the request, what would do with the result

borkdude13:05:51

think of it as an alternative way of getting clj-kondo analysis information, from some other source than clj-kondo

ericdallo13:05:07

hum, I think cider does that right?

ericdallo13:05:32

looks kind of huge parse the NREPL response and handle as analysis

borkdude13:05:59

well, the script would just give you the processed data, the script would then be responsible for the nREPL stuff

borkdude13:05:08

and clojure-lsp can be agnostic about nREPL

borkdude13:05:27

the script could provide it in the same format as clj-kondo analysis

borkdude13:05:43

so instead of saying: give me a classpath to analyze, you could say: give me the analysis

ericdallo13:05:10

I see, it could work, but it looks too work for user, isn't ? I don't see too many users using that feature

borkdude13:05:49

the complexity would be contained in the script, which can be offered as a library. users use complex libraries all the time without understanding what it does internally

borkdude13:05:25

just a thought, you're probably right that this isn't used much

ericdallo13:05:20

Yeah, but it looks to me a nice extension for clojure-lsp, giving this optional power to users, thanks for the idea 🙂 Maybe in the future I could do some POC

richiardiandrea18:05:06

Hi there, noticed the following - I was expecting a map in output:

ls | bb -i -O '{:report true :paths (vec *input*)}'
[:report true]
[:paths ["bin" "README.md" "resources" "scripts" "setup" "src" "target" "test" "tests.edn"]]
Should I file a bug or it is the expected behavior?

borkdude19:05:26

This is a bug. There is a predicate that determines if multiple objects should be printed and currently this is coll?, but obviously that is wrong. Probably sequential is better

👍 1
borkdude19:05:35

Issue welcome.

richiardiandrea22:05:36

I have fixed it quickly, sorry I might have broken the tests - will try to improve the PR later with more time at hand https://github.com/babashka/babashka/pull/853

Akiz18:05:49

Hi there, I am trying to uberjar my script clojure -e "(require 'postgryoshka.core) (compile 'postgryoshka.core)" but I am getting Could not locate babashka/fs__init.class, babashka/fs.clj or babashka/fs.cljc on classpath. Uberscript and running bb script works. What should I do to create *.jar with all the dependencies?

borkdude19:05:16

To create an uberjar with dependencies for babashka itself, you should only have to do:

bb uberjar foo.jar

borkdude19:05:51

I'm not sure why you are trying to do clojure -e etc

Akiz21:05:04

I used an example from babashks book ... To exclude these dependencies, you can use the following :classpath-overrides in your deps.edn...

Akiz21:05:34

`$ rm foo.jar $ bb -cp $(clojure -A:remove-clojure -Spath) uberjar foo.jar -m foo $ bb foo.jar :hello $ ls -lh foo.jar -rw-r--r-- 1 borkdude staff 871B Aug 19 17:07 foo.jar`

borkdude21:05:54

This was written before bb.edn was there. You can now just put your deps inside bb.edn and run bb uberjar foo.jar

👍 2
borkdude21:05:16

I will update the book with this info.

👍 2
Akiz05:05:20

Works... babashka is really cool. Thanks

Greg Rynkowski21:05:19

Hi, how to source some of the existing bash script file and run specific bash function from that file in babashka?

borkdude21:05:18

@grzegorzrynkowski_clo babashka doesn't interpret bash, it is a Clojure scripting environment

borkdude21:05:57

if you want to run bash, use bash by shelling out to it using babashka.process (or if you are using tasks: shell)

borkdude21:05:30

e.g. you can do (shell "bash -c 'source foo && foo_fn'")

Greg Rynkowski21:05:12

makes sense, thanks a lot 🙇🙏

kokada22:05:25

I was trying babashka.curl on Windows (just for curiosity, not that I care) and got the following error:

user=> (curl/get  "")
clojure.lang.ExceptionInfo: curl: option --compressed: the installed libcurl version doesn't support this
curl: try 'curl --help' for more information
[at <repl>:1:1

borkdude22:05:12

I wonder if it's possible to probe curl if it supports this option, so we can cache that for the next call

kokada22:05:17

This works, but it be good to detected it instead:

(curl/get "" {:compressed false})

kokada22:05:53

> I wonder if it's possible to probe curl if it supports this option, so we can cache that for the next call Maybe with --version ?

curl.exe --version                               
curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL Release-Date: 2017-11-14, security patched: 2019-11-05
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL  

borkdude22:05:37

I get this on macOS:

curl 7.64.1 (x86_64-apple-darwin19.0) libcurl/7.64.1 (SecureTransport) LibreSSL/2.8.3 zlib/1.2.11 nghttp2/1.39.2
Release-Date: 2019-03-27
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS GSS-API HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL UnixSockets

kokada22:05:58

I think it is the libz feature. On NixOS:

curl --version
curl 7.74.0 (x86_64-pc-linux-gnu) libcurl/7.74.0 OpenSSL/1.1.1k zlib/1.2.11 libssh2/1.9.0 nghttp2/1.41.0
Release-Date: 2020-12-09
Protocols: dict file ftp ftps gopher http https imap imaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz NTLM NTLM_WB SPNEGO SSL TLS-SRP UnixSocke

borkdude22:05:07

ok, I'll make an issue for this tomorrow if you don't beat me to it :)

kokada22:05:04

> Another interesting thread: https://bugzilla.mozilla.org/show_bug.cgi?id=1605540 So it seems it needs to check both for libz and brotli :thinking_face: > ok, I'll make an issue for this tomorrow if you don't beat me to it 🙂 I don't care too much about Windows support, but I can make an issue

👍 1
🙏 1
borkdude12:05:33

I have made a check in a delay so it checks for this only once during the duration of a bb program, but it adds 10ms or so when doing the first request.. Perhaps the result could be cached in a file, but this leads to problems like: what if the system updates the curl binary.

borkdude12:05:38

on linux systems this time is probably lower

borkdude12:05:22

perhaps it's worth it though

borkdude12:05:32

and usually requests take anywhere between 50-100 ms anyway

borkdude12:05:45

perhaps catching the exception and then retry also a way to handle it

borkdude12:05:30

or perhaps only applying this check on Windows

borkdude13:05:34

ironically, I have Windows tests in place but apparently appveyor has a curl installed that does support this

borkdude13:05:14

hmm, maybe I should just add some docs for Windows 10 that they should either upgrade curl or use :compressed false

borkdude13:05:30

and not bother trying to work around some edge case in windows

borkdude13:05:26

### Compression

From babashka 0.2.4 onwards, this library will call `curl` with `--compressed`
by default. To opt out, pass `:compressed false` in the options.  On Windows 10
the default installed curl does not support this option. You can either upgrade
curl or perform all requests using `:compressed false`.

kokada19:05:42

Well, seems fine to me While the behavior right now is not really portable, at least it is consistent and the error is clear. Checking for features could lead to too much magic where people that expected e.g. that the compression worked would suddenly have a surprise when their scripts doesn't have compression on Windows

littleli22:05:20

@borkdude How complete is the included implementation of agents in bb? I can create new instance of an agent. I can send and send-off, but not send-via and if I put agent into failed state, I cannot recover with restart-agent . I cannot tell if it's a feature or a 🐛

borkdude06:05:03

Please include more details. What error do you see, etc. Perhaps it’s just a missing function.

borkdude07:05:13

@UBLU3FQRZ These functions should be in babashka/impl/clojure/core.clj in core-extras, I think it's just a matter of adding them

littleli07:05:05

Should I create an issue on GH?

littleli07:05:27

My concern is to have full support for agents if possible.

littleli08:05:41

or not having them at all, which would be a loss in my opinion, as they are great worker pattern.

borkdude08:05:29

@UBLU3FQRZ I have created a branch here: https://github.com/babashka/babashka/tree/more-agents Please inspect the commit to see if I have caught them all

borkdude08:05:41

and then you can go to appveyor to test it (if you run on Windows)

borkdude08:05:48

or CircleCI for macOS and linux

borkdude08:05:54

or get it from #babashka-circleci-builds

borkdude08:05:10

I would also be interested in hearing about your pattern :)

littleli08:05:04

I have no pattern 🙂 it's just that agents have nice properties.

borkdude08:05:24

you said: great worker pattern

borkdude08:05:30

so I was curious how you are using them

borkdude08:05:54

Perhaps I could have used them to implement parallel tasks, I see await is a lot simpler than what I did with core.async ;)

littleli08:05:54

to be honest, I was just learning agents and I was curious how to use them with bb... so sorry, I but don't have anything specific 😄

borkdude08:05:06

no worries, I think it's good to add support for it

borkdude08:05:18

please test :)

borkdude08:05:31

hmm, I need to patch these functions for binding conveyance as well.

littleli08:05:36

there are more functions to add...

await-for
agent-error
error-handler
error-mode
get-validator
shutdown-agents

add-watch # ?
remove-watch # ?

borkdude08:05:39

I'll make an issue for that

borkdude08:05:44

I did add agent-error

borkdude08:05:48

what is handler-error?

borkdude08:05:01

can you use the full var names?

littleli08:05:53

it's error-handler... sorry

borkdude08:05:26

add-watch and remove-watch are already there

👍 2
borkdude08:05:48

Added these now:

+   'await-for (copy-core-var await-for)
+   'error-handler (copy-core-var error-handler)
+   'error-mode (copy-core-var error-mode)
+   'get-validator (copy-core-var get-validator)

borkdude08:05:44

ok, I have them all now

borkdude08:05:16

there is a small patch needed for binding conveyance. when you have a dynamic var binding, the started agent thread also needs to see it. but this should be doable

borkdude08:05:41

for now you can test the build when it's ready

littleli08:05:08

there is also agent-errors, but is marked as deprecated in clojure, and I don't think it should be included.

borkdude09:05:18

So the problem I'm fixing right now:

$ clojure -M:babashka/dev -e '(def ^:dynamic *foo* 1) (def a (agent nil)) (binding [*foo* 2] (send-off a (fn [_] *foo*))) @a'
1
This should be 2

borkdude09:05:48

Luckily this is relatively easy

littleli10:05:24

Nice PR. I'm surprised it makes sense to my untrained eyes.

borkdude12:05:06

ok, I merged it now