Fork me on GitHub
#lsp
<
2022-01-06
>
Phillip Mates16:01:29

Hey, wanted to share a clojure-lsp api experience @borkdude recommended I check out the clojure-lsp clean-ns api + bb pods for running a CI format check step. I gave it a shot with

(ns clean-ns
  (:require [babashka.pods :as pods]))

(pods/load-pod 'com.github.clojure-lsp/clojure-lsp "2022.01.03-19.46.10")

(require '[clojure-lsp.api :as api])

(defn dry []
  (api/clean-ns! {:dry? true}))

(defn fix! []
  (api/clean-ns! {}))
And ran into an error that I don't get when I do clean-ns from the clojure-lsp binary:
bb lint                                                         
[  0%] clojure-lsp
[ [ SubstrateSegfaultHandler caught a segfault in thread 0x00000000085acb00 ] ]

...

Top frame info:
  Does not look like a Java Frame. Use JavaFrameAnchors to find LastJavaSP:
  Found matching Anchor:0x00007fff1611ea80
  LastJavaSP 0x00007fff1611ea70
  LastJavaIP 0x00000000016fe60f

Threads:
  0x00000000085acb00 STATUS_IN_JAVA (safepoints disabled) "main" - java.lang.Thread@0x00007f1d77746428, stack(0x00007fff15922000,0x00007fff16120000)

VM thread locals for the failing thread 0x00000000085acb00:
  0 (8 bytes): com.oracle.svm.jni.JNIThreadLocalEnvironment.jniFunctions = (bytes)
    0x00000000085acb00: 0x00007f1d76596010
  8 (8 bytes): com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.stackBoundaryTL = (Word) 1 (0x0000000000000001)

...

Stacktrace for the failing thread 0x00000000085acb00:
  SP 0x00007fff1611e8c0 IP 0x00007f1d7b1556e4  IP is not within Java code. Trying frame anchor of last Java frame instead.
  SP 0x00007fff1611ea70 IP 0x00000000016fe60f  [image code] com.oracle.svm.core.posix.headers.Pwd.getpwuid(Pwd.java)
  SP 0x00007fff1611ea70 IP 0x00000000016fe60f  [image code] com.oracle.svm.core.posix.PosixSystemPropertiesSupport.userHomeValue(PosixSystemPropertiesSupport.java:52)
  SP 0x00007fff1611eab0 IP 0x00000000016e43a6  [image code] com.oracle.svm.core.jdk.SystemPropertiesSupport.userHome(SystemPropertiesSupport.java:240)
  SP 0x00007fff1611eab0 IP 0x00000000016e43a6  [image code] com.oracle.svm.core.jdk.SystemPropertiesSupport$$Lambda$c08be315aa20ccffc6d99c8ceeebd4e4a45b68c0.get(Unknown Source)
  SP 0x00007fff1611ead0 IP 0x00000000016e4a4e  [image code] com.oracle.svm.core.jdk.SystemPropertiesSupport.initializeLazyValue(SystemPropertiesSupport.java:216)
  SP 0x00007fff1611eb00 IP 0x00000000016e4913  [image code] com.oracle.svm.core.jdk.SystemPropertiesSupport.getProperty(SystemPropertiesSupport.java:169)
  SP 0x00007fff1611eb20 IP 0x0000000001d90544  [image code] java.lang.System.getProperty(System.java:287)
  SP 0x00007fff1611eb40 IP 0x00000000013348a6  [image code] clojure_lsp.config$get_property.invokeStatic(config.clj:28)
  SP 0x00007fff1611eb40 IP 0x00000000013348a6  [image code] clojure_lsp.config$get_property.invoke(config.clj:27)
  SP 0x00007fff1611eb50 IP 0x0000000001334529  [image code] clojure_lsp.config$get_home_config_file.invokeStatic(config.clj:37)
  SP 0x00007fff1611eb90 IP 0x00000000013342d3  [image code] clojure_lsp.config$get_home_config_file.invoke(config.clj:33)
  SP 0x00007fff1611eba0 IP 0x0000000001336be8  [image code] clojure_lsp.config$resolve_for_root.invokeStatic(config.clj:60)
  SP 0x00007fff1611ebf0 IP 0x0000000001336b26  [image code] clojure_lsp.config$resolve_for_root.invoke(config.clj:58)
  SP 0x00007fff1611ec00 IP 0x000000000134c6d1  [image code] clojure_lsp.crawler$initialize_project.invokeStatic(crawler.clj:195)
  SP 0x00007fff1611ed10 IP 0x000000000134c57a  [image code] clojure_lsp.crawler$initialize_project.invoke(crawler.clj:193)
  SP 0x00007fff1611ed20 IP 0x000000000145a551  [image code] clojure_lsp.internal_api$setup_analysis_BANG_.invokeStatic(internal_api.clj:62)
  SP 0x00007fff1611eda0 IP 0x0000000001459c76  [image code] clojure_lsp.internal_api$setup_analysis_BANG_.invoke(internal_api.clj:54)
  SP 0x00007fff1611edb0 IP 0x000000000144d9fd  [image code] clojure_lsp.internal_api$diagnostics.invokeStatic(internal_api.clj:171)
  SP 0x00007fff1611ee30 IP 0x000000000144d7f6  [image code] clojure_lsp.internal_api$diagnostics.invoke(internal_api.clj:170)
  SP 0x00007fff1611ee40 IP 0x0000000001329dac  [image code] clojure_lsp.api$diagnostics.invokeStatic(api.clj:127)
wanted to share this to say that it seems like a cool setup but I ran into some issues with it. And maybe see if someone has pointers 🙂

ericdallo16:01:59

@borkdude any clue of what is that exception?

ericdallo16:01:41

It seems to be a issue when reading the clojure-lsp home config file

ericdallo16:01:56

On the getProperty of the user dir

ericdallo16:01:02

Are you aware of any issues with babasha pods when accessing JVM properties?

Phillip Mates16:01:29

it fails on the CI differently

ericdallo16:01:02

This is the step where bb downloads the pods from pod registry

ericdallo16:01:18

Maybe some network issue with GitHub or something?

ericdallo16:01:26

Maybe a Too many requests?

borkdude16:01:39

> This is the step where bb downloads the pods from pod registry No, you can see the pod was already started in the 0%, 5%, etc output

borkdude16:01:17

I'm not sure why this happens in the pod, I don't see any reason it should behave differently, except that this is the static-compiled binary

borkdude16:01:32

So you might be able to repro this using the static binary as well.

Phillip Mates16:01:50

(println (.exists (io/file (System/getProperty "user.home") ".config"))) ;; => true is able to run fine in the context that then blows up

borkdude16:01:24

I'm not sure if @UKFSJSM38 compiles this using musl or ...

borkdude16:01:14

I would either use musl or -H:+StaticExecutableWithDynamicLibC

borkdude16:01:00

The --static option without other options isn't recommended anymore and is likely to cause errors. https://www.graalvm.org/reference-manual/native-image/StaticImages/

borkdude17:01:23

I bet @UFDRD93RR can help you with getting the musl environment going

ericdallo17:01:50

Musl doesn't work with datalevin

borkdude17:01:15

ok, then try -H:+StaticExecutableWithDynamicLibC

borkdude17:01:29

don't use --static without any other opts, it's buggy, they don't recommend it

ericdallo17:01:41

Good to know

borkdude17:01:15

but it would be good if @U02G8N0EX44 could try to reproduce with the static binary first. I expect it would behave the same as the pod (as that is the static binary!)

borkdude17:01:33

I mean, invoke it as the command line tool

Phillip Mates17:01:37

I downloaded that .zip and ran it on machine and it crashed too

borkdude17:01:01

cool, so now we know the --static binary is b0rked and it isn't pod-specific

borkdude17:01:13

are you using nixos @U02G8N0EX44?

borkdude17:01:46

oh yeah. well, @UKFSJSM38 could you add that flag? then we can try again and see if this fixes it.

ericdallo17:01:10

yes, I'll add to the CI and then open a branch just to compile the native one

kokada17:01:23

Keep in mind that -H:+StaticExecutableWithDynamicLibC will make the binary not work in NixOS or any distro that doesn't use glibc

👍 1
kokada17:01:32

(Alpine for example)

borkdude17:01:52

yes, but it doesn't work in alpine now anyway

ericdallo17:01:05

I think it does

borkdude17:01:41

the issue is that I have to select one specific binary for the pod, and I chose the static one as that is the most likely to work in most environments

kokada17:01:42

--static without -H:+StaticExecutableWithDynamicLibC should work in Alpine because it links glibc statically

kokada17:01:09

This is not recommended even by glibc folks, but kinda works unless you do things like DNS resolution

kokada17:01:55

BTW, not really sure if NixOS support is that much interesting, because NixOS has a proper clojure-lsp package

borkdude17:01:58

but this binary crashes so what is the use

kokada17:01:01

The only issue would be Alpine

ericdallo17:01:14

yes, I agree

borkdude17:01:33

we can at least try if that flag "fixes" it

borkdude17:01:43

and else we'll have to make an issue with graalvm

ericdallo17:01:47

yes, I'm on it

borkdude17:01:31

build finished, how can we download?

ericdallo17:01:37

I forgot to add the upload facepalm

ericdallo17:01:19

With the new approval github actions, I think we can create a action that trigger only with manual input, so we can have a action to build graalvm from master only when we want, it'd be handy

ericdallo18:01:15

Here you can download the static binary with that new flag: https://github.com/clojure-lsp/clojure-lsp/actions/runs/1664186105

ericdallo18:01:40

(BTW I changed clojure-lsp native binary build on CI to build the static one always)

ericdallo18:01:46

Just tested with your bb script + that binary and it worked to me

ericdallo18:01:39

Actually, I tested with this one:

(ns clean-ns
  (:require [babashka.pods :as pods]))

(pods/load-pod "/tmp/clojure-lsp/clojure-lsp")

(require '[clojure-lsp.api :as api])

(defn dry []
  (api/clean-ns! {:dry? true}))

(defn fix! []
  (api/clean-ns! {}))

(boolean (dry))

borkdude18:01:59

@UKFSJSM38 I don't see where I can download the binary. Do you have a direct link?

borkdude19:01:48

awesome! @U02G8N0EX44 can you try that one and see if it doesn't crash?

borkdude19:01:05

you can also load it as a pod by doing (pods/load-pod "./clojure-lsp") on the local binary

☝️ 1
Phillip Mates09:01:54

yes, that new binary works on the CI, thanks!

Phillip Mates09:01:34

is there a good way to completely skip the analysis step when doing things like clean-ns? or does it need it to remove unused ns requires?

borkdude09:01:32

@UKFSJSM38 Yes, I think there should be an option to skip analysis, which isn't necessary for clean-ns and formatting I think. Or perhaps the analysis can be scoped to only the project sources (which should take only a few seconds).

borkdude09:01:01

Good news! I'll change the pod to use the "mostly static" binary. Is that now the default setting for the "normal" linux binary @UKFSJSM38?

ericdallo10:01:12

@U02G8N0EX44 I have plans to improve that in the future 🤞

nice 1
borkdude10:01:10

ok, we'll wait for the next release then :)

Phillip Mates16:01:57

with that out of the way, I also wanted to say that I finally set up clojure-lsp in my nvim editor and it is sooooo great! Blown away by how awesome having things like "bind-to-let" and "extract-to-function" etc, so thanks to everyone who has made this! ❤️

gratitude 2
❤️ 1
ericdallo16:01:37

Thank you gratitude glad you like it!