Fork me on GitHub

Hey all, I don't know if this query is relevant to this channel so please redirect me if I am in the wrong place. I was using on windows and can't get my commands to run. I get the error "The system cannot find the file specified" For example:

user=> (shell "where npm")
D:\Program Files\nodejs\npm
D:\Program Files\nodejs\npm.cmd
{:proc #object[java.lang.ProcessImpl 0x1a291b28 "Process[pid=28444, exitValue=0]"], :exit 0, :in #object[java.lang.ProcessBuilder$NullOutputStream 0x19faa00c "java.lang.ProcessBuilder$NullOutputStream@19faa00c"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x57fe8cde "java.lang.ProcessBuilder$NullInputStream@57fe8cde"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x57fe8cde "java.lang.ProcessBuilder$NullInputStream@57fe8cde"], :prev nil, :cmd ["where" "npm"]}

user=> (shell "npm") Cannot run program "npm": CreateProcess error=2, The system cannot find the file specified [at <repl>:2:1]


For other commands, it works

user=> (shell "where clj-kondo")
{:proc #object[java.lang.ProcessImpl 0x7e17f120 "Process[pid=11688, exitValue=0]"], :exit 0, :in #object[java.lang.ProcessBuilder$NullOutputStream 0x19faa00c "java.lang.ProcessBuilder$NullOutputStream@19faa00c"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x57fe8cde "java.lang.ProcessBuilder$NullInputStream@57fe8cde"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x57fe8cde "java.lang.ProcessBuilder$NullInputStream@57fe8cde"], :prev nil, :cmd ["where" "clj-kondo"]}       
user=> (shell "clj-kondo")       
clj-kondo v2021.06.18

Lots of lines at the end removed 


I'm pretty sure all are in the path (as where could find it), but some commands seem to work while others get "The system cannot find the file specified"


Not much experience with windows, but it does sound like an issue with paths, even if where can find it outside of babashka. You may want to check if the path has been exported to babashka - which runs a sub-shell. In *nix shells, typically custom set path/env variables will not be passed on to subshells if they are not exported. It depends on what startup files a shell uses for config - some come from /etc, some from $HOME.


In babashka shell, try echo-ing the $PATH to confirm, maybe?


The thing is, I couldn't even get echo to work without calling the command prompt from within babashka

user=> (shell "echo %PATH%")       Cannot run program "echo": CreateProcess error=2, The system cannot find the file specified [at <repl>:2:1]
user=> (shell "cmd /C \"echo %PATH%\"")  
C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Python39\Scripts\;C:\Python39\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;D:\Program Files (x86)\Brackets\command;C:\Program Files\PuTTY\;D:\Program Files\PostgreSQL\12\bin;D:\Program Files\PostgreSQL\12\lib;C:\Program Files (x86)\gettext-iconv\bin;D:\Users\super\AppData\Roaming\npm;D:\Program Files (x86)\Yarn\bin\;C:\Program Files\dotnet\;D:\src\flutter\bin;D:\Program Files\Calibre2\;D:\Program Files\nodejs\;C:\ProgramData\chocolatey\bin;D:\Users\super\AppData\Roaming\npm-cache;C:\Users\super\scoop\shims;C:\Users\super\AppData\Local\Programs\Python\Python37\Scripts\;C:\Users\super\AppData\Local\Programs\Python\Python37\;C:\Users\super\AppData\Local\Programs\Python\Python38\Scripts\;C:\Users\super\AppData\Local\Programs\Python\Python38\;C:\Users\super\AppData\Local\Microsoft\WindowsApps;C:\Users\super\AppData\Local\Programs\Microsoft VS Code\bin;D:\Users\super\AppData\Local\Programs\Git\cmd;C:\Users\super\AppData\Roaming\npm
{:proc #object[java.lang.ProcessImpl 0x443ea63f "Process[pid=16732, exitValue=0]"], :exit 0, :in #object[java.lang.ProcessBuilder$NullOutputStream 0x19faa00c "java.lang.ProcessBuilder$NullOutputStream@19faa00c"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x57fe8cde "java.lang.ProcessBuilder$NullInputStream@57fe8cde"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x57fe8cde "java.lang.ProcessBuilder$NullInputStream@57fe8cde"], :prev nil, :cmd ["cmd" "/C" "echo %PATH%"]}


Ah. Don’t have a windows machine handy to help, but honestly, I don’t know what is the default shell when invoking in windows, and whether that is configurable. And as I re-read your original, I realize I did not completely parse your output. So, interestingly, where worked from babashka alright! Interesting.


No worries @U06KGH6LB, maybe someone else will be able to help Now that I've explored it more, the problem can also be seen when using , which makes me think it is a Java / java config thing in windows.

user=> (shell/sh "where" "npm")
{:exit 0, :out "D:\\Program Files\\nodejs\\npm\r\nD:\\Program Files\\nodejs\\npm.cmd\r\n", :err ""}

user=> (shell/sh "npm")         
Execution error (IOException) at java.lang.ProcessImpl/create (REPL:-2).
CreateProcess error=2, The system cannot find the file specified 
user=> (shell/sh "where" "clj-kondo")
{:exit 0, :out "C:\\Users\\super\\scoop\\shims\\clj-kondo.exe\r\n", :err ""}

user=> (shell/sh "clj-kondo")         
{:exit 0, :out "clj-kondo v2021.06.18\r\n\n\nOptions:\n\n  --lint <file>: a file can either be a normal file, directory or classpath. In the\n    case of a directory or classpath, only .clj, .cljs and .cljc will be\n    processed. Use - as filename for reading from stdin.\n\n  --lang <lang>: if lang cannot be derived from the file extension this option will be\n    used. Supported values: clj, cljs, cljc.\n\n  --filename <file>: in case stdin is used for linting, use this to set the\n    reported filename.\n\n  --cache-dir: when this option is provided, the cache will be resolved to this\n    directory. If --cache is false, this option will be ignored.\n\n  --cache: if false, won't use cache. Otherwise, will try to resolve cache\n  using `--cache-dir`. If `--cache-dir` is not set, cache is resolved using the\n  nearest `.clj-kondo` directory in the current and parent directories.\n\n  --config <config>: config may be a file or an EDN expression. See\n    \n\n  --config-dir <config-dir>: use this config directory instead of auto-detected\n    .clj-kondo dir.\n\n  --parallel: lint sources in parallel.\n\n  --dependencies: don't report any findings. Useful for populating cache while linting dependencies.\n\n  --copy-configs: copy configs from dependencies while linting.\n\n  --fail-level <level>: minimum severity for exit with error code.  Supported values:\n    warning, error.  The default level if unspecified is warning.\n\r\n", :err ""}


Perhaps npm isn’t a normal executable?


Perhaps try npm.cmd ?


Damn that was it


Adding .CMD to a command would probably make it stop working on Linux/macos though. Do you know of any elegant solutions? ( Right now can't think of much more than an if statement 😅 )


I usually use a predicate that checks for Windows + if


See borkdude/jayfu’s bb.edn

Karol Wójcik10:07:04

I'm looking for universal way (graalvm + babashka) of setting an environment variable from Clojure code. Does anyone have some experience in it?


probably needs a custom bb build?


This is unsupported, it doesn't work in combination with System/getenv


unless you write your own C-based getenv, but libraries all go through System/getenv, so it doesn't compose

Karol Wójcik10:07:37

I need something which does work with System/getenv


Java caches the environment at startup, this is why it doesn't work

Karol Wójcik10:07:10

I don't know whether it could work that way


well yes, you can spawn a new process of course, that always works. but it's a new process

Karol Wójcik10:07:06

What about this one:

@SuppressWarnings({ "unchecked" })
  public static void updateEnv(String name, String val) throws ReflectiveOperationException {
    Map<String, String> env = System.getenv();
    Field field = env.getClass().getDeclaredField("m");
    ((Map<String, String>) field.get(env)).put(name, val);

Karol Wójcik10:07:28

I will check that out

Karol Wójcik10:07:26

This one works

Karol Wójcik10:07:47

But not for babashka..


Stackoverflow mentions it also doesn't work on Windows

Karol Wójcik10:07:45

That's not a problem

Karol Wójcik10:07:49

It will run on lambda environment

Karol Wójcik10:07:16

I need to figure it out how to use it on babashka side

Karol Wójcik10:07:59

For obvious reasons I cannot import that class in babashka. Any ideas how to overcome this?


which class?

Karol Wójcik10:07:17

I have made this class:

package fierycod.holy_lambda;

import java.lang.reflect.Field;
import java.util.Map;

public class UpdateEnv {
  @SuppressWarnings({ "unchecked" })
  public static void updateEnv(String name, String val) {
    try {
      Map<String, String> env = System.getenv();
      Field field = env.getClass().getDeclaredField("m");
      ((Map<String, String>) field.get(env)).put(name, val);
    } catch (ReflectiveOperationException e) {
      System.err.println("[holy-lambda] Unable to patch in memory environment variable: " + name + ".Exiting!");

Karol Wójcik10:07:44

I have compiled it an checked whether it works on Clojure side

Karol Wójcik10:07:07

It worked, so I switched to babashka and I couldn't import this class

Karol Wójcik10:07:20

Of course I know why


you can translate this code to Clojure perhaps


so you don't have to compile Java


This works:

(set! *warn-on-reflection* true)

(defn set-env [name val]
  (let [env (System/getenv)
        clazz (class env)
        field (.getDeclaredField clazz "m")
        _ (.setAccessible field true)
        the-map (.get field env)
        _ (.put ^java.util.Map the-map name val)]))

(set-env "FOO" "BAR")


but the output is:

$ clj -M /tmp/setenv.clj
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by user$set_env to field java.util.Collections$UnmodifiableMap.m
WARNING: Please consider reporting this to the maintainers of user$set_env
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release


so I don't think this is a good idea ;)

Karol Wójcik11:07:21

Yeah. I have tried it. It works on Clojure side. On bb I got Unable to resolve classname: java.lang.reflect.Field

Karol Wójcik11:07:59

I have to somehow change the environment variable. I know that it's a hack. I really do, but I don't know how to do it otherwise


it's a hack that will only work in java 11, but when the next LTS comes out it won't work anymore, so it's better to not go this way


I just listened to a podcast "Java Insiders" about this whole topic

Karol Wójcik11:07:14

In lambda context it doesn't matter I think. I'm forcing use of java8 in HL


"Inside Java" it's called


well, bb is compiled with Java 11 already


and it will probably upgrade to the newest LTS. Note that on macOS only Java 11 is available


btw, in bb I get:

$ bb /tmp/setenv.clj
----- Error --------------------------------------------------------------------
Type:     java.lang.NoSuchFieldException
Message:  m
Location: 4:3
with the code I posted above


so I bet this won't work in a native image anyway


unless you perhaps allow reflection on UnmodifiableMap something something in the reflection config


but it's a hack

Karol Wójcik11:07:44

Which you probably won't accept on bb side


well, perhaps first try to make it work in a normal native image and see what happens. I can't promise anything


why don't you launch another process with the correct environment set?

Karol Wójcik11:07:22

Wouldn't it be slower?

Karol Wójcik11:07:38

I think I don't much understand your question


why do you need to set an environment variable during the duration of the program. can you set it before launching the program?

Karol Wójcik11:07:01

Because in the loop I'm fetching new AWS events, which contain a AWS_TRACE_ID, that I have to attach to environment variable


don't all Java / native image programs have this problem? why must this be solved through env vars? which libraries are reading these env vars?

Karol Wójcik11:07:10

Some code for context.

(defn- next-iter
  [routes env-vars]
  (let [runtime (get env-vars "AWS_LAMBDA_RUNTIME_API")
        handler-name (get env-vars "_HANDLER")
        aws-event (fetch-aws-event runtime)
        handler (get routes handler-name)
        iid (:invocation-id aws-event)]

    ;; aws-event contains a header with trace-id
    ;; I need bound the trace id to environment ariable right here.

    (when-not handler
      (send-runtime-error runtime iid (->ex (str "Handler " handler-name " not found!")))
      (System/exit 1))

    (when-not iid
      (send-runtime-error runtime iid (->ex (str "Failed to determine new invocation-id. Invocation id is:" iid)))
      (System/exit 1))

    (when (and (:invocation-id aws-event) (u/success-code? (:status aws-event)))
      (process-event runtime iid aws-event env-vars (u/call handler)))))
AWS SDK v2 aws-xray library. It's an must have in Custom Lambda runtime, as specified here in the documentation: • Propagate the tracing header – Get the X-Ray tracing header from the `Lambda-Runtime-Trace-Id` header in the API response. Set the `XAMZN_TRACE_ID` environment variable locally with the same value. The X-Ray SDK uses this value to connect trace data between services.

Karol Wójcik11:07:51

aws-xray now does not allow to overwrite trace id AFAIK 😂


Well, this design obviously doesn't play well with Java programs. I suggest discussing this with this AWS architect we had this discussion with on the GraalVM slack

Karol Wójcik11:07:01

For instance quarkus substitutes the xray class which their own implementation of TraceHeader class which takes the trace id not from environment variable, but from outer context 😂

Karol Wójcik11:07:20

Yeah I'm talking with Mark right now about it.

Karol Wójcik11:07:36

Thank you @U04V15CAJ! You convinced me about not making a hack!

Karol Wójcik13:07:09

It seems that I don't longer need setting an environment variable. 😄 AWS x-ray is going to introduce a system property check for trace id! @U04V15CAJ, you always save me from making those little dirty hacks! Thank you so much for you patience and hard work! I do learn a lot from you 🙂

👍 4
Nicolas Estrada10:07:26

Hi everybody has anyone been able to get bb to work in a synology linux distrib?

Nicolas Estrada10:07:52

$ uname -a
Linux moria 3.2.40 #25556 SMP PREEMPT Thu Mar 4 17:49:46 CST 2021 i686 GNU/Linux synology_evansport_415play


What is special about this linux? os it glibc based?


ah i686, this is 32 bit, probably not supported

Nicolas Estrada10:07:12

too bad, I was hoping with replacing my groovy scripts with bb ones


bb depends on graalvm which only has compilers for 64 bit:

Nicolas Estrada10:07:31

yes I understand oh well, I'll make do and hope one day there will be a graalvm for IoT or older architectures 😅


I was very close to releasing bb 0.5.0 but last minute I detected a problem with timbre: it's bloating bb with 45 MB and spikes compile time. A problem I didn't have before so I'm backtracking/trying to find what it pulls which causes this and trying to fix it. It may take a few hours.

👍 6

I've built bb from master and tried vault-clj again - however it failed with Unable to resolve classname: . which is kind of strange - I expected that class to be so common that it would be part of bb. Also:

ls -lah /usr/local/bin/bb 
-rwxr-xr-x 1 ieugen ieugen 132M iul 14 21:34 /usr/local/bin/bb


@eugen.stan Not all classes are available in bb, they are only added if necessary, on a case by case basis. It's best to make your own branch and see how far you get by adding the class to classes.clj, so you can experiment on your own and then at the end see what should be added and if it's worth it in terms of added binary size, compile time. I'm currently fixing a different problem (see last message).


yes, I understand - however I imagined that BufferedInputStream is a more common occurence - and I was surprised - that's all .

👍 2

Nobody ever needed it so far, apparently


vault-clj does not seem to work with bb because of clj-http - it requires some java libraries - quite a few of them. see issue for details.


I responded in that github issue. you can try to replace the usage with a built-in http client


you are fast 🙂


btw, there is a talk tonight about mirabelle at the Dutch Clojure Meetup, a monitoring system in clojure, might be interesting for you


I am not faimilar with the meetup


do you have a link?


the talk is already done


found the link


I joined the group for future


London Clojurians is also a nice online meetup


(I'm doing a talk next week about bb tasks...)


I replaced clj-http with httpkit (API seems to be compatible) - at least tests pass


I'm thining how I can test this since vault-clj provides leiningen, not deps.edn


maybe make a deps.edn


I guess I could provide deps.edn myself locally or just add vault-clj/src to bb paths?


yes, both should work


httpkit returns delays btw, not direct responses


so there is a difference in API regarding that


babashka.curl might be closer


but if you have to touch every spot anyway, it's only adding an @ which makes the difference with httpkit


ok, thanks


there is just one case where this needs to be done and a lot in tests


so no issue here I believe


I've ended up getting to log errors:

Checking out:  at 2d9a59d9d29bea963c6371b45bf93e25fd3ff297
----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:  Could not resolve symbol: log/trace
Location: /home/ieugen/.gitlibs/libs/amperity/vault-clj/2d9a59d9d29bea963c6371b45bf93e25fd3ff297/src/vault/lease.clj:202:4
Phase:    analysis

----- Context ------------------------------------------------------------------
200: (defn ^:no-doc maintain-leases!
201:   [client window]
202:   (log/trace "Checking for renewable leases...")
        ^--- Could not resolve symbol: log/trace
203:   ; Check auth token for renewal.
204:   (let [auth @(:auth client)]
205:     (when (and (:renewable auth)
206:                (expires-within? auth window))
207:       (try


ah is this from tools logging?


trace should be available on master


I've built babashka locally from master


master newest?


bb --version 
babashka v0.4.7-SNAPSHOT


commit 2932e246522f976af1409ee9346a56fc1439bd55 (HEAD -> master, upstream/master)
Author: Michiel Borkent <[email protected]>
Date:   Wed Jul 14 16:31:56 2021 +0200



will build again


ok, I just pushed


another commit


which fixes the BLOAT PROBLEM


ok, will test with this, rebuild and test again. if that does not work maybe you can clone and run

bb vault-get-dev
after you change the git sha commit in bb.edn to point to latest branch commit


first build same issue, building with latest commit


;; Type bb tasks to see all tasks
;; Type bb <task-name> or bb run <task-name> to run a task
{:min-bb-version "0.4.0"
 :paths ["script"]
 :deps {amperity/vault-clj {:git/url ""
                            :sha "2d9a59d9d29bea963c6371b45bf93e25fd3ff297"}
        org.clojure/tools.logging {:mvn/version "1.1.0"}}
 :tasks {:requires (
                    ;; [vault.core :as vault]
                    ;; [vault.client.http]
                    [ :as log])
        ;;  :init     (do
        ;;              (def client (vault/new-client "")))
        ;;  ;; Helpers
        ;;  vault-get-dev {:doc "Get dev secrets from vault - requires vault cli."
        ;;                 :task (do
        ;;                         (client))}
         log-me {:doc "Log test"
                       :task (do
                               (log/info "hello")
                               (log/trace "hmm"))}}}
bb log-me
----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:  Could not resolve symbol: log/trace
Location: <expr>:24:25
Phase:    analysis

----- Context ------------------------------------------------------------------
20: nil
21: (def log-me (binding [
22:   babashka.tasks/*task* '{:name log-me, :doc "Log test", :task (do (log/info "hello") (log/trace "hmm"))}]
23:   nil
24: (do (log/info "hello") (log/trace "hmm")))) log-me


there is an issue with trace


do you have a way to add tests for bb features ?


tests that can run after build?


@eugen.stan Yes. trace is missing. I will add it


I do have tests, but not for everything regarding logging yet, since it is so now


I added all of these:

['debug 'debugf 'info 'infof 'warn 'warnf 'error 'errorf
                                              'logp 'logf '*logger-factory* 'log*]


but trace wasn't there yet


@eugen.stan pushed the trace commit

👍 2

I'm debugging vault-clj


ok, I will finish the release and will continue tomorrow.


so vault-clj works with babashka now and I hope to upstream some of the changes there


it's going to require some cleanup though


nice work on your side too


it's nice you got so far. I guess you can already start using your fork locally for testing right


btw, the IllegalArgumentException change might not be needed: this is available in bb


yep, that one works now - IllegalStateException does not


I'm getting a bit sick and tired of this timbre problem, which showed up just before release.


24d1de51a018fc0f3576febe6ae0eee76f6619d4 was the commit which didn't exhibit this problem


3d45f62dd20d4592dea8e0106af4ca46fca95974 started showing the bloat (which you can see in #babashka-circleci-builds, binary size went up with 45mb), but I don't see a logical explanation of that


yep, 24d1 seems ok locally as well still


I'm taking very very tiny steps and compiling to native at every step...


a very exciting activity I imagine 🙂


very tedious and tiring. I have spent half my day on this :-S

😞 2

and @U051T4Q05 might make the talk available later


oh wrong thread


I think I finally found it.


Yes, I FOUND IT!!! I ran into an issue with pprint but the loading order of namespaces messed it up. I describe that issue here:

🎉 8
🎊 2
❤️ 2

0.5.0 coming!

👏 6

Haha, I renamed babashka.impl.timbre to babashka.impl.logging and then sorted my namespaces automatically... which changed the order of loading namespaces ;)


oh wow, so the logging loading first pulled in the unpatched version of pprint?


that is some subtle bug


I now introduced a namespace called a1 which loads the patch first 😅

🅰️ 4
😲 2

(for lack of a better name)


one alternate idea is aaaa-this-has-to-be-first-because-patches


I like that name a lot better


no need for a comment above it


will change it after the release. this was one bumpy release, it took me all day 😅 (not only this issue)

🙏 4

there was another hard problem to get pprint/formatter-out working which is used in


which is a compile-time pprint thing


but now we can load from source, for whatever that's worth ;)


and there is a new << macro in Selmer which allows you to template based on local bindings and var values, which needed a couple of iterations to get working (both in Selmer and bb)


very cool


🔥 14
🙌 10
🚀 20
💥 16

This is huge 👏


🎉 🚀 amazing! thank you for all the hard work!


is there anything in bb for encoding a string (utf8/ascii) to hex?


@kevin842 perhaps this bit of code is useful: I added this to Selmer to avoid needing some apache lib for it. You can copy and run it in bb.

👍 4