Fork me on GitHub
#babashka
<
2023-11-22
>
ritchie11:11:17

is it considered bad form to add one's own meta data to bb.edn files? and generally to deps.edn files? is there a blessed way of doing this or just don't do it

borkdude12:11:35

It’s fine as along as you use an alias or a qualified keyword eg with your org name in it

ritchie12:11:01

great thanks! that's what is was hoping for

borkdude12:11:10

To avoid conflicts with either deps.edn itself or other tools that do the same

mmer13:11:49

I have noticed (I raised this before) that running bb tasks take some time to actually start. I put a println at the start and end of the :init sections expecting to see that happen straight away, however, the delay to start seems to happen before the :init. (I don't know if println buffers so that might be skewing what I am seeing)

borkdude13:11:05

if you have a repro I could help

borkdude13:11:14

without a repro, don't know what to say actually

mmer13:11:31

OK I will try to put something together.

borkdude13:11:39

what OS + version are you using

mmer13:11:28

Windows. Babashka v1.3.182 running on powershell. I have about 50+ tasks running against a set of clojure files that are about 2300 lines of clojure in about 10 files.

borkdude13:11:12

I bet it's the amount of code being loaded in top level :requires that takes the time. You can verify this by doing a manual:

bb -e '(ns foo (:require ...))
` of those :requires

borkdude13:11:26

but without a repro it's again hard to say

mmer13:11:15

Sorry this in in house. That was my suspicion - I am not complaining just an observation.

borkdude13:11:04

in in house - do you mean, the code is in your house? :)

ritchie13:11:13

@U4C3ZU6KX just in case - if you're using taoensso.timbre logging, I've found it really slows down loading

mmer13:11:41

I mean it is inside my company and I can't share it.

borkdude13:11:47

You can maybe lazy-load some namespaces to make startup faster using (requiring-resolve 'foo.bar/the-function)

mmer13:11:56

Thanks @U05NY1NK44W I am not using that.

borkdude13:11:24

I am not sure how taoensso.timbre would make stuff load slower since the logger is baked in

borkdude13:11:20

@U4C3ZU6KX ok, but then try to remove the top level requires. It will break your code, but you can test if that is it by printing something in your :init

borkdude13:11:30

something along those lines

mmer13:11:39

Thanks will do.

👍 2
mmer10:11:18

It turned out that I must have thought of using taoensso.timbre as I had it included in the bb files requires - even though I am not using it in the other parts of of my code (I must have started and not carried it through) taking it out has changed the start time for the better. Thanks @U05NY1NK44W

🎉 1
borkdude10:11:17

@U4C3ZU6KX @U05NY1NK44W This surprises me. Can you please make a repro that shows how taoensso.timbre slows down startup time?

mmer10:11:44

I will look to do this sometime soon - will let you know when I have it working.

borkdude10:11:59

$ time bb -e "(require '[taoensso.timbre :as t])"
bb -e "(require '[taoensso.timbre :as t])"   0.02s  user 0.01s system 89% cpu 0.026 total

borkdude10:11:36

$ time bb -e "(require '[taoensso.timbre :as t]) (t/info :hello)"
2023-11-23T10:35:29.571Z m1.local INFO [user:1] - :hello
bb -e "(require '[taoensso.timbre :as t]) (t/info :hello)"   0.02s  user 0.01s system 82% cpu 0.037 total

mmer10:11:22

Thanks @U04V15CAJ All I can say is that with tasks startup it made a difference. Thanks for looking.

ritchie12:11:45

@U04V15CAJ, for me it added a 3 second startup time.

mmer12:11:45

That is what I am seeing.

borkdude12:11:35

@U05NY1NK44W the exact example I wrote above?

ritchie12:11:16

no, when i added it to my application

borkdude12:11:30

please provide a repro

borkdude12:11:38

if you have time to make one

borkdude12:11:46

so we can solve this problem for everyone

mmer12:11:29

Here is the start of my bb.edn file - it has some

{:paths ["src", "test"]
 :deps {com.rpl/specter {:mvn/version "1.1.4"}
        org.clj-commons/clj-http-lite {:mvn/version "1.0.13"}
        metosin/malli {:mvn/version "0.11.0"}
        clj-commons/clj-yaml {:mvn/version "1.0.26"}
        etaoin/etaoin {:mvn/version "1.0.40"}
        com.github.oliyh/martian-babashka-http-client {:mvn/version "0.1.25"}}
 :tasks {:requires ([etaoin.api :as e]
                    [ogsetup.core :as  core]
                    ...
                    [testing :as testing]
                    [cheshire.core :as json]
                    [clojure.tools.cli :as cli]
                    [babashka.curl :as curl]
                    [babashka.fs :as fs]
                    [taoensso.timbre :as timbre])
         :init (do
                 (def cli (cli/parse-opts *command-line-args* core/cli-options))
                 (def options (:options cli))
                 (utils/set_config (utils/read-edn-file "" (:config options)))
                 (utils/set-user (:user options))
                 (when (:info options) (do (utils/set-verbosity :info) (add-tap println) (timbre/set-level! :info)))
                 (when (:debug options) (do (utils/set-verbosity :debug) (add-tap println) (timbre/set-level! :debug)))
                 (when (:help options) (println (:summary (cli/parse-opts *command-line-args* core/cli-options))))
                 (utils/tapd-> (utils/get_config (utils/get-user)))
                 (utils/tapd-> options))
         help (println (:summary (cli/parse-opts *command-line-args* core/cli-options)))}

ritchie12:11:33

will do, it won't be today though

👍 1
borkdude12:11:15

preferably a github project I can run locally

ritchie12:11:17

fyi, i don't use the logging from tasks,

mmer12:11:22

Sorry it has cruft that you won't need but just removing the timbre lines - improved the performance

ritchie12:11:45

@U04V15CAJ just tried your example

ritchie12:11:50

time bb -e "(require '[taoensso.timbre :as t]) (t/info :hello)" 2023-11-23T12:59:13.896Z UnknownHost INFO [user:1] - :hello ______________________________________________________ Executed in 5.01 secs fish external usr time 8.64 millis 384.00 micros 8.26 millis sys time 3.83 millis 117.00 micros 3.72 millis

ritchie13:11:35

bet you it's the unknown host, some timeout trying to get hostname or something

borkdude13:11:08

@U4C3ZU6KX A standalone example would be something like:

{:paths ["src", "test"]
 :deps {com.rpl/specter {:mvn/version "1.1.4"}
        org.clj-commons/clj-http-lite {:mvn/version "1.0.13"}
        metosin/malli {:mvn/version "0.11.0"}
        clj-commons/clj-yaml {:mvn/version "1.0.26"}
        etaoin/etaoin {:mvn/version "1.0.40"}
        com.github.oliyh/martian-babashka-http-client {:mvn/version "0.1.25"}}
 :tasks {:requires ([etaoin.api :as e]
                    [cheshire.core :as json]
                    [clojure.tools.cli :as cli]
                    [babashka.curl :as curl]
                    [babashka.fs :as fs]
                    [taoensso.timbre :as timbre])
         :init (do
                 (def cli (cli/parse-opts *command-line-args* {}))
                 (def options (:options cli))
                 (when (:info options) (timbre/set-level! :info))
                 (when (:debug options) (do (add-tap println) (timbre/set-level! :debug)))
                 (when (:help options) (println (:summary (cli/parse-opts *command-line-args* {})))))
         help (println (:summary (cli/parse-opts *command-line-args* {})))}
 }
without any external cruft. Either that or provide a github repro I can run locally. Can't reproduce with this example.

ritchie13:11:29

i running on fedora, but i do have a hostname ok

mmer13:11:48

Ok sorry @U04V15CAJ, I will try.

ritchie13:11:12

@U4C3ZU6KX, did you run @Borkdudes example, do you have a hostname ok?

borkdude13:11:40

Can you try:

bb -e '(.getHostName (java.net.InetAddress/getLocalHost))'

ritchie13:11:09

ritchie@blackdog ~> bb -e '(.getHostName (http://java.net.InetAddress/getLocalHost))' ----- Error -------------------------------------------------------------------- Type: java.net.UnknownHostException Message: blackdog: blackdog: Try again Location: <expr>:1:1 ----- Context ------------------------------------------------------------------ 1: (.getHostName (http://java.net.InetAddress/getLocalHost)) ^--- blackdog: blackdog: Try again ----- Stack trace -------------------------------------------------------------- user - <expr>:1:1

ritchie13:11:15

that's the problem

borkdude13:11:25

and how long does it take?

ritchie13:11:26

it took a few secs

ritchie13:11:53

Executed in 5.01 secs fish external usr time 6.38 millis 0.00 micros 6.38 millis sys time 3.72 millis 508.00 micros 3.21 millis

ritchie13:11:21

ok, so maybe a dns misconfiguration

mmer13:11:27

I am running on windows and so might that might be the cause.

borkdude13:11:49

@U4C3ZU6KX try the snippet and find out

ritchie13:11:45

if i add my hostname into /etc/hosts file then it works fine

ritchie13:11:09

i probably altered my hostname manually at some point in the past,

ritchie13:11:30

but /etc/hosts only contained localhost.localdomain et al

ritchie13:11:42

time bb -e '(.getHostName (http://java.net.InetAddress/getLocalHost))' "blackdog" ______________________________________________________ Executed in 11.89 millis fish external

borkdude13:11:52

might be worth filing an issue with timbre about this

ritchie13:11:24

well, it's really a local misconfiguration in a way

borkdude13:11:40

cool, maybe something to add in the timbre README as a gotcha

mmer13:11:08

powershell:

Measure-Command {start-process bb get-help -Wait}           


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 2
Milliseconds      : 348
Ticks             : 23487645
TotalDays         : 2.71847743055556E-05
TotalHours        : 0.000652434583333333
TotalMinutes      : 0.039146075
TotalSeconds      : 2.3487645
TotalMilliseconds : 2348.7645

borkdude13:11:29

2.3s seems like the same problem

borkdude13:11:39

and what was the return value

ritchie13:11:43

summary: my hostname in /etc/hostname was blackdog, but this was not entered into /etc/hosts

mmer13:11:57

I think this may be because on windows the concept of hostname may be different so could be the same issue.

ritchie13:11:02

adding blackdog to the 127.0.0.1 list of names fixed the issue

borkdude13:11:20

@U4C3ZU6KX what was the return value...?

mmer13:11:09

It would normally have given the cli help output, but Measure-command seems to swallow the return value.

borkdude13:11:28

try without the time then?

borkdude13:11:52

I'll boot my own windows machine

ritchie13:11:21

yay! i can use logging again

borkdude13:11:36

C:\Users\borkdude>bb -e "(.getHostName (java.net.InetAddress/getLocalHost))"
"DESKTOP-JN2UNTV"

borkdude13:11:18

@U4C3ZU6KX can you try the same thing as I did above in cmd.exe or powershell?

borkdude13:11:24

and paste the result here

mmer13:11:22

Which command?

mmer13:11:38

the getlocalHost worked as yours did.

borkdude13:11:07

What did you get as a return value (third time is a charm...)?

mmer13:11:34

"DESKTOP-..."

borkdude13:11:45

alright. in both powershell and cmd.exe?

mmer13:11:29

Yes same.

mmer13:11:50

Sorry I am in a work meeting

borkdude13:11:02

no problem. then I think the root cause of your delay might not be timbre

borkdude13:11:09

or it's a different issue in timbre