Fork me on GitHub
#babashka
<
2020-10-16
>
borkdude11:10:48

If I would do one more talk this year about bb, would you like me to go into: 1) babashka / sci internals 2) dedicated talk on babashka pods 3) other (respond in thread) Vote here: https://twitter.com/borkdude/status/1317072035542171650

borkdude11:10:48

If I would do one more talk this year about bb, would you like me to go into: 1) babashka / sci internals 2) dedicated talk on babashka pods 3) other (respond in thread) Vote here: https://twitter.com/borkdude/status/1317072035542171650

mitchell_clojure14:10:03

I notice that *file* isn't bound when I am using the nrepl server (which makes sense). Is the best way to work around this to just manually bind it during my repl sessions?

nate14:10:21

I would recommend something like this, where most of the code that deals with the filename takes it as an argument:

(defn process
 [filename]
 (println filename))

(when *file*
  (process *file*))

borkdude15:10:26

@ Are you running into an unexpected error because of this?

mitchell_clojure15:10:56

Thanks, @, I think that is the way I am going (or setting a default value for *file* if not bound). @ no breaking errors because of this, just trying to make my REPL flow ergonomic. I noticed that evaluating the buffer from cider did not produce the same result as running the script. It was simple to track down the cause and work around. I am not sure if the bb nrepl server can see which files forms come from, but that is the only place where I think this could be fixed

mitchell_clojure15:10:29

it is small potatoes regardless and maybe not worth rebinding the var root every time the nrepl server gets a message

borkdude16:10:19

@ Can you give an example of how to reproduce this difference? I do have emacs + CIDER

borkdude16:10:52

I see, can reproduce it

mitchell_clojure16:10:08

ah cool, was just about to send the repro

borkdude15:10:15

I've made a couple of updates to babashka process: https://github.com/babashka/process. Breaking changes: - use slurp to realize output into strings, :out and :err are always streams - :exit is always a delay, forcing the delay will wait for the process to end - :throw is removed, use (wait (process ...) {:throw true}) to explicitly throw

borkdude16:10:57

I think I solved most of the problems I had doubts about in this lib. Might be ready for inclusion in bb. One issue I could not solve: https://github.com/babashka/process#piping-infinite-input It seems that io/copy of an infinite input stream never flushes to the output stream or something?

borkdude17:10:31

One thing of doubt: should (wait (process ...)) throw by default when the exit code is non-zero? Now it only does so when (wait (process ....) {:throw true})

borkdude17:10:34

I think it might make sense to throw by default...

(-> (process ["ls" "foo"]) (wait) (process ["cat"]))
would then succesfully crash because there is no foo

nate18:10:50

I think throw by default makes sense

nate18:10:19

can you do a {:throw false} if you don't want it to throw?

borkdude18:10:01

The above can now also be written as:

(-> @(process ["ls" "foo"]) @(process ["cat"]))
because of an IDeref implementation :-) Yes, throw false is an option.

nate18:10:26

hm, a couple of thoughts to consider: 1. default in bash is not to throw an exception (as in it doesn't exit the script) 2. defaults in bash might not be a sensible thing to follow

nate18:10:08

does clojure.java.shell throw if the command errors?

nate18:10:27

I can't remember

borkdude18:10:43

no, it doesn't. which is something I probably always wrap myself, or not, if I'm too lazy

borkdude18:10:39

I'm trying to completely mimic clojure.java.shell either though. if you want to use that, use that

borkdude18:10:23

From the docs: The function wait takes a process, waits for it to finish and returns it. When the exit code is non-zero, it will throw, which can be turned off with `{:throw false}`.

clojure
user=> (-> (process ["ls" "foo"]) wait :exit deref)
Execution error (ExceptionInfo) at babashka.process/wait (process.clj:74).
ls: foo: No such file or directory
clojure
user=> (-> (process ["ls" "foo"]) (wait {:throw false}) :exit deref)
1
The return value of process implements clojure.lang.IDeref. When dereferenced, it will execute wait with default option :throw true:
clojure
user=> @(process ["ls" "foo"])
Execution error (ExceptionInfo) at babashka.process/wait (process.clj:74).
ls: foo: No such file or directory

nate18:10:21

Looks very good

borkdude20:10:04

I found the issue with piping infinite streams. If I put a flush in io/do-copy for stream -> stream:

(defn copy [in out opts]
  (let [buffer (make-array Byte/TYPE (buffer-size opts))]
    (loop []
      (prn :copy)
      (let [size (.read in buffer)]
        (when (pos? size)
          (.write out buffer 0 size)
          (.flush out)
          (recur))))))
then it works. I wonder if this is a good idea or not... Probably not, but now I know the issue: buffering

borkdude21:10:56

Amazing, thanks!

borkdude21:10:10

Any reason why not to use keywords as keys instead of strings btw?

isak21:10:10

Oh no, I guess it probably works either way, let me verify and update the readme

borkdude21:10:04

Just built the project on macOS as well: ` $ dotnet build Microsoft (R) Build Engine version 16.7.0+7fb82e5b2 for .NET Copyright (C) Microsoft Corporation. All rights reserved. Determining projects to restore... Restored /private/tmp/pod_sql_server/pod_sql_server.csproj (in 3.27 sec). pod_sql_server -> /private/tmp/pod_sql_server/bin/Debug/netcoreapp3.1/pod.xledger.sql_server.dll Build succeeded. 0 Warning(s) 0 Error(s) Time Elapsed 00:00:05.23

borkdude21:10:18

This works:

(require '[babashka.pods :as pods])

(pods/load-pod ["dotnet" "bin/Debug/netcoreapp3.1/pod.xledger.sql_server.dll"])

(require '[pod.xledger.sql-server :as sql])

borkdude21:10:34

But I don't have a sql server db. So I just tried the first example to see if it would produce an error, but it seems to hang

isak21:10:24

Ah interesting. Do you think that form is better? I was going to try and eventually merge it all to one .exe (it isn't yet), because I think they plan to release stuff that lets you bake in the dotnet dependency, so even people without it can run the code

isak21:10:54

Oh strange, I'll try to fix to repro + fix that

borkdude21:10:23

which form?

isak21:10:34

dotnet my.dll

borkdude21:10:47

no, on macOS it just baked a .dll, I can't run .exe here

isak21:10:57

ohh gotcha

isak21:10:52

I wonder if the IntegratedAuthentication would work on Mac, I'd guess not because I think it relies only on active directory

borkdude21:10:13

not sure. but anyway, I put some logging in the catch of the try/catch of HandleVar_Execute_Internal and it doesn't seem to reach there, it's waiting for something

borkdude21:10:12

Not sure if that's the same for you since I'm on macOS, which is a but unusual for this pod maybe

isak21:10:22

Strange, I get the exception. Does it help if you add await _writer.FlushAsync(); at the end of SendException?

borkdude21:10:50

} catch (Exception ex) {
                Console.Error.WriteLine("in try");
            
                await SendException(id, ex.Message);
            }
It doesn't print "in try"

borkdude21:10:21

but it does print earlier logs

isak21:10:45

oh. I wonder if it is a different and bizzarely high connection timeout

borkdude21:10:40

could be, I can let it sit there for a few minutes

borkdude21:10:19

Ah, after a while it seems to get stuck in a loop

isak21:10:37

Oh, for what code?

borkdude21:10:59

async Task SendException(string id, string exMessage, object exData = null) {
            Console.Error.WriteLine("dude");
        
It prints dude repeatedly

isak22:10:52

Hm I guess it must be that outer exception handler catching a new exception when sending the exception message

borkdude22:10:21

ah if I remove that line, then I get this:

7: (sql/execute! {:connection-string "Data Source=my.db.host;Application Name=my.script;Initial Catalog=my_db_name;Integrated Security=True"
   ^--- A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 35 - An internal exception was caught)
8:                :command-text "select top 1 * from sys.objects"
9:                :multi-rs true  ;; Return multiple result sets, or just the first?
10:                })
which seems to be correct

borkdude22:10:56

but I do need to kill dotnet since it seems to stay running

isak22:10:00

Ok yea that is what I get too

borkdude22:10:01

and activates my CPU :)

isak22:10:22

strange, is that related to the pod load/unload thing?

borkdude22:10:44

no, I only removed my own Console.Error.WriteLine("dude");

borkdude22:10:06

yeah, could be related to the unload error

borkdude22:10:16

let me me try with a fresh bb

borkdude22:10:56

yep, that's probably it :)

borkdude22:10:01

sorry for the hassle

isak22:10:34

np, thanks for testing it

borkdude22:10:01

I wonder if it would work with a real SQL Server DB... is there like a free Azure thing I could connect to? I'd be really cool to have non-Windows users use this too

isak22:10:35

There might be a trial or something, and that would probably work with this

isak22:10:56

Otherwise it looks like you can install SQL Server on mac if you have docker: https://database.guide/how-to-install-sql-server-on-a-mac/

borkdude22:10:22

hmm yeah, docker!

borkdude22:10:28

going to test this tomorrow, thanks man