Fork me on GitHub
#nbb
<
2022-11-01
>
Hankstenberg13:11:31

Did anybody manage to run an nbb-based lambda function with serverless recently? There's some examples out there (like this one: https://github.com/vharmain/nbb-serverless-example/blob/main/index.mjs) that maybe worked some time ago, but for some reason, .mjs files are currently simply ignored by serverless. If I try to run this very example I'm getting: "Error: Cannot find module './index.js'". Every attempt to work around this problem: renaming index.mjs to index.js means I cannot "import" in it because it's not an ESM Module, turning the entire thing into an ESM module via package.json/type:module leads to other issues (apparently serverless adds some code using "require" during the build). I just can't seem to make it work and I can't tell why.. when it looks like it did work 10 months ago.

borkdude13:11:33

@roseneck It might help to put "type": "module" in your package.json

borkdude13:11:55

at least for the "cannot import init because ESM" thing

Hankstenberg13:11:56

That's what I thought too, but while running: "serverless invoke local --function hello" A file called "s_hello.js" is generated that has a "require" in it. And then the following error is thrown: ✖️ ReferenceError: require is not defined in ES module scope, you can use import instead This file is being treated as an ES module because it has a '.js' file extension and '/.../package.json' contains "type": "module". To treat it as a CommonJS script, rename it to use the '.cjs' file extension. at file:///.../s_hello.js:2:21

borkdude13:11:26

@roseneck Did you research their github issues? I'm sure someone else ran into this. But what you can do instead of do this in a .js file which is a commonJS one:

async function foo () {
  await import("./foo.mjs")
}

foo()

1
borkdude13:11:02

Perhaps @U6N4HSMFW knows something here

Hankstenberg13:11:48

Do you think I could just do it like this: in my index.js async function loadStuff () { const nbb = await import('nbb'); const cljsFile = await nbb.loadFile('./function.cljs') } exports.handler=loadStuff();

borkdude13:11:09

I think you might

Hankstenberg13:11:31

Yea, that's one of several issues that kind of fit, but I didn't find a solution yet.

borkdude13:11:40

without the () though

borkdude13:11:08

exports.handler=loadStuff;

1
borkdude13:11:50

it is better to the loading of the file once at startup instead of in the handler

borkdude13:11:02

so you might be able to cache the result

borkdude13:11:41

This is a workaround that we had in https://github.com/vharmain/nbb-lambda-adapter before we could just use ESM

borkdude13:11:15

hah, you might still be able to use that as a workaround for serverless

Hankstenberg13:11:45

Looks like calling import as an async function did the trick!

borkdude13:11:28

yes nice! but now I think it's good to add some caching, since you don't want to do all this work on every lambda invocation

borkdude13:11:54

So something like this:

var cachedHandler;

async function loadStuff () {
  if (cachedHandler) {
    return cachedHandler;
  }
  const nbb = await import('nbb');
  cachedHandler = await nbb.loadFile('./function.cljs');
  return cachedHandler;
}

exports.handler=loadStuff;

Hankstenberg13:11:23

You mean if the function is still warm? Yea, good idea!

Hankstenberg13:11:56

Thanks a lot for your help! 🙂

👍 1
borkdude13:11:49

I'm not entirely sure the above is 100% correct since handler is now set to a function that returns a promise of the handler, but you can probably figure this out :)

borkdude13:11:42

E.g. this may work better:

var cachedHandler;

async function handler(payLoad) {
  if (cachedHandler) {
    return cachedHandler(payLoad);
  }
  const nbb = await import('nbb');
  cachedHandler = await nbb.loadFile('./function.cljs');
  return cachedHandler(payLoad);
}

exports.handler=loadStuff;

Hankstenberg13:11:56

Hm.. .it says: TypeError: cachedHandler is not a function

borkdude13:11:33

What do you return from function.cljs?

Hankstenberg13:11:47

(ns function) (defn handler [event _ctx] (js/console.log event) (js/Promise.resolve #js{:hullo "world"}))

Hankstenberg13:11:58

still the example code

Hankstenberg13:11:18

Is it cachedHandler.handler(payload)?

borkdude13:11:50

I think you need to add something like:

#js {:handler handler}
at the bottom and then write:
cachedHandler = (await nbb.loadFile('./function.cljs')).handler;

borkdude13:11:54

When you have defn as the last expression, the return value is a SCI Var which JS doesn't recognize as a function

Hankstenberg13:11:02

I gotta to go and will fiddle around more later, thanks a ton for your help!

borkdude13:11:04

user=> (defn foo [] 1)
#'user/foo
user=> (set! (.-foo js/globalThis) foo)
nil
user=> (js/eval "foo()")
1
user=> (set! (.-foo js/globalThis) #'foo)
nil
user=> (js/eval "foo()")
"TypeError: foo is not a function"

valtteri14:11:54

I will try the example later when I get home and check if some issues have popped up

👍 1
valtteri14:11:56

I’d expect it to just work as long as the lambda Nodejs runtime supports top-level await.. since 14.something.something

Hankstenberg14:11:58

Yea I would totally expect that too and lambda definitely supports esm (zipping it up and uploading it manually works).

Hankstenberg14:11:39

It must be something about serverless, but I can't tell what I'm doing wrong.

valtteri16:11:32

I just tried a fresh clone from the repo and ran the instructions in the example and it works :thinking_face:

valtteri16:11:23

I can maybe help debug what’s wrong with your setup @roseneck. Could you please paste your serverless.yml contents here if possible?

borkdude16:11:09

and the version of serverless

borkdude16:11:11

and the Node.js version

valtteri16:11:30

And how you’re trying to run it 🙂

valtteri16:11:58

> npx sls --version > Framework Core: 2.72.3 (local) > Plugin: 5.5.4 > SDK: 4.3.2 > Components: 3.18.2

valtteri16:11:11

AHH sorry didn’t notice that this was about serverless invoke local . I just tested that the deployed one works. I’m not sure if the invoke local has ever worked since I don’t use it.

valtteri16:11:17

I’m afraid that if you want to use invoke local your options are: • patch serverless with this unmerged PR https://github.com/serverless/serverless/pull/11366/files • use nbb-lambda-adapter instead that should work with non-`.mjs` stuff https://github.com/vharmain/nbb-lambda-adapter

Hankstenberg16:11:04

It's not just local.. I just used local for faster feedback. I just did a fresh checkout of your repo and did a deploy with the following versions: Operating System: linux Node Version: 18.10.0 Framework Version: 2.72.3 (local) Plugin Version: 5.5.4 SDK Version: 4.3.2 Components Version: 3.18.2 Here's what I got: ~ serverless invoke --function add  ✔️  2m 19s  Running "serverless" from node_modules Serverless: Using provider credentials, configured via dashboard: https://app.serverless.com/hankstenberg/apps/add/nbb-serverless-example/dev/eu-west-1/providers { "errorType": "Error", "errorMessage": "Cannot find module './index.js'\nRequire stack:\n- /var/task/s_add.js\n- /var/runtime/UserFunction.js\n- /var/runtime/Runtime.js\n- /var/runtime/index.js", "trace": [ "Error: Cannot find module './index.js'", "Require stack:", "- /var/task/s_add.js", "- /var/runtime/UserFunction.js", "- /var/runtime/Runtime.js", "- /var/runtime/index.js", " at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)", " at Module.require.i.require (/var/task/serverlesssdk/index.js:9:73131)", " at require (internal/modules/cjs/helpers.js:101:18)", " at Object.<anonymous> (/var/task/s_add.js:25:23)", " at Module._compile (internal/modules/cjs/loader.js:1085:14)", " at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)", " at Module.load (internal/modules/cjs/loader.js:950:32)", " at Function.Module._load (internal/modules/cjs/loader.js:790:12)", " at Module.require (internal/modules/cjs/loader.js:974:19)", " at require (internal/modules/cjs/helpers.js:101:18)" ] }

Hankstenberg16:11:07

My node Version is v18.10.0

valtteri16:11:34

That’s a bummer. 😕 invoke without local seems to suffer from the same defect

valtteri16:11:24

But I assume that if you use curl the lambda itself does work

valtteri16:11:25

Hmmm invoke without local seems to work with the latest serverless

valtteri16:11:47

npx serverless invoke --function add --data '{"body":"{\"x\":1,\"y\":2}"}'
{
    "statusCode": 200,
    "body": "{\"result\":3}"
}

valtteri16:11:20

> npx sls --version > Framework Core: 3.23.0 (local) > Plugin: 6.2.2 > SDK: 4.3.2

valtteri17:11:09

I just updated the example repo • latest serverless and nbb • add example how to use invoke • add mention that invoke local does not work https://github.com/vharmain/nbb-serverless-example Thanks for notifying about the issue!

🙌 1
Hankstenberg18:11:02

I really don't understand what is wrong with my setup. So I checked out your updated repo. I cd'ed into it. I ran: • "npm update". • npx sls --org my-org • I said: "yes, I want to deploy now" • Deployed successfully • I ran npx serverless invoke --function add --data '{"body":"{\"x\":1,\"y\":2}"}' And I'm getting: "errorType": "Error", "errorMessage": "Cannot find module './index.js'\nRequire stack:\n- /var/task/s_add.js\n- /var/runtime/UserFunction.js\n- /var/runtime/Runtime.js\n- /var/runtime/index.js", "trace": [ "Error: Cannot find module './index.js'", I have just the same versions: Framework Core: 3.23.0 (local) Plugin: 6.2.2 SDK: 4.3.2

borkdude18:11:25

Node.js version?

valtteri18:11:14

What does your git status output look like?

Hankstenberg18:11:42

On branch main Your branch is up to date with 'origin/main'. Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git restore <file>..." to discard changes in working directory) modified: serverless.yml Untracked files: (use "git add <file>..." to include in what will be committed) package-lock.json no changes added to commit (use "git add" and/or "git commit -a")

Hankstenberg18:11:06

diff --git a/serverless.yml b/serverless.yml index f613ea5..c9bbec3 100644 --- a/serverless.yml +++ b/serverless.yml @@ -1,3 +1,5 @@ +org: hankstenberg +app: add service: nbb-serverless-example frameworkVersion: '2 || 3'

valtteri18:11:53

Could old serverless still be lurking in package-lock.json? You can try nuking it and running npm install

valtteri18:11:50

Ah no, your serverless version was new

Hankstenberg18:11:12

Yes, I still tried, but no

valtteri18:11:05

My Node.. Don’t expect that to be the issue but you never know

node --version
v18.9.1

Hankstenberg18:11:07

Note sure how to downgrade to that specific version right now... but I really don't think that this can be the issue. I'll try some other things first. Like... I don't know.. restarting my computer. 😉

valtteri18:11:37

I’m also starting to run out of ideas. 😕 I’m on MacOS. Please let me know if there’s something more I can provide for help

Hankstenberg18:11:16

Thanks a lot for all the support, at least I know that it is something about my setup now that helps a ton! I'll start ruling out things one by one. ❤️👍

❤️ 1
valtteri18:11:38

I can only think of the “traditional rituals” with Node ecosystem • rm -rf node_modules • rm package-lock.json • npm install • wait for all the planets to be aligned