Fork me on GitHub
#joyride
<
2022-12-01
>
pez17:12:16

What should I do to require node-fetch in Joyride?

; Evaluating file: util_aoc.cljs
; require() of ES Module /Users/pez/Projects/joyride-aoc/.joyride/node_modules/node-fetch/src/index.js from /Users/pez/.vscode/extensions/betterthantomorrow.joyride-0.0.24/out/joyride.js not supported.
; Instead change the require of index.js in /Users/pez/.vscode/extensions/betterthantomorrow.joyride-0.0.24/out/joyride.js to a dynamic import() which is available in all CommonJS modules.
My ns form looks like so:
(ns util-aoc
  (:require ["vscode" :as vscode]
            ["node-fetch" :as fetch]
            [clojure.edn :as edn]
            [clojure.string :as string]))

borkdude17:12:43

in electron you can't require ES modules, so you either have to find out how to require the CJS module, or do this:

(promesa.core/let [mod (js/import "node-fetch") fetch (.-fetch mod) response (fetch ...)] ...)

pez17:12:55

Oh, wow. We should update the docs. 😃

borkdude17:12:30

you can also use axios

borkdude17:12:21

luckily import is idempotent; the second time you do dynamic import on an already loaded module, it's memoized

skylize17:12:48

Or skip adding extra deps; and just use Node built-in util.promisify to transform the Node built-in http.get into a using a Promise API.

🤯 1
pez17:12:26

Does it have a way to provide a cookie?

pez17:12:41

Yes, it has, but in a very roundabout way... I'll go with dependencies 😃

skylize17:12:55

What does node-fetch offer for cookies that Node's http and https libs don't do? It's just one of the headers.

skylize17:12:52

In fact looking at the docs for node-fetch, looks like it makes dealing with cookies harder, by not saving cookies by default, and requiring you to use a separate "raw" response to read them.

skylize18:12:14

Answering your question directly, "Does it have a way to provide a cookie?": I haven't done that recently, but if memory serves, you should be able to just include a headers object in the options map. So in JS, with get already promisified, that would look like:

get(url, {headers: {'Cookie': 'myCookie=myvalue'}})
  .then(handle-response)

🙏 1
pez21:12:01

Having troubles with importing modules from npm here. I have this structure:

~/Projects/joyride-aoc/.joyride(:|✔) % tree -CI node_modules 
.
├── package-lock.json
├── package.json
└── scripts
    ├── aoc2022_1.cljs
    ├── clojuredocs.cljs
    ├── london_examples.cljs
    ├── next_slide.cljs
    ├── reset_prezo.cljs
    ├── showtime.cljs
    ├── terminal_demo.cljs
    ├── typist.cljs
    ├── util_aoc.cljs
    ├── webview
    │   ├── example.cljs
    │   └── page.html
    └── workspace_activate.cljs
🧵

pez21:12:54

in util_aoc.cljs, this ns form:

(ns util-aoc
  (:require ["vscode" :as vscode]
            ["requestify" :as fetch]
            [clojure.edn :as edn]
            [promesa.core :as p]
            [clojure.string :as string]))

pez21:12:36

In aoc2022_1.cljs this content:

(ns aoc2022-1
  (:require [util-aoc :as aoc]))

(def input (aoc/values+ 1))

pez21:12:25

Now starting from a fresh REPL, I load aoc2022_1.cljs, and get:

; Evaluating file: aoc2022_1.cljs
; Cannot find module 'requestify'
; Require stack:
; - /Users/pez/Recordings/util_aoc.cljs
; Evaluation of file aoc2022_1.cljs failed: #error {:message "Cannot find module 'requestify'\nRequire stack:\n- /Users/pez/Recordings/util_aoc.cljs", :data {:type :sci/error, :line 2, :column 3, :message "Cannot find module 'requestify'\nRequire stack:\n- /Users/pez/Recordings/util_aoc.cljs", :sci.impl/callstack #object[cljs.core.Volatile {:val ({:line 2, :column 3, :file "/Users/pez/Projects/joyride-aoc/.joyride/scripts/aoc2022_1.cljs", :ns #object[T1 aoc2022-1]} {:line 2, :column 3, :file "util_aoc.cljs", :ns #object[T1 util-aoc]})}], :file "util_aoc.cljs"}, :cause #object[Error Error: Cannot find module 'requestify'

pez21:12:03

I can load util_aoc.cljs and use requestify just fine.

borkdude22:12:03

dunno, if you can make a issue repro, I can take a look sometime soon, but you could also try debugging the :load-fn yourself where it loads the npm libs. could be an issue with resolving in the wrong directory or so

🙏 1
pez21:12:49

Also, did we ever find a way to unwrap a promise? I fail to find a way to look at the values of my aoc-input in the REPL

borkdude21:12:07

@pez you can do (.then #(def res %)) and then inspect res

pez22:12:06

I'm doing something that maybe is similar:

(ns aoc2022-1
  (:require [promesa.core :as p]
            [util-aoc :as aoc]))

(defn foo []
  (p/let [input (aoc/values+ 1)]
    (def input input)))

(foo)

borkdude22:12:23

yeah, that's similar

pez22:12:47

But (.then (aoc/values+ 1) #(def input %)) is neater. Thanks!

pez22:12:23

Maybe we could have that as a repl utility in Joyride somehow?

borkdude22:12:47

I've recommended doing this as a macro in nbb before we had the async thing.

(defmacro defp [name & body]
  `(.then (do ~@body) #(def ~name %)))

pez23:12:30

This would be in Joyride or can a Joyride script do it?

borkdude23:12:27

you can script this

pez23:12:04

I get errors:

; Evaluating file: util_aoc.cljs
; Could not resolve symbol: clojure.core/unquote-splicing
; Evaluation of file util_aoc.cljs failed: #error {:message "Could not resolve symbol: clojure.core/unquote-splicing", :data {:type :sci/error, :line 10, :column 14, :message "Could not resolve symbol: clojure.core/unquote-splicing", :sci.impl/callstack #object[cljs.core.Volatile {:val ({:line 9, :column 1, :ns #object[T1 util-aoc], :file "/Users/pez/Projects/joyride-aoc/.joyride/scripts/util_aoc.cljs"} {:line 10, :column 3, :ns #object[T1 util-aoc], :file "/Users/pez/Projects/joyride-aoc/.joyride/scripts/util_aoc.cljs"} {:line 10, :column 10, :ns #object[T1 util-aoc], :file "/Users/pez/Projects/joyride-aoc/.joyride/scripts/util_aoc.cljs"} {:line 10, :column 14, :ns #object[T1 util-aoc], :file "/Users/pez/Projects/joyride-aoc/.joyride/scripts/util_aoc.cljs"})}], :file "/Users/pez/Projects/joyride-aoc/.joyride/scripts/util_aoc.cljs", :phase "analysis"}, :cause #error {:message "Could not resolve symbol: clojure.core/unquote-splicing", :data {:type :sci/error, :line nil, :column nil, :file "/Users/pez/Projects/joyride-aoc/.joyride/scripts/util_aoc.cljs", :phase "analysis"}}}

borkdude23:12:25

you need a syntax quote before (.then

borkdude23:12:07

(edited my original post)

borkdude23:12:18

tested it in nbb:

user=> (defp x (p/delay 1000 :hello))
#<Promise[~]>
user=> x
:hello 

pez23:12:31

Sleep tight!

borkdude22:12:59

and then:

(defp x (some-promise-thing))

borkdude22:12:41

but having some REPL utility is probably better for this: (joyride.repl/await ...)

borkdude22:12:32

since the REPL is already async, you can unwrap promises. in nbb the whole evaluation is async (because ES modules forced that upon nbb, which is not the case with electron + cjs modules), this is why you can use it in normal code on the top level as well

pez22:12:34

Can you complete the example (joyride.repl/await ...)? 😃

pez22:12:01

Maybe await-def?

borkdude22:12:26

not sure. in any case we could document the defp macro

skylize22:12:42

I seem to be the only person in the world who thinks await was always just a bad idea from the start. Sticking with the monadic wrapper it is built on top of (a Promise) is so much more honest about what your code is actually doing, while still giving you everything you need to get the work done. But even Haskellers seem to love using do notation everywhere; using imperative code to describe their functional transformations. So what ground do I have to stand on?

pez22:12:54

Here's my use-case for await-def in Joyride. (Spoiler alert, AOC 2002 p1-1)🧵

pez22:12:22

(ns aoc2022-1
  (:require [promesa.core :as p]
            [util-aoc :as aoc]))

;; Unwrapping the a promise is a bit involved...
;; Read this as something like:
#_(await-def real-input (aoc/values+ 1))
(.then (aoc/values+ 1) #(def real-input %))

(def test-input [1000
                 2000
                 3000
                 nil
                 4000
                 nil
                 5000
                 6000
                 nil
                 7000
                 8000
                 9000
                 nil
                 10000])

(defn part-1 [input]
  (->> input
       (partition-by nil?)
       (map (partial apply +))
       (apply max)))


(comment
  (part-1 test-input)
  :rcf)

(part-1 real-input)

pez22:12:03

That is, my solution code can be promise free.

borkdude22:12:35

same over there

borkdude22:12:07

but in node you can also use readFileSync

pez22:12:25

Haha, so similar to mine!

(def fetch-input+
  (memoize fetch-input'+))

(defn values+ [day]
  (p/let [values (fetch-input+ day)]
    (->> values
         (string/split-lines)
         (map edn/read-string))))

pez22:12:42

I should make my slurp take a Url too of course...

(defn slurp+ [ws-path]
  (->
   (p/let [ws-root (-> vscode/workspace.workspaceFolders first .-uri)
           uri (vscode/Uri.joinPath ws-root ws-path)
           data (vscode/workspace.fs.readFile uri)
           text (.decode (js/TextDecoder. "utf-8") data)]
     text)
   (p/catch (fn [e]
              (println "Ho, ho, ho! Path not found:" ws-path e)
              (throw (js/Error. e))))))

(defn- fetch-input'+ [day]
  (->
    (p/let [cookie (slurp+ ".aoc-session")
            response (.get requestify
                           (str "" day "/input")
                           #js {:cookies #js {:session cookie}})]
      (.getBody response))
    (p/catch (fn [e]
               (println "Ho, ho, ho! Did you forget to populate `.aoc-session` with your AOC session cookie?" e)
               (throw (js/Error. e))))))

(def fetch-input+
  (memoize fetch-input'+))

(defn values+ [day]
  (p/let [values (fetch-input+ day)]
    (->> values
         (string/split-lines)
         (map edn/read-string))))

borkdude22:12:48

yeah, you can download those inputs separately from running your solution

borkdude22:12:55

and then your solution becomes sync

pez22:12:54

I sort of do it separately, with that memoize.

pez22:12:40

And the await-def you handed me works just fine for making it sync in my solution code.

borkdude22:12:25

it's not sync, the def is still being defined at a point in the future, but for REPL purposes it works

pez22:12:00

It works when I run the script as well. But that's in the REPL too, right?

pez22:12:05

What I mean is that my solution code will look exactly the same whether the input is defined now or at a point in the future.

borkdude22:12:21

well, as long as the input is there before the puzzle runs...

borkdude22:12:30

which might not be the case the first time

pez22:12:35

I see....

pez22:12:11

I'll solve that with some instructions.

pez22:12:17

Or, maybe I should allow my solutions to be async instead... Hmmm...

pez22:12:56

Then await-def is a bad name for this, right?

skylize22:12:15

Regarding the original concern of how to "unwrap" a promise, I would favor simply peeking inside.

(defn tap [x] (tap> x) x)

(-> my-promise
  (p/then tap)
  (p/then whatever-i-was-doing-with-result))

👍 1
borkdude22:12:41

yeah, I think that's better

pez23:12:46

We don't have tap> in Joyride, have we?

skylize23:12:20

hmm. That's unfortunate.

skylize23:12:58

prn to the rescue, I guess...

borkdude23:12:15

we can add tap> of course

borkdude23:12:00

just add it to the config in joyride.sci

skylize23:12:31

goodnight blob-wave

pez23:12:46

Issue please, @U90R0EPHA. And PR too, if you like. 😃

skylize01:12:08

https://github.com/BetterThanTomorrow/joyride/issues/112 My best interpretation of the SCI docs and existing code is that I should be able to add it to the map after`'clojure.core` at https://github.com/BetterThanTomorrow/joyride/blob/8de67c5eb6eec93767fcd9df05a710ce5d274da6/src/joyride/sci.cljs#L78.

; formatted to far-left for readability in Slack

:namespaces
{'clojure.core
 {'IFn (sci/copy-var IFn core-namespace)
  'tap> (sci/copy tap> core-namespace)
  'add-tap (sci/copy-var add-tap core-namespace)
  'remove-tap (sci/copy-var remove-tap core-namespace)}
But my trial shows symbol as still unresolvable. No other ideas seem obvious to me on how to patch this.

pez07:12:02

This stuff is not hot-reloaded. Just saying that, I don't think you actually assumed it was, or that this is the issue.

borkdude08:12:38

> My best interpretation of the SCI docs and existing code is that I should be able to add it to the map after`'clojure.core` at https://github.com/BetterThanTomorrow/joyride/blob/8de67c5eb6eec93767fcd9df05a710ce5d274da6/src/joyride/sci.cljs#L78. Correct

skylize13:12:10

> This stuff is not hot-reloaded. Oh yes. I forgot to build before testing. :face_palm: So all the relevant symbols exist now. But I am not really sure how to try setting a print target with bound-fn* not available.

=> (add-tap prn)
nil
=> (tap> :foo)
true
; no print output

1
skylize18:12:11

Side note: once tap> is working, my suggestion of an explicitly defined tap function could be optionally replaced by an inline #(doto % tap>).

borkdude19:12:23

My rule of thumb is to not add things to clojure.core in SCI that are not in clojure core.

skylize19:12:21

Is that in reference to`bound-fn*`? That is in Clojure core (https://github.com/clojure/clojure/blob/5ffe3833508495ca7c635d47ad7a1c8b820eab76/src/clj/clojure/core.clj#L2030*), though seems to be missing in ClojureScript. There is a mk-bound-fn added to Cljs (https://github.com/clojure/clojurescript/blob/e7cdc70d0371a26e07e394ea9cd72d5c43e5e363/src/main/cljs/cljs/core.cljs#L9742-L9746), which I guess is likely intended as a replacement. It takes 3 arguments though. I'm not sure how that translates. Honestly I have zero understanding why it is even needed. All I know is I can get a basic tap-to-print working in clj project by wrapping whatever print-fn in bound-fn*. If I don't, then it doesn't work there either. Rather than requesting to add more stuff to SCI, I think I was more hoping for guidance from an expert on how this should work without bound-fn*.

borkdude19:12:41

I have no clue what you are even doing to be honest. Can you state a problem statement as if I'm a complete noob?

skylize19:12:40

The primary goal at the moment is just to verify that tap> is working correctly in Joyride, so I can submit a PR for it. Because the easiest way to test this seemed to be printing a tap, that led to a separate goal, which is: I want to be able to use tap> in my code, and have that output to a built-in print function, such as prn, even if I am not running on JVM Clojure.

borkdude19:12:03

The reference for joyride is CLJS, not the JVM. So if you have a working example in CLJS (or nbb), we can make that the same in joyride

borkdude19:12:03

in nbb:

user=> (add-tap prn)
nil
user=> (tap> 1)
true
user=> 1

skylize19:12:43

:thinking_face: That is exactly what I have added to Joyride, but does not seem to work, neither in Joyride REPL ...

user=> (add-tap prn)
nil
user=> (tap> 1)
true
user=> 
... nor with Joyride: Run Clojure Code.
=> nil

=> true

borkdude19:12:17

can you try a different tap function? it might be something weird with the print-fn

borkdude19:12:39

e.g. the vscode information message thing

skylize20:12:22

(v/window.showInformationMessage "foo")
;; foo

(add-tap v/window.showInformationMessage)
(tap> "foo")
;; nothing

(add-tap #(v/window.showInformationMessage %))
(tap> "foo")
;; nothing
Also tried with all print variants and js/console and wrapping in another function (defn prnt [x] (prn x)). 😞

pez20:12:28

I think it's best to file a PR, and get feedback on that one. Draft it, if you think that's appropriate.

skylize20:12:29

The issue is not in the printing.

(def n (atom 0))
(add-tap #(swap! n + %))
(tap> 5)
@n       ; => 0