Fork me on GitHub
#clojurescript
<
2020-03-16
>
kwrooijen08:03:55

Hi all, i'm working with a Javascript object that seems to be recursive. If I println it, I get a too much recursion error. When I use js/console.log it gets printed, and there's one key that keeps nesting itself repeatedly. It seems to go infinitely. However my problem is that I want to access a specific portion of this data in the form of..

(-someChildren (-key1 obj))
This will give me a collection of objects, which also recur endlessly! But each one of these children has a property that I want. Basically I want to do this..
(map -someProp (-someChildren (-key1 obj)))
Of course this doesn't work, since it's a Javascript array. And I can't use js->clj or array-seq because of the infinite recursion. What's the best way to tackle this problem? One solution I could think of is using Clojure's loop + .pop until the array is empty, but maybe there's a cleaner way?

fricze08:03:47

if it’s a JS array you can just call JS Array.map , can’t you?

(.map (-someChildren (-key1 obj))
      #(-someProp %))

kwrooijen08:03:53

Ah wow, I feel pretty silly haha. Forgot JS has a map method. I think that would work, thanks 🙂

fricze09:03:42

glad I could help 😉 cheers

fricze09:03:38

JS interop is really quite nice in ClJS, but it’s easy to forget about when you write Clojure all the time 😉

kwrooijen10:03:44

Well I'm honestly not an expert in Javascript anyway 😄

ABeltramo11:03:18

Hello everybody, I need some help in end to end testing in clojurescript using shadow-cljs. I saw this https://github.com/bensu/doo/wiki/End-to-end-testing-example for doo is it possible to do something similar for shadow-cljs? I would like run the cljs tests from the test environment of the clojure backend, is that possible or should I migrate to doo?

jjttjj13:03:02

is there an example or walktrhough somewhere of starting a cljs repl from a clojure repl (not from the command line)?

jjttjj13:03:31

But it seems like this might just start a server?

Alex Turok13:03:49

I haven’t done any cljs for long, but used to do things as outlined in weasel’s readme. Worked for me - you may try it: https://github.com/nrepl/weasel/blob/master/README.md

jjttjj13:03:10

Thanks, but I'm specifically trying to get it to work with no external deps besides cljs. Which I know is possible somehow 🙂 It works if I use clj -m cljs.main from the command line. Just trying to cheat on reverse engineering that

Edmund14:03:49

Hi all. Having headaches with cljs-http. I'm playing with ring-compojure and got a server running on localhost:3000. I can make requests to this server via postman / direct in the browser and getting valid json responses. If I try the same request with cljs-http, I'm getting an empty response: '{:status 0, :success false, :body "", :headers {}, :trace-redirects ["http://localhost:3000/..." "http://localhost:3000/...."], :error-code :http-error, :error-text " [0]"} ' what could I be missing. I've looked around to no avail. This "(go (let [response (<! (http/get "" {:with-credentials? false }))] (prn response)))" however works. Only when I use my endpoint do I get this funny empty error.

Edmund15:03:25

Turned out to be a CORS issue. Adding cors middleware on the ring server fixed the problem.

paul93122419:03:45

how do I convert an asyncIterable in a core.async channel? or implement for await... of ?

sully10:03:45

As far as I know there's no libraries that do this for you, but based on the https://github.com/tc39/proposal-async-iteration#async-iterators-and-async-iterables it should be simple to make a function that makes such a channel.

sully12:03:28

My own try at this based on the current spec. Definitely not tested but this should do what you need with some minimal debugging. It takes the same arguments as https://clojuredocs.org/clojure.core.async/onto-chan and operates in a similar fashion. I didn't detail handling errors, so figure out what's appropriate for your use case.

paul93122414:03:34

I didn't succeed. I think I didn't fully grasp my problem. I have this socketcluster app which I want to use in clojurescript. I have the working server.js code which I want to rewrite to clojure. But I am stuck this part:

// HTTP request handling loop.
(async () => {
  for await (let requestData of httpServer.listener('request')) {
    expressApp.apply(null, requestData);
  }
})();

// SocketCluster/WebSocket connection handling loop.
(async () => {
  for await (let {socket} of agServer.listener('connection')) {
    // Handle socket connection.
  }
})();

paul93122414:03:12

I tried your code, but it says (.listener http "request") is not an asyncIterable, I guess this is right. How can I implement this node snippet?

paul93122414:03:28

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for-await...of How would you do this in interop?

(async function() {
   for await (let num of asyncIterable) {
     console.log(num);
   }
})();

sully19:03:52

Clojurescript doesn't have any constructs that directly translate to for await. My code should work with any async iterable object, but if you want to use that particular syntax you'll have to do it in JavaScript itself.

sully19:03:26

@U1YTC9FU5 I did forget one important detail, Symbol.asyncIterator points to a function that returns the iterator. I fixed my code so that it handles this correctly

sully19:03:51

I went to look at what that .listener method returns, it does have a Symbol.asyncIterator defined so it should work https://github.com/SocketCluster/consumable-stream/blob/master/index.js

sully19:03:26

the implementation is provided by the server class, so don't worry about the method stub in that class file

paul93122419:03:21

Thank you very much for giving this your time! I have an error though. consumable-stream tries to use createConsumer on nil. This is the error:

SHADOW import error /Users/paulcristianmartin/FaasKitchen/paulhub-wss/minimal-shadow-cljs-nodejs/.shadow-cljs/builds/app/dev/out/cljs-runtime/server.main.js


JS reload failed TypeError: Cannot read property 'createConsumer' of null


    at output (/Users/paulcristianmartin/FaasKitchen/paulhub-wss/node_modules/consumable-stream/index.js:23:12)
    at Function.cljs$core$IFn$_invoke$arity$variadic (/Users/paulcristianmartin/FaasKitchen/paulhub-wss/minimal-shadow-cljs-nodejs/.shadow-cljs/builds/app/dev/out/cljs-runtime/server/main.cljs:152:12) 

paul93122419:03:45

I guess because of this part

[Symbol.asyncIterator]() {
    return this.createConsumer();
  }

sully19:03:23

it's just strange to see that if your previous example worked, because that's the same function regular JS calls if you do for...await. Could you try in javascript

myListener = server.listener('request');
myListener[Symbol.asyncIterator]();

sully19:03:36

if that returns an error as well, socketcluster has some kind of bug that's not spec-compliant

paul93122419:03:39

this is javascript code, where do you want me to try it? in a separate node application?

sully19:03:27

you could also try passing the result from .listener directly to my async-iter-next function with the proper arguments, as it has the .next method as well

paul93122420:03:52

I did exactly that:

(def one-chan (async/chan))
(async-iter-chan one-chan (.listener http-server "request"))

paul93122420:03:12

I tried it in a separate node project, in the socketcluster one, I pasted what told me, it works.

sully20:03:23

try this (async-iter-next one-chan (.listener http-server "request") true js/console.error)

sully20:03:57

async-iter-next instead of chan

paul93122420:03:08

yes, it has no problem, no output

paul93122420:03:54

the aget starts the function createConsumer, but doesn't have this ?

sully20:03:53

aget just expands to JS code like this (aget thing foo) => thing[foo]

sully20:03:38

so (aget listener js/Symbol.asyncIterator) will return the function that makes an async iterator (eg listener[Symbol.asyncIterator])

sully20:03:39

if there's no errors now, try taking stuff from that channel (go (js/console.log (<! one-chan)))

paul93122420:03:22

I have to write it in a go block, doesn't work with def for me.

sully20:03:32

yeah sorry, forgot about that

sully20:03:04

did you manage to get anything?

paul93122420:03:17

(go
  (println (<! one-chan)))

(async-iter-next one-chan 
 (.listener http-server "request") 
 true js/console.error)

paul93122420:03:26

this is how I tried

paul93122420:03:30

this is much harded then I anticipated. Till now I only dealt with clojurescript SPA-s and some clojure backend. Never needed more the core.async

sully20:03:51

yeah, I think this is just some bad interop getting in the way. I got one more thing to try and I think I'm out of tricks from there

(defn async-iter-chan
  "Take an async iterable `iter-obj` and push it to channel `output`. Closes
  `output` when the iterable is exhausted. Provide a false `:close?` argument to
  leave the output open."
  [output iter-obj & {:keys [close? rejected] :or {close? true rejected js/console.error}}]
  (async-iter-next 
     output (js-invoke iter-obj js/Symbol.asyncIterator) close? rejected)
  output)

sully20:03:13

I changed that iter-chan function to use this https://cljs.github.io/api/cljs.core/js-invoke

sully20:03:48

so the js-invoke should expand to exactly iter_obj[Symbol.asyncIterator]()

sully20:03:39

idk why the other stuff wouldn't work, but hopefully this does

paul93122420:03:44

this seems to work, at least I don't see any errors

paul93122420:03:21

bless you man, I hope I manage from here on 😄

sully20:03:43

no problem, just glad to see people using clojure(script)!

paul93122420:03:40

from 5 years now, want to keep it that way! Still many things to learn, especially about interop.

sully20:03:29

it's pretty sweet overall, but nasty stuff like this bindings do tend to screw it up. but that's true of javascript as well

sully20:03:43

in any case, doing (async-iter-chan ...) is about as close as you'll get to for...await using core.async. Feel free to dm me if you hit another related roadblock.

paul93122420:03:04

Thank you, I will!

paul93122421:03:46

It had one more little error, for anyone who will need it later

(and (not (.-done next-elem)) (>! output (.-value next-elem)))

to

(and (not (.-done next-elem)) (async/put! output (.-value next-elem)))

sully21:03:27

hmmm, it's fine if that works for your use case, but I did >! so the go block would park until the value was taken. If you do use put! instead you'll get as many puts as your iterable will produce without waiting for takes. You'll need a buffered channel to do that without a problem.

paul93122421:03:31

Oh, I thought that's the reason that I saw only the first request in my log, my bad.

paul93122421:03:16

I see, now that I go-loop through the channel it works alright. What about parallelism? Will be okay for a lot of clients? How much can it handle?

sully21:03:09

I think asyncIterable specifies a need for internal buffers when there's more than one consumer, so it should work OK. Plus javascript is single-threaded, so only one consumer can be running at any given moment in time.

sully21:03:32

Sorry @U1YTC9FU5 was talkin some mad lies that I deleted. If you want to consume N elements at once, just use my provided code with a buffered channel. >! only parks when the buffer is full, so it will take N elements at once given a buffer of size N https://clojure.github.io/core.async/#clojure.core.async/%3E%21

paul93122422:03:50

I didn't see those, but thanks, now it's clearer

paul93122422:03:31

Is this valid? When I refresh or leave the page it goes in an infinite loop of null events. Can I brake out the loop?

(go-loop [x 1]
  (let [data (<! wss-connections-chan)]
       ;listener on socket of this connection
       (async-iter-chan 
         wss-communication-chan
         (.receiver (.-socket data) "simple-code"))
       (recur (inc x))))

sully23:03:40

I'm guessing the iterable might not actually be setting done correctly when the socket closes. If there's a callback for when the serverside of the socket closes, you can close the async iterator channel there.

sully23:03:56

You'd need to create an intermediate channel that you pipe to your output eg

(let [client-receive (async/chan)]
  (async-iter-chan
    client-receive
    (.receiver (.-socket data) "simple-code"))
  ;; You'd need to use the right callback here, just doing a strawman placeholder
  (.-onClose data #(async/close! client-receive))
  ;; I'm passing false at the end to prevent wss-communication-chan from closing
  (async/pipe client-receive wss-communication-chan false)
  ;; ...rest of the code here)

sully23:03:41

maybe a simpler solution would be to close the channel inside async-iter-next when you get a nil, but that's probably some kind of error condition you'd want to deal with

paul93122423:03:09

no, this solution is perfect, thank you very much. You saved my project !

sully23:03:30

excellent!

paul93122423:03:47

on leaving the page I've got null instead of nil, and couldn't (= js/null (.-value elem))

paul93122423:03:23

and it looks way much cleaner this way 🙂

sully23:03:18

null and nil are the same value, they just have different names between js and cljs (like in regular clojure/java). Glad it works!

sully03:03:28

I decided to clean this up and make a gist, would be great if you can throw some usage onto it https://gist.github.com/jjsullivan5196/0904057a2ec3eb080de5c4d6f45da630

paul93122411:03:11

I finish testing it and I will make a repo for the socketcluster clojure translation, and mention it in a comment, if that's what you mean by usage.

sully11:03:03

yeah, put a link to your repo in the gist, it would help other people out

paul93122413:03:07

I totally agree. I make the front-end part aswell with this solution, then I'll push it, with some example buttons and stuff.

paul93122419:03:04

I am almost ready, I got stuck again at .setMiddleware()

(let [client-receive (async/chan)]
           (.setMiddleware 
              ag-server 
              (.-MIDDLEWARE_INBOUND ag-server)
              (fn [middleware]
                  (async-iter-chan
                     client-receive
                     middleware))))
This works, with multiple clients also, but if I refresh the page or leave it, the events stops working.

paul93122412:03:19

oh yes, I did it, I am so happy I was able to figure it out

(.setMiddleware 
  ag-server 
  (.-MIDDLEWARE_INBOUND ag-server)
  (fn [middleware]
     (let [client-receive (async/chan)]
        (async-iter-chan
          client-receive 
          middleware)                                   (async/pipe 
  client-receive 
  wss-middleware-chan 
  false))))
Of course with the help of your examples :D

jjttjj20:03:24

Anyone know how to use macros from an external, depended upon library with cljs.js/eval-str ? Is it possible to do it without using the source string? Ie, I want to use the library https://github.com/hoplon/javelin inside cljs.js/eval-str how do I do this?

(cljs.js/eval-str state
                  "
(do 
(ns myns (:require-macros [javelin.core]))
(def xx (javelin.core/cell 123)) 
@xx ;;works
;;(def yy (javelin.core/cell= (+ xx 123))) ;;cell= is a macro
;;@yy ;;doesn't work
)"
                  'foo.bar
                  {:verbose    true
                   :source-map true
                   :eval       cljs.js/js-eval 
                   }
                  (fn [{:keys [error] :as res}]
                    (if error
                      (do
                        (println error)
                        (println (.. error -cause -stack)))
                      (println res))))

kaosko22:03:59

anybody working with three.js & shaders / cljs / reagent? I've kind of gotten a hang of it but I'm still looking for guidance on how you've packaged your shaders and how you are manipulating the values? seems obvious to put the manipulatives in an atom but if there are any examples on how you've maybe structured the manipulation code and multiple shaders together, I'd be interested in taking a look

darwin22:03:41

this is maybe too far from your model, but I ported a few react-three-fiber demos to cljs: https://github.com/binaryage/cljs-react-three-fiber it is using modern react, so all shader updates would be implemented in react hooks - but none of ported demos needs that

darwin22:03:10

also using shadow-cljs so I’m able to keep shader sources as separate files and embed them into compiled cljs via inline macro: https://github.com/binaryage/cljs-react-three-fiber/blob/master/src/app/rtf_examples/demos/refraction.cljs#L44-L48

darwin22:03:36

you could do it in clojure macro as well, but this method is compilation-cache friendly

kaosko22:03:33

thanks @U08E3BBST last time I looked at the react-fiber not to mention the cljs pieces, it didn't look quite ready for primetime. shader updates in react hooks sounds great though. can you say anything about the performance compared to the stack renderer, perhaps especially related to three.js?

darwin22:03:52

unfornutaly I cannot, this is the only toy project I have at the moment, and I’m new to three.js

kaosko22:03:56

alright, thanks anyway. I'll keep your project in mind