Fork me on GitHub
#clojurescript
<
2018-07-30
>
pesterhazy08:07:29

Is there a reason to use cljs.nodejs/require (http://cljs.github.io/api/cljs.nodejs/) rather than simply js/require?

pesterhazy11:07:19

Another q — can anyone recommend a library/implementation for locking (mutex) for JavaScript/ClojureScript?

leonoel11:07:10

missionary has a semaphore implementation https://github.com/leonoel/missionary

thheller11:07:33

you can't lock anything in JS given that there is only one thread

pesterhazy11:07:01

that's incorrect. You still need locking for critical sections when making async fn calls

thheller11:07:05

nope. no such thing in JS. only one thing can happen and if there were locks you'd deadlock 100% of the time since the lock holder can never release.

pesterhazy11:07:58

@thheller again wrong (why not think about it for a second before replying?). Check this example for example: https://www.npmjs.com/package/async-lock#why-do-you-need-locking-on-single-threaded-nodejs

thheller11:07:15

ok you are right. why not just this this lib then?

thheller11:07:04

> In computer science, a lock or mutex (from mutual exclusion) is a synchronization mechanism for enforcing limits on access to a resource in an environment where there are many threads of execution.

thheller11:07:52

thats my understanding of mutex but I can you can stretch callsback to mean "many threads of execution"

pesterhazy11:07:03

I think async-lock is a good choice, I was just wondering if people have used something smaller (perhaps included in GCL?)

pesterhazy12:07:28

yes, JavaScript has concurrency (but not parallelism)

pesterhazy12:07:37

As I understand it, locking or other mutexes are required when a common resource is used, not only by parallel processes, but also by concurrent processes (which JS has)

avfonarev12:07:41

When I create an empty project with the following deps.edn, I weirdly get an Error building classpath. What am I doing wrong?

{:deps
 {org.clojure/clojurescript {:mvn/version "1.10.339"}
  org.clojure/core.async {:mvn/verison "0.4.474"}}}

pesterhazy12:07:46

In each case a critical section is protected from concurrent access by processes. In JS a "process" is a series of callbacks, so multiple "processes" using the same resource can interleave.

abdullahibra12:07:06

i would like to handle login + refresh page still logged in, what i have tried is: 1- define js variable var identity = {{identity|json|safe}}; in the base.html template 2- wrap-identity middleware which try to add identity key like: (defn wrap-identity [handler] (fn [request] (prn request) (binding [identity (get-in request [:session :identity])] (handler request)))) actually i'm following Web Development with Clojure book example but i still get identity = null after successfully login session

abdullahibra12:07:36

did anybody face this problem and successfully fix it?

abdullahibra12:07:09

or how do you guys handle login + session management to not loss session info after reloading the page ?

manutter5113:07:24

I would recommend using let instead of binding. The binding function is mostly for testing and/or for those rare cases where you want to override the value of some global var.

manutter5114:07:56

Also, I'd say you might want to avoid using the -> threading operator in early stages. If you use let and assign each value one at a time, you can inspect each step and see what's going on better.

manutter5114:07:12

That's the general advice, now for your specific question: What should be happening here is that some other middleware should be adding the session to your request before your wrap-identity middleware gets called.

manutter5114:07:25

If the user has already logged in, the session value should contain the identity, so you should be able to get it using

(let [session (:session request)
      identity (:identity session) ...]

manutter5114:07:51

In the code you posted, you're passing request to the handler function, which means you're getting back a response. You're then getting the body of the response, and calling parse-string on it, which is probably not what you want.

manutter5114:07:02

I'm guessing that your response body contains <html ..., or in other words, the rendered page. It might be a web page or an error message, but it's not what you're looking for. Also, since you're only looking at the response coming back, it's probably not useful to get the identity at that point, because all your processing has finished, and you're in the middle of returning the web page back to the client now

Empperi14:07:42

trying to include https://pushjs.org/ via :npm-deps but I get:

[eval]:85
            !id.startsWith(goog:);
                           ^^^^

SyntaxError: missing ) after argument list
    at new Script (vm.js:74:7)
    at createScript (vm.js:246:10)
    at Object.runInThisContext (vm.js:298:10)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at evalScript (internal/bootstrap/node.js:563:27)
    at startup (internal/bootstrap/node.js:248:9)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:596:3)

Empperi14:07:12

I've simply tried to require ["push.js" :as Push]

Empperi14:07:21

I can see that it is downloaded via npm

Empperi14:07:24

it looks like a bug in cljs to me but I can't be certain

abdullahibra14:07:11

@manutter51 thanks for this advice, but actually i'm defining identity as global var in to be used with render, i have another question if the user has already logged in how the session contain the identity ?

abdullahibra14:07:22

the flow as i understand: 1- make request, user send creds to the server 2- response should have {:session {:identity "email"}}

manutter5114:07:03

The basic idea is that somewhere early in your chain of middlewares you have a session handler middleware like ring-session that adds the session to the request on the way in, and then extracts the session from the response on the way out. Then in your login handler, you verify the user’s credentials, and add them to the session under the :identity key. So something like

(if (valid?)
  (let [session (:session request)
        identity (get-user-identity-somehow)
        session (assoc session :identity identity)]
    (-> (make-response)
      (assoc :session session)))

manutter5114:07:21

That’s code out of my head so apologies for any typos/bugs, but that’s the general idea — you pull the session out of the request, add the identity to it, then add the session to the response. The ring-session middleware takes care of storing the session data between requests, so the next request that comes in should have your session (and identity) added automatically.

manutter5114:07:56

So you don’t really need a global var for identity, you can always get it from the request with (get-in request [:session :identity])

abdullahibra14:07:52

how can you set the identity to the request from the client?

abdullahibra14:07:07

(session/put! :identity id) -> after successful login ?

manutter5114:07:34

You do not do it from the client. It has to be done from the server. It would be a security vulnerability if the client could set the identity because then it would be exposed to hackers.

manutter5114:07:15

The way sessions work with clients is that the ring-session middleware sets a cookie with a SessionID and sends that to the client. The client sends back the same SessionID in the cookie, and the ring-server middleware uses that to look up the session data for that specific client.

manutter5114:07:44

Right, that’s the basic idea. It’s important to understand that all that is happening on the server, not the client.

justinlee16:07:57

@pesterhazy regarding locking, you can often solve the problem with a simple promise queue. (also be sure to think about whether a locking library will solve the problem when you have more than one node process)

richiardiandrea16:07:47

We do not need locking because we have atoms no? ...having said that I do not have any experience report with trying atom coming back from concurrent promises

justinlee16:07:03

locking across events. I don’t even think the concurrency guarantees of atoms are necessary in js

Empperi16:07:30

Atoms do not save every scenario in true concurrency, for ClojureScript they are more than enough

justinlee16:07:11

I’m trying to think if atoms help in any scenario in cljs. Synchronous code cannot be interrupted in javascript.

Empperi16:07:37

They are practically irrelevant

Empperi16:07:17

JavaScript as a language has been designed to be concurrency safe... By being single threaded

justinlee16:07:32

Only within a given synchronous block of code. In node, where you will do lots of i/o, you can do things that are not safe and require synchronization.

Empperi16:07:01

I/O is handled via event loop mechanism

Empperi16:07:24

It's still single threaded on language level

lilactown16:07:34

I think atoms are helpful in that they allow people to pass around a box as a value and change the thing within it

Empperi16:07:00

@lilactown yeah agree but that has nothing to do with concurrency

Empperi16:07:10

They are a nice abstraction

Empperi16:07:22

"This is mutable"

lilactown16:07:04

but yeah the whole STM thing doesn't really exist in CLJS AFAIK

justinlee16:07:08

if you do something like read a file and then write some files based on that read, you’ll probably need to lock around that piece of code. atoms and single-threadedness will not help you

Empperi16:07:07

@lilactown well, there are no refs in cljs so yeah, obviously

justinlee16:07:23

there was a discussion in here about how to deal with that. a simple solution is to create a little queue at the beginning of the function and chain promises until the queue is empty. but that won’t help if there’s more than one node process

justinlee16:07:35

that’s what I was talking about

lilactown16:07:31

that's the case in a language with threading and locking too, right?

justinlee16:07:54

I think that’s right: a locking library in js would only get you to the same level as a threaded language. it seems to me that it wouldn’t often be useful for just that reason, but I well could be wrong about that. i always use the database as synchronization

pesterhazy16:07:51

@niklas.collin it's a common misconception that there are no concurrency issues in JavaScript because it's single-threaded

Empperi16:07:21

I never said that

pesterhazy16:07:43

"JavaScript as a language has been designed to be concurrency safe"

Empperi16:07:51

You can of course end up into sync issues via I/O

Empperi16:07:14

Take XHR for example

Empperi16:07:43

But you cannot end up into such cases via js logic alone

pesterhazy16:07:37

If multiple processes (as in composites of async callbacks) are using a common resource, interleaving access from multiple concurrent processes (in the same single-threaded environment) can cause concurrency problems

Empperi16:07:13

Web workers give you concurrency but due to their design you really can't fuck things up

pesterhazy16:07:36

A single browser window gives you concurrency

pesterhazy16:07:46

No web workers needed

justinlee16:07:40

@pesterhazy but how can that help you if you have multiple node processes?

pesterhazy16:07:29

@lee.justin.m I'm actually not thinking about Node but the browser - multiple processes accessing IndexedDb

pesterhazy17:07:02

even in Node, there are resources that are local to a single process

justinlee17:07:04

oh that makes sense. (you had asked about nodejs require earlier so I thought that’s what you were doing)

Empperi17:07:16

Naturally in cases like in that Redis example you might get into trouble, that's why there's database transactions

pesterhazy17:07:17

right, I see how that was confusing, sorry about htat

Empperi17:07:22

But that's exactly in the I/O department, you are accessing an external resource

Empperi17:07:02

Same with IndexedDB

pesterhazy17:07:12

@lee.justin.m thanks for the pointer to promise-queue, that's very helpful

justinlee17:07:38

it may be what you want on the browser side. it’s trivial to implement.

justinlee17:07:41

it can be used for concurrency > 1 too (which is actually where i use it)

pesterhazy17:07:33

@lee.justin.m when max-queue-length is reached, does it "block" (i.e. wait until the queue has space and then fire the promise) or throw?

justinlee17:07:31

in my case i’m not worrying about having a queue too long, but if I wanted to be more robust I think you’d want to throw

justinlee17:07:07

maybe you mean something different by block

justinlee17:07:27

oh i’m not even using a library.

justinlee17:07:20

when you enter the function, you just check to see if there is a slot. if so, you decrement a slot counter and go. if not, you throw the arguments on a queue. then you throw a then at the end of the function that checks the queue and repeats

justinlee17:07:45

you can make it general. i think there’s a library called throat.

pesterhazy17:07:54

>> throat(concurrency) This returns a function that acts a bit like a lock (exactly as a lock if concurrency is 1).

pesterhazy17:07:23

very useful pointers

justinlee17:07:23

yea. and lindesay forbes is the boss. that library is widely used and probably pretty robust

pesterhazy17:07:55

this kind of stuff is known too little in the cljs community (people are too quick to jump to core.async for these issues)

leonoel08:07:50

This kind of stuff is precisely what leads the cljs community to look for alternatives to mainstream paradigms. When you encounter the need to wrap promises in thunks, you're basically lazifying a reification, and at this point it should be common sense to start questioning your design choices.

pesterhazy17:07:23

if you dig a bit, npm is a gold mine of solid libraries

Empperi17:07:37

Does anyone have any idea on the problem I asked few hours ago?

Empperi17:07:13

I'm guessing it's a problem with the cljs automatic commonjs analysis which surfaces with that lib

pesterhazy17:07:25

@niklas.collin the short answer is that npm-deps has rough edges so you may have an easier time with the alternatives (shadow-cljs or using a webpack bundle) — not sure if you knew this

pesterhazy17:07:00

(if you're interested in making it work with npm-deps, that's also great, just noting that it's not the path of least resistance)

jplaza17:07:18

Hi all! Any suggestions for a good replacement for Bigdecimal in CLJS?

pesterhazy17:07:01

@jplaza I think I tried https://github.com/MikeMcl/big.js/ some time ago and it worked fine

jplaza17:07:26

Thanks @pesterhazy! I’m using that, just wanted to know if there was a “CLJS way”

pesterhazy17:07:17

it has a ton of github stars so it must be good 🙂

Empperi17:07:19

@pesterhazy was afraid that would be the answer

pesterhazy17:07:45

@jplaza my advice would be to pick a lib off NPM and use it directly, via JS interop

mv17:07:07

Is there any talk of having cljs support webasm?

mv17:07:24

The last few forum posts I have seen suggest it isn’t being considered

lilactown17:07:08

what's the point?

🎯 1
mv17:07:01

I was hoping that if it can output wasm, then I could use cljs to write smart contract code

mv17:07:23

But that’s not enough of a draw for general users

justinlee17:07:36

my understanding is that you’d have to implement your own gc with webasm in the main thread, so that’s a big obstacle

dnolen17:07:03

it’s come up in the webasm future plans - but I’m not holding my breath

mv17:07:27

Understandable

dnolen17:07:34

webasm is an evolution of asm.js

dnolen17:07:39

and I brought this up 5 years ago

mv17:07:00

Yes, yours were the forum posts I last saw 🙂

dnolen17:07:13

so I say another 2-3 years for the idea to even be approved

dnolen17:07:20

then another 2-3 for it to be implemented

dnolen17:07:29

and then 1-2 years to assess whether it’s actually practical

dnolen17:07:57

this aside - doing a webasm DSL in ClojureScript does sound fun / useful - probably even easy given it’s a s-expr based thing

mv17:07:10

Do you have any recommended resources for writing DSLs in clojure/script?

dnolen17:07:54

not on hand - but there’s been a few talks at Clojure conferences over the years on this subject

dnolen17:07:22

I guess core.async would not be bad thing to look at

dnolen17:07:37

for inspiration

dnolen17:07:54

specifically the Clojure side

dnolen17:07:22

we did reconcile the ClojureScript AST, so it should be informative

mv17:07:51

Thanks!

dnolen18:07:36

np, if Webasm gets GC, calling into JS (which isn’t too painful), and multithreading then I could see us going for it - but this is a long way off

mkvlr18:07:28

btw, I saw this talk earlier this month on the webassembly roadmap: https://www.youtube.com/watch?v=WTxPqDBo8CI&amp;t=1830

mkvlr18:07:59

so v2 is due next year targeting high level langauges, v3 later (so prob at least two years away) targeting dynamic languages

pesterhazy19:07:13

You can’t say they are rushing things

bbss22:07:01

I want to do (into #js [] [{} {}])

bbss22:07:34

I have a clojurescript collection but I just want to clj->js for one level deep

bbss22:07:01

How would I go about that?

bbss22:07:23

I've been able to do it by copying the clj->js function and adjust it to only do something on the (coll? x) branch.

mfikes23:07:05

@bbss Check out cljs.core/into-array

👍 1
😁 1