Fork me on GitHub
#hoplon
<
2020-04-10
>
dave12:04:26

has anyone experienced hoplon's prerendering turning < in inline scripts into <? i'm running into trouble with my app's content security policy because the prerender step is changing the content of an inline script that contains < so that it contains < instead

dave12:04:32

which seems like a bug. the content of script tags should not be html-escaped

micha12:04:40

the prerenderer specifically skips scripts

micha12:04:46

in both the head and the body

dave12:04:51

in this case, the scripts have :prerender-keep

micha12:04:46

yeah but that code doesn't look like it knows about :prerender-keep

micha12:04:12

it's suspicious that it broke all of a sudden

micha12:04:20

in the middle of changing all the tooling

dave12:04:45

i think it's been broken the whole time, we just didn't notice in this particular project because the existing scripts we had with :prerender-keep happened not to contain &, <, or >

dave12:04:56

the new one we're adding wants to do a less-than comparison

micha12:04:11

can you make a minimal repro case?

dave12:04:38

to be clear, i've put the debooted branch aside

dave12:04:45

this is all using the existing boot pipeline for geir

micha12:04:47

like a helloworld that exhibits the problem

dave12:04:10

i'll see if i can take a stab at that when i have time

dave12:04:28

for now, i think i might need to add a postprocessing step that re-encodes < as <

micha12:04:30

i don't see how it can possibly be including those scripts

dave12:04:36

i guess inside of script tags only

micha12:04:59

there are better ways, you can use the baackslash hex encoding if you need to

micha12:04:04

in the js

dave12:04:13

ah! i was wondering if there were another way to encode it in the JS

micha12:04:26

like \x12 or whatever

micha12:04:43

but link me to the script tag in the cljs code

micha12:04:53

the problematic one

micha12:04:08

and also have you tried with prerendering off?

dave12:04:24

sending it to you on adzerk slack

dave12:04:50

i've actually been wondering if we really need prerendering

dave12:04:20

since we're getting caching via cloudfront, i'm not sure if prerendering is helping much beyond that

micha12:04:40

prerendering helps with the flash of unstyled content

micha12:04:53

have you built it with prerendering off though?

micha12:04:06

to eliminate other issues unrelated to prerendering that may be causing it

dave12:04:08

yeah, without prerendering the problem goes away

dave12:04:20

my first thought was that it might be an advanced optimizations issue, but that wasn't it

dave12:04:03

hmm, it looks like \x3C is the escape sequence for <, but it doesn't seem to work outside of javascript string literals

dave12:04:19

i've only experimented in the JS console, though. not sure if it behaves differently in an inline script tag

dave12:04:48

in the JS console you can do escape("<"), which returns "%3C"

dave12:04:06

then you can do "\x3C", and that returns "<"

micha12:04:18

you should be able to put that in the js source, no?

dave12:04:24

i'll give it a try

micha12:04:00

you need to escape the backslash because clojure strings interpret it

micha12:04:20

like (script "foo \\3C ...")

dave12:04:44

makes sense

dave12:04:50

that doesn't appear to work

micha12:04:29

what does it look like when you do "show source" (not the elements pane of the devtools)

dave12:04:35

i changed the offending part of the code to for(w=0,x=v.length;w\\x3Cx;++w) whereas the \\x3C used to be <

dave12:04:54

i haven't tried prerendering yet, so show source doesn't include the script tag

dave12:04:21

i suspect the result would be the same

micha13:04:41

what does the script tag look like in the elements pane

micha13:04:31

the easiest thing to do really is to make your own html file and then mount the app in a div inside there

micha13:04:21

then you don't need prerendering because you can pull out the things you want to have prerendered into the html file

dave14:04:43

looks like what i expected: w\x3Cx intsead of w<x

dave14:04:21

making our own html file is a good suggestion. i've been thinking about that in the back of my mind

dave14:04:41

i would just need to do some post-processing to insert the right cache keys for cache busting

dave14:04:46

could another solution be to put the inline JS into a file instead? or would that potentially screw up the load order?

dave15:04:54

putting the JS into a file got me un-stuck

alandipert15:04:17

just catching up,was the problem literla JS inside a script tag in a cljs file?

dave15:04:26

with :prerender-keep

dave15:04:45

although i guess i'm not 100% sure if :prerender-keep was related

alandipert15:04:38

i think i've run into that before,with google analytics or something

alandipert15:04:02

i wonder about using js* to inline JS strategically

alandipert15:04:23

that would roll it into the cljs source

alandipert15:04:39

i guess sometimes you need JS to run before the cljs kicks in tho

dave15:04:26

does it need to be inline for that to be the case?

dave15:04:16

like is the load order for this:

<script>alert("lol");</script>
<script src="index.cachekeygoeshere.html.js"></script>
any different than the load order for this?
<script src="alert_lol.js"></script>
<script src="index.cachekeygoeshere.html.js"></script>

alandipert15:04:41

should be no different

alandipert15:04:31

i was suggesting using js* to put alert("lol") in index.cachekeygoeshere.html.js

dave15:04:43

ah, i see

dave15:04:49

we're actually doing that for google analytics in geir

dave15:04:09

it's just that it only works because the inline JS doesn't include &, <, or > which hoplon's prerenderer would escape

alandipert15:04:33

seems to me like the prerender shouldn't escape script tag innards

alandipert15:04:14

from a pure xml sense it make sense, and is a good default. maybe a bug in prerender?

dave15:04:43

i'm thinking it's a bug, too

dave15:04:03

like if the type of the DOM node is "script," then the escaping shouldn't happen

alandipert15:04:24

absolut nyet

alandipert15:04:50

anothre option is golf that code to not use any escaped chars like goog did

alandipert15:04:17

partially as entertainment 😆

dave15:04:29

haha, nothing like a rousing game of code golf 😄

dave15:04:47

i'm sure there is some too-clever way to get "less-than" semantics without using <

alandipert15:04:59

function lt(x, y) { var z = (y - x); return z != "0" && z.toString()[0] !== "-"; }

alandipert15:04:15

crap,ampersand

dave15:04:20

function lt(x, y) { return (x - y).toString()[0] == "-"; }
would this work?

alandipert15:04:52

that would be <= because if x == y, [0] !== "-"

alandipert15:04:22

close but no banana

dave15:04:34

hmm, i tried that in my browser and lt(2, 2) is false like i expected

alandipert15:04:20

oh i see, yes, it should work

alandipert15:04:36

my thing was lte, yours is lt

alandipert15:04:17

x is < y iff (y - x) < 0, e.g. its string rep. has a leading dash

alandipert15:04:51

i bet google did that intentionally man

alandipert15:04:10

i wonder if a tool of theirs does this kind of rewriting even

dave15:04:25

lt=(x,y)=>(x-y).toString()[0]=="-"
golfed a bit further 😄

alandipert15:04:39

aha that => tho

alandipert15:04:45

gotta keep it es3

dave15:04:56

i keep forgetting that JS has had that for like 5-10 years lol

alandipert15:04:20

a few years ago at my new job i spent a few weeks relearning JS starting from 2012 or something

alandipert15:04:23

it has all the things now pretty much

alandipert15:04:53

in addition to all the crazy things it always had

alandipert15:04:45

you know, i guess you can't make a real short-circuting && as a function, because the real one delays evaluation

alandipert15:04:51

the best you could do is wrap every argument in a function at the call site

dave15:04:15

for the purposes of <= and >=, you could just implement < and > and then call it with !

alandipert15:04:25

lol this is unfolding into a great big tech company interview question

dave15:04:54

"please sort this linked list in JS, and you can't use &, <, or >"

alandipert15:04:58

oh sweet, yes good point

alandipert15:04:49

you could even say it's not a contrived question, it actually came up in the course of work haha