Fork me on GitHub
#hoplon
<
2015-08-23
>
xifi07:08:29

@alandipert: yesterday you mentioned your demo hoplon contacts project. It looks nice, thanks! Is there one that would also show off the server-side? Is Castra used for that? An example of that would be cool to have.

xifi09:08:26

I'm trying to set up a project where I use boot-http with a ring handler. I set resource-paths to #{"resources"}, ring's middleware resources to "public" (so that the ring server will serve static resources from resources/public), create a resources/public/js/main.cljs.edn so that my main.js will end up there. When I start the server if I go to localhost:3000/js/main.js it is indeed there which is cool, but the index page can't find goog's base.js. The created main.js points to main.out/goog/base.js while it should be pointing to js/main.out/goog/base.js. Is this something I can set up or is the compiler not configured correctly?

xifi09:08:20

ah, there's a separate boot channel here on slack, sorry

micha14:08:34

@onetom: yesterday pushed alpha6 with FOUC fixes

onetom14:08:17

@micha: im testing that for an hour now with different version number combinations. i read the discussion on #C053K90BR too.

onetom14:08:41

im seeing all sorts of weird behaviour, eg all pages are compiled into 1

micha14:08:48

example of how i'm using it

onetom14:08:10

and as the page loads the title (on the chrome tab) is iterating thru all the page titles and settles with the last one

micha14:08:30

what version of boot-cljs?

onetom14:08:03

tailrecursion/boot-hoplon 0.1.2

onetom14:08:29

but i reverted to exicon/boot-hoplon 0.1.2-SNAPSHOT because prerendering was broken again otherwise

onetom14:08:03

i've checked you kept our make-parents but yet it didnt work with hoplon alpha6

micha14:08:39

hm maybe i botched the merge?

onetom14:08:58

i got a similar error as before:

Prerendering HTML files...
• newsletters/index.html
             clojure.lang.ExceptionInfo: java.io.FileNotFoundException: /Users/onetom/.boot/cache/tmp/Users/onetom/exicon/homepage/8a8/-e9icqa/newsletters/index.html (No such file or directory)

micha14:08:00

if you remove prerender task is everything ok?

micha14:08:19

hm let me test this

onetom14:08:21

yes, without the prerender it seems fine

micha14:08:33

ok i may need to push a new version

micha14:08:48

the merge might have been done incorrectly

micha14:08:46

sorry about that

micha14:08:12

we need tests simple_smile

onetom14:08:41

'[[exicon/boot-hoplon "0.1.2-SNAPSHOT"]
    [tailrecursion/hoplon "6.0.0-alpha6"]
    [adzerk/boot-cljs "1.7.48-SNAPSHOT"]
this version combo works with prerendering and there is no observable FOUC either

onetom14:08:08

well, clearly tests are necessary simple_smile

micha14:08:26

we need a test harness for boot tasks

onetom14:08:32

btw, i looked into what are the latest version of some indirect deps and i was wondering if there is any reason for holding back on these upgrades:

[tailrecursion/javelin "3.8.0"]
    [cljsjs/jquery "2.1.4-0"]
u mentioned some change in jquery, but it worked for stuff

micha14:08:34

ok i verified, the merge was botched

micha14:08:54

javelin can be updated for sure

micha14:08:08

the api hasn't changed in over a year

micha14:08:25

if jquery works for you we can update

micha14:08:40

there were some issues aroung attr/prop i think

micha14:08:57

but keeping up to date with jquery is important especially for mobile

micha14:08:13

for desktop not so much

onetom14:08:20

what is really used from jquery, btw?

micha14:08:35

only the low-level DOM manipulation stuff

micha14:08:00

you'd think that goog closure would be able to do that but i found it to be only partial implementation

micha14:08:19

it didn't compare to jquery in terms of efficiency and reliability

onetom14:08:24

that's exactly what my thinking was simple_smile

micha14:08:42

i ran into problems with setAttribute, which is a total no-go

micha14:08:56

like setting data-* attributes was unreliable

micha14:08:10

because goog had a hardcoded list of attrs that i guess they use at google

micha14:08:23

and it did weird things with any attr it didn't have hardcoded in

micha14:08:27

things like that

micha14:08:45

i figured that jquery is the most solid low-level dom library there is

micha14:08:56

so it makes sense to use it

onetom14:08:34

the guy mentions some document.querySelector or what

micha14:08:39

is this the "it's a library" vs "it's a framework" debate?

onetom14:08:05

havent checked it yet but that sounded like what $(<string>) does

micha14:08:30

we use $(<string>) in only one or two places in the entire thing

micha14:08:00

what jquery does provide though is really solid stuff like set attribute on element

micha14:08:12

where you need hacks around various browser bugs

micha14:08:00

we can replace it though, i'm not married to jquery

micha14:08:09

it's just a time management thing

micha14:08:13

jquery just works

micha14:08:38

i spent a lot of time working around goog closure bugs and whatnot

onetom14:08:47

i was not aware of this built in function

micha14:08:03

yeah it's a thing, and jquery uses it

micha14:08:20

jquery has a highly evolved thing that uses whatever is best

micha14:08:25

and it has caching and whatnot

micha14:08:19

in a modern fully compliant browser jquery is simply a thin layer on top of querySelector and all those things

onetom14:08:55

okay. it's just dnolen was advertising learning the "awesome closure lib" and using goog.object.get instead of aget and stuff like that, that's why i was wondering...

micha14:08:16

it might have improved

micha14:08:27

but i found it lacking compared to jquery

micha14:08:33

for dom stuff anyway

micha14:08:36

my rationale is that the most time consuming bugs are the browser compatibility ones

micha14:08:54

using jquery pretty much eliminates those

onetom14:08:09

i got you, no need to further elaborate. just push the fixed boot-hoplon so i can test alpha6 😉

micha14:08:38

he's talking about web components

onetom14:08:44

should i also upgrade cljs to -2 or -3 from -SNAPSHOT?

micha14:08:58

hoplon is the dream of web components but in a way that actually works

micha14:08:10

you should try latest cljs

onetom14:08:17

ah, yeah, that video was nothing very convincing

micha14:08:18

if there are bugs we need to fix them

micha14:08:39

alan and i are pretty convinced that any js solution will be lacking

micha14:08:52

js doesn't even have a sane equality semantic

micha14:08:08

so any framework you build on it will be coconuts

micha14:08:45

ok i'm going to update all this stuff

onetom14:08:35

yeah, that guy was showing of templating, but immediately made me remember coconut lisp too

onetom14:08:08

but then again, it was new to me that the <template> is default in browsers

micha14:08:15

it's not really

micha14:08:20

only in some versions of chrome

micha14:08:38

as i understand it google pissed everyone off by moving forward with their own thing

onetom14:08:44

so he is practically lying then?

micha14:08:46

and everyone else is refusing to implement it

micha14:08:55

when is the talk from?

onetom14:08:11

well the guy works for google...

micha14:08:16

as i understand it ie declares they will never support it

micha14:08:19

and ff also

onetom14:08:47

yeah he mentions what FF said and he said it's easy to polyfill it so it's not a big issue... simple_smile

onetom14:08:57

the whole talk was a bit fishy

micha14:08:04

yeah i looked into it a while ago

micha14:08:12

thinking to have hoplon elements be those

onetom14:08:15

even at the end a guy kinda nailed him badly...

micha14:08:20

but the polyfill only does the easy part

micha14:08:39

with polyfills it's way less powerful than hoplon already is

micha14:08:49

and the import href business is insane

micha14:08:04

dependency management is a critical piece of the puzzle

micha14:08:21

you need a really solid thing for dependencies

micha14:08:27

which maven is ideal for

onetom14:08:28

yeah, he was keep saying "just stiching the parts together"...

micha14:08:41

yeah how does that work with transitive deps

onetom14:08:51

made me think he doesn't seem to know wtf is he talking about

micha14:08:12

i think a lot of these guys don't work on big applications

micha14:08:19

they work on demos or something

onetom14:08:51

what worries me is this new ES6 is getting accepted and the webcomponents has started to gain traction

onetom14:08:54

at the same time it's all half arsed solutions really. like coffeescript goes well beyond what ES6 provides, what's the point of trying to close the gap between the 2 just a liiiill bit? 😕

micha14:08:10

yeah the es6 stuff is laughable

micha14:08:27

they're arguing for years about the syntax of let and fat arrows

micha14:08:46

in lisp you write a macro in less than 5 minutes and move on with getting work done

micha14:08:20

and the one thing we really could use, weak references, are not a thing

micha14:08:54

you can't build really complex things without weak references

micha14:08:14

i mean you can, but your abstractions suffer

micha14:08:46

0.1.3 deployed

micha14:08:38

i found another bug btw in the prerender task yesterday, it was caching prerendered html based on the mtime of the input html file

onetom14:08:38

can u recommend anything i can read up on weak references so i can understand what r u talking about?

micha14:08:00

which was incorrect because the stuff we're prerendering has nothing to do with the initial html

micha14:08:06

it's all generated by js

micha14:08:11

so i removed the caching

onetom14:08:23

ah we are also seeing a strange caching when i use panderio/boot-http

onetom14:08:37

the browser is using fuzzy caching TTL

micha14:08:37

it should now prerender every time

onetom14:08:46

because there is no explicite cache-control header

micha14:08:03

in chrome you can have it disable caches while dev tools window is open

micha14:08:07

that's what i do

onetom14:08:13

if only last modified time is specified, that's not enough

micha14:08:15

i haven't had a cached page issue in a long time

onetom14:08:42

okay, maybe we will do that too

micha14:08:52

there must be a ring handler for etag maybe?

micha14:08:22

there is also a secret thing that alan showed me the other day

micha15:08:02

when you have dev tools open you can right click on the reload button and select "Empty Cache and Hard Reload"

micha15:08:23

but i think if you have caching disabled when dev tools is open it's just a no-op

onetom15:08:26

i think even the etag was not enough

micha15:08:27

to make you feel better

micha15:08:03

are you sure it wasn't the bug in the perrender task?

micha15:08:14

it wasn't actually prerendering unless the initial html file changed

micha15:08:32

which would completely ruin prerendering for hoplon

onetom15:08:39

we had to add this to our nginx config to make sure content gets updated quickly in the visitors' browsers:

expires 1m;
		add_header Cache-Control "public";

micha15:08:42

because the initial html file never changes

micha15:08:03

ah ok, this is for production, not dev?

onetom15:08:34

actually as u r asking it now, i think we experienced that phenomena too sometimes

onetom15:08:52

but what i was talking about is prod, yes

micha15:08:00

we need to fix this in hoplon, use some kind of cache busting url scheme for the js file

onetom15:08:26

eg we released a newer pricing page and we really wanted to old one go away asap...

alandipert15:08:29

maybe the url is the content hash?

micha15:08:47

we need to compute the url before we know the contents of the js probably

micha15:08:49

the time has come, my hoplon friends, to talk of many things; of nginx and cdns, of cabbages and kings

micha15:08:21

we're planning to deploy our frontend to s3 website hosting

micha15:08:36

talks to backend cojure application via CORS

onetom15:08:06

S3 is on our horizon too

onetom15:08:33

the CORS thing works well for about 2 months now with castra 3 and transit

micha15:08:04

are you using the wrap-castra-session middleware or straight CORS?

micha15:08:37

i had so many annoying issues with cookies and CORS that i just said fu 🍪

micha15:08:50

and made a thing using localstorage instead

micha15:08:02

alandipert: do you have ideas wrt deploy to s3 + cdn?

micha15:08:25

like is index.html on the cdn?

micha15:08:55

if not then we need to rewrite all urls to fetch from cdn instead of relative uris

onetom15:08:58

(ns appboard.api
  (:require
    [ring.middleware.cors :refer [wrap-cors]]
...
(def ring-handler
  (-> (not-found "Not found")
      (wrap-castra
        'appboard.api
        'leaderboards.api
        'app-builder.api
        'distribution.api)
      (wrap-utf-8)
      (wrap-session
        (node-mongo-session (config :db) "sessions" "appboard.v1"))
      (wrap-cors (re-pattern (str "^" (config :frontend-url) "/?$"))
                 (re-pattern (str "^" (config :homepage-url) "/?$")))))

micha15:08:28

oh awesome, and cookies are working correctly of course?

micha15:08:33

that's great

onetom15:08:02

where wrap-cors come from

[jumblerg/ring.middleware.cors "1.0.1"]

micha15:08:06

the browser doesn't give any helpful insight into why it doesn't send cookies with CORS, it just silently doesn't when it doesn't want to

micha15:08:24

yeah that's a good CORS middleware

onetom15:08:34

we are setting the cookie on *.http://exiconglobal.com so {www,app,api,appboard}.http://exiconglobal.com can all access it

micha15:08:30

my backend is on a completely different domain

micha15:08:19

we also want to be able to support whitelabeling, so we don't know what domain the frontend will be on

micha15:08:47

the backend is multi-tenant of course

onetom15:08:47

meaning u would need to accept * as the CORS origin?

micha15:08:54

no, we'd have a whitelist

micha15:08:36

for the user's security

micha15:08:08

we don't want random domains on the internet to be able to trick people into logging into our backend from their page

onetom15:08:41

but shouldnt that whitelist be exposed to every client then?

micha15:08:58

no, the CORS middleware handles that very elegantly

micha15:08:19

it generates the CORS headers dynamically

micha15:08:24

for each request

micha15:08:40

only containing the info necessary to set that specific request up

micha15:08:33

the whitelist would be for the middleware to use to deetrmine whether to generate accep-origin headers for that request or not

onetom15:08:33

shouldnt the Access-Control-Allow-Origin: header contain the list of all possible frontends?

micha15:08:46

no, only the origin the client sent

micha15:08:52

if it's in the whitelist

micha15:08:10

the client doesn't need any of the other domains

onetom15:08:43

so those regexes i set in the example above are not sent back, just the requesting domain IF that domain matched any of those regexes?

micha15:08:52

indeed, if you set it to #".*" in the middleware it doesn't send *

micha15:08:02

it sends the origin the client sent, but for any client

micha15:08:11

yes, what you said

onetom15:08:34

interesting. i think i should read the code. it makes a lot of sense of course.

micha15:08:35

accept * will correctly block cookies from being sent

micha15:08:44

so we don't send that

micha15:08:00

the client won't send credentials to accept-origin:*

micha15:08:25

and it shouldn't because that would be a nasty trick to pull on the user

micha15:08:47

i think CORS is not well understood

onetom15:08:58

so * is meant for publicly readable apis?

micha15:08:20

it's meant for APIs that are safe to use on other people's behalf without their knowledge

micha15:08:37

like the github public api, for example

micha15:08:43

like to get info about public repos

micha15:08:48

all GET requests

micha15:08:54

no credentials

micha15:08:23

the api key used is associated with the domain of the sender for acocunting purposes, not for authentication

micha15:08:33

rate limiting, etc

micha15:08:22

CORS is a mechanism by which the browser prevents "confused deputy" attacks

micha15:08:49

the confused deputy is the situation where the user has authroized the browser to perform certain actions on their behalf

micha15:08:59

like sending cookies with each request for example

micha15:08:34

you "deputize" the browser such that once you log into a site the browser can cache some token and automatically send it on subsequent requests

micha15:08:05

however, suppose you log into a site in one tab

micha15:08:23

and in another tab you navigate to a different site that wants to make AJAX requests to the first site

micha15:08:45

you've delegated the cached credentials management to the browser

micha15:08:12

the browser now needs to know whether it should send credentials or allow site 2 to make the AJAX request to site 1

micha15:08:41

the way it does this is by adding an additional CORS preflight step to site 2's request

micha15:08:07

where it asks site 1 whether site 2 is within the same authroization realm, basically

micha15:08:25

like will site 1 accept credentials from people on site 2

micha15:08:43

so CORS stuff is not for backend security or authroization

micha15:08:08

it's metadata the browser needs to determine whether credentials associated with one domain are related to another domain

micha15:08:42

so the browser can decide what the security implications are of allowing or not allowing access from site 2 to resources on site 1

onetom15:08:58

so whats the misunderstanding?

onetom15:08:17

because what u described so far is known to me at least

micha15:08:28

a lot of times it's seen as a way to prevent unauthorized access to the backend

micha15:08:35

which it totally is not

micha15:08:51

a hacked client can circumvent CORS easily

onetom15:08:27

yup, im clear on that i think, but thanks for the explanation irregardless. my colleagues might found your explanation more enlightening that what they read about cors so far

micha15:08:59

also the preflight thing

onetom15:08:05

im trying alpha6 w boot-hoplon 0.1.3 and boot-cljs 1.7.48-2

micha15:08:08

like why the need for the OPTIONS request

micha15:08:24

mostly misunderstood

onetom15:08:19

it does that "load all pages" thing again///

micha15:08:36

which optimizations setting are you using?

micha15:08:57

also can you look in the html file and see if there are multiple js script tags?

micha15:08:18

can you try with :none please?

micha15:08:26

i think there is an issue with optimizations

micha15:08:35

does it happen without prerendering too?

micha15:08:21

oh also, some of the experimental features added to alpha6 might be useful for static site generators

micha15:08:30

the (static ...) macro specifically

onetom15:08:42

i will try in all combinations, it just takes 5mins to run a :simple opt + prerendering

onetom15:08:15

i read your commit message and i will check your gist too soon, im just having dinner atm

micha15:08:25

haha no rush simple_smile

micha15:08:29

enjoy your dinner

onetom15:08:32

i was also wondering in our specific case i would be probably a lot better off if i would just generate 1 page with sections which depend on the URL, but not on the #hashfragment but on the /path/part...

onetom15:08:11

i know u r against this HTML5 history api, but for such a limited case it might work, no?

onetom15:08:24

to simulate independent pages for SEO purposes

onetom15:08:03

i reverted to [adzerk/boot-cljs "1.7.48-SNAPSHOT"] and it's not fckd now

micha15:08:15

interesting

micha15:08:36

are you building multiple things concurrently?

micha15:08:46

multiple pages i mean

onetom15:08:57

well, do i have an option?

onetom15:08:07

i dont know how to turn off parallel builds

onetom15:08:44

im still talking about our public homepage which defines ~15 hoplon pages

micha15:08:48

no i was just curious

onetom15:08:24

but i slowly feel like if i would break our repo into 15 repos for every page we would have less issues simple_smile

micha15:08:41

how do you mean?

micha15:08:04

we can add the --ids functionality to hoplon pretty easily

onetom15:08:09

all of them could be on different hoplon, cljs, etc versions and could be deployed "independently"

onetom15:08:23

the only problem is that the header and the footer are shared

micha15:08:36

boot hoplon --ids products/index cljs ...

micha15:08:51

that would compile just products/index.html et al

micha15:08:06

the other tasks support this as well

onetom15:08:24

sounds something which could significantly speed things up for us in development

micha15:08:47

it would be added to the hoplon task

onetom15:08:53

is there a way to set task options from the command line?

micha15:08:00

how do you mean?

onetom15:08:36

we already have a pipeline setup which has hoplon in the middle

onetom15:08:49

how can i just alter the options for hoplon?

onetom15:08:58

from the command line i mean

micha15:08:05

you mean like this?

micha15:08:28

derp disregard where it says "http://hoplon.io"

micha15:08:39

i must have copy pasted that a long time ago

micha15:08:49

this is the dev task for my current project at adzerk

onetom15:08:34

just without being explicite and coaxing the options out

micha15:08:06

hm, that's the only way i know of

micha15:08:22

this way you get useful --help and whatnot

micha15:08:24

for all tasks

micha15:08:44

which is important on my project because we have people testing who aren't familair with the project

onetom15:08:27

so with this --ids option can i specify multiple pages? since we usually want to build the main page and navigate from there to the other page we are working on

onetom15:08:01

great, in that case we would use it for sure

micha15:08:17

(comp ... (hoplon :ids #{"index" "products/index" ...}))

onetom15:08:01

beautiful!

micha15:08:57

so how important is it to you to be able to have pages at paths like products/42/index.html etc

micha16:08:04

the 42 in there specifically

micha16:08:37

i'd like to do away with the tailrecursion.hoplon.app-pages._products_42_index_DOT_html namespace names

micha16:08:54

but that only works if your pages are valid namespace symbols

micha16:08:07

meaning you can't have a namespace named products.42.index

onetom16:08:31

we dont have such needs at the moment

onetom16:08:46

if we would have we would just prefix the number with something

micha16:08:49

perhaps we can escape those yeah

onetom16:08:03

it's not like we are trying to emulate a RESTful API w static files simple_smile

micha16:08:11

yeah that's my thinking as well

micha16:08:22

and the SPA entry points are few

onetom16:08:31

but i can imagine that another important use-case would be building blogs a'la jekyll

onetom16:08:05

where the article indexes are typically under /2015/08/11 pathes...

onetom16:08:02

in thai even in speech the colors are prefixed with the word "color" so i dont see why u shouldnt prefix it here 😉

micha16:08:02

/post-2015-08-11

onetom16:08:54

imean they dont say "red" they say "color red"

micha16:08:14

does the word for "red" have other meaning without the prefix?

onetom16:08:30

mostly not

micha16:08:37

man, i recently did a little reading about the Finnish language

micha16:08:41

really fascinating

micha16:08:00

there are so many ways to say things in that language

micha16:08:29

apparently even google searches are somewhat complicated by the different forms for each word

micha16:08:39

@onetom: why do you use :simple optimizations during development?

onetom16:08:42

i dont use simple during dev. i used it for prod (because :advanced was buggy with some of the hoplon versions)

onetom16:08:27

where is this fascination of yours w natural languages comes from? i was dating linguist girls for years...

micha16:08:38

man i need to do that

onetom17:08:52

[adzerk/boot-cljs "1.7.48-2"] works with :optimizations :none but the *re*compilation time is abysmal; it's 54seconds, while it's "only" 16seconds with 1.7.48-SNAPSHOT

micha17:08:47

you're saying it's actually slower in parallel?

onetom18:08:32

why, the difference between -2 and -SNAPSHOT is parallelism? -SNAPSHOT also saturated all of my cores.

micha18:08:31

maybe too much parallelism

micha18:08:59

the difference is separating each build into its own pod

onetom18:08:45

im on a 2.6GHz i5 MacBook Pro with 4 cores but "only" 8GB ram.

micha18:08:03

yeah i don't get it

micha20:08:31

@onetom: i think the multiple pages issue is now fixed in cljs