Fork me on GitHub
#hoplon
<
2016-03-27
>
thedavidmeister11:03:37

so, if I create a new local storage cell like this:

thedavidmeister11:03:55

(def stored-cell (local-storage default-cell ::my-cell))

thedavidmeister11:03:01

what’s the best way to reset that to the default value of my-cell?

thedavidmeister11:03:10

is (reset! stored-cell default-cell) enough, or will that cause problems with the local storage handling?

thedavidmeister11:03:48

soz, (reset! stored-cell @default-cell)

micha13:03:07

the storage cell should be like a normal cell

micha13:03:59

swap and reset work the same

thedavidmeister13:03:45

is there any special behaviour for storage cells?

thedavidmeister13:03:52

i’m thinking of doing a writeup on it

thedavidmeister13:03:43

@levitanong: it would be nice to get the basic usage of hoplon documented too, so when people become curious they have a smooth “on ramp” simple_smile

levitanong14:03:27

@thedavidmeister: Isn’t README.md sufficient? simple_smile

thedavidmeister14:03:34

i would say no, because there are many useful functions and macros and techniques provided by hoplon core that are not documented in either README or the wiki

thedavidmeister14:03:53

and because i see a lot of very similar questions being asked in this chat each week

thedavidmeister14:03:07

and even though a lot of this stuff might exist as part of one of the demos in the demo repository, many people won’t find/understand it in that format

levitanong14:03:26

I see! Okay, I’m game to do it. If you could list down those common questions, that would be great so I could structure it better.

thedavidmeister14:03:54

i’m just going to be casually putting stuff up on the hoplon repo as issues simple_smile

thedavidmeister14:03:03

everything related to performance is a big one

thedavidmeister14:03:48

I only just realised that the DOM is built statically and then do! assigns watchers for cells recently

thedavidmeister14:03:18

if you don’t know that, and you don’t know about javelin propagating values, I can see how all the performance stuff is confusing

thedavidmeister14:03:34

and i’m sure i’m only just scratching the surface with that 😉

levitanong14:03:53

yup, and that will prolly dovetail nicely into best practices also

thedavidmeister14:03:33

also, storage cells, for which there are quite a few different backends for, aren’t documented afaics ^^

thedavidmeister14:03:56

and smaller things

thedavidmeister14:03:11

with-timeout, detecting page load, how to use the route-cell, etc...

thedavidmeister14:03:50

@levitanong: i put up a couple of issues for some docs, feel free to have a crack at it simple_smile

micha14:03:03

@thedavidmeister: one thing about the storage atom that was recently(ish) added is the :ignore-existing keyword arg

micha14:03:23

normally the storage atom will initialize itself with the value stored in local storage

micha14:03:59

but sometimes you want to ignore what's in local storage and start with whatever is in the cell you're wrapping instead

micha14:03:04

that's what that's for

thedavidmeister14:03:49

@micha: ah cool, i’ll paste that explanation onto the issue and write up later simple_smile

thedavidmeister14:03:36

@micha: what about an explanation on how to implement your own storage backend?

micha14:03:17

a storage backend is basically a js object that has .getItem(key) and .setItem(key, value) methods

micha15:03:03

so like suppose i define js/firebaseStorage for example

micha15:03:31

(function() {
  var firebaseStorage = {
    getItem: function(key) {...},
    setItem: function(key, value) {...}
  };
  window.firebaseStorage = firebaseStorage;
})()

micha15:03:44

then i could do:

micha15:03:49

(defn firebase-storage
  [atom k & {:keys [ignore-existing]}]
  (html-storage atom js/firebaseStorage k :ignore-existing ignore-existing))

levitanong15:03:14

@thedavidmeister: thanks for the issues! Will work on it as soon as I can simple_smile

thedavidmeister15:03:12

@micha: ah ok, so that bit is usually done in raw JS and then just wrapped up for hoplon?

micha15:03:14

usually it comes from some other library

micha15:03:27

like the localstorage and session storage are built into the browser

micha15:03:46

you can use deftype to make a js type from cljs though

thedavidmeister15:03:48

do you know an example of that I can review?

micha15:03:23

hm so there is also another way

micha15:03:51

you could implement IStorageBackend protocol on anything

micha15:03:04

like this:

micha15:03:39

(defrecord FirebaseStorage [key]
  hoplon.storage-atom.IStorageBackend
  (-get [this not-found] ...)
  (-commit! [this value] ...))

(defn firebase-storage
  [atom k & {:keys [ignore-existing]}]
  (hoplon.storage-atom/store atom (FirebaseStorage. k) :ignore-existing ignore-existing))

micha15:03:07

i wonder if there are interesting uses for the storage atom that i haven't seen yet?

micha15:03:22

seems like i don't use it much

micha15:03:49

particularly for live-reload

micha15:03:57

it seems very handy for that

thedavidmeister15:03:57

well, i think it’s a nice easy way to save stuff

micha15:03:21

i really want to make an experiment to test google's javascript crawling performance

micha15:03:38

they claim to index single page apps with no shenanigans needed now

micha15:03:53

like they load the page now with a full js environment

micha15:03:04

but people are afraid of it

micha15:03:32

it would be interesting to make a single page app with lots of content and see how google really indexes it

micha15:03:53

like no html pushstate of any of those kinds of bakcend hacks

thedavidmeister15:03:21

webmaster tools seemed to show it working OK for me simple_smile

thedavidmeister15:03:28

although i do have pre-render on

thedavidmeister15:03:52

but, how would they know about “pages” that are just created from URL fragments

micha15:03:59

they'd have full js

micha15:03:14

so if they see a link to #/foo/bar they know what to do

micha15:03:25

and how to index it, presumably

micha15:03:06

an interesting thing is how do they know when the page is loaded?

micha15:03:29

like with a SPA the page is actually finished loading at some time after the html and js are downloaded

micha15:03:42

unlike a php app or whatever, where everything is rendered on the server

micha15:03:52

with an SPA the rendering takes place in js

micha15:03:33

there is no universally established way for an application to signal that the DOM is built

micha15:03:07

and the built-in events don't really line up with how the application works

thedavidmeister15:03:45

i specifically added some stuff to the page after window load

thedavidmeister15:03:54

so that’s a good point

micha15:03:25

they deprecated the #!/foo/bar thing, the ajax crawling protocol

micha15:03:34

which seems weird to me

micha15:03:43

since that was definitely by far the cleanest solution

micha15:03:14

shifting the responsibility for rendering the js to the server rather than to google's crawler

micha15:03:05

now they need to worry about apps that are slow to load and things like that

micha15:03:28

whereas with the ajax crawling protocol the site could cache those pages and whatnot

thedavidmeister15:03:09

maybe they just don’t crawl you if you’re too slow 😛

micha15:03:24

yeah they're missing out on a lot of content that way

micha15:03:32

and content is their business

micha15:03:41

i guess they must know what they're doing lol

thedavidmeister15:03:35

well i know they don’t like slow pages

micha15:03:50

yeah you fix that by caching for them

micha15:03:11

seems reasonable

thedavidmeister15:03:43

yeah, i’m using cloudflare

micha15:03:49

if you crawl faster pages more you will end up with skewed content

thedavidmeister15:03:49

which is easy enough to setup simple_smile

micha15:03:02

probably have mostly spam and linkfarms

micha15:03:27

because people with the best content are not so concerned with page load times, since they don't have a high bounce rate

thedavidmeister15:03:43

ah, but it’s all mobile first, etc.

thedavidmeister15:03:52

so i think part of the argument is that page speed is “part of the content"

thedavidmeister15:03:00

well, part of the quality of the content anyway

micha15:03:51

i guess it depends on what you do

micha15:03:05

if you're a serious business with specific content you're not going to be mobile first

micha15:03:27

if you're like facebook or something that people just flip to when they're bored then yes

micha15:03:56

but the best content is not something people will flip to for 0.2 seconds and then click on something else with their ADD

thedavidmeister15:03:28

depends where you live

thedavidmeister15:03:35

i’m sure if you’re in america it’s all fine

thedavidmeister15:03:02

but many sites i go to can take a long time to load

thedavidmeister15:03:07

all that latency to american servers

thedavidmeister15:03:01

will be nice when everyone is doing http2

thedavidmeister15:03:19

fewer roundtrips will be a great thing simple_smile

micha15:03:31

with like a SPA

micha15:03:43

like a hoplon app, you'd have the whole app on cloudflare

micha15:03:48

it's just html and js

micha15:03:02

the latency would be for api requests to the backend

micha15:03:16

which is less of an issue, no images etc

micha15:03:24

just compressed json

thedavidmeister15:03:52

that has been nice

micha15:03:03

so you should be able to load the app from a local cache wherever in the world you are

thedavidmeister15:03:15

yeah, i’ve been playing around with that

thedavidmeister15:03:22

it’s been good

thedavidmeister15:03:31

one more reason to use hoplon simple_smile

thedavidmeister15:03:10

actually, that’s probably worth mentioning on the performance writeup

micha15:03:47

yeah i think it's very useful to think of the SPA more like a java program for example

micha15:03:58

than a web application

thedavidmeister15:03:08

i was looking at electron

thedavidmeister15:03:20

to see how hard it would be to setup hoplon in that

micha15:03:30

that's the nodejs thing?

thedavidmeister15:03:31

and have an offline version

thedavidmeister15:03:44

uh, i think there’s some node in it

thedavidmeister15:03:51

but from what i read it’s basically just index.html

thedavidmeister15:03:58

wrapped up in an app

micha15:03:08

ah should be perfect for hoplon then

thedavidmeister15:03:10

with a few node functions to bootstrap the right bits

thedavidmeister15:03:27

i was thinking of having different storage backends depending on whether it’s online or offline

thedavidmeister15:03:46

but pretty much everything else working the same

micha15:03:54

yeah that's a hard problem, depending on what your app does

micha15:03:00

a distributed systems problem

micha15:03:10

like what to do when the app is offline

micha15:03:29

it's basically a partitioned replica then

thedavidmeister15:03:52

i was thinking fully offline

micha15:03:00

so when it comes back online you have to reconcile its state with all the things that happened in the rest of the world while it was offline

thedavidmeister15:03:01

like, you could use it as an app

thedavidmeister15:03:16

i’m cool with a “save” button

micha15:03:33

if users don't interact with each other then it's easier

micha15:03:49

but you still have the issue of like two versions of the app are being used by the user

thedavidmeister15:03:51

yeah, i’m trying to minimise direct user interaction

micha15:03:00

and they click save on both

micha15:03:04

and they disagree

micha15:03:32

the "multiple tabs" problem

thedavidmeister15:03:35

yeah, i’ve got a few options

thedavidmeister15:03:41

i’m not at that point yet

thedavidmeister15:03:46

that’s a problem for future me

micha15:03:51

haha right

thedavidmeister15:03:41

right now, i’m still just experimenting with solving different problems in hoplon

thedavidmeister15:03:02

and refactoring everything i do every week or so, lol

thedavidmeister16:03:49

anyway, i gotta cruise

thedavidmeister16:03:52

it’s late here

micha16:03:56

btw i sent an invite for the hoplon org on github

raywillig16:03:21

I have a single page app with LOTS of content

raywillig16:03:39

@micha how would we run this experiment?

micha16:03:51

is it live and being indexed by google?

micha16:03:30

the specific things to verify are how google indexes #/foo/bar type links in a SPA

micha16:03:02

like if i link to in my blog, does google index that correctly?

micha16:03:35

will google send you to that link in the search results? or does it just send you to

micha16:03:25

google analytics seems like a nice way for google to get feedback with SPAs

micha16:03:38

like when you fire off a synthetic pageload event in google analytics

micha16:03:50

they could totally cross reference that against the way they index your site

micha16:03:10

like your GA setup is giving them insight into the topology of your application

micha16:03:24

since the GA setup reflects how you organize the structure of your app

raywillig16:03:44

it’s not live live. but the dev version is on the interwebs. isn’t there some way to tell google about a new website so it can scan?

micha16:03:24

yeah you can do it in the webmaster tools

micha16:03:31

send them a link to it

micha16:03:52

to test though you need to build up links to the app from outside, too

micha16:03:51

but getting them to index it would give you insight

levitanong18:03:48

hey @micha, is it really necessary for hoplon to redefine certain methods like insertBefore?

micha18:03:14

it is, but it needs to be done a little better

levitanong18:03:27

I ask this because it seems to be interfering with some of d3.js’s behaviors

micha18:03:43

yeah the problem is in the merge-kids function i think

micha18:03:13

without the insertBefore stuff you can't do something like this:

levitanong18:03:17

when not using hoplon, when d3 creates and inserts an element, the newly created element has parentNode defined. when using hoplon and d3 creates and inserts an element, it’s null.

micha18:03:51

(ul
  (li "this is static")
  (loop-tpl :bindings [x xs]
    (li :text x))
  (li "this is also static"))

micha18:03:44

are you doing the d3 stuff in the body?

micha18:03:53

or is it operating in a div that's inside the body

micha18:03:15

because there is a bug with insertBefore that affects things that are direct children of the body

levitanong18:03:18

the operations are indeed on body

micha18:03:27

can you make a wrapper div?

micha18:03:33

to test this theory?

levitanong18:03:17

done. the issue still happens 😞

micha18:03:37

what hoplon is doing there is making a layer on top of the DOM api

micha18:03:52

so when you do (.appendChild elem foo)

micha18:03:04

it actually adds foo to an atom

micha18:03:11

the atom contains a vector of the children

micha18:03:25

then a watcher on that atom does the actual appendChild if necessary

levitanong18:03:47

that’s why it stays in the kind of… limbo state

micha18:03:51

the reason why this is necessary is because of things like loop-tpl

micha18:03:05

like in the case i showed you above

micha18:03:15

how does hoplon know where to put the tpl elements?

micha18:03:22

like in between those two li's

micha18:03:35

what hoplon does there is have the atom like this

micha18:03:52

[(li "this is static") [...] (li "this is also static")]

micha18:03:00

that's the child vector for that ul element

micha18:03:09

the vector in the middle is the loop-tpl elements

micha18:03:15

that could even be empty

micha18:03:19

which is critical

micha18:03:30

because the empty vector saves the place where the elements will go

micha18:03:39

if the loop-tpl becomes non-empty

levitanong18:03:44

makes sense.

micha18:03:55

there is no such thing as a placeholder in html dom unfortunately

micha18:03:58

or a generic container

micha18:03:06

like in svg you have the g element

micha18:03:10

which is a generic container

micha18:03:16

but in html you have either div or span

micha18:03:26

and you can't use either univerasally

micha18:03:20

perhaps loading d3 after hoplon is finished with the dom might help?

levitanong18:03:28

will give it a try

levitanong18:03:36

just wondering though, how it’ll affect the other libraries

micha18:03:45

yeah it's a tradeoff

micha18:03:54

it makes it much easier to do things in hoplon though

micha18:03:07

and most libraries in js are of pretty low quality

levitanong18:03:29

alas, i went through hell trying to figure out what was going on in the d3 source

levitanong18:03:33

so much mutation @_@

micha18:03:39

yeah it's a whole thing

micha18:03:47

it has its own web framework in there

micha18:03:55

hundreds of thousands of lines of js

levitanong18:03:57

when you say loading it after hoplon is finished

levitanong18:03:04

do you mean putting it on a setTimeout?

micha18:03:15

yeah that would be easy to try

micha18:03:22

like a 2000 ms timeout or something

micha18:03:11

this is interesting

micha18:03:17

perhaps we could do something similar

micha18:03:56

actually, svg might have issues

micha18:03:23

we made one little modification to jquery for svg support

micha18:03:32

because jquery doesn't really support svg properly

levitanong18:03:26

just tried putting it in a timeout. now it can’t find d3 😛

levitanong18:03:00

also just read your link. Did he basically reimplement the DOM parts of d3 in react? o_O

micha18:03:30

i guess with hoplon the real problem is that hoplon exposes a dom that has more fundamental features than the browser dom

micha18:03:49

like the generic container (the vector in the children atom)

micha18:03:13

there is no way to expose the normal DOM in a way that doesn't lose information

micha18:03:25

so like d3 is both writing to and reading from the dom

micha18:03:47

does your example application have any loop-tpl or anything in there?

levitanong18:03:50

Time to make my own charting library 😛

levitanong18:03:10

The example I made for this issue doesn’t have loop-tpl. It’s just a reduced case. The reason I’m working on this, however, is an app I’m making for work.

levitanong18:03:35

Ideally, I’d avoid reinventing the wheel, and just use d3 for rendering the data vis stuff

micha18:03:39

do you have a demo i can use to reproduce the error?

levitanong18:03:11

but given the issues with insertBefore, I’m just using d3 to run some of the scaling calculations, and implementing the visualizations entirely in hoplon.

levitanong18:03:52

Since in my country, the past few days has been a series of national holidays, I decided to do due diligence to see if there’s anything at all I can do to get d3 to work with hoplon

levitanong18:03:01

as for a demo

levitanong18:03:21

I can send you a zip

levitanong18:03:49

will that suffice?

levitanong18:03:32

well alternatively i can just push it to github

levitanong18:03:35

actually i’ll do that

micha18:03:02

yeah the insertBefore logic in hoplon is convoluted

micha18:03:11

i'm sure it can be simpler but it's hard to think about

levitanong18:03:40

there will be a lot of console.logs scattered around the d3.js file

raywillig18:03:20

@levitanong: are you looking to make normal charts and graphs for business? bar, line, pie, etc?

micha18:03:19

d3.js is the regular d3 source code, right?

levitanong18:03:00

@raywillig: i am, but with some oddly client specific stuff, so I can’t use c3 😞

levitanong18:03:34

@micha: yes. It’s slightly modified to get rid of some of the tertiary conditionals

levitanong18:03:57

for debugging

micha18:03:00

are those modifications necessary? or was it for testing

levitanong18:03:17

If you like, I can replace it entirely with stock d3

micha18:03:18

ok the first thing i recommend is using this: https://github.com/cljsjs/packages/tree/master/d3

micha18:03:26

instead of script tag

levitanong18:03:29

I was actually using that 😛

micha18:03:34

because the script tag will load asynchronously

levitanong18:03:36

for my project

levitanong18:03:50

but for the reduced case, I wanted to do everything on the console

micha18:03:38

i don't see any d3 stuff in the repo though

levitanong18:03:12

ah yes, i forgot to mention

levitanong18:03:37

just copy and paste the following in the console

levitanong19:03:46

i was comparing this to a setup wherein i wasn’t using hoplon

levitanong19:03:54

but used the exact same d3.js file

levitanong19:03:14

and same snippet in the console

levitanong19:03:31

this is how i concluded that hoplon was doing something that made certain methods behave differently

micha19:03:49

ok so i can do things like this

micha19:03:52

d3.select("body").append("div").text('ok')

micha19:03:56

that works

levitanong19:03:13

yeah, the simple stuff works

levitanong19:03:20

it totally breaks down when working with axes.

levitanong19:03:56

Do you think it’s possible to make a kind of “island” macro?

levitanong19:03:20

basically a piece of code that is isolated from the rest of hoplon, where it’s just vanila DOM

micha19:03:34

it should be possible

levitanong19:03:06

because I think hoplon development would stall if we had to bend over backwards trying to accommodate these JS libraries

micha19:03:10

a special element, like (foreign :id "foo")

levitanong19:03:18

and then it just evaluates the body

levitanong19:03:26

(with-foreign) maybe

micha19:03:30

the problem is when d3 does things later though

micha19:03:40

the context is lost

micha19:03:25

we can mark foreign nodes by adding a .isForeign property to them

levitanong19:03:29

perhaps some kind of binding that allows some communication between that island and everything else?

micha19:03:38

and any new node gets that set if its parent has it set

micha19:03:56

then the shim insertBefore etc can check that

micha19:03:08

and delegate to the native browser methods if it's a foreign element

micha19:03:21

instead of doing the atom children thing

levitanong19:03:29

sounds like a lot of work 😛

levitanong19:03:50

but that would be seamless

micha19:03:50

it would be good to know why d3 is failing though

micha19:03:10

because it's proabbly a bug that we can just fix and then maybe everything works fine

levitanong19:03:49

well, the farthest i’ve gone is their d3_selectionPrototype.insert method

micha19:03:57

have you tried iframes?

micha19:03:08

i wonder what the situtaion is there

levitanong19:03:12

i have not tried iframes.

micha19:03:28

whoa what is that

micha19:03:34

d3_selectionPrototype

levitanong19:03:04

iframes are scary, but maybe they can work as the island...

levitanong19:03:59

anyway, what i found is that in d3_selectionPrototype.insert, the call to this.insertBefore returns different results for hoplon and vanilla.

levitanong19:03:21

i suppose the next step is to put in some log statements inside hoplon’s .insertBefore, that’s alien territory for me at the moment

levitanong19:03:53

anyway, it’s getting pretty late here, will have to sign off. Thanks so much for your time, @micha!

micha19:03:29

later! i'll see if i can find anything

levitanong19:03:00

Much appreciated!

micha19:03:56

this is where we want unit tests simple_smile

micha19:03:02

i do see a bug in the implementation

micha19:03:19

foo.insertBefore(newNode, null)

micha19:03:31

that should be the same as foo.appendChild(newNode)

micha19:03:16

:highfive:

micha19:03:20

i will make a branch with the fix

levitanong19:03:45

@micha whoa that was fast! Now I can sleep easy

levitanong19:03:04

Excited for the branch :D

levitanong19:03:38

Also, unit tests, you say? @thedavidmeister will be pleased

micha19:03:00

yeah we could really use some unit tests for that kind of stuff

micha19:03:04

the dom impl

levitanong19:03:22

Alright, off to bed for me. May the rest of your day be great, @micha!

micha19:03:33

see you later!

levitanong19:03:42

couldn’t help it. had to test it. IT WORKS

micha19:03:59

good news eh?