Fork me on GitHub
#hoplon
<
2015-12-20
>
laforge4902:12:30

I've reworked castra-chat to use the current hoplon-castra template: https://github.com/laforge49/castra-chat

micha02:12:20

@laforge49: the wiki page is great

laforge4903:12:02

Glad you like it

micha03:12:25

sent an invite on github

micha03:12:36

feel free to push updates if you like

laforge4903:12:05

nice. But it will be quite a while methinks!

micha03:12:13

no worries!

laforge4903:12:08

Mostly I'm going to have lots and lots of questions going forward. But I'm more than happy to create demos and docs as I learn.

micha03:12:33

i'd like to track down that problem with the target directory on windows

micha03:12:50

if you could file an issue with the boot project with a stacktrace that would be awesome

micha03:12:04

boot -vv ... will enable full traces

laforge4903:12:14

I am in the process of building a db of my own: https://github.com/laforge49/aatree

laforge4903:12:32

This is a rework of some old Java code, and I still have a long way to go.

laforge4903:12:05

But it needs a front-end. So that will be my first project in hoplon.

onetom03:12:38

@flyboarder: https://github.com/degree9/silicone excites me too because i've ditched material ui a year ago in favour of semantic ui, because at that time material ui was just not mature enough (neither our clojure knowledge to integrate js libs) by any chance do u have any example which uses silicone?

onetom03:12:12

@laforge49: u made a very informative and attractive intro for aatrees! do u know http://picolisp.com/wiki/?home ? it's a lisp built with 1st class db support. certain style of symbols are pointing directly into the db on disk, but they are treated equally with symbols bound to in-mem values.

flyboarder03:12:49

@onetom: yes and no, im refactoring all the code right now but our flagship product is using it, it’s a very basic state right now and highly broken but most of the repo’s under https://github.com/degree9 are related the core project is https://github.com/degree9/theLounge

flyboarder03:12:15

by the end of the year i’d like the full alpha working

laforge4903:12:47

I've also found that things are working better now, so I simplified the wiki page: https://github.com/hoplon/hoplon/wiki/Running-on-Windows

flyboarder03:12:58

@onetom: im also interested in refactoring the importing so all the polymer imports are part of the static page prior to hoplon loading

onetom03:12:48

how big polymer is now? (gzipped minified)

flyboarder03:12:02

not sure im using bower

onetom03:12:21

and are you thinking about using it as is or hoplonifying the behaviour parts and just use the CSS?

micha03:12:24

@laforge49: the init-params shenanigans is no longer necessary with boot-jetty?

micha03:12:52

also aatree looks fascinating

laforge4903:12:35

And no, I am unfamiliar with picolisp.

flyboarder03:12:40

@onetom: well the idea is that you are able to select the exact components you want to load from the library so you dont need all of it

flyboarder03:12:10

micha gave me some direction on fixing the interop between hoplon and polymer but most everything work the hoplon way

micha03:12:36

i'm going to do some experimentation next week with the script loading

micha03:12:47

i'll have some vacation time coming up

laforge4903:12:53

my db is still in the toy stage. takes a lot to move it forward. Having an haplon front end will help!

onetom03:12:06

@flyboarder: if u could keep the development open, we would very like to be helping out

flyboarder03:12:42

@onetom @micha all our source code will be open the idea is a community developed infrastructure automation software

flyboarder03:12:38

getting it up and going is the current in progress, the libs we make will be for everyone 😀

micha03:12:17

@laforge49: is the idea to have clojure data that might not all fit in memory but can be lazily evaluated and modified?

flyboarder04:12:47

@onetom: silicone 0.3.0 was pushed ill throw together a little sample app sometime

laforge4905:12:48

@micha Transparently loaded on demand, cached and transactionally modified

micha05:12:24

even read-only with transparently loaded on demand would be enough for me

micha05:12:39

out-of-band modification via rpc calls, decoupling the "query" form the "command"

laforge4906:12:43

looking at the castra-chat, my first question is "must we poll for notifications?" I see a 200 ms poll loop. Ouch!

laforge4906:12:35

@micha I found this and was wondering if any work was done integrating hoplon with websockets: http://hoplon.discoursehosting.net/t/polling-vs-server-push/227/11

laforge4906:12:58

The alternative of course would be to aggregate events on the server side for each client and have the client fetch them by polling. Then at least there is a single pole rather than one for each possible event. For reliability, each poll could also serve as an ack previously received events. Numbering events per destination client means you can ack an event and all prior events, so the response would only contain higher events and the acked events get dropped.

laforge4906:12:04

So it looks like a first project would be a smart chat which works this way. Messages are then only sent once, though batched.

laforge4907:12:18

@tbrooke @alandipert What is the latest on integrating haplon and firebase? Any progress?

laforge4914:12:28

@mynomoto Impressive. And beyond what I want to attempt at this time. I just lack the background. I'll stick with castra for the time being. Enough to learn there and I really want to get back to working on my own stuff. On the flip side, websockets are the obvious way to go and I'd love working on some joint projects, especially small ones.

mynomoto14:12:58

Yeah, maybe some simple examples using websockets, sse with hoplon could be useful, including the server side. Right now I lack the time to do that but if you try and get stuck I should be able to help.

micha15:12:55

@laforge49: i like the idea of aggregating

micha15:12:00

on the server

laforge4916:12:18

castra-chat demo question--about defrpc: What is that vector at the beginning of the defrpc body? (defrpc get-state [& [user]] {:rpc/pre [(nil? user) (logged-in?)]} (let [user (or user (:user @session)) users (->> @db :users keys sort) convos (->> (->> @db :messages keys (filter #(contains? % user))) (select-keys (:messages @db)) (map-v (comp reverse (partial take 10))))] (when user {:user user, :users users, :messages convos})))

laforge4916:12:28

It is assigned in a map the key :rpc/pre. But beyond that I don't see anything.

micha16:12:34

that's a precondition

micha16:12:52

the idea is that you could have rpc functions that are composable, or you can call them from the repl

micha16:12:06

this means that you need to do authentication in a more nuanced way

micha16:12:26

the :rpc/pre preconditions are only evaluated when calling the endpoint from HTTP

micha16:12:56

so if you have an rpc endpont foo that calls another endpoint bar internally, the preconditions are only applied to foo

micha16:12:24

this way you can compose them without needing to mock state or carry global state around

micha16:12:05

there is also :rpc/query which is used to return the response to the client

micha16:12:22

but again only the actual rpc endpoint the client called will evaluate the query

micha16:12:19

the high level design of it is that we want to establish "domains" of state

micha16:12:35

like consider a simple web app that has user authentication

micha16:12:56

so the client will let the user log in, and then in the app there are states that depend on who the user is

micha16:12:19

so we'd have a cell that contains the user info, like their name, etc

micha16:12:39

but there are multiple rpc endpoints that relate to operations on the user

micha16:12:47

each one of those will return the same query

micha16:12:54

this is a stateless query over the database

micha16:12:02

i.e. the new user state

laforge4916:12:15

I did find this in castra.core: (when pre# @(when pre [`(assert pre)]) but had not yet figured out pre#. Your comment helps.

micha16:12:25

but the "command" (the rpc function body) is completely decoupled from the response "query"

micha16:12:37

the thing that's returned is just a view over the data

micha16:12:48

that may or may not change as a result of the rpc body evaluation

laforge4916:12:48

Are there any examples of :rpc/query anywhere?

micha16:12:04

hm there should be

micha16:12:44

separating the rpc/query into metadata on the rpc function has the advantage that the function can return something else when called as a regular function

laforge4916:12:48

not anywhere in castra-chat!

micha16:12:12

which is good because on the server you have synchronous access to the function and can catch exceptions synchronously

micha16:12:43

so it's good to have that, like for example when a new user registers for the first time

micha16:12:59

you generally want to create their user data in the database, and then log them in in one go

micha16:12:17

most web apps don't do this because it's problematic with the way they do authentication

laforge4916:12:17

I was saying that it is not used anywhere in the castra-chat project

micha16:12:23

that's possible

micha16:12:41

it might have been added later

micha16:12:58

that was a very early example that i made in an hour as a demo for a job

micha16:12:05

*job interview

laforge4916:12:06

but without docs it is all there is. 😞

micha16:12:08

the :rpc/pre is an expression that is evaluated in the context of the rpc function body

laforge4916:12:13

Or am I missing something?

micha16:12:18

it's transformed into an assertion as you saw

laforge4916:12:36

yes I got that figured out now. many thanks

micha16:12:40

you can refer to arguments there, etc

micha16:12:02

if you throw an exception there it will be transported back to the client

micha16:12:08

so the client can catch it and handle it

micha16:12:23

if you throw a generic exception it will be wrapped

micha16:12:39

the idea is that the exception message is always acceptable to show to the user

micha16:12:51

you can throw a castra exception with your own message

laforge4916:12:51

so the same for :rpc/query? just add another key to the dictionary where :rpc/pre is associated?

micha16:12:33

i will write docs today

micha16:12:55

we've had like 2 years amnesty, it's time lol

laforge4916:12:59

Now in get-state I see this:

laforge4916:12:04

(when user {:user user, :users users, :messages convos})

laforge4916:12:12

Oh that would be super!

micha16:12:42

i'll have something in an hour or two

micha16:12:49

with a new demo

laforge4916:12:53

This is the return value of the get-state rpc.

laforge4916:12:18

so when :rpc/query is not used, the function return value is used instead?

micha16:12:36

that get-state function is the root domain one

micha16:12:08

the other functions in that domain would have :rpc/query (get-state)

laforge4916:12:23

So there are many api methods, but only one result returned for everything?

micha16:12:32

well you divide into domains

micha16:12:53

like all the endpoints that operate on the user would return get-user for instance

micha16:12:06

the chat demo is very simple so i think i threw the whole thing in one domain

micha16:12:17

that returns the entire state of the application

micha16:12:34

but in a more realistic app you'd have multiple domains that are orthogonal

micha16:12:52

and you'd be getting a window of data too

laforge4916:12:58

so the value of an rpc return depends on its domain?

micha16:12:16

the return value establishes a domain, you could say

micha16:12:22

like for example

micha16:12:38

i have get-user, login, logout, and register endpoints

micha16:12:48

maybe that's all the operations you can do on a user

micha16:12:16

get-user is the endpoint that returns a query over the database for the current user's data

micha16:12:27

it gets the current user from the session perhaps

micha16:12:48

then login, register, etc all have :rpc/query (get-user)

laforge4916:12:53

I don't see get-user anywhere

micha16:12:06

i haven't looked at that program in a while

micha16:12:16

but in my current project for example

micha16:12:41

the idea is that during the design phase we identified this orthogonal concern

micha16:12:46

the "user" state

micha16:12:06

so when we implement it on the server we start with the getter rpc endpoint

micha16:12:18

which is a query over the database

micha16:12:45

SELECT name, age, karma FROM user WHERE id = ?

micha16:12:13

now suppose we want to make an rpc endpoint that lets the user change their age

laforge4916:12:27

I notice the .hl file does not have any response logic tied to the forms except for the rpc call.

micha16:12:35

yeah that's key

micha16:12:57

in the cqrs model the commands are completely decoupled from the query

micha16:12:11

the query here is formulas on cells in the client

laforge4916:12:17

that solves the synchronous model problem

micha16:12:18

the command is the rpc call

micha16:12:26

there are nuances

micha16:12:00

like it's good to separate the comand from the query, because then in the client the thing that sends the command to change the age doesn't need to know about the things that will update as a result of the age in the user cell changing

micha16:12:13

it just knows how to send the command

micha16:12:24

if the command succeeds then the user cell will change

micha16:12:33

but it doesn't know about that, or need to know

micha16:12:54

however, the piece of program that sends the command usually needs to handle errors locally

laforge4916:12:04

how big a user base do you have with castra???

micha16:12:06

becauase it's going to encapsulate that state

micha16:12:14

haha i don't know

micha16:12:35

so anyway the rpc function in the client, when you call it, does return a promise

micha16:12:45

you can use this promise to handle exceptions

micha16:12:53

you may need to retry, for instance

micha16:12:13

sometimes you also need to deviate from the cqrs model

micha16:12:21

and you can do that with the promise

laforge4916:12:12

but I am not seeing any of that in the chat demo, so it is hard to infer!

laforge4916:12:26

Is there something else to look at?

laforge4916:12:46

or should I just wait? simple_smile

micha16:12:06

this is the data flow

micha16:12:17

blue is the command line

micha16:12:22

red is the query line

micha16:12:51

"stem cell" would be a cell that contains the domain state

micha16:12:58

like the user data for example

micha17:12:06

there could be more than one of them of couse

laforge4917:12:44

Where is this gif referenced???

micha17:12:56

it was removed from http://hoplon.io at some point

micha17:12:09

it should be in the castra repo probably

micha17:12:16

i'll add it to the docs

jethroksy17:12:47

this is awesome

laforge4917:12:53

so I take it there is some library function somewhere which gets that promise from a command.

micha17:12:08

that's what the rpc call returns in the client

micha17:12:21

like if you do (login! "foo" "bar") in the client

micha17:12:28

where login! is an rpc fn

micha17:12:35

what that returns is a promise

micha17:12:44

you can add a .fail handler to it for instance

micha17:12:53

and do something if there's an error

laforge4917:12:57

yeah, but it is done via a submit in a form.

micha17:12:07

how do you mean?

micha17:12:34

the form isn't actually POSTing to the backend though

micha17:12:52

it's just firing the submit event, and your handler is calling the rpc function

laforge4917:12:55

(form :submit #(log-reg! @user @pass @pass2)

micha17:12:06

right yeah

laforge4917:12:10

and what handler is that?

micha17:12:13

so if you want to say retry

micha17:12:16

for example

laforge4917:12:58

this? (def handler (-> app-routes (castra/wrap-castra 'demo.api.chat) (castra/wrap-castra-session "a 16-byte secret") (d/wrap-defaults d/api-defaults)))

laforge4917:12:08

I don't understand the handler.

micha17:12:26

(form
  :submit #(.fail (log-reg! @user @pass @pass2) (fn [& args] (.log js/console "error!"))))

micha17:12:53

that's building the web server

micha17:12:10

it's using the ring model

micha17:12:26

you compose middleware to build the webserver

laforge4917:12:30

(form :submit #(log-reg! @user @pass @pass2) (div (label "Username") (input :type "text" :autofocus "autofocus" :value user :change #(reset! user @%))) (div (label "Password") (input :type "password" :value pass :change #(reset! pass @%))) (div :toggle register? (label "Confirm") (input :type "password" :value pass2 :change #(reset! pass2 @%))) (button :type "submit" "submit") (a :href "javascript:void(0)" :click toggle-register! (text "~{reg-link-text}")))

laforge4917:12:30

So I need to study ring then, hm?

laforge4917:12:40

--I am such a newbie!

micha17:12:54

luckily ring is very simple!

micha17:12:04

especially if you have experience with the java end

micha17:12:20

most people have a hard time with the servlets and jetty and whatnot

micha17:12:28

but you're probably all good there

laforge4917:12:38

very possibly

micha17:12:40

ring itself is utterly simple

micha17:12:09

basic idea is they made a servlet that gets a request

micha17:12:18

and transforms the request into a clojure map

micha17:12:27

then calls a clojure function with that map as argument

micha17:12:36

that function is expected to return a response map

micha17:12:46

which is transformed into an http response in the servlet

micha17:12:12

since the servlet is delegating to a clojure function to handle the request you can do lots of interesting things

micha17:12:18

like wrap the function in another function

micha17:12:23

middleware

laforge4917:12:26

yeah but I thought the promise was handled in the .nl code somewhere. Or does that also use ring?

micha17:12:35

the promise is a clientside thing

micha17:12:54

the promise is a js construct that represents a future async response

laforge4917:12:57

we were talking about the client when you mentioned ring

micha17:12:05

ring is only on the server

micha17:12:16

you pasted the ring source is why

laforge4917:12:22

so back to the submit code?

laforge4917:12:35

I pasted the form in the .hl file

micha17:12:00

the (def handler (-> ... is in a .hl file?

laforge4917:12:23

no that was eariler, or perhaps I'm just very confused simple_smile

micha17:12:03

the second image might be illuminating

laforge4917:12:07

you said the fail was caught in thehandler, and the only handler I know is server side, so I got confused.

micha17:12:34

i meant the onfail callback

laforge4917:12:53

so there is no client side handler code in the chat demo

micha17:12:56

there probalby isn't one in the demo

laforge4917:12:03

must be in a library somewhere?

micha17:12:05

it was simple enough to handle errors globally

micha17:12:11

that application

micha17:12:21

i was pressed for time simple_smile

laforge4917:12:45

I should just shut up and let you write the doc! 😄

micha17:12:56

i'll have something in an hour or two

laforge4917:12:04

bye for now then!

laforge4917:12:23

--always so much fun!

micha22:12:01

temporarily disabling slack notifications