Fork me on GitHub
#untangled
<
2016-09-21
>
mitchelkuijpers09:09:54

Just a general question, I am kind of running into problems with the way om.next handles getting remote data. I just read some code of untangled and that makes a lot more sense to me. Is it possible to use that stuff without completely using untangled?

mitchelkuijpers09:09:21

I mean the way it sends a mutation to load a remote field

ethangracer13:09:27

@mitchelkuijpers not sure that anybody has tried. just looking through the code it doesn’t appear that it has any strict expectations that an untangled application is being used. you would have to set up your networking to support the data fetch api, which given the amount of time we spent getting it right, I probably wouldn’t recommend just from an effort/benefit tradeoff perspective. I’d definitely be curious to hear what others in the channel think

tony.kay15:09:16

@mitchelkuijpers So, Untangled exists because Om next has some complexity to it that we believe has a reasonable general-purpose solution. In particular, the need to write a parser is a trade-off that Untangled believes is too costly from a general engineering perspective. At the end of the day, what you have to do (in your parser or in your database) is transform the raw database objects into a shape that will work for your tree. In Untangled, we prefer those transforms happen on user-driven events in a direct data manipulation manner (easy to reason about, lower cost on rendering overhead, generally easier). Loads in Om Next are always some sequence of: mutate something in app state to track that you are loading, return remote from the parser, get/merge the data (typically over top of the place where you marked that you needed to load), etc. It's the same thing over and over, but you end up peppering that logic about in a parser. Untangled wraps the need to do the above sequence in load-data and load-field. They do what you're going to have to do in Om Next...they just make it less work. They also deal with the potential problems of out-of-order completion, parallel queries for the same object (asking for different attributes), deep merge, and other issues that would normally show up in your program as bugs that would be mysterious to you. So, I don't have much interest in supporting the use case of "what can I pick out of Untangled?" The answer is: nothing useful. The main advantage of untangled is reducing the central complexity of Om Next: removing the need for a parser and lots of networking/load logic.

tony.kay16:09:13

There are trade-offs, and there are missing features that still need added, but we're having great success with it. The problems we've run into (and others using it commercially) have been easy to work with, or resulted in extensions to the framework.

mitchelkuijpers16:09:14

Aha that makes sense, I'll try it out and maybe switch over then. Thank you for your detailed answer much appreciated! @tony.kay

tony.kay16:09:23

Check out the getting started videos, for sure

tony.kay16:09:40

and the tutorial, and docs, etc.

tony.kay16:09:54

I also gave a talk at an Unsession in Seattle that is up on YouTube that details some more rationale

tony.kay16:09:20

I've been doing what I can to make it approachable. Lots of demands on my time, but I do understand that there is a hurdle to Om Next and Untangled. Trying to get people over the hump. This stuff really is so nice to work with. There are "easier" front-end things to get started with, but as soon as you try to do something "real" the easy goes away.

tony.kay16:09:53

Untangled/Om Next keep the overall picture much simpler

therabidbanana16:09:04

We just had our company hackday - I used untangled and I surprised myself by how amazingly fast and easy it was to work with now that I (mostly) get it.

tony.kay17:09:59

Yeah I was coding something up last night from scratch and was just flying and giggling maniacally about my super-powers 😉

tony.kay17:09:47

:man_in_business_suit_levitating:

mitchelkuijpers17:09:10

I will try it @tony.kay I have had some big problems with the way remote reads work with vanilla om.next. But it mostly solves a lot of other problems. We are moving away from re-frame to om.next and maybe untangled 🙂

tony.kay17:09:21

@mitchelkuijpers At the very least: with Untangled you write Om Next code: you just avoid the overhead of writing a parser and a bunch of plumbing. Worst case is you abandon Untangled and write a parser...so you really have nothing to lose

tony.kay17:09:37

it really is additive

mitchelkuijpers17:09:58

Damn it.. Now I have to migrate some code

mitchelkuijpers17:09:08

I guess you are right

tony.kay17:09:42

And if you abandon it, well, then you "get to play" with process-roots to fix your UI queries for your server, and implement your own remote loading scheme, etc. 😉

mitchelkuijpers17:09:42

Is this still the case? IMPORTANT: NOT READY FOR USE. Clone TodoMVC from github as a template instead or can I try the template?

tony.kay17:09:55

you can try the template: FYI, I am working on the template TODAY

mitchelkuijpers17:09:56

Yeah that is exactly what I am fighting with

mitchelkuijpers18:09:16

especially the cb that sometimes needs a query and sometimes does not need a query

tony.kay18:09:18

just testing it now

mitchelkuijpers18:09:25

Awsum I will test it with you then

tony.kay18:09:36

I can send you a tarball of it in a few if you'd like

tony.kay18:09:52

it will take more time to integrate my work with the lein template system

mitchelkuijpers18:09:54

Cool I would like that

tony.kay18:09:06

making it deployable to Heroku as well

mitchelkuijpers18:09:08

I just want to test some stuff out

mitchelkuijpers18:09:36

Aha won’t use that but that’s pretty cool

mitchelkuijpers18:09:16

I am going to eat dinner and then I will play around with it, thank you so much for your time @tony.kay

tony.kay18:09:15

still in the works

tony.kay18:09:41

having trouble with server code refresh, but it mostly works. Feel free to check it out, and you can just pull as I work on it

mitchelkuijpers18:09:37

@tony.kay do I need to make a index-dev.html in the resources folder?

mitchelkuijpers18:09:13

I have it working nvm

mitchelkuijpers19:09:54

Very impressed so far.. This could have saved me a lot of time

tony.kay19:09:02

yeah, it is nice to work with. Turns out my problem was with intellij. pushed a few minor patches to project file, but otherwise it should be pretty good

mitchelkuijpers19:09:16

The InitialAppState idea is pretty damn nice

tony.kay19:09:25

yeah...makes things SO fast to prototype

tony.kay19:09:43

I threw together that whole UI so far in about an hour with server interactions

tony.kay19:09:48

another bit of time for css

mitchelkuijpers19:09:27

It seems like you pretty much only write mutates and run txes to get remote data am I correct in assuming that?

mitchelkuijpers19:09:57

Cool website btw

tony.kay19:09:14

ooops...seems like I lost the HTML files

tony.kay19:09:25

do you have them?

mitchelkuijpers19:09:39

I made one myself

tony.kay19:09:47

that was an accident

tony.kay19:09:57

I wiped em unintentionally 😞

mitchelkuijpers19:09:51

No worries, only cost me 5 minutes. untangled will probably save me hours

tony.kay19:09:20

html pushed

mitchelkuijpers19:09:24

What is the main pattern to load data? just put it in the componentWillMount?

tony.kay19:09:34

there are 4 files. One for specs, one for devcards, one for production, and one for dev

mitchelkuijpers19:09:55

Aha cool i will pull them in

tony.kay19:09:59

load data happens mainly in two places: 1. There is a start-callback in core.cljs

tony.kay19:09:08

2. Due to user events

tony.kay19:09:57

it is possible to put them in componentWillMount, but I have not needed to. Another option is you can write a mutation that uses the helper functions load-data-action to emit a remote request from any arbitrary mutation.

tony.kay19:09:07

See the cookbook for examples

tony.kay19:09:56

So, imagine your app just loaded: use started-callback to get initial server state. Then you nav to some page you want to lazy load data for. Make a nav mutation that checks app state for the data, and if it is missing issues a load-data-action call from the :action of the mutation.

tony.kay19:09:23

that gets you your "load on demand"

mitchelkuijpers19:09:37

Yeah we currently use secretary, so that would fit very nice

tony.kay19:09:40

which always want to look into app state to see if it is already there

tony.kay19:09:37

yeah, with secretary, you just pass the app to transact! to trigger your nav mutation

tony.kay19:09:56

If you have not checked out the specs and devcards, be sure to try them

tony.kay19:09:04

cards.html and test.html

mitchelkuijpers19:09:13

I have to plug this into a existing re-frame codebase so I will probably do it like that

tony.kay19:09:45

you have to be running all three figwheel builds (the -D options in the instructions)

mitchelkuijpers19:09:23

Oh I have just started it on the repl

mitchelkuijpers19:09:22

I actually use ohai-emacs from Bodill

tony.kay19:09:36

cool. Glad to hear it works for u

adambros19:09:18

you should be able to just call (start-figwheel) and it will start up everything or you can call it with a vector of build id strings eg: (start-figwheel [“dev”])

mitchelkuijpers19:09:45

I will probably put untangled in our boot.build. Would you ever be interested in a boot option for the template?

mitchelkuijpers19:09:00

Yeah works like a charm @adambros

adambros19:09:22

i wouldnt mind, just shoot a PR on untangled-template whenever you got it

adambros19:09:46

ill be updating it once tony figures out what he wants with his version

mitchelkuijpers19:09:02

Cool will put that on my todo list

tony.kay19:09:09

yeah, boot is on the TODO list, but none of us use it (yet), and we don't have the horsepower to maintain it. So at present any Boot pointers will be included sort of as documentation YMMV

mitchelkuijpers19:09:30

Sure I wouldn’t mind helping out

mitchelkuijpers19:09:40

I’ll have todo it anyways

mitchelkuijpers19:09:10

Am I correct in assuming that I could just use my own endpoint on which I have an om/parser running which returns transit?

tony.kay19:09:11

um...sort of.

tony.kay19:09:24

let me think...there was some caveat

tony.kay19:09:48

Oh. The error handling mechanism establishes some standards...which are undocumented

tony.kay19:09:01

but for the most part, yes

tony.kay19:09:17

You'd probably want to dupe the unhappy path stuff, but it is a small amount of stuff

tony.kay19:09:28

and it needs a bit more baking anyhow

mitchelkuijpers19:09:56

No problem.. we are currently using catacumba but figuring out how to integrate will not be hard I think

mitchelkuijpers19:09:04

I guess I have a nice weekend project

tony.kay19:09:29

no, the server stuff is all convenience except for some of the error handling formats that Untangled expects at the network layer. Easy enough to emulate

mitchelkuijpers19:09:13

Oh I see the code

mitchelkuijpers19:09:22

That stuff is trivial to reuse

mitchelkuijpers19:09:41

Cool I don’t see any roadblocks, thank you so much for convincing me

tony.kay19:09:50

yeah, for that matter, just hook our api handler into your stack

mitchelkuijpers19:09:38

Yeah I already found that, I can simply install ring middleware in catacumba

tony.kay19:09:45

welcome. I think a lot of ppl still see Untangled as a competing framework to Om Next...we're just trying to make it easier

mitchelkuijpers19:09:25

Yeah, I also saw it that way

tony.kay19:09:45

Not sure how to fix that messaging. Maybe make that more prominent on the home page

mitchelkuijpers19:09:43

I’ll tell it to everyone 😉

tony.kay19:09:51

@ethangracer Did you just patch post-sweep?

tony.kay19:09:03

@mitchelkuijpers thanks! Word of mouth doesn't hurt 🙂

tony.kay19:09:19

I see the Jira, but no PR

ethangracer19:09:42

@tony.kay are you talking about the story in jira? I’m pretty positive we fixed it ages ago, I haven’t seen mutation symbols in app state for quite some time

tony.kay19:09:59

oh...ok, I honestly hadn't checked 😕

tony.kay19:09:12

I just saw the story close and didn't see a UC change

ethangracer19:09:26

yeah I marked it as already fixed

ethangracer19:09:41

not sure when but I think it happened during a data fetch refactor

tony.kay19:09:53

just FYI, I fixed the related network bug where the app stops trying to do network ops when it gets a malformed response

tony.kay19:09:32

just added an exception try/catch...whcih should have been there anyway.

ethangracer19:09:50

didn’t even realized that was a thing

tony.kay19:09:18

you remember seeing (in console) malformed response '<' or something like that?

tony.kay19:09:37

yeah...if you got that, you were dead. Reload required

ethangracer19:09:47

awesome. yeah that was a pain

tony.kay19:09:29

on 0.5.6 SNAPSHOT on clojars

ethangracer19:09:01

also meant to ask, given the om is going to beta soon, where are we with untangled going to “beta”?

ethangracer19:09:48

it would be nice to have more docs but the code itself feels pretty solid

tony.kay19:09:57

Largely a change to public facing messaging and docs, yeah

tony.kay19:09:12

Feel free to bang out more docs 🙂

tony.kay19:09:26

I keep finding more stuff to do 😕

tony.kay19:09:40

want to get the ref guide and tutorial beefed up for sure

tony.kay19:09:00

We have a lot of cool stuff that isn't even documented

tony.kay19:09:05

protocol support

tony.kay19:09:16

extra routes

ethangracer19:09:25

do we have protocol tests for todomvc?

tony.kay19:09:30

don't think so

ethangracer19:09:54

ok, I can write those up pretty quickly in the near future

ethangracer19:09:15

and definitely want to add more to the ref guide as well

adambros19:09:20

config env vars

ethangracer19:09:10

untangled datomic migrations

tony.kay19:09:35

yeah...all of those

tony.kay19:09:29

initial app state - that one was HUGE on impact to devs

tony.kay19:09:52

I've documented that in cookbook at least

mitchelkuijpers20:09:02

I have a problem where my server gives me 500 with a load-field and then the client keeps trying eternally

mitchelkuijpers20:09:15

i try to run this: (f/load-field this :users :fallback 'users/server-down)

tony.kay20:09:33

what does your server-down do?

mitchelkuijpers20:09:33

and this is my query

(query [this]
    [:id
     [:current-user '_]
     {:users (om/get-query User)}])

mitchelkuijpers20:09:49

I added that because I thought that might solve the issue

tony.kay20:09:54

and what triggers your load-field?

tony.kay20:09:03

looping is caused by your code

mitchelkuijpers20:09:03

componentDidMount

mitchelkuijpers20:09:09

lol yeah that is my mistake

tony.kay20:09:10

oh yeah, like I said, don't do that 😉

mitchelkuijpers20:09:21

yeah you are right

tony.kay20:09:50

you have to have an ident for load-field to work, and you have to implement the server bit correctly. 500 is all you too 🙂

mitchelkuijpers20:09:19

No that is because the query is invalid

mitchelkuijpers20:09:23

I implemented the server bit

tony.kay20:09:34

The generated server query will be a join on the ident { [:user 1] [...] }

tony.kay20:09:49

which is why it needs an ident

tony.kay20:09:08

load-field is really just a helper for load-data that derives the query for you

mitchelkuijpers20:09:23

So I should not need it?

tony.kay20:09:36

you don't "need" it...but it does make things easier

tony.kay20:09:49

read the code...it is only a couple of lines

tony.kay20:09:05

pull the query, focus it on the field, join it with the ident

tony.kay20:09:10

then do what load-data does

mitchelkuijpers20:09:51

Ah ok I’ll read it

mitchelkuijpers20:09:30

Ah now I get it.. It is for loading extra data of certain entities with a id, so for example to get the posts of a user

tony.kay20:09:47

lazy load stuff you didn't know you needed yet, related to something you already have

tony.kay20:09:16

also puts a marker in place so you can easily show a spinner in that spot

tony.kay20:09:43

(marker in app database where the data will appear)

tony.kay20:09:51

examples in cookbook

tony.kay20:09:05

gotcha there is to remember to query for UI fetch-state

grzm21:09:00

I'm trying to reset my app state on user log out and getting an Invariant Violation: malformed Ident. error. I'm using the same state that I'm initializing the app with, so I'm kinda confused at what the difference would be resetting the state as opposed to initializing it. Would the difference have to do with me initializing with a data structure (which is denormalized) as opposed to an atom?

grzm21:09:08

You guys are great rubber duckies. I'm thinking that I need to swap it with the normalized state, not the denormalized one I initialized it with. And I can probably use tree->db to normalize my initial state.

grzm21:09:41

Well, that was a good idea, but still got the same error 😕

tony.kay21:09:26

@grzm interesting. Are you forcing a re-render from root?

tony.kay21:09:37

could be Om is trying to render a sub-tree and there is no state for it

tony.kay21:09:56

I'd unmount it and re-mount it

grzm21:09:26

That could be. I'll give that a shot. Where would I put the code to unmount and remount the root?

tony.kay21:09:47

make some top-level function. Make sure you're keeping the app in an atom

tony.kay21:09:58

then just run umount/mount on the app

tony.kay21:09:07

I've never tried it...you're in untested territory 🙂

grzm21:09:07

Cool. That makes sense.

grzm21:09:28

Yay! I was pretty happy that I figured out how to reset history.

tony.kay21:09:31

you will need to make the normalized db, and there you're right: db->tree

tony.kay21:09:22

ah. those might be nice to have in the lib, now that you mention it

tony.kay21:09:44

in fact, it totally makes sense what you're trying to do on logout

tony.kay21:09:59

clean it up nicely and perhaps we can integrate it as part of the app interface

grzm21:09:10

I think it should even be added to om proper, actually. It's independent of untangled.

tony.kay21:09:22

good luck there 🙂

grzm21:09:28

But, yeah, will do 🙂

tony.kay21:09:55

default db format vs custom format. Not general enough for Om, and initial app state isn't available in a general sense liek it can be with Untangled

grzm21:09:12

Oh, I meant just the reset history function.

tony.kay21:09:18

oh, yeah, that one I agree

grzm22:09:02

@tony.kay Is this what you had in mind for remounting?

(defn remount-app! []
  (let [reconciler (:reconciler @app)]
    (om.next/remove-root! reconciler (gdom/getElement app-mount-point))
    (om.next/add-root! reconciler ui/Root (gdom/getElement app-mount-point))))

tony.kay22:09:26

oh, no, sorry

tony.kay22:09:38

there is a mount/unmount in Untangled

tony.kay22:09:44

part of the application interface

tony.kay22:09:25

actually, looking back at the interface:

grzm22:09:27

I see mount. I don't see unmount

tony.kay22:09:30

we have reset-state

tony.kay22:09:37

you're right on the unmount...oversight I think

grzm22:09:49

I tried that, too. That gave me the same invariant error.

tony.kay22:09:42

If you do your remount (with a reset-state in the middle), that is what I was talking about

grzm22:09:46

I'll pound on it some more

grzm22:09:04

gotcha. (I think)

tony.kay22:09:57

yeah, looking over the code to refresh my memory. What you're suggesting (with a reset-state using db->tree) will likely work fine.

tony.kay22:09:05

Technically, it would work "better" to do this through a mutation, and just include a root query key as a follow-on read

tony.kay22:09:53

you can do a swap! on state in a mutation, query the Root query and initial app state to make the normalized "new" state

tony.kay22:09:07

As I'm thinking about it, that would be the better approach.

grzm22:09:27

That makes sense.

grzm22:09:40

And feels cleaner

grzm22:09:10

Is there a general root query key or should I just add a key for something the root requires?

grzm22:09:33

For the follow-on read