Fork me on GitHub
#untangled
<
2017-04-14
>
robert-stuttaford14:04:57

hey @tony.kay šŸ™‚ say i wanted to get started with Untangled, using everything according to the latest understanding. is there perhaps a lein template that brings it all together? particularly interested in the UI stuff

gardnervickers14:04:59

@robert-stuttaford I'm not sure if there is a lein template to generate a project but there is this template project which is pretty great to clone and modify for your use case https://github.com/untangled-web/untangled-template/blob/develop/README.md

robert-stuttaford14:04:03

i see this one doesnā€™t have the support-viewer or the new ui stuff, iā€™m guessing it predates those

gardnervickers15:04:41

Ahh yup, admittedly I haven't used the UI components much yet or the support viewer. The form support is very interesting though.

gardnervickers15:04:52

@robert-stuttaford if you're just getting started with Om.next and Untangled I highly recommend the Untangled video series. It was pretty crucial to my understanding of Om.next and the boundaries Untangled hooks into.

robert-stuttaford15:04:05

yeah, iā€™m looking into those now

robert-stuttaford15:04:33

i have mucked around with omn before, but found the parser stuff to be quite heavy. keen to see Untā€™s take

robert-stuttaford15:04:14

my need is that we need to build a lot of db admin stuff pretty quickly and iā€™m really not interested in reinventing anything. done enough of that šŸ™‚

robert-stuttaford15:04:33

just looking for something with batteries to get stuff built

gardnervickers15:04:03

Yup building the parser emitters in pure Om.Next on the client side was rather cumbersome, I quite like Untangled's approach. Some things that bit us early on were not having a great idea of what our entities were in the system (data backed by an ident), and having too many different ident backed data structures. We've since taken a different approach where we use normalization/idents only where it's necessary for performance or deduplication. Keeping the count of different ident types in your application low helps reasoning tremendously.

robert-stuttaford15:04:40

happily we understand our data tremendously well; itā€™s 4 years old šŸ™‚

robert-stuttaford15:04:00

weā€™ve lots of fixed-scope tools to build

robert-stuttaford15:04:08

which each focus on a small part of the whole

robert-stuttaford15:04:56

our tooling has lagged behind our primary feature dev and operation quite a lot. so we know our data, and what we want to do to manage it, we just need a sensible, scalable toolkit to get it done

robert-stuttaford15:04:53

literally things as simple as capturing SSO integration configs, support tools like reviewing a userā€™s profile, emails weā€™ve sent them, stuff like that

gardnervickers15:04:10

Ah yup, that sounds like the sweet spot for Om.Next/Untangled. Developing both your UI and data-model at the same time is more difficult than other frameworks because you need to be fairly precise with queries describing your component's data dependencies. Large changes take longer because you need to not only change the UI logic but also the query layout among several components.

gardnervickers15:04:49

We've found that once the data model solidifies into something that makes sense, change becomes much easier and more reliable.

tony.kay15:04:09

Iā€™m interested @gardnervickers: do you choose not to use idents on things that need to update in the UI? Or are you saying that with nested data that comes from the server for pure display you donā€™t bother?

tony.kay15:04:07

@robert-stuttaford The untangled-ui project needs a little polish, but it is usable as-is. Just add it as a dependency and use the stuff outlined in the devcards. The forms stuff doesnā€™t have any server adapters yet, but the ā€œdiff protocolā€ is sends is really easy to reason about.

tony.kay15:04:00

Iā€™d say the very biggest problem Iā€™ve seen with new users is not leveraging union queries to keep the query low-overhead. The new defrouter macro facilitates that. If you just make one big query without that your UI performance will suck

tony.kay15:04:16

and the components guide

tony.kay15:04:32

the former is pure css, the latter is more wrappers that emit dom +css

robert-stuttaford15:04:54

cool. iā€™m going to work through all your videos, and read through all the devcards, and then try building something. i will have questions šŸ™‚

robert-stuttaford15:04:21

i guess it doesnā€™t matter (because itā€™s easy for me to do differently), but why did you stick with the verbose om.dom rendering code?

tony.kay15:04:31

note that the Untangled in the Large videos mention making defrouterā€¦defrouter is now a formal part of Untangled, with a slightly improved API than shown there

tony.kay15:04:47

server-side rendering

tony.kay15:04:56

not adding more deps for ppl

tony.kay15:04:05

the current syntax doesnā€™t bother me šŸ™‚

tony.kay15:04:27

so, basically what you said: it is easy for you to do differently

robert-stuttaford15:04:29

ah šŸ™‚ it bothers me, heh. iā€™m all for concision in naturally verbose code. html is just one of those.

robert-stuttaford15:04:45

yep. happy to deal with it my own way!

tony.kay15:04:48

overhead as well. didnā€™t want to add possible performance problems

robert-stuttaford15:04:19

really nice work btw. i can see a huge effort here. whether we end up using it or not, kudos!

gardnervickers15:04:43

@tony.kay For example, if our UI tree looks like

{:users {:user/id 1 :user/phone {:phone/type :work :phone/number "4014770248"}}}
We would have a table of :user/by-id but we wouldnā€™t, for example have a table of :phone.number/by-number.

gardnervickers15:04:19

That might be a bad example, but essentially we were finding that we were using idents/normalization where it didnā€™t benefit us all that much.

gardnervickers15:04:43

Not every level of map nesting needs to be normalized

robert-stuttaford15:04:49

kinda like only using :avet in datomic when you genuinely need to

tony.kay15:04:51

I see. The normalization really helps with things like ā€œforms supportā€

tony.kay15:04:08

e.g. I could not easily write something like that without normalization

tony.kay15:04:21

it would be a lot more complex (youā€™d have to be able to declare ā€œtype ofā€ nesting, I think)

gardnervickers15:04:30

So that only works on flat maps with no nesting?

tony.kay15:04:11

the forms support lets you wrap normalized entities, because I can easily write the mutate/read code ā€œfor youā€. If you arbitrarily denormalize, it would be a lot more complicated

tony.kay15:04:21

In general, if it is editable, Iā€™d recommend normalizing it.

tony.kay15:04:10

and for new users, Iā€™d recommend making something with defrouter firstā€¦learn how to do that immediately. Iā€™ve seen too many people write an app that queries the ā€œwhole UIā€ for every frame, and that leads to unusable UIs

tony.kay15:04:43

I need to make a new stater tutorial and video series that stresses that. I think it is the most common place where early failure will happen with new users

robert-stuttaford15:04:52

is defrouter covered in any of the docs or videos?

tony.kay15:04:20

the Untangled in the Large shows what it does (I made that while developing it). The API is a bit improved, adn the devguide covers it

tony.kay15:04:13

that last one is the in-the-large video on ui routing

gardnervickers15:04:35

@tony.kay Iā€™ll try and get a devcard together this weekend showcasing how we using an approach similar to initial-state to compose url->app-state and app-state->url serialization/deserialization for routing. We even hook into :tempid migration to automatically place the correct resource URL when the server returns the generated ID.

tony.kay15:04:25

Basically, a Union query is one in which only a single part of the UI (and query) is visible at onceā€¦like a switching mechanism (it can also be used for hetero lists). Only one branch is evalā€™d at once, so it is a way to ensure your UI only queries for the part on-screen.

tony.kay15:04:54

if you donā€™t use them, you end up querying everything that could possibly be on screen every frameā€¦which can be horribly slow.

tony.kay15:04:51

BTW, Iā€™m combining all the recipes into a devcards project. I could give you access to that if youā€™d like to put it there. It is private while Iā€™m working on it

tony.kay15:04:04

OH, devcards wonā€™t work for that. You cannot change the URI or devcards pukes

gardnervickers15:04:30

Oh interesting. Let me get a gist up first, not sure if Iā€™ll have time to refine it much since Iā€™m with the family for Easter this weekend. Iā€™ll keep you in the loop.

tony.kay15:04:55

@robert-stuttaford So, if you can get a handle on the app database graph format (via the devguide) and queries, then most of the rest falls into place. Perhaps watching the https://www.youtube.com/watch?v=mT4jJHf929Q&amp;t=1s&amp;list=PLVi9lDx-4C_T_gsmBQ_2gztvk6h_Usw6R&amp;index=8 video about the data lifecycle will tie things together as well.

robert-stuttaford15:04:15

iā€™m sure that it will, thank you! i have that video queued

tony.kay15:04:55

There really are not very many core concepts. I think it gets ultra-simple once youā€™ve got those bits down.

robert-stuttaford15:04:00

i have lots of om.previous and rum experience, so i understand dealing with the normalised database vs ui tree graph mismatch quite well

robert-stuttaford15:04:33

just need to become familiar with the idioms and overall organisation, and build that first something

tony.kay15:04:37

yeah, Iā€™m really stoked with the result. Every time I go to do something Iā€™m amazed at how easily it falls out, and how little (incidental) complexity there is.

tony.kay15:04:27

@gardnervickers the other bit about normalization: you canā€™t easily relocate a UI component that isnā€™t normalized, because the mutations become non-portable.

robert-stuttaford15:04:36

what iā€™m hoping for is that we get one or two tools built, establishing some patterns, and then we can tear through the backlog from there. like i said earlier, we know our data well (and itā€™s all in Datomic šŸ˜Ž)

tony.kay15:04:21

Oh, also make sure you know about defmutation. Much nicer IDE support for the mutations. That is another one I show in-the-large that is now part of the lib.

gardnervickers15:04:33

@tony.kay Just to clarify, we do use normalization quite a bit. We just donā€™t normalize things that are not shared but still nested.

tony.kay15:04:11

got it. Yeah, I just wanted to point out the important aspects of normalizationā€¦more for whoever might be lurking and not speaking šŸ˜‰

robert-stuttaford16:04:20

it feels like overkill for the toy app (root + list + item), but i can see the value for non-trivial apps

donmullen16:04:55

@tony.kay The first two video links after the routing guide link where not public links - will you share those again?

tony.kay16:04:32

@robert-stuttaford it does feel like overkill for small thingsā€¦but nothing stays small šŸ™‚

robert-stuttaford16:04:40

nothing useful šŸ™‚

tony.kay16:04:46

help me outā€¦which ones in particular?

tony.kay16:04:02

oh, I seeā€¦

tony.kay16:04:45

thatā€™s the whole playlist

donmullen16:04:03

Great - thanks.

tony.kay16:04:04

thatā€™s the routing one

tony.kay16:04:13

those work?

tony.kay16:04:05

and for anyone reading the stuff aboveā€¦the defrouter thing is important, but you arenā€™t going to understand it until you get queries/ident/db format. Those foundational pieces are necessary to build anything. But then make sure you get defrouter.

pedroteixeira17:04:10

hello! watched a few talks about untangled today, thanks a lot for sharing the knowledge and the code! a question here.. any one knows of an example using a library such as martinklepsch/derivatives (used by rum) with untangled? I'm considering using untangled for a next project, but trying to see a solution for managing derived values in more declarative ways (i.e. instead of having to call functions on various mutations) - looking into re-frame`s reaction abstraction as well

pedroteixeira17:04:36

i think for Om Next it would be just an impl. detail of a mutation.. but was curious if there is already such a pattern in the untangled ecosystem or someone working on a reference integration šŸ™‚

gardnervickers17:04:09

@pedroteixeira In Om.Next, the read parsers would be what generate your ā€œderivativesā€. Untangled provides a default read parser that is backed by the app db, so you need to generate your ā€œderivativesā€ using a mutation.

pedroteixeira17:04:01

hm, ok! starting to read some of the code, thanks for the pointer. I was curious if there were examples of expressing derived values across the app state. For ex: instead of having to call update functions in various mutations, hook up something more generic in Om.Next to listen to changes in data and apply futher changes according to rules but in the same "transact".

tony.kay18:04:19

Right, derivatives are just represented in the graph as data. Everything goes in mutations. Of course, if it is a simple derivation, you could calculate it as part of the view.

tony.kay18:04:15

The basic idea is that you have to make it somewhereā€¦if you do it in a parser, then you add the complexity of figuring out when (and caching for performance), and mix that into a parser abstraction with graph navigation. In Untangled, the idea is that since youā€™re mostly going to want to cache the result of your computation, why not just make that the ā€œwayā€, and elide the need for more complex mechanisms. Lifecycles of views are pretty easy to manage, in that you have mount/unmount from React, and UI navigation events (e.g. clicking on a tab to go to X).

tony.kay18:04:20

Generate the ā€œinitial viewā€ where it makes sense (e.g. on load or view routing), update it where it makes sense. It is your data, and limiting the ways in which you have to think about it keeps things naturally simple for the most part

urbank20:04:57

It's probably just me, and tony.kay has helped elucidate quite a bit already, but I find untangled a bit daunting to use for an application where everything in the database is editable all the time and editable from multiple places in the app at the same time. It kind of breaks normalization as soon as one edit component has state in the app-db.

urbank20:04:34

On the other hand, the app is also liable to need caching at some point, so perhaps I might as well bite the bullet now.

urbank20:04:03

I'm thinking that perhaps I can compute everything on the fly if I use virtualization like https://github.com/bvaughn/react-virtualized

tony.kay21:04:24

@urbank I really donā€™t know what you mean ā€œIt kind of breaks normalizationā€ā€¦you mean you might have to have more than one graph edge that points to the same thing? That is still normalizedā€¦but it does mean you have to manage more than one link, but the fact is that you can still locally reason about those graph edges from that componentā€™s viewpoint. It is true that a list of idents might need an update in multiple places, but again that is a composeable task: compose together the list refreshed into the data lifecycle mutation of the mutations that post-mutate of the load

tony.kay21:04:50

OR use a root (link) query and just keep shared lists in the root node

tony.kay21:04:18

e.g. [ [:my-list '_] ] in a query anywhere in the app pulls in the edge from the root node

tony.kay21:04:12

so, if it is true that you need to use the same ā€œcollectionā€ or ā€œedgeā€ from multiple places you can just do thatā€¦.or even join on some specific entityā€™s edge [{[:table id] (om/get-query Table)}]

tony.kay21:04:51

to get the entity, then pull out the edge from those propsā€¦all sorts of possibilities

urbank21:04:29

@tony.kay Yeah, sorry - loose choice of words. Mostly talking about components with opaque state in the app-db (like calendar) that edits some other part of the app-db via a callback, but internally updates its "local state". If the entity it targets is also edited by another component, similar to the calendar, the calendar has to be updated whenever that other component updates the entity, else it displays stale data.

urbank21:04:31

Sill, your post gives me ideas. I'll continue playing around with it