Fork me on GitHub
#fulcro
<
2018-02-28
>
tony.kay00:02:07

an assert would probably be possible…

wilkerlucio00:02:57

we have to check if the component implements IQuery as well before checking the arg, because for non-stateful components this still a valid thing to do

tony.kay00:02:49

Fulcro spec 2.0.3 is on clojars. Found a bug with server-side test runner that was causing it to crash internally. Fixed.

tony.kay00:02:23

Fulcro SQL 0.3.2 is also up. There was a keyword inconsistency when selecting alternate db drivers.

tony.kay02:02:59

argh…2.0.3-1 for spec…I forgot to include the compiled server test renderer XD

tony.kay03:02:04

New Workflow video on YouTube https://youtu.be/xaM7sqXk32U. Starts where part 1 left off, and adds server code to deal with a query and mutation. Demonstrates a little of fulcro-sql, fulcro-spec for the server, and how devcards and live development of the app can be done at the same time.

tony.kay05:02:50

One more: Processing queries on the server. Shows techniques of dealing with graph queries on the server without having to write manual parsers. Uses SQL as the database, and covers a few different techniques, including the Connect system of pathom, which can easily be adapted to any data store: https://youtu.be/gbrdnSsUerI Companion files at https://github.com/awkay/fulcro-pathom-sql-demo

myguidingstar09:02:09

@tony.kay the problem with solving SQL N+1 problem using SELECT ... WHERE id IN(1, 11, 111) is that we can't limit number of rows per id which can become a disaster if the table is big. Even if we try to use LIMIT in that query, chances are some ids will take up the limit while others are left out. For the case where some LiMIT is specified, the SQL query will be UNION of the generated queries:

(SELECT ... WHERE id = 1)
UNION
(SELECT ... WHERE id = 11)
UNION
(SELECT ... WHERE id = 111)

myguidingstar09:02:04

walkable needs a big refactor then 😄

wilkerlucio15:02:49

@tony.kay Thanks so much for making a video about pathom! I would like to comment a few things that might help the future viewers: - you can return nil from the resolvers when it's unable to fulfil the query, must work the same as blank map - 20:13 - pathom connect has a helper function for using the resolve symbol as dispatch, so the multimethod could be defined as: (defmulti entity-resolver pc/resolver-dispatch) - 24:28 - when dealing with to-one relations (account->settings), you can just return the :settings/id on the same level, this will enable you flatten out every to-one relation in a single map, this is more flexible than forcing user to descend a level, and if you need to descend a level (to accommodate UI relations) you can use placeholder nodes, they are there just to solve this. - 31:27 - for caching, connect already has a built-in caching mechanism, you just have to add the plugin and it works out of the box (the basic pathom cache plugin), it is based on the input for the resolver, so the params you used might be a problem to it, but I think with a little structural change you can remove the param dependency so the cache can flow, usually params are better to use on things like sorting, query search and other things, but if it's data it's better to propagate via entity information (clarified later: tony was talking about a different type of cache that requires a more elaborated implementation) Great video!

tony.kay17:02:50

@wilkerlucio Thanks. I've added your comments to the description. When I mention caching, I'm talking about the N + 1 problem, not the re-query problem. For example, if someone is running the "invoices that have an item" query, I know at the very first level that I'm going to need item details, and could even pre-fetch them in that first level (I might even choose to over-query in anticipation of what is wanted, because I may know the most common usage pattern). In other words, I might want to design a mechanism that lets me gather up data that I suspect might be needed later in the query as a pre-caching move. That way it doesn't degrade into a single individual SQL query for each and every item's detail. In fulcro-sql, I actually accomplish this by gathering IDs at the "root set" so I can run subqueries with IN conditions. I'm sure similar patterns can be implemented in pathom, but it was too much to cover in that video.

wilkerlucio18:02:37

interesting, thanks for clarifying, you are right, that's why I think more specialized systems (like SQL) need to have more structure behind, so we can have special caches that know how to optimize for common patterns (like to-many calls). the env is open, so you just need to add a map there and you have the building block for a new cache (this is how the current cache works as well)

wilkerlucio18:02:26

but about the overfetch, we can also look deeper down on the query, since you have the AST we can know ahead of time which fields are going to be needed down the road, so the resolver can use that information and fetch only what its actually going to be needed

tony.kay18:02:58

That's true, but that mixes layers of abstraction (parsing the remainder of the query and caching) in a way that makes the software complex. Locality of reference will often make minor over-querying have a negligible effect, while continuing to keep source clean. It's a balance.

tony.kay18:02:18

A given entry point, like "invoices with an item" will have common usage patterns. Sometimes coding in a specific solution is better than general, because it is simple, easy to understand, and effective.

tony.kay18:02:07

The down-side, of course, is it isn't "general"...but "fully general" solutions can be a great way to spend a lot of money on development for very little improvement 🙂

wilkerlucio18:02:45

yeah, but we can help ourselves to make this easier, names are key here I think, like on the graphql connect case, we use a cheap detection method (keyword namespace prefix) to get what part of the subquery is relevant, I'm not trying to say we can do everything upfront, but I think a lot of common cases can be optimized on this way, so we can same some hybrid optimizations (combinding pre-built common and specialized ones coming from the user)

tony.kay18:02:16

Yes, those kind of things are relatively easy ideas as well

tony.kay18:02:15

Converting the graph query to an sql query directly is also another approach for that specific data store. I wrote such a thing, but the problem is the table output has a ton of duplicated content in it

tony.kay18:02:49

and I think the overall system impact would be worse

wilkerlucio18:02:53

also, you usually don't want to expose the whole database (like user tables), so some constraints must be added

wilkerlucio18:02:14

and we didn't even started talking about security 😛

tony.kay18:02:35

I've talked about that before in demos and docs

wilkerlucio18:02:48

I mean in this conversation, hehe

tony.kay22:02:25

I’m trying to generate more awareness of Fulcro. Anyone willing to up-vote this or post something somewhere of their own is appreciated. https://www.reddit.com/r/Clojure/comments/81062n/playlist_of_fulcro_videos/