Fork me on GitHub
#untangled
<
2016-08-16
>
jasonjckn18:08:32

is GC run after every merge?

tony.kay18:08:42

no such thing

jasonjckn18:08:04

i thought there was something that crawls links and deletes data that has no links to it

jasonjckn18:08:11

how do I cleanup normalized data?

tony.kay18:08:38

Nope, there is no way to not make that buggy.

jasonjckn18:08:06

if I merge data {:id 3 :foo {:id 4}} and it's normalized to {:x {3 ..} :y {4 ..} } and then I merge data {:id 3 :foo nil}

tony.kay18:08:11

e.g. I have a list of users. I run a filter to show those that start with 'A'. Now my database only has links to a subset...do I clean those just because a "GC" runs?

jasonjckn18:08:18

how do I cleanup :y automatically

tony.kay18:08:32

Untangled is not involved. It is your database.

jasonjckn18:08:00

ok, this makes me want to not normalize data 😞

jasonjckn18:08:05

so I don't have to deal with this pain

jasonjckn18:08:17

this is a trait of OM in general I understand

tony.kay18:08:25

There is no free lunch in programming. What is a trait?

tony.kay18:08:59

if you don't normalize data, then you have other more serious problems.

jasonjckn18:08:00

well there's some logic you have to write that is essentially checking if table is not referenced by any links, and seems like i have to write this over and over again for each situation that arises

jasonjckn18:08:10

like a search engine that loads search results

jasonjckn18:08:14

each search query loads new search results

jasonjckn18:08:17

so you have to cleanup after yourself

jasonjckn18:08:24

which search results are no longer referenced

tony.kay18:08:33

So just clean the entire table as part of the query logic

jasonjckn18:08:57

in my example, each time the search query changes, I empty the search results table?

tony.kay18:08:59

that is much simpler and more reliable than an automatic GC that you have to program

tony.kay18:08:53

I mean, I guess you could write some kind of GC of your own...it is a simple algorithm if you make certain assumptions (e.g. no refs means cleanup, which isn't good enough for all apps, but might be ok for yours).

tony.kay18:08:34

but unless you're worried about killing a javascript VM with data size, I would not even worry about it.

tony.kay18:08:56

most apps don't have huge data sets, and js can have a pretty darn large memory footprint.

tony.kay18:08:24

anyway...if you write GC, just transact on the reconciler once every 15 minutes or something.

jasonjckn18:08:41

"So just clean the entire table as part of the query logic" in my example this means in the search mutation, I empty the search-results table first before doing load-action ?

jasonjckn18:08:57

i don't have time to write GC, for some reason I thought you wrote one 😄

jasonjckn18:08:11

i'll make do with the other methods

tony.kay18:08:35

we have been writing pretty large apps, and have not found a need for general GC yet, nor is there a great way to solve it in general, as I said.

tony.kay18:08:54

You could do things like mark tables with metadata (no-gc), etc.

tony.kay18:08:08

but I don't tend to do work unless I need it 🙂

jasonjckn18:08:08

what about stale data interfering with newly merged data

jasonjckn18:08:28

anything to worry about there?

tony.kay18:08:47

If you queried for it, it will overwrite it. How can it be stale?

tony.kay18:08:11

Untangled does include logic to stomp on stuff that has disappeared (was queried for, but didn't appear)

tony.kay18:08:20

but that is at an attribute level, not entity

tony.kay18:08:46

(so a ref might disappear, but not the row in the table)

tony.kay18:08:59

As I said: no way to be sure (from a library) that a given row in your database is gone, unless the application can prove it. Just because you don't have a UI ref to a user object doesn't mean it is "gone" in your application (might still be persisted on server).

tony.kay18:08:24

Add in websockets and entity subscriptions with datomic: now we're talking

tony.kay18:08:41

but then you'd have to manage subscriptions 🙂

tony.kay18:08:53

so it really just pushes the problem off by one step

tony.kay18:08:20

leaking subscriptions instead of rows

jasonjckn18:08:32

i didn't normalize my search results and so far it's been fine

tony.kay18:08:39

that's ok too

jasonjckn18:08:50

i'm using normalization on a case by case basis

jasonjckn18:08:56

"will it help me solve this problem"

tony.kay18:08:57

if you don't need data in more than one place, it is perfectly acceptable to treat it as a blob

tony.kay18:08:34

well, mutating it gets more complex and less modular...but if it is read-only search result data, who cares

grzm19:08:42

I've got a UI that's not updating after a post-mutate after a data-fetch. Looking at the app-state, the data is there where I expect it to be. When I save a file and the react-key is updated, the data shows up in the right place. Any suggestions where I might look?

grzm19:08:21

Something's not triggering a refresh, right?

ethangracer20:08:08

@grzm try overriding shouldComponentUpdate to return true — see if that causes the re-render

ethangracer20:08:37

if it does, then the issue is with om’s implementation of shouldComponentUpdate. we’ve seen the behavior before, isolated one case of it, but it’s possible we haven’t found all of them

grzm20:08:07

@ethangracer: Thanks for the idea. I suspect it's more something in my particular application, and that this is just a symptom of something I've done wrong.

ethangracer20:08:27

@grzm fair enough, I’ve run into that issue plenty of times. you’ve already verified that the data is showing up in your app state, meaning the network request completed. if the request completed and the data was put in your app-state, then the data fetch called merge! with your data on the reconciler. Which schedules a re-render through om. So if the re-render isn’t happening, it’s either because you haven’t included a necessary key in the :refresh parameter to load-data, or because something’s confusing om’s shouldComponentUpdate

grzm20:08:18

Thanks. I'll definitely look into :refresh. I suspect that's it.