This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-07-07
Channels
- # admin-announcements (2)
- # boot (111)
- # capetown (5)
- # cider (15)
- # clara (15)
- # cljs-dev (8)
- # clojure (78)
- # clojure-brasil (1)
- # clojure-dev (2)
- # clojure-greece (25)
- # clojure-hk (1)
- # clojure-russia (5)
- # clojure-seattle (1)
- # clojure-spec (120)
- # clojure-sweden (3)
- # clojure-uk (63)
- # clojurescript (161)
- # data-science (1)
- # datomic (21)
- # editors (43)
- # emacs (3)
- # funcool (1)
- # hoplon (72)
- # instaparse (11)
- # jobs (1)
- # off-topic (2)
- # om (212)
- # onyx (9)
- # other-languages (129)
- # proton (5)
- # re-frame (15)
- # reagent (18)
- # slack-help (4)
- # spacemacs (18)
- # untangled (224)
- # yada (21)
so I think I may finally be sort of grokking this...
1) my (server) call gets data into app-state at :sources
2) the reconciler normalizes and stuffs the real data into :sources/by-id
and replaces the entries in :sources
with ids - this works because I've defined Source as a component that has initial state and query
3) I run a :post-mutation
to move the :sources
list to [:sources-tab 1 :sources]
4) Profit?
@curtosis: In plain Om, the general idea is that you merge it in and use a parser to morph it to a different form on the fly. In Untangled, we've found it easier to deal with it this way
everybody understands data transforms and moves. It denormalizes your database a bit, but that is tractable to manage.
for performance of your UI, I kinda felt like you were either going to be doing memoization (and managing the caching of that) in the parser or denormalizing the database. Given that cache invalidation is one of the "hard problems" of CS...a little denormazliation seems pretty tolerable.
now the tricky bit is that the Source component itself doesn't actually render ... it's a placeholder for the query, ident, etc. It actually gets rendered by a table component, so I need to deref it (`db->tree`, I think) to get access to the actual data for the cells.
You should not need to make the tree. Untangled already has db->tree internally to make the tree. You need to make a subgraph in your db that matches your UI query.
Right. Typically you have some sub-portion of the UI that represents real persisted data. And then there is all the local crap that makes up the other bits of UI.
I have a Sources
component (is a tab) that has a :sources
prop that gets its query from a Source
component.
Right, and that query needs to compose up the tree to root. If you use InitialAppState to build state up the tree
Sources
renders a table, but because of the underlying component (fixed-data-table) I don't render it a Source
at a time.
oh, so you're saying your getting a subgraph from your query, because your query doesn't walk (query for) the entire tree of data?
Sources
has a list :sources [[:sources/by-id 1] [:sources/by-id 2] ... ]
... but I have to provide a cell function that essentially maps (id, column-key) -> renderable Cell
just keep composing it until you've got components that mirror the data all the way down
hmm... would you define a different Cell component per column type? e.g. NameCell, FilenameCell, etc?
If so, that is a union query, just like in tabs, but where the cell type varies over the collection.
It's just like the tabs example, but instead of a single ident you have a vector of them (for each row)
That may be part of your data transform: make it look the way you need it to look for the UI
The "magic" has to happen somewhere. In Untangled: it usually happens in a mutation (after a load or event)
yeah, that part makes sense. I'm just trying to adapt it to how this 3rd-party component expects things.
Ah, then what you want to do is this: Don't write query stuff for the parts that go to that component at all. Just drop a blob of JS data there
a "property" in Om/Untangled can have any value: js Date, js map, Datascript database, whatever
If your goal is to interact dynamically with the data, then that is more complicated
in my case, the table display is just a summary. the real point is to edit the data.
In that case you're probably going to find a parity mismatch with the design of the pre-built component and Om/Untangled.
(which I fear leads me down the nested tabs route, but that's a problem for tomorrow...)
1. Query and normalize the data 2. Write the UI and mutations that modify that data 3. Write an "update-table" mutation that transforms (via db->tree and clj->js, etc) into a form usable by the display widget 4. Trigger that mutation with modifications, and include a follow-on read for re-rendering the table
So, you'll basically keep updating a "display version" of the information separate from the database table objects.
In fact, that's how I imagine it...in SQL terms. There are the real tables, and there are views.
I can't figure out where/why my data/props are getting rewritten... I have a :post-mutation
to put loaded data into the right place for the tab to grab it. Three scenarios:
1) Straightforward untangled style: (assoc-in state [:sources-tab 1 :sources] idents)
- works fine, Sources
tab component gets the idents.
2) Manually stash the real data: (assoc-in state [:sources-tab 1 :sources] (values sources-by-id))
- data is there in app-state; Sources
component gets [{} {} {} {} {} {} {} {} {} {}]
in the sources
prop
3) Manually stash the real data at app-state root: (assoc-in state [:sources-raw] (values sources-by-id))
- data is there in app-state; Sources
component gets [nil nil nil nil nil nil nil nil]
in the sources-raw
prop (which has a corresponding link '[:sources-raw _]
in its query.
I could sort of expect #2 if the query is filtering what comes from the app-state, but #3 should be coming through intact, it seems.
I think this might be something I've seen before trying to fight post-mutations - but I'm probably getting my explanation of what I was fighting way wrong: basically it seems like something that has an ident on the component you write the query for must be normalized to be queryable.
I was trying to do something like scenario #3 you describe, and seeing the same thing
hmm... but that doesn't make sense to me. I mean, in #3 it's just data I've stashed and could be anything. Why is the query machinery mucking about in it?
I think I just had an array of dumb objects and not even a component with an ident, and it hated that too - just having an array of objects was enough to cause issues, and the fix was to give it the full om treatment (in my case, a list of users, had to build a User component with ident and use that query)
As to the why, I'm not entirely sure, didn't really make sense to me either. 😄
I've got another set of random data getting stuffed at app-state root that works fine
Is it a *vector? I was seeing the effect not happen on *maps. (I also had a current-user I was pulling via a link to root query that worked wonderfully)
Yeah, that was the thing I was fighting for sure - for some reason it didn't act the same because it was a vector
Like some underlying part of the system wanted to force me to normalize anything in a vector, was the sense I got.
Maybe it has to do with the default read untangled provides, or maybe it's underlying how om queries work when they encounter *vectors, not sure
Definitely not for me either - I spent a good half day at least banging my head against my desk with this. 😄
can't stuff arbitrary data into the component's map, though (#2) which is, more or less, expected; that part of app-state is "owned" by the component and the query machinery.
I think that's the same root cause - if you put idents there instead, you might have better luck
maps at top level keywords accessed by links will work
maps nested in vectors anywhere in the app-state will not work
it’s an om issue with db->tree
it assumes that vectors contain idents when the query has a join
yeah, the idents worked fine (that's #1). My problem is I need to get at the underlying data, and I feel like om/ref->any
is the "wrong way"
I was talking more about #2
I think
I’ve never tried or run into #3
It's basically the same thing because you join - just to a root area instead of further in the tree
oh, ok
I imagine
This definitely seems like a rough edge worth pointing out in the tutorial or something (maybe it was and I didn't pay attention? Lots of good info there)
I wouldn’t be surprised if it were missing
it’s one of those things that’s more of a function of untangled being an alpha framework… operating on top of another alpha framework
we could definitely benefit from a known issues section
Yeah, for sure - but surprisingly this was probably the first thing that acted completely unexpected. Once I got the hang of how queries work, for the most part everything has made a ton of sense.
that’s awesome to hear
the changes tony has mode most recently in 0.5.3 really helped to simplify things too
we’re really happy with how much simpler it’s becoming
@curtosis: did any of that help at all or are you still stuck?
is that because :stuff
doesn't show up in any component or Root and so om just doesn't care?
I think it's more because you're not joining on :stuff. As long as you don't try to write a join query that looks into the :stuff key you'd probably be okay
So you can treat :stuff
as a raw value and that'd probably be okay
But if you tried to go {:stuff [:key]}
, then you'd need to normalize the things in that array
It generally makes sense I suppose if you think of any join syntax as its equivalent in SQL - you'd need anything you join to be in normalized into a separate table, so if you do a join in your query, then that's expected behavior. Hopefully that'll help me think about it in case I run into this again.
^^ yes that is my understanding as well
@curtosis: your first example doesn’t work because om assumes that the vector is a list of idents, and tries to denormalize data that is already denormalized
the second one works because :sources-data
is a map, not a vector — om doesn’t make assumptions about its structure
@ethangracer: right, but the gotcha is that :stuff
is a vector, and the magic is that by not invoking any query machinery on :stuff
it gets ignored.
yup… definitely an esoteric detail
there is one way it kind of makes sense — if you don’t include a nested query, om just dumps what it finds
if you DO include a nested query, it goes through denormalization
if the data isn’t normalized, there’s no need to add a nested query
I keep reminding myself that this is so much simpler once I get through the mind warp. 😛
(and, admittedly, my current problem is trying to deal with a 3rd-party react component that doesn't map nicely to my data component hierarchy)
the details might still be scroll-up-able, but the short version is: in a normal Sources table you'd iterate over Source rows, no problem. But fixed-data-table
doesn't do that; it iterates over cells... so I could either decompose everything down to individual Cell components or just side-load everything in a format that is easier to work with.
I don't think post-mutations can take params @w1ng
@w1ng nope, no post-mutation parameters. what parameters apart from the app-state do you need?
im writing an autocomplete component and want to pass the path where to merge the loaded data
in untangled-todomvc, send-support-request mutation returns the ID in the mutation,
(defmethod apimutate 'support-viewer/send-support-request [e k p]
{:action
(fn []
(let [_ (swap! last-id inc)
id @last-id]
(timbre/info "New support request " id)
(swap! requests assoc id p)
id))})
how do I get access to that ID in the client?as a workaround i made a mutation which sets the params i want in the post-mutation in the app state and the post-mutation reads the params from there, but i dont think thats a good solution
@w1ng so you have an autocomplete component that might merge data to different places in the app state depending on the data. currently your workaround is to store that location in app-state so the post-mutation knows where to put the data. am I getting that right?
is there no way to tell where the data belongs based on what’s returned from the server?
just trying to get a sense of how to suggest alternative approaches
@jasonjckn: you should be able to access it from app-state
@ethangracer: where is it loaded?
the return values of action thunks are dropped
oh, you’re not swapping on the app state
I haven’t played much with the support viewer
yeah not too sure about the support-viewer
I know that you have to return a map from the mutation, keyed either by :tempids
or :value
so the last line of your code above would have to be:
{:value id}
(defmethod mutate 'permalink/create [{:keys [ast state] :as env} k p]
{:remote (df/remote-load env)
:action (fn []
(df/load-data-action state '[:permalink]
:marker false
:params {:permalink {}}))})
gives client-side error [ 1.214s] [om.next] transacted '[(permalink/create)], #uuid "80e58bd9-b41c-4ee8-aa38-844e749449a0"
user.cljs:96 APP-STATE DIFF: {[:ui/loading-data] [:+ true :- false]}
core.cljs:3081 Uncaught Error: Doesn't support namespace:
same problem with
#(transact! [(untangled/load {:query [:permalink]
:params {:permalink {}}})])
@ethangracer: yes exactly. the location im storing in the app state has two keys :from and :to . the idea was to have a generic autocomplete component which can handle different types of data (clients, products,...) based on the data passed to it and the transactions passed to onChange
@ethangracer: heres my code:
(defmethod m/mutate 'app/set-load-targets [{:keys [state] :as env} k {:keys [from to]}]
{:action (fn []
(swap! state assoc-in [:load-data-targets :to] to)
(swap! state assoc-in [:load-data-targets :from] from)
)})
(defmethod m/mutate 'app/merge-load-targets [{:keys [state] :as env} k _]
(let [current-tab (get-in @state [:current-tab 0])
from (:from (:load-data-targets @state))
to (:to (:load-data-targets @state))]
{:action (fn []
(swap! state assoc-in [current-tab :tab to] (from (:load-data @state)))
)}))
(autocomplete/ui
{:floatingLabelText "Clients"
:onSelect #(m/set-value! this :ui/selectclients-val %)
:onUpdateInput (fn [search-text data-source]
(m/set-value! this :ui/selectclients-text search-text)
(om/transact!
this
`[(app/set-load-targets {:from :clients
:to :ui/selectclients})
(untangled/load {:query
({:load-data
[(:clients
{:fuzzysearch
~search-text})]})
:refresh [:current-tab]
:post-mutation app/merge-load-targets})
:current-tab ]))
:data (map client-select-item selectclients)
:searchText selectclients-text})
@jasonjckn: yes, I’ve been seeing this ^^ too
@w1ng: so what you’re saying is you’d like to have some way of getting the :from
and :to
keys into the mutation. interesting. let me think about that for a bit
your workaround is how I’d approach it as well, need to spin some brain cycles on whether that’s a desirable pattern or not
In our app we've got some attributes in client side queries that are not actually retrievable with the pul API. For example the query [:dashboard/title :dashboard/created-at]
title
works with datomic.api/pull
but created-at
is derived with a query over the :db/txInstant
.
Ideally we'd like to have an extensible pull wrapper, call it pluck, where we could specify custom ways to query keys like :dashboard/created-at
but the results get merged back in such that it looks like a regular pull API call.
@ethangracer @tony.kay: maybe you've run into this usecase? Any suggestions for how to address it?
I suppose I should also ask in the datomic channel.
I'm thinking an extensible API like so would be useful.
@currentoor: definitely have run into the use case, we are post-processing data on the server to rename the keys as the client expects them to appear
I like the idea of that kind of api
@ethangracer: if you change the queries on the client will the server omit them or send back extra data?
I exclude them from the server query using :without
on data fetch
then when the server returns the data it normalizes it using the full UI query
oh, actually i’m using a client-side post process as well
right we have some issues like that also
like querying :dashboard/by-id
will always return created-at updated-at whether the client asks for them or not
i could go into the server side read function and put some conditional logic for these two keys but that seems way to special-casey
a pull api wrapper would be much preferable
let’s think about that then, how would it work
the UI query would have to remain intact for normalization
so either before the query is sent to the server, or after the query hits the server, we would have to intelligently replace keys that correspond to the database schema
then once the data is returned, we’d replace the data returned from the database with the keys required by the client query
missing anything?
yeah that sounds about right
i was thinking maybe a ring middleware type approach would be appropriate
since we need to do pre and post processing
with multimethods for the individual keys
which would have access to that pluck api you mentioned?
yeah I think i’m with you
interesting
why is this more preferable than manual modification by query?
but the tricky bit is traversing the tree in this ring-middleware thing and then putting stuff back in the right spot
right
oh because it might not be just a simple query attribute re-naming
like :dashboard/created-at
there is not created-at
attribute in the dashboard model
it's derived from the transaction time
yeah i’m still not totally clear on what you mean by that
it sounds like you have some datomic schema for the namespace dashboard
but created-at
is not one of them
correct?
so you want to write a custom query to fill that field
created at is retrived like so
(d/q '[:find
?created-at .
:in $ ?d
:where
[?d :dashboard/organization _ ?org-tx _]
[?org-tx :db/txInstant ?created-at]]
db
eid)
but on the client created-at
is a regular field like anything else on the dashboard
right
yeah we haven’t run into this exact issue — we have some reporting stats that we calculate using aggregates, so we have to invent keys to return to the client, since there is somewhat significant server-side processing involved in formatting that aggregated data properly
so similar idea, I guess I’m wondering if this api you’re suggesting could be easily extended to support that use case as well
I definitely see how it could be useful, my only remaining question would be where it belongs. untangled-datomic? untangled-server? would this apply to other DB models?
my guess would be untangled-datomic unless we can make it super general
i'm going to try and pair with @therabidbanana on this
hopefully we can come up with something that covers both our usecases
sounds good, let me know if I can do anything to help
@currentoor: snuck in a bit of time from iCTO
2 thoughts. one is that a created-at date derived from a database transaction is unreliable, since any subsequent modification of that attribute would change the transaction time
so a true query to get the initial created-date would be somewhat involved, its easier to store a date as a separate attribute on the entity
second thought: we don’t want to introduce more middleware that is only going to be used for certain kinds of queries, because it’ll just slow server response times for the general use case queries that don’t have this concern
Yeah created-at
in general should use the history db and grab the smallest timestamp, but for our usecase the organization is only set once and we have validations preventing it from being updated.
Also I meant ring middleware style decorator pattern, not actual server middleware.
Lol that would be ridiculous! But I can see how what I said was confusing.