Fork me on GitHub
#om
<
2016-11-17
>
tianshu02:11:55

@isak Sounds great! Is the parser open source? If not, is there any example for usage, so I can learn to write myself.

isak02:11:15

@doglooksgood: its not open source, because it is written in f# (.NET), so there probably wouldn't be enough interest to warrant the effort, but i can give a couple of examples

tianshu02:11:34

that will be a great help!

isak02:11:35

1) simple example with multiple 1-to-1 joins: http://i.imgur.com/P7heGui.png

isak02:11:19

More complex example that includes a 1-to-many join, and also search conditions: http://i.imgur.com/WK9xOYR.png

isak02:11:51

the strategy i took was 1) parse om query 2) generate a query plan from om query 3) translate om query to a SQL syntax tree 4) render sql from syntax tree 5) execute sql 6) use query plan to know how to read the results back into the right shape

tianshu02:11:49

Is that always use {:table [& :fields]}, so no need to use namespaced keyword?

isak02:11:39

yea, that is what i have done, so i haven't needed namespaces for keywords

isak02:11:03

fields or nested joins

isak02:11:11

and then i have another place with metadata about joins: 1) from table 2) to table 3) join name 4) join condition

tianshu02:11:43

the part [and ....] come from the om query?

isak02:11:21

thats actually a custom thing, because search conditions other than idents are not part of the grammar

isak02:11:54

and it probably should have just been (and ...), but won't make too much difference

tianshu02:11:42

but, it seems your client have the fully control to the query? is there any server-side part? I mean, some idea like: client decide the data they want, and server decide the entities they can provide?

isak02:11:54

yes for us the client fully specifies everything. the only thing controlled by the server is permission checking, and which company the query is relative to

isak02:11:34

for permission checking, we check both at the row level, and at the column level (also for the search conditions)

isak02:11:25

but thats not really necessary, because you need to do everything in stages anyway, and can choose to add/remove search conditions for the sql on the server

tianshu02:11:40

the checker walk through the query expression before parsing query to sql?

tianshu02:11:06

yes, the server can still modify the query expressions.

isak02:11:08

yea, at least before executing it

isak02:11:52

but it may be easier to start with having the query specify more of the conditions

tianshu02:11:44

how you specify offset and limit in query? any example?

isak03:11:03

in the second example, i have :top, which is the sql server equivalent of limit in postgres

isak03:11:36

for offset, i haven't needed it yet, but probably would be done in a similar way

isak03:11:31

:top is pretty easy, except when your query includes a 1-to-many join, in which case you have to be careful not to chop off child records if there is a limit

isak03:11:49

that is why the TOP is in the subquery in the second sql query

tianshu03:11:45

It seems, with this parser, you only need to write permission check for every read.

tianshu03:11:18

thanks, this is great!

tianshu03:11:27

I think I know what I should do.

isak03:11:38

glad to hear it, np 🙂

danielstockton08:11:03

Really struggling with how to handle authentication with compassus. I created a Wrapper component and have a (when token (factory props)) in render. Trouble is, it is trying to read the query for the route before I've authenticated. @anmonteiro any advice on how I should do this? How should I delay the read until after I have fired a login request and received a token?

danielstockton08:11:36

My only idea is to have two separate apps and mount one or the other, depending on whether im authenticated.

danielstockton08:11:35

I see if I don't set route-dispatch: false I can add a read method that dispatches on the route, perhaps I can use this to change behaviour based on whether I'm authenticated or not?

anmonteiro10:11:05

@danielstockton :route-dispatch false could probably help but you still need to read after you have authenticated

anmonteiro10:11:46

that follow-on read could either be done in merge! or by transact!ing a key to read

danielstockton11:11:05

I tried adding a key to read at the end of the transact, but it doesn't trigger the read. Perhaps because it isn't a key in the wrapper's query?

danielstockton11:11:45

The Wrapper is re-rendering after the transaction, but doesn't seem to be re-reading the query for the route

anmonteiro11:11:15

@danielstockton are you on 1.0.0-alpha2?

anmonteiro11:11:38

try upgrading and see if it works 🙂

danielstockton11:11:51

I also tried with :route-dispatch false and adding my own read function for the route, I got into a muddle with user-parser, ast, route, query....

danielstockton11:11:55

Ok, i'll try 🙂

danielstockton11:11:48

Tried different keys after transact, the route key and the key in the query of the route view

anmonteiro11:11:18

can you put together a minimal repro I can look at?

anmonteiro11:11:54

I don't think there's a bug in Compassus anymore, might be related to your app

danielstockton11:11:20

Ok, i'll try to do that

danielstockton11:11:28

This shows the idea: https://gist.github.com/danielstockton/cb3a26c7dc1768f895956c2e2cfc77bc Need to set it up in a new project and try to run it, maybe you can spot something clearly amiss

anmonteiro11:11:02

@danielstockton maybe. try putting it in didMount instead?

danielstockton11:11:34

No difference. Am I right with the key I'm passing to transact (:index), or should I be passing a key in the child component query?

anmonteiro11:11:28

:index doesn't mean anything to the Indexer

anmonteiro11:11:57

you should be queueing :user/token for example

anmonteiro11:11:22

but I think merge! will queue that automatically hrm

danielstockton11:11:24

well I have a custom merge fn on the reconciler

danielstockton11:11:13

I did try the other keys though

anmonteiro11:11:41

make sure you return :keys from your custom merge

danielstockton11:11:44

i only customized merge-tree actually, and its using a simple clojure merge

anmonteiro11:11:10

if your gist exhibits the problem I'll try it out and let you know

danielstockton11:11:54

ok, ill try to test it

danielstockton12:11:20

you're probably safe, i can't reproduce it 🙂 i'll try to find out why

danielstockton12:11:53

it works in my test case, at least when i pass a key to transact

danielstockton12:11:40

main difference is that there's no ajax call

anmonteiro12:11:15

you can try a setTimeout or something

danielstockton12:11:06

works perfectly with a 5 second timeout too, nevermind, ill keep digging

danielstockton12:11:40

I found the problem. I'm not sure how to access the app-state in the send function, so I was setting an atom with the token value and firing the request when the token was present. Due to the asynchronous nature, I was trying to fire the request before i reset that atom with the token...

danielstockton12:11:01

Is there a better way to do this? I think I need to modify the read function to pass the token through to send somehow

anmonteiro12:11:49

@danielstockton can't you send the token as a param in the query?

anmonteiro12:11:28

you can attach the token to the AST and read it back in your send function

danielstockton12:11:25

Probably, I haven't quite got to grips with the parser yet. My query looks like [:totals] in send at the moment and my read function is:

(defmethod read :totals
  [{:keys [state ast] :as env} key params]
  (let [st @state]
    {:api true
     :value (get st :totals)}))

danielstockton12:11:40

Instead of :api true I need to do something with the ast and pass that?

anmonteiro12:11:35

cljs.user=> (om/ast->query (assoc (p/expr->ast :totals) :params {:token "xxx"}))
(:totals {:token "xxx"})

anmonteiro12:11:01

so in short you just need to (assoc ast :params {:token "foo"})

danielstockton12:11:23

Ok great, i'll try that, thanks

snoonan12:11:34

Query question: I am trying to pivot a vector. What I have is one entity (with rendering component) with an array of owners, and I would like to also have the array of owned when rendering the owners.

snoonan12:11:56

I can give an example if it will help.

snoonan12:11:26

I am trying to just even query the db directly right now but even that is failing me.

anmonteiro12:11:59

perhaps an example would help

snoonan12:11:50

{:players [{:name "me"}
            {:name "other guy"} 
            {:name "them"}]
      :companies [{:name first :owners ["me" "other guy"]}
                  {:name "second" :owners ["me "them"]}]}

snoonan12:11:30

I am looking to get an result for "me" that contains "first" and "second" and a result for "them" that contains "second"

snoonan12:11:57

At this point I don't really even care much about the form of the result, just that it exists.

snoonan12:11:05

ultimatly, I would like to support duplicates in the :owners list, and retrieve a list of "me"s from each company, even it is was empty, but I can deal with that later

snoonan12:11:17

Something like:

[{:name "first" :owners ["me"]} {:name "second" :owners ["me"]}]
is what I really want. I have tried various forms of
[{:companies [:name [:owners "me"]]}]
trying to get the [:field value] syntax to work for me, but failing

danielstockton14:11:54

This: https://gist.github.com/danielstockton/ca8b8d7c1fbbe19f3674f6fd9cbe794f#file-test-cljs-L29 The read is happening before the token is merged into the state...I don't know how I can delay it.

danielstockton14:11:50

Shouldn't any read-keys be read after the state has been updated?

anmonteiro14:11:42

can you describe the problem you're having?

danielstockton14:11:44

With the when-let above, the read never happens because token is always nil. I'm mutating a token on to state and telling it to re-read :user/name afterwards, it seems that read happens before token is merged onto the state

danielstockton14:11:38

Somehow, I need to wait until state has a token and then re-trigger the read...

danielstockton14:11:43

Or rethink what im doing

anmonteiro14:11:39

right but is :user/name coming from the remote?

anmonteiro14:11:21

@danielstockton the re-read won't go back to the remote, if that's what you want

anmonteiro14:11:24

Om Next won't queue a 2nd remote call if you specify :user/name to be re-read

anmonteiro14:11:31

you need to do a 2nd transaction for that to happen

anmonteiro14:11:37

does this sound like the problem you're having?

danielstockton14:11:13

how should i trigger a second transaction to trigger the remote read? i have two remote reads, getting the token and then using the token in an ajax request

anmonteiro14:11:29

@snoonan it's unclear to me what your real problem is. I think you just want to query for [{:companies [:name :owners]}] but filter the parser result based on some "name" parameter

anmonteiro14:11:53

@danielstockton I think the easiest way is to override :merge in the reconciler

danielstockton14:11:17

even if it doesn't re-read from the remote, it doesn't also call the read function?

danielstockton14:11:34

i have a print in there and it doesnt seem to trigger it, which implies it doesnt do any kind of read?

anmonteiro14:11:52

what read function?

anmonteiro14:11:02

defmethod read :user/name ?

danielstockton14:11:48

this print is never triggered, even though i tell the transact to re-read :user/name

anmonteiro14:11:28

I see a println

anmonteiro14:11:09

I see it printing, is what I meant

danielstockton14:11:52

let me try a lein clean and restart figwheel..

danielstockton14:11:07

maybe it didnt properly update to alpha2

danielstockton14:11:50

still not working for me 😕 what can that be

anmonteiro14:11:36

I don't know... Om / ClojureScript version?

danielstockton14:11:25

oh boy, i had a typo in my :clean-targets

danielstockton14:11:10

still doesnt work though

danielstockton14:11:11

[org.clojure/clojure "1.8.0"] [org.clojure/clojurescript "1.9.293"]

anmonteiro14:11:20

hrm, I might have pasted an outdated version of your gist. I don't see the printing after all 🙂

anmonteiro14:11:22

you're not crazy!

anmonteiro14:11:36

your :user/name read is not getting triggered

anmonteiro14:11:42

but your default read is

anmonteiro14:11:00

which makes sense because the only thing that's queued for re-reading is :user/token, not :user/name

snoonan14:11:04

Another way of asking, how would you do 2d normalization. Each cell is a cross of owner and company and represents a portion of ownership (10% in my case).

anmonteiro14:11:49

if you normalize your state, both owner and companies will get the relevant data (the ones matching component Idents) normalized

danielstockton14:11:16

@anmonteiro what about the :user/name key after the transact!

danielstockton14:11:22

should that not trigger a :user/name read?

anmonteiro14:11:48

yeah, it's triggering a :user/name read

anmonteiro14:11:59

but that read happens before the remote has responded

danielstockton14:11:10

yeah, this is my problem

snoonan14:11:25

My trouble with accomplishing that is I don't have a good name (Ident) that both can use. I'll try to figure out something that will work. Thanks

anmonteiro14:11:47

what you need to do is add :user/name to :keys in the reconciler's :merge function

anmonteiro14:11:01

so that it gets read after the remote result returns

danielstockton14:11:13

do you have an example of something similar?

anmonteiro14:11:24

there's something else you can do too

anmonteiro14:11:00

you can use the :value {:keys ...} that your remote mutation returns to auto-queue them automatically in an overridden merge function

anmonteiro14:11:39

this way if you add {:value {:keys [:user/name]}} to your 'user/login mutation on the server, merge will know to re-read those because they arrive in the remote result too

anmonteiro14:11:46

I'm afraid I don't have a concrete example to show you, but Compassus does something with :keys in merge which maybe can give you some insight into how it works: https://github.com/compassus/compassus/blob/master/src/main/compassus/core.cljc#L295-L298

danielstockton14:11:25

in my case, every remote read should be re-read once I have a token and adding all the keys from the remote doesn't seem nice

danielstockton14:11:36

it should re-read the query for the current-route, in the general case

danielstockton14:11:06

i only have :user/name for now but this is the simplest case

anmonteiro14:11:16

that's something you can also do easily

anmonteiro14:11:24

you have the reconciler and the state in merge

anmonteiro14:11:47

so you can get the current route, and obtain the query of the component which pertains to that route

danielstockton14:11:41

and then just update :keys to be that query?

anmonteiro14:11:52

not quite 🙂

anmonteiro14:11:11

if the query is all keywords, then yes

anmonteiro14:11:25

if not you need to extract their "dispatch-key"

danielstockton14:11:37

right, so query->ast and map :dispatch-key

anmonteiro14:11:38

e.g. {:some/join [:foo :bar]} -> :some/join

anmonteiro14:11:17

(map (comp :dispatch-key om.next.impl.parser/expr->ast) query)

anmonteiro14:11:20

something like this ^

danielstockton14:11:33

i'll see what i can work out

denik14:11:04

What could be the reasons for the (str "No queries exist for component path " cp " or data path " path’) exception?

denik15:11:28

we have one read-key, that when added in a transaction which makes most of the app re-render and throws this exception at the (full-query component) call of a leaf component

danielstockton15:11:23

I see this quite a lot, is there a reason to prefer (update-in [:keys] over (update :keys ?

danielstockton15:11:23

@anmonteiro https://gist.github.com/danielstockton/ca8b8d7c1fbbe19f3674f6fd9cbe794f#file-test-cljs-L48 It looks like {:keys [:user/name], :next {:om.next/tables #{}, :compassus.core/route :index, :user/token token}, :tempids {}}, still not seeing :user/name being re-read

tianshu15:11:23

I think there should be a style/indent metadata on defui. For Emacs users, without any customize, this can provide a comfortable indentation, rather than this:

(defui Person
  static om/Indent
  (indent [{:keys [name]}]
          [:person/by-name name]))
people always prefer following I think
(defui Person
  static om/Indent
  (indent [{:keys [name]}]
    [:person/by-name name]))
the style/indent should be [1 :form :defn].

tianshu15:11:02

BTW, is it possible to have a monospaced font on slack in browser?

danielstockton15:11:54

I got it working, now I have the remote read problem (doesn't trigger a remote read but the read function is triggered)

danielstockton16:11:55

@anmonteiro why exactly doesn't a re-read trigger a remote read? and how can i trigger that with a transact! ?

bbss16:11:48

@danielstockton I could be wrong but I think re-read does trigger another remote read if the read returns a remote key-key with true or an AST

bbss16:11:16

You can trigger that with a transact by mutating and specifying the key that should be re-read IIRC.

anmonteiro18:11:34

@danielstockton because it could introduce loops

anmonteiro18:11:53

if your re-reads trigger remote reads, then they'll be re-read over and over in a loop

danielstockton18:11:59

Is there nothing special about a reread from a mutation compared to a normal read

danielstockton18:11:12

I mean, can a distinction not be made somehow

amann20:11:31

Hey #om, I’m quite new here so please excuse me if this is not at all the right place for this. I'm trying to do some leg work into investigating some performance issues in FireFox over at CircleCI. Has anyone here seen instances where Om Next triggers a requestAnimationFrame or a bunch of Non-incremental GC actions and FireFox siezes, dropping the FPS down to 0...for like 20 seconds. I'm curious if anyone has run into a similar issue with using Om/Next in FireFox. I've done profiling side by side in Chrome and it is unhappy with a few little things but it doesn't seem to even blink at the load. It also has a GC spike but it takes 300 ms vs 20 s.