The loading mutation is run without waiting for the file-upload one to finish

I am seeing a successful OPTIONS request, basically file-upload OPTIONS fires, file-upload POST is pending, but the load is fired at that point

what does your file upload mutation look like

ah, does it matter that the mutations have different remotes?

(defmutation upload-file [{:keys [ident value]}]
  (action [{:keys [state]}]
    (println "upload-file")
    (swap! state set-value* ident value))
  (file-upload [env] true))

(defmutation load-adapted-data [{:keys [file-id] :as params}]
  (action [{:keys [state] :as env}]
    (println params)
    (println env)
    (println "load-adapted-data")
    (println (get @state :file/by-id))
    (df/load-action env :adapted-data nil
      {:params {:file-id file-id}}))
  (remote [env] (df/remote-load env)))

(prim/ptransact! this '[(upload-file …) (load-adapter-data …)]

@tony.kay ^

hmmm…run this function and tell me what the output is:

(prim/pessimistic-transaction->transaction `[(upload-file {}) (load-adapted-data {})]
  {:valid-remotes #{:remote :file-upload}})

sorry, you need that valid remotes for it to work right

in my ptransact! ?

no, just log the output of that

that’s the function ptransact! uses to transform your tx

I want to see what it transforms it to

the ... can just be empty maps…

ug…typos

re-read that message, I edited it, or perhaps you saw the typo

[(contaico.api.mutations/upload-file {:ident [:file/by-id #fulcro/tempid["aa3bfdfb-f46b-462c-9fc0-159a4e81d839"]], :value {:db/id #fulcro/tempid["aa3bfdfb-f46b-462c-9fc0-159a4e81d839"], :file/type :file.type/main, :file/name 2018-05-alegra-invoices.csv, :file/size 561887, :file/uploaded? false, :file/js-file {}}}) 
(fulcro.client.data-fetch/deferred-transaction {:remote :file-upload, :tx [(contaico.api.mutations/load-adapted-data {:file-id #fulcro/tempid["aa3bfdfb-f46b-462c-9fc0-159a4e81d839"]})]})]

hm…that looks ok

and it is using the correct remote for the deferred

you didn’t specify parallel on the remote did you?

no

like, you’re using the normal http remote from fulcro, just with middleware for the req/resp

I’m using this

(def api-request-middleware (-> (net/wrap-fulcro-request) wrap-auth-token))
(defn api-response-middleware [app]
  (-> (net/wrap-fulcro-response) (handle-token-expiration app)))

(defn api-remote [app endpoint]
  (net/fulcro-http-remote
   {:url                 (str "" endpoint)
    :request-middleware  api-request-middleware
    :response-middleware (api-response-middleware app)}))

But it looks like it is not waiting on the file middlewares no?

wait, this might be fishy

(defn handle-token-expiration [handler app]
  (fn [{:keys [status-code status-text] :as req}]
    (if (and (= status-code 401)
             (= status-text "Unauthorized"))
      (prim/transact! (:reconciler @app) `[(token-expired)])
      (handler req))))

I’m transacting within a middleware

no…it could be that ptransact requires the remotes to all be the same…let me refresh my memory

i don’t think so @tony.kay

yeah, it uses the load queue to do the defer…so it has to be the same remote

I think the new incubator stuff will work for your use-case

I’ll take a look

@currentoor was just doing some of this…I’ll defer to his more recent knowledge 🙂

so i’ve got this mutation that uses the file-upload remote

(defmutation upload-file
  "Mutation: Start uploading a file. This mutation will always be something the
  app itself writes, because the UI will need to be updated. That said, adding
  the file to the client database is done via the helpers add-file*."
  [{::file-upload/keys [id size abort-id] :as file}]
  (action [{:keys [state]}]
    (swap! state (fn [s]
                   (-> s
                     (file-upload/add-file* file file-upload/MAX-FILE-SIZE)
                     (assoc-in [::camera-btn :singleton :file]
                       [::file-upload/by-id id])))))
  (file-upload [{:keys [ast]}]
    ;; don't even start networking if the file is too big.
    (when-not (>= size file-upload/MAX-FILE-SIZE)
      (-> ast
        (m/with-abort-id abort-id)
        (m/with-progressive-updates `(file-upload/update-progress ~file))))))

and i’ve got this other mutation that was built for the incubator’s pmutat!

(defmutation new-wash-&-unconfirmed-vehicle
  [{:keys [wash file-id]}]
  (action [{:keys [state]}]
    (log/info "Creating unconfirmed vehicle with file id" file-id "wash-id" (:wash/id wash))
    (swap! state add-wash* wash))
  (ok-action [{:keys [state ref] :as env}]
    (let [unconf-v (get-in @state (conj ref ::pm/mutation-response))
          ident    (prim/get-ident u.m.s.confirmation/UnconfirmedVehicle unconf-v)]
      (swap! state
        (fn [s]
          (-> s
            (prim/merge-component u.m.s.confirmation/UnconfirmedVehicle unconf-v)
            (assoc-in [:wash/id (:wash/id wash) :ui/unconfirmed-vehicle] ident))))))
  (error-action [{:keys [state reconciler] :as env}]
    (u.flash/show! reconciler {:error (util/default-error @state) :duration 0})
    (r/route-to-impl! env {:handler :camera})
    (u.m.state-machine/set-impl! env {:value :base}))
  (remote [env] (pm/pessimistic-mutation env)))

which definitely relies on the upload-file mutation happening on the server a priori

and i’m able to ensure this only with the incubators ptransact! not the one built into fulcro proper

(pm/ptransact! this
                        `[(r/route-to {:handler :packages})
                          (u.m.state-machine/set {:value :photo-taken})
                          (upload-file ~file)
                          (new-wash-&-unconfirmed-vehicle
                            {:file-id ~id :wash {:wash/id    ~wash-id
                                                 :wash/state :initial}})])

i’m not sure how (if) tempid resolution is affected across remotes however, i’m using client generated UUIDs which i use as primary keys everywhere

@pvillegas12 does that make sense?

yeah, @currentoor I’ll try using the incubator ptransact! I can get the tempid mapping from the state if they are properly chained

ok, do you need tempids?

switching away from them to client generated random-uuids did save me some headache and complexity

this is the first time tempids are biting me 😛

are you using datomic also?

but I do see how the client uuids can help here

yes, using datomic

yeah same for me

The tempid remapping is quite harmless 🙂

yeah in most cases i agree it works, but it is a non-zero amount of extra complexity

and i’m not sure how much benefit it provides, it’s just one more thing that can go wrong

Do you have the convention of some field for all types of objects?

yeah it was @tony.kay’s suggestion, we use <entity-name>/id

so a user entity will have an attribute called :user/id which will be a UUID with a uniqueness constraint (indexed)

I believe @wilkerlucio uses that too?

yeah, that looks like a better way to handle client generated tempids

i’d recommend going one way or another early on, refactoring in a larger project won’t be fun 😅

but do what works for you

I like the :user/id format, reduces the translations of attributes to do on server side when using pathom

oh yeah i forgot, it’s better for working with pathom (which is pretty cool by the way)

Incubator’s ptransact! works 😄

When using defrouter would it be normal to define your "navigation handler" function in the root component and pass it all the way down to the place where it's needed? So you might be using prim/computed going down through several levels of UI components?

@ben735 no

for a component that needs to know the current route you add [fr/routers-table '_] to it’s query

say the relevant router is ::admin-router

(defrouter WashProgramAdminRouter ::admin-router
  (fn [_ props] (tab-ident props))
  ::details WashProgramDetailsTab
  ::subscriptions WashProgramSubscriptionsTab
  ::reports WashProgramReportsTab)

then in render you can get the current tab like so

(let [routers-table (get props fr/routers-table)
      active-tab    (first (fr/current-route routers-table ::admin-router))]
  ...)

what if it needs to change the route though? Won't optimisation prevent the root component from re-rendering with the new route displayed if the mutation is made from a nested UI component?

if you want to change the route you have two choices

fulcro.client.routing/route-to mutation

or fulcro.client.routing/route-to-impl! helper function can be called inside one of your mutations

> Won’t optimisation prevent the root component from re-rendering with the new route displayed if the mutation is made from a nested UI component? the root component queries for the top level router right? it should update

does that make sense?

I'm currently using routing/route-to, but was finding that while the DB was changing, the router wasn't being re-rendered. I might need to do some more testing. Thanks for your help

hmm yeah it should work probably need some debugging, unless i misunderstood something 😅

definitely try to avoid passing routes directly through computed props

have you read the book’s sections on routing?

and no worries

oh but i just realized my router uses html5 routes, that might change things

you can make your own mutations that calls r/route-to-impl and use declarative refresh http://book.fulcrologic.com/#_declarative_refresh

I've read bits and pieces about routing in the book, but I probably need to go through it all again

if the mutation changes something the root depends on, and you declare that key in it’s refresh section then it definitely will re-render

I've just tried calling (prim/transact! this [(routing/route-to {:handler :list})])` on a component a few levels below the router and it's definitely changing the DB but not re-rendering the router

Where this is the nested component

@currentoor if I add a follow-on read for something on the root component it works though

@tony.kay I’m looking at the fulcro book, and it says somewhere that if I want to deal with REST, I should be using the fulcro legacy networking API. Could you elaborate on why this is the case? I ask because I’ve been using the new networking API with REST just fine, and I’m wondering if I’m missing something.

Any tips on how to make a controlled input field that syncs remotely onBlur? I'm currently passing an extra parameter to my mutation called local-only in onChange, which seems to work ok

I'm basically trying to avoid making a server request on every keystroke

@ben735 http://book.fulcrologic.com/#_the_ui_and_post_mutation

@ben735 perhaps you could have the onChange only update local react-state and bind the mutation to onBlur ?

@ben735 You just need to add a follow on read for something in the parent, or even fr/current-route…the routing stuff in Fulcro doesn’t invalidate the rest of the UI refresh story.

@ben735 mutations refresh section is a follow on read <EDITED>

I don’t follow your comment @currentoor…

remote? Do you mean refresh?

yeah oops

@currentoor the incubator functions don’t help either, a composition of mutations with different remotes does not respect pessimism 😞 I went with returning the data I need from the first remote instead of separating it out

strange it’s working for me

Looking at the above snippet, how do you actually rely on the file upload finishing?

If you are using a client generated file id for the optimistic part of the UI you don’t really need the server to respond from the file upload

m/with-target is not working for me in the file-upload remote, any ideas how to debug this?

so, all of the returning/targeting stuff works by modifying the AST of the remote request

chances are you’re middleware is morphing the query/ast and dropping the data

read the source of with-target @pvillegas12, and follow the data through your middleware

Now I’m getting Query ID received no class, how do I ensure meta-data is preserved when copying the right data? In this case it’s 'query from the ast

if you make a new query, just copy the metadata over from the old one

the query will also have metadata for :component

which is the secret sauce for normalization

In this case, I want to put raw data into a specific path

when you get to the networking layer, things get a bit more involved 🙂

yes 🙂

Right now I’m doing

(cond-> (assoc resp
                      :error :none

                      :transaction [(merge (select-keys ast [:query])
                                      {(file-path real-id) [:db/id :status]})])

@tony.kay what else should I be doing apart from this merge?

So, here’s the core thing you need to know: The return value from your middleware goes directly to prim/merge!

your query is the query to merge, and the data is the data to merge

(Right now the data is getting merged, however it is being merged at root)

get-query adds metadata to the query to let it know how to normalize..and the shape has to match

so normalization is ok

no normalization (big payload)

This is the file upload use case I was discussing before

I’m returning the computation with the tempid mapping

(couldn’t get two remotes to behave :()

prim/merge-mutation-joins is the processing for a mutation that has a returning

you’re doing a mutation for uploads if I remember right

yes, I’m currently doing

(file-upload [{:keys [ast]}]
    (println "upload-file")
    (println parent-id)
    (m/with-target ast [:integration-details :root-router])))

for the remote part of the mutation

so targeting won’t work unless the mutation looks like a mutation join…because otherwise there is nothing to move

your query should look something like [{(upload-mutation param-map) ['*]}]

http://book.fulcrologic.com/#MutationJoins

or whatever query matches the thing your returning besides [*]

then targeting is added on the metadata of THAT query…e.g. (vary-meta (get-query ReturnType) assoc ::fdi/target [:target :path])

where the get-query gets you a query that has :component metadata on all of the recursive things in the query

but it no normalization, then you could just use ['*] for the query I think

@tony.kay I got it working 😄 that was intense 😛

happy to hear feedback on making it easier…also, I should probably see if I can fix ptransact across multiple remotes…it should actually work

Namely I aligned these two pieces

(assoc resp
                  :error :none
                  :transaction [{(file-path real-id) [:db/id :status]
                                 target              [:pipeline-data]}])

(assoc resp :body
    (merge body {(file-path id) {:db/id id :status status}
                 target         (select-keys body [:pipeline-data])})))

The problem was an understanding of how the middleware (the file upload one) was manipulating the query merging

https://github.com/fulcrologic/fulcro/issues/254

@tony.kay doing this manual merging, how do I get rid of the edge on the root? I am seeing pipeline-data in the correct path, but root has the same data too.

dissoc it

😜

I mean, if you’re properly using the targeting system, it should do that for you

(defn- body-response [{:keys [body] :as resp} target id status]
  (assoc resp :body
    (merge body {(file-path id) {:db/id id :status status}
                 target         (select-keys ('upload body) [:pipeline-data])})))

(def file-response-middleware
  "Response middleware that manipulates the merging of data (mainly the datomic
  id of the file and the computed data from the pipeline) by handcrafting
  :transaction in the response."
  (->
    (fn [{:keys [body error] :as resp}]
      (let [ast     (prim/query->ast1 (:transaction resp))
            id      (get-in ast [:params :value :db/id])
            target  (-> ast :query meta vals last)
            real-id (get-in body ['upload ::prim/tempids id])]
        (cond-> (assoc resp
                  :error :none
                  :transaction [{(file-path real-id) [:db/id :status]
                                 target              [:pipeline-data]}])
          (= error :network-error)      (body-response target id :network-error)
          (= error :http-error)         (body-response target id :network-error)
          (and real-id (= error :none)) (body-response target real-id :complete))))
    (net/wrap-fulcro-response)))

That’s what I’m doing (similar to the one in the file upload repo)

I got around it my hiding the data in 'upload 😛

I feel I’m tiptoeing the targeting system, but I don’t think it can be avoided

I don’t see you using targeting anywhere in that code

Implicitly using it here target (-> ast :query meta vals last)

you’re reading it, but you’re not letting Fulcro process it for you

right, that’s because I’m creating a new :transaction if I understand correctly

and vals last is not reliable

maps have a “predictable order” for internal reasons you should not rely on

nor can I guarantee that order won’t change due to my own code changes to Fulcro

yeah, you’re right

so, you’re not changing the body, but your code isn’t enough for me to understand things…what is IN body?

and what target are you passing?

body

{upload
 {:fulcro.client.primitives/tempids
  {#fulcro/tempid["8e9d4a09-e06d-451f-8d7b-f10a178a6f9c"]
   #object[Object 28684059345420642]},
  :pipeline-data [...]

(defmutation upload-file [{:keys [ident value parent-id]}]
  (action [{:keys [state]}]
    (swap! state set-value* ident value))
  (file-upload [{:keys [ast]}]
    (println "upload-file")
    (println parent-id)
    (m/with-target ast [:integration/by-id parent-id])))

From what I can see, if I change :transaction within the response, I need to manually merge the data I care about

no, you change the transaction to include the mutation merge “query” that will support targeting

you want something like (in this last defmutation) (-> ast (m/returning ['*]) (m/with-target [:integration/by-id parent-id]))

then your middleware pretends to be the server, and returns

{`upload-file {data map of stuff}}

actually returning takes a component, I think

so you might need to write a placeholder component with that query

and I don’t touch the transaction in the response then in the middleware?

for that matter, if you write the placeholder and it has an ident function that could generate the “target”, you’d be all set I think

don’t think you need to, do you?

are you adding other data?

no, that’s the entire implementation above

the db/id status thing, perhaps?

the db/id is tempid remapping

nah, I mean the ident-based merge you’re doing for the file upload entity

is pipeline data a single map? You’re showing it as a vector

it is a vector of maps

but it should not be normalized

if you want to return that shape from the server, then :pipeline-data is going to end up nested in the entity map

but your with-target isn’t doing anything because there is no query…you have to use returning

or with-target is a noop

as far as it’s concerned, there is nothing to move, because you’re returning “void”

I was trying to do this http://book.fulcrologic.com/#_demo_of_mutation_joins

The error-message retargetting

oh…hm

ah, I was misremembering I guess…

but then I’m not understanding why you had to do all of this manipulation

just the with-target part would not re-target the data, the pipeline-data would remain placed in the root

and then I realized that the middleware was modifying the :transaction and that led me to the above

I see…but the problem wasn’t the targeting…it was the manipulation of the transaction

it was throwing out the metadata

I also tried adding the metadata back in to the query part

did you try just pulling the incoming transaction and doing a conj for the file upload status?

something like

:transaction [{(file-path real-id) (with-meta ['*] (meta :query resp)     }])

let me try that

or just (update resp :transaction conj ...)

the original mutation query would have been correct for the targeting, and you didn’t need to throw it out…just add in the extra stuff you want to cause to be merged

@tony.kay

(defn- body-response [{:keys [body] :as resp} id status]
  (assoc resp :body
    (assoc body (file-path id) {:db/id id :status status})))

(def file-response-middleware
  "Response middleware that manipulates the merging of data (mainly the datomic
  id of the file and the computed data from the pipeline) by handcrafting
  :transaction in the response."
  (->
    (fn [{:keys [body error] :as resp}]
      (let [ast     (prim/query->ast1 (:transaction resp))
            id      (get-in ast [:params :value :db/id])
            real-id (get-in body ['upload ::prim/tempids id])]
        (cljs.pprint/pprint body)
        (cljs.pprint/pprint (:transaction resp))
        (cond-> (assoc resp
                  :error :none
                  :transaction (conj (:transaction resp)
                                 {(file-path real-id) [:db/id :status]}))
          (= error :network-error)      (body-response id :network-error)
          (= error :http-error)         (body-response id :network-error)
          (and real-id (= error :none)) (body-response real-id :complete))))
    (net/wrap-fulcro-response)))

not working 😞

The transaction before is

[{(contaico.api.mutations/upload-file
   {:ident
    [:file/by-id
     #fulcro/tempid["49514c48-f4f8-42ec-886c-c86b5e13ba12"]],
    :parent-id #object[Object 62469852643721535],
    :component #object[contaico.ui.file-upload.FileUpload],
    :value
    {:db/id #fulcro/tempid["49514c48-f4f8-42ec-886c-c86b5e13ba12"],
     :file/type :file.type/main,
     :file/name "2018-05-alegra-invoices.csv",
     :file/size 561887, 
     :file/uploaded? false, 
     :file/js-file {}}}) 
  [*]}]

is the problem that the query is not including the :pipeline-data?

@pvillegas12 your problem I think is body-response

well, maybe not…reading again

I’m not seeing the problem 😕

weirdness, at least the other version is working

wait…is that the right order for the middleware

I always have trouble thinking about that…

I used the same one here https://github.com/awkay/file-upload-demo/blob/master/src/main/file_upload/file_upload.cljc#L179

I guess it is

is your server changing the name of the mutation?

you’re reading 'upload out

but that’s not the mutation name you showed in the response

mmm, can you elaborate?

contaico.api.mutations/upload-file vs upload

your server isn’t responding with the same mutation name as the client is putting in the request

ah 🙂

I guess this is taken care of by the macros like defquery-root on the server right?

yes…the parser period

The network protocol is very simple…you send a query for :x or ’mutation, and the response has to include {:x value 'mutation value}.

not hard really 🙂

things just have to match

and of course queries can nest, so there is that complication, but in principle, it’s very simple

but yeah, for return value processing and targeting, the internals cannot assume the targeting on one mutation applies to some other, just because it’s the only one there 🙂

right

thanks for the help!

thanks for your help @currentoor,@pvillegas12, @pontus.colliander and @tony.kay!

@pvillegas12 did that fix it?

Haven’t tried yet, @tony.kay I’ll let you know if that does it

cool

So @pvillegas12 @currentoor, I’ve just run a test on Fulcro on ptransact! against two diff remotes. It behaves as I would expect, and defers properly. I’m confused as to why either of you had problems.

i’m in the middle of something, but i can verify later

wait does original ptransact! support ok-*?

no

I’ll expand my tests and see if I can reproduce

I’m getting this error when i run the clj side of fulcro-spec (#:fulcro-spec.selectors{set-active-selectors #:fulcro.client.primitives{:error java.lang.AbstractMethodError}} "Parser error:\n" {:status 400, :body #:fulcro-spec.selectors{set-active-selectors #:fulcro.client.primitives{:error {:type "class java.lang.AbstractMethodError", :message nil}}}})

but i’m not sure what i’m doing wrong

yeah, sorry…I think there are certain things your tests can do that break the runner over there…I’ve seen that as well…fixing it is usually commenting out some tests

and then trying to figure out what’s wrong with the one failing

Marking bug 254 as “cannot reproduce”. I wrote a full-stack example with two remotes, and it works as documented.