This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-01-21
Channels
- # beginners (73)
- # capetown (1)
- # cider (13)
- # cljsrn (4)
- # clojure (56)
- # clojure-russia (2)
- # clojure-uk (1)
- # clojurescript (50)
- # community-development (3)
- # cursive (1)
- # datomic (80)
- # defnpodcast (2)
- # emacs (2)
- # fulcro (16)
- # graphql (8)
- # hoplon (206)
- # immutant (43)
- # keechma (4)
- # lumo (4)
- # off-topic (26)
- # perun (2)
- # re-frame (2)
- # reagent (4)
- # remote-jobs (2)
- # rum (4)
- # shadow-cljs (82)
- # spacemacs (5)
- # vim (6)
@chromalchemy i think you need to profile what's actually going on
i'm assuming you're calling filter
or remove
under the hood in those fns
which just walk the sequence and apply their predicate to each item, which should be fast enough
you could maybe get some minor speed benefits doing a single pass with a combined test, e.g. transducers
but not 5-10s worth i wouldn't have thought...
datascript/datomic/datalog is an example of something more generalised for querying the state of your items
@chromalchemy if you're rearranging list items and you have internal state for your items then you'll have a much easier time managing that state with a keyed list https://reactjs.org/docs/lists-and-keys.html
even if you carefully wrap all your own state in shared scopes that you manage properly, there's still browser state like focus, values of inputs, cursor positions, selections, etc. (much of which JS cannot even see) that can get mixed up if you can't lock a specific DOM element to a keyed item
I have a UUID for each item. I've been using this to query the list and get/set/test map vals. I was assuming that this was not great, that constantly scanning the list for item id's was inefficient, and that I should be using the implicit index in the vector. But I'm seeing maybe I was on the right track?
I see that the key does not necessarily have to be a UUID, just locally unique.
Am I correct that since my "items" each have an unique :id, then it is a "keyed list"?
@chromalchemy in the keyed-for-tpl you provide a key fn
if you don't provide one then identity is used
which would really only make sense if your items never changed, but you wanted to rearrange them
the tests use keywords as an example of that, so a list like [:a :b :c :d]
the key for an item is itself
for your case, :id
would be your keyfn
so the work needed is something like
- map keyfn to the list
- walk this list of keys and lookup to see if we already have an el in the cache or we need a new one
- return a new list of els based on the key lookups
- call merge-kids
to actually put the els in the DOM because there's no guarantee they line up with the last iteration
building els and calling merge-kids
at the end is probably the slowest parts of that
but its all pretty new!
in the future, merge-kids
could potentially even become "key aware" and do smarter/faster diffs based on that
Ok thanks, I'll try to walk through, digest that process.
I'm a bit confused about keyfn though. What does it return? Or does it apply the key/val to each list item?
given an item, return its key
or its "identity" in a sense
you're just telling hoplon what about the item makes it unique
if you have a uuid that's easy, just return that
The id value itself right? Not simply the keyword the value is on?
ah well keywords are fns
(keyfn myitem) => 23
so :id
is both the key and the function to lookup the key 😛
yeah exactly
Ok, just wanted to make sure, lol
yup, but let's say you needed to combine two things
like um :id
and :type
or something
you could do (juxt :id :type)
for your keyfn
like how a primary key in a db can be a combination of a few fields
So basically the keyfn is the "query" on the list?
uh, it is, but specifically it's the query to tell hoplon how to line things up internally
say i had [{:a 1 :b 2} {:a 3 :b 4}]
and i passed that to the loop
the loop needs to build an el for each item
but it also needs to know how to retrieve that same el next time it gets a new version of the list
so if the next iteration of the list looks like
[{:a 1 :b 5}]
what do we do?
if the keyfn is :a
then we re-use the first el from before and remove the second el
if the keyfn is :b
we need a brand new el and remove all the other els
if the keyfn is (juxt [:a :b])
then we need a new el, and we'll never reuse this new one until we see a new item with both :a 1
and :b 5
if the keyfn is identity
(the default) then we'll only reuse the el if we see a new item that is =
to the entirety of the current item
ok. And this is all in the context of keyed-for-tpl
? And "re-use" means the same dom node, but it can have updated values (outside it's key)?
yeah, same DOM node new values
the important difference between a normal for-tpl
is that the mapping between items and DOM nodes is based on the "identity/key" of the item rather than the position in the list
which is crucial for re-ordering, because if you think about it, the whole concept of "reorder" is grounded in the idea that we have some "unique things" that can be reordered in the first place 😛
True, sounds obvious in retrospect. So the keyfn returns an item's unique value. And the keyed-for-tpl tests for equality of that value across updates to better manage (diff) the dom.
but currently it's more about ensuring sensible state downstream than efficiency
although i'm sure we could get improvements on the diffing later too 🙂
With the normal for-tpl, I would filter the list outside of the loop, then feed the filtered results in. But my understanding is that each iteration of any outside filtering, generates a whole new list in memory under the hood. Even if you only add/remove one thing? There's no sharing, and potentially a memory overload if you have too many filterings going on.
depends how you do it
subvec for example uses structural sharing, so it's O(1)
but it's not really a filter per se
calling filter
yes, that will make a new list each time
actually tbh i'm not even 100% sure of that, lol
I've been using Specter a lot, so not sure what's going on under it's hood.
part of clojure's immutability is they do stuff "under the hood" to make things faster
even if filter
currently makes a new list, it might not in a future version...
i'd be surprised if the making of a new list was causing problems
Like changing [:a :b :c]
to [:a :c :b]
results in 2 lists in memory?
i dunno, i think "it depends"
i think you have to research structural sharing in clojure to find out
Hmm. Ok I assumed some clojure immutable magic was going on. It only just looks like I'm duplicating all over the place.
yeah exactly
i think there's a lot of magic making it safer than it looks 😛
regardless, you can use transducers if you're worried about this
i highly recommend profiling before you go down this road though, it sounds like a micro optimisation to me
e.g. i'd expect the process of walking the list and applying a predicate to every item to decide what to keep is going to be riskier for CPU than the mechanical process of building the resultant list
even if you only remove one item, filter still needs to check every item
What do you think of the cascading formula cells to filter a list, (per above meta-code), before feeding to for-tpl. If I change the params around I get a lot of different versions of the list. But it seemed like you didn't think this would necessarily lead to a memory explosion in the for-tpl.
i do things like that reasonably regularly
i also use tools that have a query language regularly
inside the for-tpl
(at least my proposed versions) i don't think there's much you can "blow up"
the for-tpl
only sees the final output of your prior manipulations, and then builds item cells based on the key/position of the items list, and maps them to DOM elements based on the key/position
this is where i got to last night with a simplified for-tpl
:
(defn loop-tpl*
[items tpl]
(let [els (cell [])
itemsv (cell= (vec items))
items-count (cell= (count items))]
(do-watch items-count
(fn [_ n]
(when (< (count @els) n)
(doseq [i (range (count @els) n)]
(swap! els assoc i (tpl (cell= (get itemsv i nil))))))))
(cell= (subvec els 0 items-count))))
there's not really a lot to break there...
and walking a list and applying a pred to filter is O(n) so it's not going to "blow up", just get linearly slower as you add more stuff to the list
"i'd expect the process of walking the list and applying a predicate to every item to decide what to keep is going to be riskier for CPU than the mechanical process of building the resultant list. Even if you only remove one item, filter still needs to check every item" I assume this is what's bogging me down more than anything. How can I avoid this constant walking (event to just pick out a single item with a given UUID)? That's why I was thinking I needed to use the implicit vector index. But the react article seemed to warn against that?
ps. I haven't had a chance to get into Datascript yet 😛
well you can't is the short answer
but looking forward to it
because if you have any downstream local state
you'll just need to do the same calculations there, but probably more than once
using my example from the issue i raised
(let [state (cell {})] ; <-- note the additional state _outside_ the defn
(defn my-item
[item]
(let [k (cell= ...) ; <-- get an id from item
expand? (j/cell= (get state k) (partial swap! state assoc id))] ; <-- e.g. javelin readme
...)))
(for-tpl [x my-list] (my-item x))
we're now just applying keyfn
ad-hoc and using it to lookup values from a state cell per DOM element
instead of doing it once up front and letting the DOM elements downstream take a consistent view of an item without additional accounting
essentially there's no such thing as "changing a list" in clojure, as lists are immutable values
you make a new list, and if there's any relationship you want to preserve between list A and list B then you need to make that explicit
which is going to involve a walk somewhere
might as well make sure it's only 1 walk and not 50
Yes, ok. I'm in the 50 walks camp right now, lulled by the idea that by mutating a named cell I have a "consistent" list.
you can use a for-tpl
and get some (probably minor) speed benefits if:
- your downstream DOM elements are totally read-only, not even a11y browser interactions are allowed, like focus states
- your downstream DOM elements have no internal state that references the identity/key of an item
I don't understand that last one. Can you give an example?
^^ like the expand?
above
like "read more" with a popdown showing more info
you want that to be tied to an item, not a DOM element
if i click "read more" on item 1, then reverse my list, after rearranging i want the last item to be expanded, not the first
i guess i'm just saying the same thing twice 😛 which is that you've got internal state of elements to worry about, which could either be cljs/cell state or browser internal state
I really don't want to worry about browser internal state if can avoid it. So by "item" you mean the item map, presumable contained in a cell. So If I have a "read more" link, I walk that cell for the "more" data, to feed to the view (which might create a new dom element)...?
ah so for the purpose of what we're talking about here there is very little difference between a keyed and non-keyed for-tpl
a (cell [ ... ])
goes in, gets turned into [(cell= ... ) (cell= ... ) (cell= ... )]
or {k1 (cell= ... ) k2 (cell= ... ) k3 (cell= ... )}
actually that's not quite right lol
hmmm oversimplified too much 😛
it's actually more like
(cell [ ... ])
goes in, then we check to see if there is an appropriate DOM element for each item, if there is not then we make one like (my-el (j/cell= (get items k nil)))
when you change the items list, we just check to see if new individual item cell/DOM mappings need to be made
if you provide a sensible key to keyed-for-tpl
you won't need to worry about browser state getting out of sync with your uuids
my point is just that even though there's some internal stuff going on to make that happen, you can't make it faster by avoiding it because either A. you'll have to do equal or greater work downstream or B. you will have subtle bugs caused by stale/inconsistent state
I think I'm doing what you talked about earlier w/ respect to downstream identity:
(defc state 1)
(defc items [{:id 1} {:id 2}])
(for-tpl [x @items]
(elem :color (cell= (when (= (:id @x) @state) :red))
(defc items [{:id 1} {:id 2}])
(keyed-for-tpl nil :id [x items]
(elem :colour (cell= (when (= (:id x) 1) :red)))))
^^ looks like this with a key
ignore the first nil
in keyed-for-tpl
for now 😛
but what's also important, and is hard to see from this example, is that often these things are in different parts of your codebase
your for-tpl
might be in some ns calling elem
from another ns, and elem
might also be called from multiple different places, so it makes it tedious to track all this state across multiple places and contexts
you really don't want your elem
to be "reaching out" to global state
in order to get the job done 😕
I think I see. But how do you change the 1 (state value) in your example?
you can make it local to elem safely
it's really
(defn my-item
[item]
(let [expand? (cell false)]
...))
(keyed-for-tpl nil :id [x my-list] (my-item x))
(let [state (cell {})] ; <-- note the additional state _outside_ the defn
(defn my-item
[item]
(let [k (cell= (:id item)) ; <-- get an id from item
expand? (j/cell= (get state k) (partial swap! state assoc id))] ; <-- e.g. javelin readme
...)))
(for-tpl [x my-list] (my-item x))
the boilerplate in the latter needs to be repeated for every single cell that i want to represent state for a given item
I think I'll have to play about a bit to fully appreciate all this. Right now it's still a bit fuzzy cause
(defc items [{:id 1} {:id 2}])
(for-tpl [x items]
(elem :colour (cell= (when (= (:id x) 1) :red)))))
looks almost the same as
(keyed-for-tpl nil :id [x items]
(elem :colour (cell= (when (= (:id x) 1) :red)))))
But I understand that this example is overly stripped down.well actually your example works in both for-tpl
and keyed-tpl
😛
because your cell attached to :colour
is using part of the value of item
that works
expand?
is different because you can't read it from item
it is related to item
but it isn't part of that data structure
more like this
(defn elem [item]
(let [toggle? (cell false)]
(div
:color (cell= (if toggle? :red :blue))
:click #(swap! toggle? not))))
(defc items [{:id 1} {:id 2}])
(keyed-for-tpl nil :id [x items]
(elem x))
when you click the elem it swaps between red and blue
so if i click :id 2
it goes red, if i then reverse the list so that :id 2
is at the start of the list, do i want the first DOM element or the second one to be red?
for-tpl
would keep the second element red, so :id 1
would be red now
keyed-for-tpl
would make sure the item with :id 2
is red, so the first element would be red now
I think I see now. I think in that situation I would have tried to create a permenant {:id 1 :toggled? true} attribute to get/set/test. But that seemed verbose and redundant and more walking..😬 Using that local state looks a lot more elegant.
yes of course you can keep injecting all this state into item
but you're going to end up with dozens or hundreds of very similar looking keys quite fast >.<
and none of those extra keys really add value to item
tbh
toggled?
has nothing to do with the fundamental definition of the item
, it has to do with the internal state of a particular DOM element that is rendering that item in a certain way
Yes, that's the feeling I was getting. My semantic data was getting littered with view-state complexity! I want to compute the view stuff implicitly, but was searching for an efficient means.
Well thank you for taking the time to explain it in depth. This will help me tremendously. My app is all about filtering and sorting a list, and this clarifies where I need to head, and free's me from some gloom about feeling I was plodding down a dead end path.
I am currently trying to update to Hoplon 7.1 but getting some .spec errors. Do I need 7.1 to use the keyed-for-tpl? or could I plug it into 7.0.3?
actually keyed-for-tpl
is not available in anything yet 😛 waiting on @flyboarder to push a snapshot when he's available 🙂
but it would be in 7.2-SNAPSHOT
getting ready for the 7.3
release
what errors are you getting from spec?
I got the one about serializing the error message, but I was able to follow your trick of inserting (prn) code into boot.cljs
ah yeah, well that error just hides another error
now I'm getting stuff like
{ :cause
"Call to clojure.core/defn did not conform to spec:\nIn: [1 0] val: {:keys [url open-new?], :or {:open-new? false}, :as attr} fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-n :bodies :args] predicate: vector?\r\nIn: [1 0] val: ({:keys [url open-new?], :or {:open-new? false}, :as attr} kids) fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: (cat :args (* :clojure.core.specs.alpha/binding-form) :varargs (? (cat :amp #{(quote &)} :form :clojure.core.specs.alpha/binding-form))), Extra input\r\n"
:data
{:clojure.spec.alpha/problems ,
(
{:path [:args :bs :arity-1 :args], :reason "Extra input", :pred (clojure.spec.alpha/cat :args (clojure.spec.alpha/* :clojure.core.specs.alpha/binding-form) :varargs (clojure.spec.alpha/? (clojure.spec.alpha/cat :amp #{(quote &)} :form :clojure.core.specs.alpha/binding-form))), :val ({:keys [url open-new?], :or {:open-new? false}, :as attr} kids), :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/args+body :clojure.core.specs.alpha/arg-list :clojure.core.specs.alpha/arg-list], :in [1 0]}
{:path [:args :bs :arity-n :bodies :args], :pred clojure.core/vector?, :val {:keys [url open-new?], :or {:open-new? false}, :as attr}, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/args+body :clojure.core.specs.alpha/args+body :clojure.core.specs.alpha/args+body :clojure.core.specs.alpha/arg-list :clojure.core.specs.alpha/arg-list], :in [1 0]})
:clojure.spec.alpha/spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x13d54144 "clojure.spec.alpha$regex_spec_impl$reify__2436@13d54144"], :clojure.spec.alpha/value ,
(link [{:keys [url open-new?], :or {:open-new? false}, :as attr} kids] (elem :td :underline :tc :blue :m :pointer :click (fn* [] (.open js/window url (if (not open-new?) "_self"))) (dissoc attr :url :open-new?) kids))
:clojure.spec.alpha/args
(link [{:keys [url open-new?], :or {:open-new? false}, :as attr} kids] (elem :td :underline :tc :blue :m :pointer :click (fn* [] (.open js/window url (if (not open-new?) "_self"))) (dissoc attr :url :open-new?) kids))}
:via
[{:type clojure.lang.ExceptionInfo
:message
"src\\elements\\misc.cljs [line 90, col 1] Call to clojure.core/defn did not conform to spec:\nIn: [1 0] val: {:keys [url open-new?], :or {:open-new? false}, :as attr} fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-n :bodies :args] predicate: vector?\r\nIn: [1 0] val: ({:keys [url open-new?], :or {:open-new? false}, :as attr} kids) fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: (cat :args (* :clojure.core.specs.alpha/binding-form) :varargs (? (cat :amp #{(quote &)} :form :clojure.core.specs.alpha/binding-form))), Extra input\r\n"}]}
But haven't had a chance to dig into it yet.{:keys [url open-new?], :or {:open-new? false}, :as attr}
Call to clojure.core/defn did not conform to spec
yeah looks like that's clojure itself complaining about some of your syntax
you probably just got bumped up a few versions of cljs to support 7.1
Yeah, I think so. Hopefully I wont hit a wall with it 😅
My current codebase is in Hoplon.UI, which hasn't been updated in a while. I'm not sure If i can update to 7.2 if there are breaking changes, at least until a major refactoring. Can I cut and paste the keyed-for-tple stuff on top of a 7.1 base. Or does it depend on 7.2 internals?
well if you get stuck let us know
yeah you can paste it in
the only thing is that it is so new that it will probably evolve a bit
so don't get too attached to the version you paste
in theory there aren't breaking changes in 7.2
, it was a big refactor internally but the idea was to keep the external APIs the same or extended with new things only
Sure. I think anything might help me out right now. Especially just learning how to orient to that keyed internal state you so eloquently demonstrated.
Ok I'll give it a try
What about using a 7.0.3 base, just in case?
ah actually hmmm
i lied
it won't totally work without the new version of merge-kids
i forgot about that
since for-tpl
never re-arranges anything, hoplon doesn't actually support re-arranging >.<
i had to add that support while i was working on keyed-for-tpl
which also sucks for me because it means i can't use it either until the snapshot goes up, lol
on that note i think i'm going to go get a ☕
don't stress too much about 7.1
-> 7.2
though, i don't see that being a major hurdle at all
Ha. Lifestyle of the bleeding edge! I'll probably refactor and catch up to 7.2/7.3 asap either way. Thank you for the help, and all your recent work!!!
np, glad to help
and also it's great if other ppl start using what i'm writing so i can get feedback and make sure its rock solid
Sure. I'll test and give some feedback as soon as I'm able. Later
:thumbsup: catch
@thedavidmeister I’ll get it in snapshot soon I promise
Nice explanation! This is a great enhancement!
@flyboarder hah, all good, i've got plenty of other things to keep me busy 😉
i think this also fits well with the "easy interop with 3rd party libs" principle, as they will often have their own internal state that hoplon shouldn't damage or need to know about - imagine a WYSIWYG editor attached to an item, we don't want to be thrashing a setData()
method on every reorder, that could be very expensive as it would internally trigger a parse/render cycle on the editor for every item
@chromalchemy https://www.youtube.com/watch?v=6mTbuzafcII <-- good watching if you're concerned about lots of list rebuilding
you're basically doing what he describes at around 6:00
Hi all. I am new to hoplon as well as clojure(script) so please excuse my naive question
I am trying to get the value of a plain text input field and create dynamically a number of text input fields based on the number I will get. I am not sure how to get this number.
(let [c (cell 0)]
(input :value c :input #(reset! c @%))
(for-tpl [i (cell= (range (int c)))]
(input)))
@corelon ^^ something like this