Fork me on GitHub
#membrane
<
2022-03-23
>
Richie00:03:03

A Rectangle "knows" how large it is. i.e. I can get the bounds of a rectangle without a backend. On the other hand, I require a backend to get the bounds of a Font. Some things like Translate, Rotate, Spacer, and Padding compose against the bounds of the child and do not require a backend themselves. i.e. (padding 5 (rectangle 10 10)) does not require a backend and I can still get the bounds: [20 20]. LabelRaw does require a backend to know its bounds. Are there other elements that require a backend or just font? What benefits would we get out of being able to render the ui without a backend? e.g. testability? I would test my event handling with a call to mouse-down, right? Is it true that I can't do that unless I have a backend since I need the bounds to find what element the mouse [x y] maps to? Is there a better way to get the benefits? e.g. simulating a mouse down with something other than [x y]. Can we use bounds to select a font size rather than using a font size to determine bounds?

phronmophobic01:03:31

There's a lot to unpack here, so I'll answer the easy questions first. > Are there other elements that require a backend or just font? For elements defined in membrane.ui, the elements that require a backend are membrane.ui/image and membrane.ui/label. Generally, it's any element that requires an external library to measure. Since anyone can implement a new element type, there's no limit to the number of elements that might require a backend.

phronmophobic01:03:57

> What benefits would we get out of being able to render the ui without a backend? e.g. testability? There's lots of reasons to want to be able to inspect views without drawing them. Testing is a big one, but here's some other examples: • analysis: eg. automatically determine if a user interface conforms to a style guide or accessibility standards • pre-rendering: You may want to partially or completely pre-render some part of the UI • optimizations: You may want to transform or do some calculations at compile time to improve performance • debugging • diagnostics, statistics, or benchmarks on views • generating sample data: since views may be inputs to other functions/libraries, you may want to dynamically generate sample UIs using spec The idea is to have a data oriented approach for all the flexibility and benefits that a data oriented approach provides.

phronmophobic01:03:57

> Can we use bounds to select a font size rather than using a font size to determine bounds? That's not what label does, but you could create a bounded-label that does exactly that. You could also wrap label with membrane.ui/fixed-bounds

phronmophobic02:03:00

> I would test my event handling with a call to mouse-down, right? That's one way to do it and it's the preferred method, but there are other options. • Views are data, so you could write your own implementation of mouse-down that just operates on the views as data. • You could provide your own implementation of IBounds for Label and Image similar to how the backends do it. • You could use with-redefs to provided alternate implementations for either IBounds or IMouseEvent that is completely separate or reuses the existing implementation with some small differences.

phronmophobic02:03:26

There's nothing special about the default mouse handler (or any of the event handlers). the skia backend makes it easier to provide a custom handler for input events (https://github.com/phronmophobic/membrane/blob/master/src/membrane/skia.clj#L1886), but the same idea could be applied to any of the backends.

phronmophobic02:03:46

I think membrane does a decent job of taking apart the pieces required for building user interfaces compared to analogous options like GTK, Swing, JavaFX, HTML, etc. However, I think there's still lots of room for improvements to break things into smaller pieces and simplify further.

phronmophobic02:03:05

> Is it true that I can't do that unless I have a backend since I need the bounds to find what element the mouse [x y] maps to? Sorta. There's multiple options here too. It seems unlikely that you wouldn't have a backend since there's a backend based on java2d which ships with the jvm and there's an implementation that runs in the browser. If for some reason you really couldn't use one of the builtin backends, you could provide your own. It's ok to implement only the parts you need. There's also options if there were some reason you really couldn't use any backend. > Is there a better way to get the benefits? e.g. simulating a mouse down with something other than [x y]. Most of the time, just plugging in random [x y] pairs would be fairly inefficient. Most mouse-event handlers don't even care what the actual coordinates are and only whether or not the coordinates are within the bounds of the handler's element. Instead of using membrane.ui/mouse-event, you could use membrane.ui/children to walk the view tree and call membrane.ui/-mouse-event in a depth first search to see which view elements respond with intents. An idea for a future improvement might be adding a click event. Just like mouse-down and mouse-up are pseudo events derived from the more generic -mouse-event, it might worth having a click pseudo event that derives from mouse-event and explicitly signifies that the handler doesn't care about the specific [x y] coordinate.

phronmophobic02:03:25

> Is there a better way to get the benefits? e.g. simulating a mouse down with something other than [x y]. Repeating the question for this second answer. The separation between event handlers and effect handlers also provides another hook for taking things apart for testing. Depending on what you're trying to test, you can skip producing a view altogether and just generate intents and feed those directly into effect handlers

phronmophobic03:03:34

A related note is that IBounds is overloaded. It tries to do too many things. It both represents hitboxes as well as bounds for painting which aren't necessarily the same. In the future, I would consider breaking IBounds into simpler pieces.

Richie11:03:47

Thanks! I really was struggling to think of benefits beyond testing but I figured there had to be more to it. There's a lot to unpack in your response as well. It helped me understand that there's still much that I can do even without knowing the font bounds.

phronmophobic17:03:43

Why don't you know the font bounds?

Richie17:03:10

Sorry, I mean that I have to ask a backend for the bounds of a font.

Richie01:03:21

Can you give an example difference between hitboxes and bounds? Would I use a hitbox larger than the bounds to be forgiving of a close mouse click?

(defrecord Rotate [degrees drawable]
  IOrigin
  (-origin [this]
    [0 0])

  IBounds
  (-bounds [this]
    (bounds drawable)))
Maybe bounds is axis aligned whereas the hitbox rotates.

phronmophobic02:03:49

What I was trying to say is that "bounds" isn't even a meaningful concept unless you qualify it further. Currently, the IBounds interface is being used for at least two different concepts (maybe more): • visual bounds • hitbox bounds A simple example when visual bounds and hitbox bounds differ is if you add a dropdown shadow to an element. Most of the time, the shadow isn't part of the hitbox. As you mentioned, Enlarging the hitbox of an element to make it easier to interact with is also an example. Even the concept "visual bounds" is overloaded. You might want to distinguish between the general area an element looks like it occupies (for comparing your UI with a style guide) and separately having a visual bounds that specifies the extent that the element will never draw outside of (for draw call optimization). There are also useful concepts that are completely missing from membrane.ui's model (like convex hull). If I ever make a membrane.ui2 model, I'll probably build it on a foundation of 2d/3d geometry. It seems like a no-brainer in retrospect, but it's shocking that geometry is almost never the foundation for UI frameworks outside of games.

Richie03:03:42

Thanks, that's enlightening. Do you know any 2d/3d geometry libraries that would be a good fit?

phronmophobic21:03:18

I haven't had a chance to fully investigate this route, but https://github.com/thi-ng/geom and https://github.com/sicmutils/sicmutils are on my list

zimablue17:03:50

re Richie's thread, but maybe worth pulling out, I implemented something that does what you were saying about bounds to select a font size rather than font size to determine bounds, for @smith.adriane maybe it's quite a common request. Not sure my implementation is any good, it was fiddly to write.

zimablue17:03:28

seperate question, the restriction that "membrane components must be called with a literal map" seems quite restrictive, or maybe I just haven't thought of a nice pattern to code around it yet

phronmophobic17:03:39

There are some improvements that can be made here. Do you have an example use case so I can make sure I'm thinking of the same problem?

zimablue17:03:20

apologies I've been away from the UI bit for a couple of weeks and just got back to it, I saw some hacks in a few places where I'd worked around it but I might not be fresh on the "real" problem I had, one was when "parents" want to call "child" components with a map which is a value in the map the parent has been passed in, so you're calling (my-child-component my-value) which is not a literal map?

zimablue17:03:36

sorry if that's stupid like I said I'm only back to ui code now

phronmophobic17:03:48

I think that's the same problem I'm thinking of.

zimablue18:03:36

the other pattern than I stumbled into immediately with the first app i'm trying to write is that you have a global "layout" that calls other highest-level components so you want to pass them the SAME map that your top-level component is dispatching off, you could work out which need which keys but it's not the first thing I'd intuitively do

phronmophobic18:03:44

Yea, that doesn't currently work, but there's no technical limitation that would prevent implementing defui to allow that.

phronmophobic18:03:30

That's definitely a use case I want to support.

phronmophobic18:03:58

I've been meaning to revisit defui to make that improvement as well as few other ideas.

zimablue18:03:16

Thanks so much for this library! I'd love to take defui apart and think about what other options there would be for component models, the first thing I was thinking to try myself was to see if I could de-macro-ify "defui" as much as possible, as in make it a macro that calls functions as much as possible. I haven't had the time to try that though, so far I just manually read the output of a defui call to try and understand it, that's when I thought that it looked like the actual "macro body" could be smaller by delegating to some functions

phronmophobic18:03:26

Yea, defui could definitely be broken down into smaller pieces. It also just outputs a lot of code. I've had some components run into jvm method size issues.

phronmophobic18:03:08

I have been happy that it can coexist happily with other component types since there's a pure function underneath

phronmophobic18:03:41

As long as everyone is creating pure functions, there should be multiple component options available that can happily coexist.

phronmophobic18:03:28

I'm not sure I would ever completely demacrofy defui since I've found the dependency tracking and bookkeeping to be really helpful and I'm not sure there's an alternative to macros to solve the problem.

phronmophobic19:03:18

Other improvements I've been thinking about: • views returned by defui components don't correctly implement assoc so you can't do (assoc (todo-view {:todos my-todos}) :todos my-other-todos), but that's something that could and should work • the analysis that defui does is useful and would be nice to expose a la carte • the component info is awkwardly stored on the component's var's meta-data • namespaced keywords aren't supported yet 😳

zimablue21:03:18

I didn't mean to demacrofy, I just meant make the macro body shorter by delegating to functions, especially where those functions don't move actual sexps around

👍 1
zimablue21:03:39

*to demacrofy completely

phronmophobic21:03:17

yea, that would be cool

zimablue21:03:17

I think my model of using the library will be to do a bit more manual juggling of the "refs" or "zippers" as they kind of are around manually, as what I'm trying to do is quite dynamic but I haven't tried for long enough to be certain of that

zimablue21:03:40

so the defui analysis would be pretty cool, and other stuff making it easier to understand that machinery

zimablue21:03:37

the component info is awkwardly stored on the component's var's meta-data I still don't really understand the difference between clojure and clojurescript here but does that mean that this HAS to be compile time in clojurescript, since it doesn't have "real" runtime vars (I think, as I say I may never understand this)

phronmophobic21:03:08

yea, clojurescript is tricky. sci is also another tricky target because it doesn't handle metadata well.

zimablue17:03:11

although actually I implemented it only for webGL which I think isn't very widely used

phronmophobic18:03:46

This is what the code looks like

😱 1
phronmophobic21:03:36

hopefully, that's not too terrifying

Ben Sless07:03:12

It's terrifying like a volcano. The impressive kind

Ben Sless07:03:43

Ofc what I really want here is the auto generating code