Fork me on GitHub
#reveal
<
2020-12-15
>
wilkerlucio02:12:11

does reveal makes use of datafy/nav protocols?

wilkerlucio02:12:52

I see a nav option in the menu, but that doens't seem to be using the nav protocol

seancorfield02:12:54

Yes, it does. I use it quite a bit with next.jdbc results.

seancorfield02:12:30

Unlike REBL, it doesn't datafy things automatically, but when you choose nav, it calls datafy and then nav.

seancorfield02:12:56

(because you can only nav things that have been datafy'd)

wilkerlucio02:12:01

kind of, but I think I'm finding a bug in reveal

wilkerlucio02:12:08

I'm looking at the nav action definition

wilkerlucio02:12:16

(defaction ::nav [x {:vlaaad.reveal.nav/keys [coll key val]
                     :or {key ::not-found
                          val ::not-found}}]
  (let [datafied-coll (d/datafy coll)]
    (when (= datafied-coll coll)
      (cond
        (not= key ::not-found) #(d/nav datafied-coll key x)
        (not= val ::not-found) #(d/nav datafied-coll x val)))))

wilkerlucio02:12:47

and after some debug, I notice the :vlaaad.reveal.nav/key comes with numbers for vector items, but comes blank for map entries

wilkerlucio02:12:55

trying to understand, maybe its because Im using a custom map type

wilkerlucio02:12:06

still digging on it

seancorfield02:12:22

That does look a bit weird... why would the datafied coll be equal to the original coll?

wilkerlucio02:12:34

I guess to make sure its datafied

wilkerlucio02:12:48

that check is to define if the action itself must be presented or absent

seancorfield02:12:00

That makes no sense. datafy can produce something completely different from the underlying data.

wilkerlucio02:12:14

but the datafy of datafy should always be the same value, right?

seancorfield02:12:46

Er, no. Are you saying it's calling datafy twice?

wilkerlucio02:12:32

I believe the intention of this check is to only offer "nav" when Reveal thinks the data is datafied, and one way to check that is calling datafy on it again, I guess the property is true: (defn datafied? [x] (= x (datafy x)))

wilkerlucio02:12:12

but I start doubting my logic now

wilkerlucio02:12:03

yeah, maybe the check should be about the parent implementing Datafiable

seancorfield02:12:25

Lots of things can implement Datafiable

seancorfield02:12:58

It looks like @U47G49KHQ has changed this at some point and now datafy is offered as a separate action, at least sometimes.

seancorfield02:12:19

(find-ns 'user) produces a value that can be datafy'd for example

wilkerlucio02:12:19

I'm trying to create my own custom action

wilkerlucio02:12:35

but I'm getting confused with the meta-data provided by Reveal, it seems incomplete for properly call nav

seancorfield02:12:09

Yeah, try this:

(with-meta {} {`p/datafy (fn [x] (with-meta {:a 1 :b 2} {`p/nav (fn [coll k v] (if (= :b k) (* 2 v) v))}))})

seancorfield02:12:52

Then right-click > datafy it. Then go to :a in the result and right-click > nav -- you'll get 1, then go to :b in the result and right-click > nav and you'll get 4.

seancorfield02:12:26

So it works, but the code in the nav action is pretty counter-intuitive.

wilkerlucio02:12:39

I just got it working

wilkerlucio02:12:49

the problem is related to lazy navigation

wilkerlucio02:12:07

with the current nav impl it doesn't properly use the nav protocol to find lazy data, as far as I understood

wilkerlucio02:12:12

I got it to work with this action:

wilkerlucio02:12:18

(rx/defaction ::my-nav
  [x {:vlaaad.reveal.nav/keys [coll key val]
      :as m
      :or                     {key ::not-found
                               val ::not-found}}]
  (when (satisfies? clojure.core.protocols/Navigable coll)
    (let [source (-> coll meta :clojure.datafy/obj)]
     (cond
       (not= ::not-found val)
       #(d/nav source x val)

       (not= ::not-found key)
       #(d/nav source key (get coll key))))))

wilkerlucio02:12:04

its similar, but note in my version I use the nav in the source object (which comes as part of the datafied data meta)

seancorfield02:12:13

That's not right.

wilkerlucio02:12:22

this way, the nav can return data that is different from what datafy returned

wilkerlucio02:12:29

allowing for lazy navigation

seancorfield02:12:32

You're trying to nav on the original object which is wrong.

seancorfield02:12:42

You should nav on the result of datafy.

wilkerlucio02:12:10

let me try to bring my situation, maybe it will make more sense

seancorfield02:12:16

In a lot of real world cases, datafy is what adds the nav protocol implementation.

seancorfield02:12:32

You can datafy some arbitrary thing and the result will be nav'able.

wilkerlucio02:12:37

the problem is that, how do I datafy without doign recursive explosion?

wilkerlucio02:12:54

in my case, I have a DS that's a map like, but it has some cached data, and some unrealized data

wilkerlucio02:12:17

if I realize everything on datafy, it causes stack overflow before it goes infinite recursion

seancorfield02:12:17

datafy is a no-op on most things but you must call datafy first before nav

wilkerlucio02:12:43

so, I need to return something to say that those keys exist, but I can't evaluate then ahead of time

wilkerlucio02:12:04

maybe I'm using it wrong, but this is the only way I found to be able to know the navigatable keys, but only realize on nav

seancorfield02:12:17

You are doing it wrong 🙂

wilkerlucio02:12:29

is there a way to solve my problem "doing it right"?

seancorfield02:12:46

Explain your problem again...

wilkerlucio02:12:06

I have this data structure that contains cached data, and some keys that are "possibly available"

wilkerlucio02:12:54

like a datomic entity, where a read may or may not cause an index download, but in my case I have an arbritary number of keys, that may return other complex data in it

seancorfield02:12:29

So datafy should produce a pure data representation that has those keys but doesn't have to realize them... so they're placeholders that can be displayed as pure data.

wilkerlucio02:12:40

my goal is to make a navigatable environment on this special map, so during datafy I need to expose all possible keys, but can't evaluate all of them (only the cached, that I know its a limited space, the "possible" is bound to infinity)

wilkerlucio02:12:52

yeah, that's what I'm doing

wilkerlucio02:12:58

I return a special keyword on those cases

wilkerlucio02:12:44

ah, I think I may realized the solution, use metadata on the values instead of the protocol on the parent type

seancorfield02:12:58

Have a look at how next.jdbc does it

seancorfield02:12:13

It does lazy navigation through a database based on assumed foreign keys.

seancorfield02:12:08

datafy adds the nav protocol implementation that does the DB fetch when invoked.

seancorfield02:12:40

If you're in doubt, fire up REBL and try it in that: it definitely does datafy/nav "the right way" so if your impl doesn't work in REBL, it's not right.

seancorfield02:12:49

Reveal is... a bit odd in that respect...

wilkerlucio02:12:56

for some reason REBL is crashing my process when I try to get it up here

wilkerlucio03:12:06

ok, I got the nav to work as expected now, thanks for pointing the direction @U04V70XH6 🙏

vlaaad08:12:14

Morning! (at least it’s morning here in Sweden)

vlaaad08:12:39

A lot to unpack here, but I’m glad you resolved your issue @U066U8JQJ!

vlaaad08:12:59

I can describe how datafy/nav works in Reveal :) When maps are printed to Reveal, map keys are annotated with corresponding values, and values are annotated with corresponding keys, and both keys and values are annotated with collection they are in. That provides all necessary information for actions to do nav (coll, k, v) when some key OR value is selected: the counterpart will be there. The relevant code is here: https://github.com/vlaaad/reveal/blob/master/src/vlaaad/reveal/stream.clj#L202-L218 Reveal also performs datafy-then-nav when it suggests nav action, but there is a twist: datafied data structure might be substantially different from the original one. For example, entity map from datomic might have only preloaded keys in its default form, but perform a request to load all its keys when datafied, so we can then lookup related values from those keys:

;; original entity:

{:db/id 123}

;; datafied entity:

{:db/id 123
 :user/name "fred"
 :user/orders "not loaded, nav this key to load"}
Reveal prints original undatafied map, and with that map it’s impossible to nav to :user/orders key because it’s not in the map — it appears when datafication happens. That’s why Reveal’s nav action does not suggest itself if datafication is different: you are missing out on other keys. Instead, you should select the coll and execute datafy action to get those keys, and only then nav in the returned data structure. This is why Reveal suggests datafy action only when datafication is different, by the way — if it’s the same, you can just nav in the printed data structure.

👍 3
seancorfield18:12:10

Great explanation! Thanks, @U47G49KHQ!

wilkerlucio05:12:44

is there a way to make the maps print in reveal with the keys sorted?

vlaaad08:12:50

I was considering doing that, but was worried it might give users an incorrect view of their program: they might get an impression all maps are always sorted. I usually use eval-on-selection and write sort when I want something sorted

wilkerlucio13:12:53

I think would be nice to have the option, because for example if I try to to do this with a Nav object, then I can't navigated on the sorted list, and I quite like the view of regular map on sorted. by personal experience with Fulcro Inspect I know that I'm mostly happier having then sorted, would you consider having this as a configurable thing?

vlaaad14:12:16

you can have custom :vlaaad.reveal.stream/type meta key for your data structure and make that sort the keys at the point of printing

wilkerlucio15:12:40

I think it would be better to have it for everything, I guess its a preference on the user side, in my case I normally deal with quite large maps, unsorted keys is really painful with those

wilkerlucio15:12:56

could be a pref like the Theme on Reveal, but I would like to apply to every map by default

vlaaad08:12:47

Putting it into pref is a good idea, thanks! I’ll give it a thought

seancorfield06:12:10

No, but you can right-click > table view and then sort keys or values

vlaaad08:12:44

or Space, type sort, Enter

seancorfield06:12:40

(my :dev alias in my dot-clojure repo has an auto-table-display function built-in)

wilkerlucio17:12:41

another thing on maps, had you consider the option to collapse map values (and sequences)?

vlaaad17:12:59

I thought about that. There is no decision yet, but table view is semantically pretty close to collapsing values to get overview of keys/items, and structural navigation allows fast jumping between values in the output panel, so that's pretty low priority for me, but I keep it in mind/todo list

wilkerlucio18:12:32

one bug I see on the table view is that the keys column content doesn't resize when you resize the column

wilkerlucio18:12:45

and since I use quite large keys that's also a problem

vlaaad18:12:28

Hmm, can you show how it looks?

wilkerlucio18:12:57

not right now, but I can send later

wilkerlucio18:12:10

in summary, it keeps "..." at the end of the keys

wilkerlucio18:12:17

and when I expand the column size, they remain as "..."

vlaaad18:12:26

Ah, now I understand! I probably should bump the max length of printed object

👍 3
genekim05:12:11

I often found myself wishing map keys were sorted as well — I’ve been learning how Fulcro and Fulcro RAD work, trying to make sense of the huge env maps being passed around. It would have been easier to make sense of the database connections and configs had they been next to each other. Funnily enough, I was about to make the same suggestion as @U066U8JQJ about not truncating the keys, because it was impossible to distinguish those two keys! 😂😂 Thanks for all your great work on Reveal!

vlaaad08:12:25

@U066U8JQJ I released 1.1.178 that bumped text max length in tables a little. Ideally that could be done as a preference, but it’s a bigger task

🙏 3
vlaaad08:12:01

@U6VPZS1EK My pleasure 😊 Unfortunately, not truncating the keys is not possible today, since keys can be so long (e.g. (range 1000000)) that trying to show them as JavaFX text nodes will kill UI performance. Maybe one day I’ll be able to implement some sort of intelligent text nodes that render themselves on-demand, but it’s a big task-small payoff, so maybe later ¯\(ツ)

vlaaad22:12:42

Hey there! I just released 1.1.177 that is almost ready to be 1.2.* release with the UI improvements in the area of result navigation: • now you can have multiple result panels by pressing Ctrl+Enter in the action popup — this will make comparisons between different results/views easier; • switching between tabs is more discoverable (there are buttons with tooltips); • in every result panel you can see a result tree that shows all items in this panel hierarchically (available with Ctrl+Up); I'll fix some minor issues and then will announce it as 1.2.*

🎉 15
seancorfield22:12:51

Very nice! Just in time for my presentation tonight, where I'll be showing off my workflow which includes Reveal!