Fork me on GitHub

does reveal makes use of datafy/nav protocols?


I see a nav option in the menu, but that doens't seem to be using the nav protocol


Yes, it does. I use it quite a bit with next.jdbc results.


Unlike REBL, it doesn't datafy things automatically, but when you choose nav, it calls datafy and then nav.


(because you can only nav things that have been datafy'd)


kind of, but I think I'm finding a bug in reveal


I'm looking at the nav action definition


(defaction ::nav [x {:vlaaad.reveal.nav/keys [coll key val]
                     :or {key ::not-found
                          val ::not-found}}]
  (let [datafied-coll (d/datafy coll)]
    (when (= datafied-coll coll)
        (not= key ::not-found) #(d/nav datafied-coll key x)
        (not= val ::not-found) #(d/nav datafied-coll x val)))))


and after some debug, I notice the :vlaaad.reveal.nav/key comes with numbers for vector items, but comes blank for map entries


trying to understand, maybe its because Im using a custom map type


still digging on it


That does look a bit weird... why would the datafied coll be equal to the original coll?


I guess to make sure its datafied


that check is to define if the action itself must be presented or absent


That makes no sense. datafy can produce something completely different from the underlying data.


but the datafy of datafy should always be the same value, right?


Er, no. Are you saying it's calling datafy twice?


I believe the intention of this check is to only offer "nav" when Reveal thinks the data is datafied, and one way to check that is calling datafy on it again, I guess the property is true: (defn datafied? [x] (= x (datafy x)))


but I start doubting my logic now


yeah, maybe the check should be about the parent implementing Datafiable


Lots of things can implement Datafiable


It looks like @U47G49KHQ has changed this at some point and now datafy is offered as a separate action, at least sometimes.


(find-ns 'user) produces a value that can be datafy'd for example


I'm trying to create my own custom action


but I'm getting confused with the meta-data provided by Reveal, it seems incomplete for properly call nav


Yeah, try this:

(with-meta {} {`p/datafy (fn [x] (with-meta {:a 1 :b 2} {`p/nav (fn [coll k v] (if (= :b k) (* 2 v) v))}))})


Then right-click > datafy it. Then go to :a in the result and right-click > nav -- you'll get 1, then go to :b in the result and right-click > nav and you'll get 4.


So it works, but the code in the nav action is pretty counter-intuitive.


I just got it working


the problem is related to lazy navigation


with the current nav impl it doesn't properly use the nav protocol to find lazy data, as far as I understood


I got it to work with this action:


(rx/defaction ::my-nav
  [x {:vlaaad.reveal.nav/keys [coll key val]
      :as m
      :or                     {key ::not-found
                               val ::not-found}}]
  (when (satisfies? clojure.core.protocols/Navigable coll)
    (let [source (-> coll meta :clojure.datafy/obj)]
       (not= ::not-found val)
       #(d/nav source x val)

       (not= ::not-found key)
       #(d/nav source key (get coll key))))))


its similar, but note in my version I use the nav in the source object (which comes as part of the datafied data meta)


That's not right.


this way, the nav can return data that is different from what datafy returned


allowing for lazy navigation


You're trying to nav on the original object which is wrong.


You should nav on the result of datafy.


let me try to bring my situation, maybe it will make more sense


In a lot of real world cases, datafy is what adds the nav protocol implementation.


You can datafy some arbitrary thing and the result will be nav'able.


the problem is that, how do I datafy without doign recursive explosion?


in my case, I have a DS that's a map like, but it has some cached data, and some unrealized data


if I realize everything on datafy, it causes stack overflow before it goes infinite recursion


datafy is a no-op on most things but you must call datafy first before nav


so, I need to return something to say that those keys exist, but I can't evaluate then ahead of time


maybe I'm using it wrong, but this is the only way I found to be able to know the navigatable keys, but only realize on nav


You are doing it wrong 🙂


is there a way to solve my problem "doing it right"?


Explain your problem again...


I have this data structure that contains cached data, and some keys that are "possibly available"


like a datomic entity, where a read may or may not cause an index download, but in my case I have an arbritary number of keys, that may return other complex data in it


So datafy should produce a pure data representation that has those keys but doesn't have to realize them... so they're placeholders that can be displayed as pure data.


my goal is to make a navigatable environment on this special map, so during datafy I need to expose all possible keys, but can't evaluate all of them (only the cached, that I know its a limited space, the "possible" is bound to infinity)


yeah, that's what I'm doing


I return a special keyword on those cases


ah, I think I may realized the solution, use metadata on the values instead of the protocol on the parent type


Have a look at how next.jdbc does it


It does lazy navigation through a database based on assumed foreign keys.


datafy adds the nav protocol implementation that does the DB fetch when invoked.


If you're in doubt, fire up REBL and try it in that: it definitely does datafy/nav "the right way" so if your impl doesn't work in REBL, it's not right.


Reveal is... a bit odd in that respect...


for some reason REBL is crashing my process when I try to get it up here


ok, I got the nav to work as expected now, thanks for pointing the direction @U04V70XH6 🙏


Morning! (at least it’s morning here in Sweden)


A lot to unpack here, but I’m glad you resolved your issue @U066U8JQJ!


I can describe how datafy/nav works in Reveal :) When maps are printed to Reveal, map keys are annotated with corresponding values, and values are annotated with corresponding keys, and both keys and values are annotated with collection they are in. That provides all necessary information for actions to do nav (coll, k, v) when some key OR value is selected: the counterpart will be there. The relevant code is here: Reveal also performs datafy-then-nav when it suggests nav action, but there is a twist: datafied data structure might be substantially different from the original one. For example, entity map from datomic might have only preloaded keys in its default form, but perform a request to load all its keys when datafied, so we can then lookup related values from those keys:

;; original entity:

{:db/id 123}

;; datafied entity:

{:db/id 123
 :user/name "fred"
 :user/orders "not loaded, nav this key to load"}
Reveal prints original undatafied map, and with that map it’s impossible to nav to :user/orders key because it’s not in the map — it appears when datafication happens. That’s why Reveal’s nav action does not suggest itself if datafication is different: you are missing out on other keys. Instead, you should select the coll and execute datafy action to get those keys, and only then nav in the returned data structure. This is why Reveal suggests datafy action only when datafication is different, by the way — if it’s the same, you can just nav in the printed data structure.

👍 3

Great explanation! Thanks, @U47G49KHQ!


is there a way to make the maps print in reveal with the keys sorted?


I was considering doing that, but was worried it might give users an incorrect view of their program: they might get an impression all maps are always sorted. I usually use eval-on-selection and write sort when I want something sorted


I think would be nice to have the option, because for example if I try to to do this with a Nav object, then I can't navigated on the sorted list, and I quite like the view of regular map on sorted. by personal experience with Fulcro Inspect I know that I'm mostly happier having then sorted, would you consider having this as a configurable thing?


you can have custom meta key for your data structure and make that sort the keys at the point of printing


I think it would be better to have it for everything, I guess its a preference on the user side, in my case I normally deal with quite large maps, unsorted keys is really painful with those


could be a pref like the Theme on Reveal, but I would like to apply to every map by default


Putting it into pref is a good idea, thanks! I’ll give it a thought


No, but you can right-click > table view and then sort keys or values


or Space, type sort, Enter


(my :dev alias in my dot-clojure repo has an auto-table-display function built-in)


another thing on maps, had you consider the option to collapse map values (and sequences)?


I thought about that. There is no decision yet, but table view is semantically pretty close to collapsing values to get overview of keys/items, and structural navigation allows fast jumping between values in the output panel, so that's pretty low priority for me, but I keep it in mind/todo list


one bug I see on the table view is that the keys column content doesn't resize when you resize the column


and since I use quite large keys that's also a problem


Hmm, can you show how it looks?


not right now, but I can send later


in summary, it keeps "..." at the end of the keys


and when I expand the column size, they remain as "..."


Ah, now I understand! I probably should bump the max length of printed object

👍 3

I often found myself wishing map keys were sorted as well — I’ve been learning how Fulcro and Fulcro RAD work, trying to make sense of the huge env maps being passed around. It would have been easier to make sense of the database connections and configs had they been next to each other. Funnily enough, I was about to make the same suggestion as @U066U8JQJ about not truncating the keys, because it was impossible to distinguish those two keys! 😂😂 Thanks for all your great work on Reveal!


@U066U8JQJ I released 1.1.178 that bumped text max length in tables a little. Ideally that could be done as a preference, but it’s a bigger task

🙏 3

@U6VPZS1EK My pleasure 😊 Unfortunately, not truncating the keys is not possible today, since keys can be so long (e.g. (range 1000000)) that trying to show them as JavaFX text nodes will kill UI performance. Maybe one day I’ll be able to implement some sort of intelligent text nodes that render themselves on-demand, but it’s a big task-small payoff, so maybe later ¯\(ツ)


Hey there! I just released 1.1.177 that is almost ready to be 1.2.* release with the UI improvements in the area of result navigation: • now you can have multiple result panels by pressing Ctrl+Enter in the action popup — this will make comparisons between different results/views easier; • switching between tabs is more discoverable (there are buttons with tooltips); • in every result panel you can see a result tree that shows all items in this panel hierarchically (available with Ctrl+Up); I'll fix some minor issues and then will announce it as 1.2.*

🎉 15

Very nice! Just in time for my presentation tonight, where I'll be showing off my workflow which includes Reveal!