Fork me on GitHub
#specter
<
2017-04-20
>
nathanmarz00:04:57

@lvh I don't think specter will be very useful for that

albaker17:04:00

@nathanmarz is there a recursive-path that will not recurse into the transformed value? i.e. (recursive-path [] p [(walker vector?) (stay-then-continue ALL p)]) will go for a while, but I'm still finding something will stackoverflow... core.async, futures, apache client. I'm thinking I need to just run transforms again and again, until it no longer transforms rather than in one big recursive blitz

albaker18:04:16

in the stack overflow error, there's lots of repeating versions of this:

nathanmarz18:04:58

@albaker that path is defined so it will continue recursing

nathanmarz18:04:19

recursive-path itself doesn't do anything besides provide you the programmer the ability to have the path refer to itself

nathanmarz18:04:36

if it's stack overflowing, then you're doing something wrong in your program

albaker18:04:47

I'm okay recursing, just not into the results, so I recurse down to the last known genration, apply and end

nathanmarz18:04:03

I don't know what that means

nathanmarz18:04:21

whatever logic you want will need to be encoded in the path

nathanmarz18:04:44

you can use value collection to keep track of how many levels you've recursed

nathanmarz18:04:15

and then check that using collected? to have the path decide whether or not to recurse

nathanmarz18:04:41

or you can encode in your data itself information to allow the path to decide whether to recurse

nathanmarz18:04:43

e.g. using metadata

albaker18:04:00

yeah, something like that - right now I'm keeping a counter ref going

nathanmarz18:04:00

yea, that would work too

nathanmarz18:04:21

lots of options available to control termination

albaker18:04:53

there is a potential for a loop, and I thought I had caught it - but perhaps not

albaker18:04:07

there a way to force the transform with recursive path to bail out?

nathanmarz18:04:29

you mean with an exception or something?

albaker18:04:46

yeah, I guess just a regular exception and wrap the transform in a try/catch

nathanmarz18:04:15

I guess you could make a navigator which does that

nathanmarz18:04:33

try catch over invoking next-fn, and when your special exception is thrown it returns the structure unchanged

albaker18:04:07

ok, I'll take a look at that - thanks!

nathanmarz18:04:49

imo that's a dirty way to solve this and you're better off more precisely specifying when you want recursion to stop

nathanmarz18:04:00

it's using exceptions for flow control

albaker18:04:16

ah sweet, found the bug, took the exception out, and now have a near general purpose specter+stardog graph walker for recursively expanding graph data

albaker18:04:41

@nathanmarz thanks for all the help - this is really nice, with like 6 lines of specter, and a few lines of SPARQL, this thing walks and queries and walks again. I'll give a shot out in the blog that'll fall out from this

nathanmarz18:04:40

@albaker no problem, happy to help

jeremyraines18:04:58

I have a (maybe similar?) question. Happy to post on StackOverflow if I can formulate it correctly. I think the imaginary code I want may explain best:

(def TreeValues
  (s/recursive-path [] p
    (s/if-path #(not (empty? (:children %)))
      (s/submap [:id :name {{ set :children to [:children s/ALL p] }}  ])
      (s/submap [:id :name :children]))))

jeremyraines18:04:18

is this possible without an inner transform? Maybe with some use of collect that I haven’t hit on yet?

jeremyraines18:04:06

essentially it’s operating on a vector of maps, where each map has a :children key with a vec of the same type of map

jeremyraines18:04:15

I’m just trying to prune each level to the keys I want

nathanmarz18:04:48

submap isn't going to prune keys

nathanmarz18:04:18

it lets you operate on a portion of the map with the result being merged back into the original map

nathanmarz18:04:02

I think you want to use continue-then-stay

jeremyraines18:04:29

hmm, OK thanks. I got the notion of pruning because this is the closet I got, but it only returns the leaves

nathanmarz18:04:35

(recursive-path [] p (continue-then-stay :children ALL p))

jeremyraines18:04:02

(def TreeValues
  (s/recursive-path [] p
    (s/if-path #(not (empty? (:children %)))
      [:children s/ALL p]
      (s/submap [:id :name :children]))))

nathanmarz18:04:17

then you could do: (transform TreeNodes #(select-keys % [:id :name :children]) data)

jeremyraines18:04:42

ok, thanks - reading up on continue-then-stay and trying it out

jeremyraines18:04:39

that works, thanks!

jeremyraines19:04:43

I’m lacking some intuition though . . . I don’t understand why this works:

(transform [ALL TreeNodes] #(select-keys % [:id :name :children]) data)
but not this:
(transform [FIRST TreeNodes] #(select-keys % [:id :name :children]) data)

nathanmarz19:04:43

what's not working?

jeremyraines19:04:03

the latter does something, I’m not sure what, on the whole data vec. I thought it would be equivalent to:

jeremyraines19:04:30

(transform [ALL TreeNodes] #(select-keys % [:id :name :children]) (take 1 data))

nathanmarz19:04:59

no, it will just do the transformation on the first element of the vector

nathanmarz19:04:11

you'll still get the whole vector back, but everything beyond the first element will be untouched

nathanmarz19:04:06

transform always operates on a select portion of the data structure, with the path specifying how to navigate to what needs to be operated on

jeremyraines19:04:07

makes sense, I think I understood that but just dropped the context of the basics when struggling with this problem. Thanks