Hi Nathan. I'm new to specter but this seems indeed like the bees knees

I have a question, a while back I posted the following s.o. question: http://stackoverflow.com/questions/42551765/idiomatic-and-concise-way-of-working-with-nested-map-vector-structures-in-clojur and I have been playing for the past few hours with using spectre for this nested traversal problem. In essense I have some json data (maps and vectors) and I need to traverse to a certain depth, do a few operations, remember context, traverse a few more levels down, do a few operations etc

I am not really interested in changing the traverse structure. I assume I should be using collectors then?

@mbjarland so to clarify, you're not transforming the nested structure, you just want to perform some operations using values inside?

this is the groovy equivalent of what I want to do:

json.issues.each { issue ->
  issue.changelog.histories.each { history ->
    def date = DateTime.parse(history.created)
    if (date < fromDate || date > toDate) return

    def timeItems = history.items.findAll { it.field == 'timespent' }
    if (!timeItems) return

    def consultant = history.author.displayName
    timeItems.each { item ->
      def from = (item.from ?: 0) as Integer
      def to   = (   ?: 0) as Integer

      timesheets[consultant].entries << new TimeEntry(date: date, issueKey: issue.key, secondsSpent: to - from)
    }
  }
}

and generate some resulting data structure (map)

it seems that the path traversals above json.issues.each, issue.changelog.histories.each, etc are a perfect match for specter

it sounds like you want to do a traverse to get to the values you care about in the order you want

and then reduce over that to generate your result

and multiple nested traverses to get hold of the different levels (issue, history, item) above?

use collect / collect-one

a simplified example:

(reduce conj [] (traverse [ALL (collect-one :name) :age] [{:name "Bob" :age 21} {:name "Alice" :age 20}]))

ok, that's probably enough to nudge me in the right direction

thank you! ...and clojure still makes me feel like an idiot at least once a day

I've been trying various methods of getting the groovy code above expressive and terse in clojure, I think specter might be the answer

your path would look something like: [:issues ALL :changelog :histories ALL (selected? :date #(> % fromDate) #(< % toDate)) (collect-one :author :displayName) :items ALL #(= (:field %) "timespent") (collect-one :from (nil->val 0)) :to (nil->val 0)]

: )

or something like that

this is by the way the json returned from a jira instance and we use the results for invoicing so it is an actual real world example

thank you, that helps a ton

cool

no problem

one more question, in your path the date comparison assumes the date is parsed

what if I need to do some computation at an intermediate level like parsing the date ?

just an inline function in the path expression?

a privilege to be able to get answers directly from the author and I have to say I'm impressed with the clojure community experience so far. Thanks again!

: ) and the more I look at the above path expression....this is beautiful!

@mbjarland you can insert (view parse-date) to parse before comparing

a function directly in a path is interpreted as a filter

ok got a last one, for context I have the following function:

(defn disect-json [json from-date to-date]
  (let [path [:issues ALL
              (collect-one :key)
              :changelog :histories ALL
              #(history-in-date-range? % from-date to-date)
              (selected? [:items ALL #(= (:field %) "timespent")])
              (collect-one (view #(parse-history-date %)))
              (collect-one [:author :displayName])
              :items ALL
              (collect-one :from (nil->val "0") (view #(read-string %)))
              (collect-one :to (nil->val "0") (view #(read-string %)))]]
        (group-by #(nth % 2)
          (reduce conj [] (traverse path json)))))

oh and the above works, so I’m happy as a clam after looking for months for a clean solution for this in clojure.

@mbjarland instead of collecting the last element, just navigate to it

terminate the path with :to (nil->val "0") (view #(read-string %))) and drop the collect-one

also you can do (view parse-history-date) to make it cleaner, same with usage of read-string

also if you just want a vector of results back, just do select instead of (reduce ... (traverse ...))

finally, you should specify the path inline with the select call so it can be properly optimized

or define the path local using specter's path macro

(let [json-path (path :issues ALL ...) ...