Fork me on GitHub
#specter
<
2017-02-10
>
bradford21:02:42

Hi all! I'm trying to get better at Specter. I'm doing a lot of parsing HAR files for metadata. I'm able to do what I want, but it feels...inefficient. like this:

(select [ALL #(or 
                   (re-find #"css" (get-in % [:response :content :mimeType]))
                   (and (re-find #"html" (get-in % [:response :content :mimeType]))
                        (re-find #"tff|woff" (get-in % [:request :url]))
                        )
                   (re-find #"font" (get-in % [:response :content :mimeType]))
                   (and (re-find #"javascript" (get-in % [:response :content :mimeType]))
                        (re-find #"js" (get-in % [:request :url]))
                        )
                   (and (re-find #"image" (get-in % [:response :content :mimeType]))
                        (re-find #"jpg|png|gif|bmp|ico" (get-in % [:request :url]))
                        )

                   )
         [:request :url]] f)

bradford21:02:50

Is there a cleaner way to do this?

bradford21:02:20

(I'm trying to extract URLs that match certain mimeTypes)

nathanmarz22:02:10

@bradford I think this is cleaner and it should be much more efficient as well:

(defn ^:direct-nav my-matcher [exp1 exp2]
  (collected? [part1 part2]
    (and (or (not exp1)
             (re-find exp1 part1))
         (or (not exp2)
             (re-find exp2 part2)))
    ))

(select [ALL 
         (selected?
          (collect-one :response :content :mimeType)
          (collect-one :request :url)
          (multi-path
            (my-matcher #"css" nil)
            (my-matcher #"font" nil)
            (my-matcher #"html" #"tff|woff")
            (my-matcher #"javascript" #"js")
            (my-matcher #"image" #"jpg|png|gif|bmp|ico")
            ))
         :request
         :url]
        f)

zane22:02:51

^:direct-nav?

nathanmarz22:02:14

minor optimization, not necessary

zane22:02:45

Is it idiomatic to use collect-one in that way if you're trying to, uh, introduce a variable that will be used by a downstream navigator?

zane22:02:32

Let me see if I can rephrase that question to make it clearer… :thinking_face:

zane22:02:40

I wand to use cond-path, but each of the cond-path tests begins with the same (view …).

zane22:02:13

What is the right way to reuse the (view …) results?

nathanmarz22:02:38

yea collect-one + collected? should do what you need

nathanmarz22:02:08

you can use DISPENSE to drop collected values later in path (so they don't reach your transform-fn, for instance)

zane22:02:34

Interesting. Let me give this a shot.

nathanmarz22:02:00

you can also use defrichnav to do more custom manipulation of collected values

nathanmarz22:02:16

look at definition of DISPENSE for an example

bradford22:02:12

@nathanmarz oh... very cool. very.

zane22:02:48

Should I be thinking of the collected values as … a stack?

zane22:02:02

Like, what happens if some upstream navigator before me also collected a few things?

nathanmarz22:02:08

@zane they're a vector

nathanmarz22:02:16

new collected values are conj'd to the end

nathanmarz22:02:25

I'm sure there's room for all sorts of new navigators to help with those sorts of issues

zane22:02:10

I see, okay.

nathanmarz22:02:13

something like (with-fresh-collected (cond-path ...))

zane22:02:22

That's what I'm itching for.

zane22:02:25

But this'll work.

nathanmarz22:02:47

should be easy to do using richnav and defdynamicnav

zane22:02:04

I haven't yet wrapped my mind around those, I think.

zane22:02:07

Baby steps.

nathanmarz22:02:09

I opened an issue for it since it's actually a pretty good idea https://github.com/nathanmarz/specter/issues/175

zane22:02:57

There's no DISPENSE-ONE?

nathanmarz22:02:07

I have a feeling the use cases for that would be better served by with-fresh-collected

nathanmarz22:02:25

not sure, this part of the design space is fairly unexplored

nathanmarz22:02:31

would need to see more use cases

zane22:02:49

defrichnav isn't documented yet?

zane22:02:35

The only way it differs is that it takes vals, which is the collected vals?

nathanmarz22:02:33

yea, that and next-fn expects to be passed in the new vals + the next value to navigate to

nathanmarz22:02:11

defnav is a thin wrapper around defrichnav

zane22:02:23

Is defnav mainly there for backwards compat at this point?

zane23:02:33

I'm not going to need late-bound-nav here?

zane23:02:44

Rather than richnav?

nathanmarz23:02:16

@zane oh yea, you'll need late-bound-richnav

zane23:02:39

Maybe I should re-read the caching implementation doc.