Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2020-03-04

Channels

#aws
<
2020-03-04
>
lgouger19:03:36

Are the paginators for an operation available via the Clojure AWS api?

lgouger19:03:19

In Javascript API they’re in client specific data files, eg. aws-sdk-js/apis/logs-2014-03-28.paginators.json

lgouger19:03:58

(this happens to be the one for the Amazon CloudWatch Logs service)

kenny19:03:42

No, nothing is built in for pagination right now.

lgouger19:03:28

ok, I’ve been making op specific lazy pagers. I’ll continue on that path for now.

kenny19:03:06

Yep, we have the same.

ghadi19:03:05

there is a function being considered for inclusion in clojure 1.11 that helps with pagination @lgouger

ghadi19:03:08

(defn filter-log-events
  "Issues a CloudWatch Logs query (with the filter syntax {$...}, not the CW Insights syntax.

   Collects all pages of results.

   `client` is an aws-api client
   `request` argument is passed directly as the initial request to the :FilterLogEvents op"
  [client request]
  (iteration (fn [token]
               (aws/invoke client
                           {:op :FilterLogEvents
                            :request (if token {:nextToken token} request)}))
             :kf :nextToken))

ghadi19:03:32

example usage ^

kenny19:03:24

I think he may be talking about a mapping of op -> request & response next token keys

kenny19:03:20

If so, there is this: https://github.com/iann0036/aws-pagination-rules. Not sure how accurate it is.

lgouger19:03:27

yeah, different apis use different names for nextToken in the request and response and where the results are. For example S3's listObjectsV2 vs logs DescribeLogGroups 

"ListObjectsV2": {
      "input_token": "ContinuationToken",
      "limit_key": "MaxKeys",
      "output_token": "NextContinuationToken",
      "result_key": [
        "Contents",
        "CommonPrefixes"
      ]
    }
and 
"DescribeLogGroups": {
      "input_token": "nextToken",
      "limit_key": "limit",
      "output_token": "nextToken",
      "result_key": "logGroups"
    }

ghadi19:03:01

Yeah - right now the paginator descriptors aren’t exposed