This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-02-01
Channels
- # aleph (2)
- # aws (23)
- # bangalore-clj (1)
- # beginners (99)
- # bristol-clojurians (1)
- # cider (3)
- # circleci (1)
- # clj-kondo (8)
- # cljdoc (4)
- # clojars (1)
- # clojure (47)
- # clojure-india (1)
- # clojure-nl (2)
- # clojure-serbia (3)
- # clojure-spec (2)
- # clojure-uk (17)
- # clojuredesign-podcast (1)
- # clojurescript (20)
- # datascript (1)
- # datomic (1)
- # emacs (1)
- # fulcro (4)
- # pathom (18)
- # ring-swagger (2)
- # shadow-cljs (31)
- # spacemacs (3)
- # tools-deps (10)
- # vscode (1)
We are considering adding a new function to Clojure called iteration
, and it would be very useful for consuming paginated APIs like Amazon's
(defn log-groups
"gets all CloudWatch Log Groups"
[client]
(->> (iteration
(fn [token]
(aws/invoke client (cond-> {:op :DescribeLogGroups}
token (assoc :request {:nextToken token}))))
:kf :nextToken
:vf :logGroups)
(into [] cat)))
^ here's how it could work with Cognitect Labs' aws-api
Nice. First thought is how do I handle anomalies? Sometimes (most times) I'd like to essentially have a reduced
if an anomaly occurs, returning the anomaly.
the iteration above will terminate on anomalies (because :kf won't return anything from an anomaly) -- the (into [] cat) is swallowing it though
Hmm, I see. Clever. Looking at the pagination code we are using, the only other thing we keep track of is page-count. In some cases we use that for logging when certain page counts are hit for long running iterations & other cases for capping pages followed. Seems like you can do that here by returning a different data structure from :kf. I think this would cover our use case.
The only other comment is I almost always prefer apis where the function takes its options in a map instead of the variadic approach. Easer to programmatically build up when needed. Was there a reason you chose the variadic args?
I suspect that this isn't going to be a case where calls are programmatically built up, but I'll take that feedback back.
user=> (transduce (halt-when odd?) (completing conj! persistent!) (transient []) [2 2 2 3])
3
user=> (into [] (halt-when odd?) [2 2 2 3])
Execution error (ClassCastException) at user/eval197 (REPL:1).
class java.lang.Long cannot be cast to class clojure.lang.ITransientCollection (java.lang.Long is in module java.base of loader 'bootstrap'; clojure.lang.ITransientCollection is in unnamed module of loader 'app')
(defn into'
([to xform from]
(let [add-meta #(with-meta % (meta to))]
(if (instance? clojure.lang.IEditableCollection to)
(transduce xform (completing conj! (comp add-meta persistent!)) (transient to) from)
(transduce xform (completing conj add-meta) to from)))))
user=> (into' [] (halt-when #{:anomaly} (fn [& args] (zipmap [:result :failing-input] args))) [1 2 3 :anomaly 4])
{:result [1 2 3], :failing-input :anomaly}