This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-06-21
Channels
- # ai (1)
- # announcements (18)
- # aws (7)
- # babashka (54)
- # babashka-sci-dev (6)
- # beginners (22)
- # calva (57)
- # cider (3)
- # clj-http (1)
- # clojure (144)
- # clojure-austin (1)
- # clojure-dev (38)
- # clojure-europe (34)
- # clojure-gamedev (4)
- # clojure-norway (8)
- # clojure-uk (5)
- # clojurescript (12)
- # clr (20)
- # conjure (4)
- # data-science (1)
- # emacs (30)
- # events (1)
- # graphql (9)
- # helix (7)
- # hyperfiddle (22)
- # introduce-yourself (4)
- # jobs (1)
- # leiningen (7)
- # malli (3)
- # off-topic (26)
- # polylith (26)
- # portal (5)
- # random (14)
- # shadow-cljs (113)
- # tools-deps (6)
- # xtdb (9)
Hey all! Howdy? When I run an AWS CLI command to retrieve values from DynamoDB, as shown below, everything works fine. The query returns 73MB of data as expected:
aws dynamodb query --table-name a-table \
--key-condition-expression "#p = :projectval" \
--expression-attribute-names '{"#p":"project"}' \
--expression-attribute-values '{":projectval":{"S":"A_PROJECT_NAME"}}'
However, when I attempt the same query in a Clojure project, using the Cognitect AWS API as follows, I encounter some issues:
(s/defn ->query :- wire.out.dynamodb/Query
[table-name :- s/Str
item :- (s/pred map?)]
{:op :Query
:request {:TableName table-name
:KeyConditionExpression "#p = :projectval",
:ExpressionAttributeNames {"#p" (str (-> item keys first name))}
:ExpressionAttributeValues {":projectval" {:S (-> item vals first str)}}}})
The resulting query only yields 1.17MB of data, which is incomplete.
Are there any additional configurations I need to adjust to retrieve the complete data set? Or is there a limitation that I should be aware of?a lot of the aws calls page and tell you that in response metadata - any chance that's happening here?
Let me check about it
I'll make a loop to scan all pagination, giving a try here
Indeed, it was the pagination (`:LastEvaluatedKey`). Now it went as expected. Many thanks!
I'll take a look on iteration
, thanks for the tip @U01GXCWSRMW! A middle aged Clojurian here 😅
This is the best documentation I’ve seen of it: https://www.juxt.pro/blog/new-clojure-iteration/
👀 2