Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2018-05-01

Channels

#portkey
<
2018-05-01
>
viesti12:05:37

np, this is small compared to your contribution 🙂

viesti10:05:55

from 39 failures, 12 errors to 15 failures, 12 errors by fixing test troll

🙌 4
viesti11:05:02

just what I was looking for too 🙂

baptiste-from-paris12:05:13

very cool 🙂 ! I go at the restaurant for 2 hours and you guys already did the review job ^^

cgrand12:05:08

>>> For some services, you must include the X-Amz-Security-Token query parameter in the canonical (signed) query string. For other services, you add the X-Amz-Security-Token parameter at the end, after you calculate the signature. For details, see the API reference documentation for that service.

cgrand12:05:22

Ok and how do I know?

viesti12:05:43

hmm, might this relate to role based access control

cgrand12:05:15

sigv4 non s3

cgrand14:05:51

all tests pass

baptiste-from-paris14:05:54

can you quickly explain what went wrong ?

viesti14:05:38

> Yay, all your tests have been fixed!

viesti14:05:47

says CircleCI too

baptiste-from-paris14:05:31

I hope they gave a test suite for signing v2

cgrand14:05:37

there were errors in the tests: • treating normalize-path.txt as a test case • multi-valued headers (both multiline value and multiple occurences) were not properly handled when generating the headers map

cgrand14:05:31

errors in signature code: • missing percent encoding for non-ASCII • improper path normalization

cgrand14:05:18

v2 is only required by simple db, no?

baptiste-from-paris14:05:38

I am not sure, that’s what I am looking at right now

cgrand14:05:18

oh and third bullet: STS token handling depending on service name and moon phase

baptiste-from-paris14:05:59

okay so it looks like sdb is the only service that supports ONLY v2

baptiste-from-paris14:05:10

which is a pretty good news

baptiste-from-paris14:05:31

as it’s also a “deprecated” service

cgrand14:05:41

and is there any compelling reason for someone to use v2 on other services?

viesti14:05:28

hmm, did we have issue for S3 support?

baptiste-from-paris14:05:42

it’s a special protocol not yet implemented

baptiste-from-paris14:05:08

there are 5 of them, we’ve done 1

baptiste-from-paris14:05:38

I have to finish the query protocol, working on it, hope to come with something in the next 2 weeks

baptiste-from-paris14:05:17

oh, I did not see the new partitions.json ^^

baptiste-from-paris14:05:04

no more resources/aws-sdk-core/apis/ ?

baptiste-from-paris14:05:13

how do we generate apis then ^^ ?

cgrand14:05:18

And I should revive the spec split

cgrand14:05:32

Most apis descriptors are now part of the Java SDK. @viesti worked on that

cgrand14:05:28

S3 is missing iirc

cgrand15:05:32

I’d rather source from aws-sdk-ruby for example; @viesti, your opinion?

viesti15:05:15

hum, it might be more complete, haven't checked

cgrand15:05:51

and what about script vs dev dep?

viesti15:05:17

dev time script could fetch the specs

viesti15:05:59

I'm a bit surprised on this amount of corners of the internet that have aws specs here and there :)

cgrand15:05:21

or why don’t they have a public repo for them?

viesti15:05:58

maybe aws-sdk-ruby as git submodule, to keep track of specific version

viesti15:05:41

from where was the initial api specs, before aws-java-sdk-models, taken from @cgrand ? (as I recall, s3 was missing from those specs too)

cgrand19:05:03

Ruby and S3 was there iiirc

baptiste-from-paris16:05:40

I don’t remember

baptiste-from-paris16:05:02

so there is no harmonized way of having some descriptions files

baptiste-from-paris16:05:10

from official sdks ?

viesti16:05:56

I guess com.amazonaws/aws-java-sdk-models is official source, but is missing S3

viesti16:05:24

apropo, resources/aws-sig-v4-test-suite should be a dev-time resource

viesti16:05:46

hmm, aws-sdk-ruby is I guess official too, since it’s under aws github organization

viesti16:05:39

but haven’t yet found a single language independent place of api specs

viesti17:05:51

So made it such. Also using classpath for resolving path to aws-sig-v4-test-suite, since remembering that in Cursive the working directory might not be the project directory (might be that this actually was for Leiningen tasks though only).

viesti17:05:36

should compare list of api specs from the ruby sdk with the maven artifact

viesti17:05:06

I think a git submodule would make sense for tracking a repo with the api specs

baptiste-from-paris17:05:32

so 2 submodules, java and ruby one

baptiste-from-paris17:05:44

one task to compare both api description

baptiste-from-paris17:05:59

note that I see only what appears to be the last version of each lib =>

baptiste-from-paris17:05:13

I compare this to the “old” way where we had all versions co-existing

baptiste-from-paris17:05:23

which is fine, I am just saying

viesti18:05:11

user> (count model-jar-entries)
128

viesti18:05:27

[email protected]  ~/programming/aws-sdk-ruby/apis(master|✔)
0% ls -C1 | wc -l
     136

viesti18:05:38

I guess we’d need only stuff from the ruby sdk

viesti18:05:48

so thinking that would make sense to keep only the latest version of the api spec

viesti18:05:19

0% du -h -d 1 | sort -h
8.0K	./.github
 32K	./tasks
 92K	./doc-src
992K	./build_tools
 27M	./apis
 53M	./gems
 58M	./.git
139M	.

baptiste-from-paris18:05:10

#{"alexaforbusiness" "opsworkscm" "application-autoscaling"
  "lex-models" "appstream" "secretsmanager" "connect"
  "mediastore-data" "fms" "resourcegroupstaggingapi"
  "AWSMigrationHub" "s3" "meteringmarketplace" "iotanalytics"
  "elasticloadbalancingv2" "iot-data" "kinesis-video-media"
  "autoscaling-plans" "pricing" "iot-jobs-data"
  "kinesis-video-archived-media" "acm-pca"}

baptiste-from-paris18:05:22

found in ruby and not in java

viesti18:05:33

that’s 22 items, there 8 item difference in the count, hmm…

baptiste-from-paris18:05:29

yes I know 🙂

baptiste-from-paris18:05:23

I might be wrong because I did a difference on a set, on directory names by excluding all the 2222-22-22 dirs

viesti18:05:52

different naming, for example iot-jobs-data vs data.jobs.iot

baptiste-from-paris18:05:52

anyway, there are more description files in the ruby repo

viesti18:05:18

do we need to generate api from versions other than the latest?

cgrand19:05:10

I don’t think so. Only from now on :-)

viesti19:05:57

user> (def ruby-sdk-apis (set (for [file (.listFiles (java.io.File. “/Users/kimmoko/programming/aws-sdk-ruby/apis”))
                                    :let [[latest-api-dir] (sort #(compare %2 %1) (.listFiles file))
                                          json (with-open [rdr ( ( latest-api-dir “api-2.json”))]
                                                 (cheshire.core/parse-stream rdr true))]]
                                (get-in json [:metadata :serviceFullName]))))
#’user/ruby-sdk-apis
user> (def aws-java-sdk-apis (set (for [entry model-jar-entries
                                        :let [json (with-open [in (.getInputStream jar-file entry)
                                                               rdr ( in)]
                                                     (cheshire.core/parse-stream rdr true))
                                              ]]
                                    (get-in json [:metadata :serviceFullName]))))
#’user/aws-java-sdk-apis
user> (clojure.set/difference ruby-sdk-apis aws-java-sdk-apis)
#{“Amazon Kinesis Video Streams” “AWS IoT Analytics” “Amazon Simple Storage Service” “Firewall Management Service” “Amazon Kinesis Video Streams Archived Media” “Amazon Connect Service” “AWS Certificate Manager Private Certificate Authority” “AWS Secrets Manager”}
user> (count (clojure.set/difference ruby-sdk-apis aws-java-sdk-apis))
8

viesti19:05:59

no S3 though 😕

cgrand19:05:10

I don’t think so. Only from now on :-)

viesti19:05:36

right, so we go back to sourcing from ruby

viesti19:05:56

is the submodule idea any good?

viesti19:05:31

was thinking that bumping the source dependency would be a bump of the submodule, would not need to write code to fetch the spec files

cgrand19:05:56

Sounds ok but I have no strong git opinions.

viesti20:05:54

trying it out

viesti20:05:56

but sleep now