Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Internal] Add methods for configuring tests to work with test proxy #15259

Merged
merged 50 commits into from
Sep 2, 2021
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
43e875b
moving into a separate PR to merge with main
seankane-msft Aug 9, 2021
ce8dfa6
adding yml and script to start server
seankane-msft Aug 10, 2021
8b716cd
Merge branch 'main' of https://github.com/Azure/azure-sdk-for-go into…
seankane-msft Aug 10, 2021
2d34f68
renaming examples for go vet
seankane-msft Aug 10, 2021
858f929
fixing stop command
seankane-msft Aug 10, 2021
0e78180
merge conflicts
seankane-msft Aug 10, 2021
abfd495
skipping problematic tests
seankane-msft Aug 10, 2021
ae991cc
bens feedback
seankane-msft Aug 10, 2021
ab34658
testproxy changes from track2-tables
seankane-msft Aug 12, 2021
464bb90
simplification
seankane-msft Aug 12, 2021
5170886
fix
seankane-msft Aug 12, 2021
1bb884d
updating to work with common
seankane-msft Aug 19, 2021
6355a03
getting main build-test.yml files
seankane-msft Aug 19, 2021
ccf37a8
whitespace
seankane-msft Aug 19, 2021
29bed05
adding tests for test proxy infra
seankane-msft Aug 19, 2021
f33a232
setting azure_record_mode to playback
seankane-msft Aug 19, 2021
0ea12a0
setting proxy cert variable
seankane-msft Aug 19, 2021
def4c27
prepend ./ to testId, use testId to determine full path from repo root
seankane-msft Aug 19, 2021
42e667a
workaround for magically dissappearing files
seankane-msft Aug 19, 2021
0455394
more tests
seankane-msft Aug 19, 2021
0f87900
dropping coverage
seankane-msft Aug 19, 2021
2921e52
removing azcore dependent code
seankane-msft Aug 23, 2021
66158cb
undoing changes to eng files
seankane-msft Aug 23, 2021
3de4936
unexporting a couple things
seankane-msft Aug 23, 2021
5feef38
fixed issue with deleting recordings
seankane-msft Aug 24, 2021
e752a6f
adding additional tests
seankane-msft Aug 24, 2021
8ec2e2a
dropping code coverage
seankane-msft Aug 24, 2021
32c6570
cleanup
seankane-msft Aug 24, 2021
297b899
richards comments
seankane-msft Aug 24, 2021
ecc9563
improving coverage
seankane-msft Aug 25, 2021
7867ab1
fixing recording options test
seankane-msft Aug 25, 2021
1453a3d
inching closer
seankane-msft Aug 25, 2021
2cee416
a
seankane-msft Aug 25, 2021
b3f7949
dropping coverage 1%, have every inch of new code tested
seankane-msft Aug 26, 2021
bd98736
adding test case for backwards slash path
seankane-msft Aug 26, 2021
9ca55c8
no newline
seankane-msft Aug 26, 2021
9421675
fixing test
seankane-msft Aug 26, 2021
e2b0ded
allowing parallel tests
seankane-msft Aug 26, 2021
6c3d638
removing snippet
seankane-msft Aug 26, 2021
b62e797
undoing change to config
seankane-msft Aug 26, 2021
f40b00d
Merge branch 'main' into test-proxy-main
seankane-msft Aug 26, 2021
a009528
fixing armcore link
seankane-msft Aug 31, 2021
2c8e939
Merge branch 'test-proxy-main' of https://github.com/seankane-msft/az…
seankane-msft Aug 31, 2021
cdca98a
Merge branch 'main' into test-proxy-main
seankane-msft Aug 31, 2021
2d96846
undoing changes to migration guide
seankane-msft Aug 31, 2021
1fd955b
merge conflict
seankane-msft Aug 31, 2021
d3719af
Update sdk/internal/recording/recordings/TestUriSanitizer.json
seankane-msft Sep 1, 2021
0a56f19
Update sdk/internal/recording/recordings/TestStartStop.json
seankane-msft Sep 1, 2021
6aba571
richards comments
seankane-msft Sep 2, 2021
a20591d
Merge branch 'test-proxy-main' of https://github.com/seankane-msft/az…
seankane-msft Sep 2, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions documentation/MIGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ To the show the code snippets for the change:
authorizer, err := adal.NewServicePrincipalToken(oAuthToken, "<ClientId>", "<ClientSecret>", endpoint)
client := resources.NewGroupsClient("<SubscriptionId>")
client.Authorizer = authorizer
```
```

**Equivalent in new version (`sdk/**/arm**`)**

Expand All @@ -54,7 +54,7 @@ For detailed information on the benefits of using the new authentication classes

### Error Handling

There are some minor changes in the error handling.
There are some minor changes in the error handling.

- When there is an error in the SDK request, in the old version (`services/**/mgmt/**`), the return value will all be non-nil, and you can get the raw HTTP response from the response value. In the new version (`sdk/**/arm**`), the first return value will be empty and you need to cast the error to `HTTPResponse` interface to get the raw HTTP response. When the request is successful and there is no error returned, you will need to get the raw HTTP response in `RawResponse` property of the first return value.

Expand Down Expand Up @@ -180,7 +180,7 @@ In new version (`sdk/**/arm**`), we use `(armcore.ConnectionOptions).PerCallPoli

Similar to the customized policy, there are changes regarding how the custom HTTP client is configured as well. You can now use the `(armcore.ConnectionOptions).HTTPClient` option in `github.com/Azure/azure-sdk-for-go/sdk/armcore` module to use your own implementation of HTTP client and plug in what they need into the configuration.

**In old version (`services/**/mgmt/**`)**
**In old version (`services/**/mgmt/**`)**
```go
httpClient := NewYourOwnHTTPClient{}
client := resources.NewGroupsClient("<SubscriptionId>")
Expand Down
2 changes: 1 addition & 1 deletion eng/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
},
{
"Name": "internal",
"CoverageGoal": 0.83
"CoverageGoal": 0.90
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This here is a positive indicator. :)

}
]
}
1 change: 1 addition & 0 deletions eng/pipelines/templates/steps/build-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ steps:
GO111MODULE: 'on'
AZURE_RECORD_MODE: 'playback'
PROXY_CERT: $(Build.SourcesDirectory)/eng/common/testproxy/dotnet-devcert.crt
${{ insert }}: ${{ parameters.EnvVars }}

- pwsh: ../eng/scripts/create_coverage.ps1 ${{parameters.ServiceDirectory}}
displayName: 'Generate Coverage XML'
Expand Down
67 changes: 33 additions & 34 deletions sdk/internal/recording/recording.go
Original file line number Diff line number Diff line change
Expand Up @@ -439,7 +439,7 @@ var modeMap = map[RecordMode]recorder.Mode{
Playback: recorder.ModeReplaying,
}

var recordMode, _ = os.LookupEnv("AZURE_RECORD_MODE")
var recordMode = os.Getenv("AZURE_RECORD_MODE")

const (
modeRecording = "record"
Expand All @@ -451,7 +451,7 @@ const (
UpstreamUriHeader = "x-recording-upstream-base-uri"
)

var recordingId string
var recordingIds = map[string]string{}

var tr = &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
Expand All @@ -466,10 +466,6 @@ type RecordingOptions struct {
Scheme string
}

func resetRecordingId() {
recordingId = ""
}

func defaultOptions() *RecordingOptions {
return &RecordingOptions{
UseHTTPS: true,
Expand All @@ -486,20 +482,22 @@ func (r RecordingOptions) HostScheme() string {
}

func getTestId(pathToRecordings string, t *testing.T) string {
if strings.HasSuffix(pathToRecordings, "/") {
seankane-msft marked this conversation as resolved.
Show resolved Hide resolved
pathToRecordings = strings.TrimRight(pathToRecordings, "/")
}
return pathToRecordings + "/recordings/" + t.Name() + ".json"
seankane-msft marked this conversation as resolved.
Show resolved Hide resolved
}

func StartRecording(t *testing.T, pathToRecordings string, options *RecordingOptions) error {
richardpark-msft marked this conversation as resolved.
Show resolved Hide resolved
if options == nil {
options = defaultOptions()
}
if recordMode == "" {
t.Log("AZURE_RECORD_MODE was not set, options are \"record\" or \"playback\". \nDefaulting to playback")
recordMode = "playback"
if !(recordMode == modeRecording || recordMode == modePlayback) {
return fmt.Errorf("AZURE_RECORD_MODE was not understood, options are \"record\" or \"playback\". Received: %v", recordMode)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No biggie, but could '%s' and use the actual constants in your error message since you're already Errorf'ing.

}
testId := getTestId(pathToRecordings, t)
seankane-msft marked this conversation as resolved.
Show resolved Hide resolved

url := fmt.Sprintf("%v/%v/start", options.HostScheme(), recordMode)
url := fmt.Sprintf("%s/%s/start", options.HostScheme(), recordMode)

req, err := http.NewRequest("POST", url, nil)
if err != nil {
Expand All @@ -512,13 +510,19 @@ func StartRecording(t *testing.T, pathToRecordings string, options *RecordingOpt
if err != nil {
return err
}
recordingId = resp.Header.Get(IdHeader)
recId := resp.Header.Get(IdHeader)
if recId == "" {
b, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
return fmt.Errorf("Recording ID was not returned by the response. Response body: %s", b)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: would it be helpful to say that the specific header was not found? Could aid in diagnostics if someone is trying to figure out what's wrong strictly by the error message.

}
recordingIds[t.Name()] = recId
return nil
}

func StopRecording(t *testing.T, options *RecordingOptions) error {
defer resetRecordingId()

if options == nil {
options = defaultOptions()
}
Expand All @@ -528,10 +532,12 @@ func StopRecording(t *testing.T, options *RecordingOptions) error {
if err != nil {
return err
}
if recordingId == "" {
var recId string
var ok bool
if recId, ok = recordingIds[t.Name()]; !ok {
return errors.New("Recording ID was never set. Did you call StartRecording?")
}
req.Header.Set("x-recording-id", recordingId)
req.Header.Set("x-recording-id", recId)
_, err = client.Do(req)
if err != nil {
t.Errorf(err.Error())
Expand Down Expand Up @@ -591,58 +597,51 @@ func LiveOnly(t *testing.T) {

// Function for sleeping during a test for `duration` seconds. This method will only execute when
// AZURE_RECORD_MODE = "record", if a test is running in playback this will be a noop.
func Sleep(duration int) {
func Sleep(duration time.Duration) {
if GetRecordMode() == modeRecording {
time.Sleep(time.Duration(duration) * time.Second)
time.Sleep(duration)
}
}

func GetRecordingId() string {
return recordingId
func GetRecordingId(t *testing.T) string {
return recordingIds[t.Name()]
}

func GetRecordMode() string {
return recordMode
}

func InPlayback() bool {
return GetRecordMode() == modePlayback
}

func InRecord() bool {
return GetRecordMode() == modeRecording
}

func getRootCas() (*x509.CertPool, error) {
func getRootCas(t *testing.T) (*x509.CertPool, error) {
localFile, ok := os.LookupEnv("PROXY_CERT")

rootCAs, err := x509.SystemCertPool()
if err != nil {
if err != nil && strings.Contains(err.Error(), "system root pool is not available on Windows") {
rootCAs = x509.NewCertPool()
} else if err != nil {
return rootCAs, err
}

if !ok {
fmt.Println("Could not find path to proxy certificate, set the environment variable 'PROXY_CERT' to the location of your certificate")
t.Log("Could not find path to proxy certificate, set the environment variable 'PROXY_CERT' to the location of your certificate")
return rootCAs, nil
}

cert, err := ioutil.ReadFile(localFile)
if err != nil {
fmt.Println("error opening cert file")
return nil, err
}

if ok := rootCAs.AppendCertsFromPEM(cert); !ok {
fmt.Println("No certs appended, using system certs only")
t.Log("No certs appended, using system certs only")
}

return rootCAs, nil
}

func GetHTTPClient() (*http.Client, error) {
func GetHTTPClient(t *testing.T) (*http.Client, error) {
transport := http.DefaultTransport.(*http.Transport).Clone()

rootCAs, err := getRootCas()
rootCAs, err := getRootCas(t)
if err != nil {
return nil, err
}
Expand Down
Loading