Test fix ups

[jillr]

SUMMARY

Enable coverage and fix nightly builds

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

tests/utils/shippable/shippable.sh

[jillr]

Units did not trigger on #38 (see 00:26 WARNING: No tests found for detected changes in build_ci) so it was not evident that the unit tests for cloudformation needed to be updated as part of that PR, and are now broken.

[tremble]

This really makes my skin crawl. Especially since we have a virtualenv we should be able to use the codecov package. pip install --user codecov && codecov -t <the-repository-upload-token>

Looking at the code I think it supports the same flags.

[jillr]

They're not exactly the same. Digging into the source it looks like the python module does not support coveragepy, but I believe pycov should do the same thing.

[gundalow]

I've checked this codecov key is correct

[jillr]

The coverage failures look to be related to smoketests that should no longer have been enabled.
ansible-collections/overview#45 (comment)

[jillr]

Codecov upload failures look to be related to a service issue on their end.
No actually the problem is with how the post url is being formatted.

02:39 .query commit=3c5f774fa13f3dccf3f3a8a368b006623b13643c&branch=master&pr=45&build_url=https%3A%2F%2Fapp.shippable.com%2Fgithub%2Fansible-collections%2Famazon.aws%2Fruns%2F222&token=<secret>&service=shippable&build=222&slug=amazon.aws&name=aws%2F2.7%2F4&flags=integration%2Cpython_2_7&package=py2.0.21
02:39     Pinging Codecov...
02:39 Error: HTTP 400
02:39 slug must match pattern ^[\w\-\.\~\/]+\/[\w\-\.]{1,255}$

https://app.shippable.com/github/ansible-collections/amazon.aws/runs/222/13/console
There's a flag for specifying the slug to use, but it doesn't seem to be doing what we need.
Switching back to the bash uploader is working though.
https://codecov.io/github/ansible-collections/amazon.aws/commit/30207ecfcc892d52146dccf08d30a6bedb307e27

@tremble While it would be vastly preferable to not curl|bash, the codecov-python package does not Just Work. I'm switching back to the bash uploader so we can make progress on getting tests stable.

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@ed25a62). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #45   +/-   ##
=========================================
  Coverage          ?   45.70%           
=========================================
  Files             ?       77           
  Lines             ?     8796           
  Branches          ?     1852           
=========================================
  Hits              ?     4020           
  Misses            ?     4264           
  Partials          ?      512           

Flag	Coverage Δ
#integration	53.20% <0.00%> (?)
#python_2_6	17.71% <0.00%> (?)
#python_2_7	48.74% <0.00%> (?)
#python_3_5	17.73% <0.00%> (?)
#python_3_6	25.66% <0.00%> (?)
#python_3_7	49.25% <0.00%> (?)
#python_3_8	25.65% <0.00%> (?)
#python_3_9	19.73% <0.00%> (?)
#sanity	17.76% <0.00%> (?)
#stub	0.00% <0.00%> (?)
#units	37.29% <0.00%> (?)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed25a62...ab3a7e6. Read the comment docs.