Update to version v5.1.0

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,31 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior.
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Please complete the following information about the solution:**
+- [ ] Version: [e.g. v1.0.0]
+- [ ] Region: [e.g. us-east-1]
+- [ ] Was the solution modified from the version published on this repository?
+- [ ] If the answer to the previous question was yes, are the changes available on GitHub?
+- [ ] Have you checked your [service quotas](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) for the sevices this solution uses?
+- [ ] Were there any errors in the CloudWatch Logs?
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem (please **DO NOT include sensitive information**).
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea for this solution
+title: ''
+labels: enhancement
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the feature you'd like**
+A clear and concise description of what you want to happen.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/PULL_REQUEST_TEMPLATE.md

@@ -2,5 +2,4 @@
 
 *Description of changes:*
 
-
 By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

CHANGELOG.md

@@ -4,6 +4,30 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+
+## [5.1.0] - 2020-04-30
+### Added
+- Default encryption to SNS topic
+- Environment variable to configure the AWS SDK to reuse TCP connections (https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/node-reusing-connections.html)
+- SQS queue options at deployment to capture workflow outputs
+- Support for accelerated transcoding in Elemental MediaCoonvert: https://aws.amazon.com/about-aws/whats-new/2019/10/announcing-new-aws-elemental-mediaconvert-features-for-accelerated-transcoding-dash-and-avc-video-quality/
+- support for Glacier deep archive
+
+### Changed
+- Lambda functions runtime to latest available (Node.js 12)
+- Build assets to include package-lock.json files
+- Build and test commands to use _npm ci_ instead of _npm install_
+- Cloudformation template to use _AWS::Partition_ instead of _aws_
+- Logic to add MediaPackage VOD as a custom origin to CloudFront (it's now done as a custom resource when the stack is created / updated)
+
+### Fixed
+- Links in README file
+- fix to buildUrl function in output-validate lambda to support non root objects (https://github.com/awslabs/video-on-demand-on-aws/issues/61)
+- fix mediainfo lambda function signing method error (https://github.com/awslabs/video-on-demand-on-aws/issues/670
+
+### Removed
+- _'use strict'_ directives
+
 ## [5.0.0] - 2019-11-20
 ### Added
 - MediaPackage VOD support
@@ -35,4 +59,3 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Lambda functions (except for mediainfo) runtime to nodejs10.x
 - Mediainfo lambda function to python3.7
 - Mediainfo executable version (from v0.7.92.1 to v19.09)
-

CONTRIBUTING.md

@@ -1,14 +1,11 @@
 # Contributing Guidelines
-
 Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
 documentation, we greatly value feedback and contributions from our community.
 
 Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
 information to effectively respond to your bug report or contribution.
 
-
 ## Reporting Bugs/Feature Requests
-
 We welcome you to use the GitHub issue tracker to report bugs or suggest features.
 
 When filing an issue, please check [existing open](https://github.com/awslabs/video-on-demand-on-aws/issues), or [recently closed](https://github.com/awslabs/video-on-demand-on-aws/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
@@ -19,7 +16,6 @@ reported the issue. Please try to include as much information as you can. Detail
 * Any modifications you've made relevant to the bug
 * Anything unusual about your environment or deployment
 
-
 ## Contributing via Pull Requests
 Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
 
@@ -39,23 +35,17 @@ To send us a pull request, please:
 GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
 [creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
 
-
 ## Finding contributions to work on
 Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/awslabs/video-on-demand-on-aws/labels/help%20wanted) issues is a great place to start.
 
-
 ## Code of Conduct
 This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
 For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
 opensource-codeofconduct@amazon.com with any additional questions or comments.
 
-
 ## Security issue notifications
 If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
 
-
 ## Licensing
-
 See the [LICENSE](https://github.com/awslabs/video-on-demand-on-aws/blob/master/LICENSE.txt) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
-
 We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

LICENSE.txt

@@ -187,7 +187,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright 2019 - 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+   Copyright 2019 - 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

NOTICE.txt

@@ -1,6 +1,6 @@
 Video on Demand on AWS
 
-Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 
 **********************
 THIRD PARTY COMPONENTS

README.md

@@ -1,7 +1,7 @@
 # Video on Demand on AWS
 
 How to implement a video-on-demand workflow on AWS leveraging AWS Step Functions, AWS Elemental MediaConvert, and AWS Elemental MediaPackage.
-Source code for [Video on Demand on AWS](https://aws.amazon.com/answers/media-entertainment/video-on-demand-on-aws/) solution.
+Source code for [Video on Demand on AWS](https://aws.amazon.com/solutions/video-on-demand-on-aws/) solution.
 
 ## On this Page
 - [Architecture Overview](#architecture-overview)
@@ -10,6 +10,8 @@ Source code for [Video on Demand on AWS](https://aws.amazon.com/answers/media-en
 - [Source Metadata Option](#source-metadata-option)
 - [Encoding Templates](#encoding-templates)
 - [QVBR Mode](#qvbr-mode)
+- [Accelerated Transcoding](#accelerated-transcoding)
+- [Source Code](#source-code)
 - [Creating a custom Build](#creating-a-custom-build)
 - [Additional Resources](#additional-resources)
 
@@ -35,6 +37,9 @@ The workflow configuration is set at deployment and is defined as environment va
 * **MediaConvert_Template_720p:**	The name of the SD template in MediaConvert
 * **Source:**	The name of the source S3 bucket
 * **WorkflowName:**	Used to tag all of the MediaConvert encoding jobs
+* **acceleratedTranscoding** Enabled Accelerated Transocding in MediaConvert. options include ENABLE, DISABLE, PREFERRED. for more detials please see: 
+* **enableSns** Send SNS notifications for the workflow results.
+* **enableSqs** Send the workflow results to an SQS queue
 
 ### WorkFlow Triggers
 
@@ -62,7 +67,7 @@ The only required field for the metadata file is the **srcVideo**. The workflow
 ```
 {
     "srcVideo": "string",
-    "archiveSource": boolean,
+    "archiveSource": string,
     "frameCapture": boolean,
     "srcBucket": "string",
     "destBucket": "string",
@@ -71,7 +76,12 @@ The only required field for the metadata file is the **srcVideo**. The workflow
     "jobTemplate_1080p": "string",
     "jobTemplate_720p": "string",
     "jobTemplate": "custom-job-template",
-    "inputRotate": "DEGREE_0|DEGREES_90|DEGREES_180|DEGREES_270|AUTO"
+    "inputRotate": "DEGREE_0|DEGREES_90|DEGREES_180|DEGREES_270|AUTO",
+    "captions": {
+        "srcFile": "string",
+        "fontSize": integer,
+        "fontColor": "WHITE|BLACK|YELLOW|RED|GREEN|BLUE"
+    }
 }
 ```
 
@@ -112,7 +122,18 @@ AWS MediaConvert Quality-defined Variable Bit-Rate (QVBR) control mode gets the
 
 For more detail please see [QVBR and MediaConvert](https://docs.aws.amazon.com/mediaconvert/latest/ug/cbr-vbr-qvbr.html).
 
-## Source code (Node.js 10)
+## Accelerated Transcoding 
+Version 5.1.0 introduces support for accelerated transcoding which is a pro tier  feature of AWS Elemental MediaConvert. This feature can be configured when launching the template with one of the following options:
+
+* **ENABLED** All files upload will have acceleration enabled. Files that are not supported will not be processed and the workflow will fail
+* **PREFERRED** All files uploaded will be processed but only supported files will have acceleration enabled, the workflow will not fail.
+* **DISABLED** No acceleration.
+
+For more detail please see [Accelerated Transcoding](https://docs.aws.amazon.com/mediaconvert/latest/ug/accelerated-transcoding.html).
+
+## Source code
+
+### Node.js 12
 * **archive-source:** Lambda function to tag the source video in s3 to enable the Glacier lifecycle policy.
 * **custom-resource:** Lambda backed CloudFormation custom resource to deploy MediaConvert templates configure S3 event notifications.
 * **dynamo:** Lambda function to Update DynamoDB.
@@ -124,19 +145,19 @@ For more detail please see [QVBR and MediaConvert](https://docs.aws.amazon.com/m
 * **profiler:** Lambda function used to send publish and/or error notifications.
 * **step-functions:** Lambda function to trigger AWS Step Functions.
 
-## Source code (Python 3.7)
-> **Note**: The _mediainfo_ function uses the python3.7 runtime since the distributable was compiled on Amazon Linux, and the [Operating System for the node version 10 runtime](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html) is Amazon Linux 2.
+### Python 3.7
+* **mediainfo:** Lambda function to run [mediainfo](https://mediaarea.net/en/MediaInfo) on an S3 signed url.
 
-* **mediainfo:** Lambda function to run mediainfo on s3 signed url. https://mediaarea.net/en/MediaInfo. bin/mediainfo must be made executable before deploying to lambda.
+> ./source/mediainfo/bin/mediainfo must be made executable before deploying to lambda.
 
 ## Creating a custom build
 The solution can be deployed through the CloudFormation template available on the solution home page: [Video on Demand on AWS](https://aws.amazon.com/answers/media-entertainment/video-on-demand-on-aws/).
 To make changes to the solution, download or clone this repo, update the source code and then run the deployment/build-s3-dist.sh script to deploy the updated Lambda code to an Amazon S3 bucket in your account.
 
 ### Prerequisites:
 * [AWS Command Line Interface](https://aws.amazon.com/cli/)
-* Node.js 10.x or later
-* Python 3.7 or later
+* Node.js 12.x or later
+* Python 3.8 or later
 
 ### 1. Running unit tests for customization
 Run unit tests to make sure added customization passes the tests:
@@ -156,14 +177,14 @@ aws s3 mb s3://my-bucket-us-east-1
 Build the distributable:
 ```
 chmod +x ./build-s3-dist.sh
-./build-s3-dist.sh my-bucket video-on-demand-on-aws v5.0.0-custom
+./build-s3-dist.sh my-bucket video-on-demand-on-aws version
 ```
 
 > **Notes**: The _build-s3-dist_ script expects the bucket name as one of its parameters, and this value should not include the region suffix.
 
 Deploy the distributable to the Amazon S3 bucket in your account:
 ```
-aws s3 cp ./regional-s3-assets/ s3://my-bucket-us-east-1/video-on-demand-on-aws/v5.0.0-custom/ --recursive --acl bucket-owner-full-control
+aws s3 cp ./regional-s3-assets/ s3://my-bucket-us-east-1/video-on-demand-on-aws/version/ --recursive --acl bucket-owner-full-control
 ```
 
 ### 4. Launch the CloudFormation template.
@@ -175,22 +196,22 @@ aws s3 cp ./regional-s3-assets/ s3://my-bucket-us-east-1/video-on-demand-on-aws/
 ### Services
 - [AWS Elemental MediaConvert](https://aws.amazon.com/mediaconvert/)
 - [AWS Elemental MediaPackage](https://aws.amazon.com/mediapackage/)
-- [AWS Step Functions](https://aws.amazon.com/mediapackage/)
+- [AWS Step Functions](https://aws.amazon.com/step-functions/)
 - [AWS Lambda](https://aws.amazon.com/lambda/)
 - [Amazon CloudFront](https://aws.amazon.com/cloudfront/)
 - [OTT Workflows](https://www.elemental.com/applications/ott-workflows)
 - [QVBR and MediaConvert](https://docs.aws.amazon.com/mediaconvert/latest/ug/cbr-vbr-qvbr.html)
 
 ### Other Solutions and Demos
-- [Live Streaming On AWS](https://aws.amazon.com/answers/media-entertainment/live-streaming/)
-- [Media Analysis Solution](https://aws.amazon.com/answers/media-entertainment/media-analysis-solution/)
+- [Live Streaming On AWS](https://aws.amazon.com/solutions/live-streaming-on-aws/)
+- [Media Analysis Solution](https://aws.amazon.com/solutions/media-analysis-solution/)
 - [Live Streaming and Live to VOD Workshop](https://github.com/awslabs/speke-reference-server)
 - [Live to VOD with Machine Learning](https://github.com/aws-samples/aws-elemental-instant-video-highlights)
 - [Demo SPEKE Reference Server](https://github.com/awslabs/speke-reference-server)
 
 ***
 
-Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.

deployment/build-s3-dist.sh

@@ -67,17 +67,22 @@ chmod +x ./mediainfo/bin/mediainfo
 
 for folder in */ ; do
     cd "$folder"
+
     function_name=${PWD##*/}
-    echo "Creating deployment package for $function_name"
+    zip_path="$build_dist_dir/$function_name.zip"
+
+    echo "Creating deployment package for $function_name at $zip_path"
 
     if [ -e "package.json" ]; then
         rm -rf node_modules/
-        npm install --production
-        rm package-lock.json
+        npm i --production
 
-        zip -q -r9 "$build_dist_dir/$function_name.zip" .
-    else
-        python3 setup.py build_pkg --zip-path="$build_dist_dir/$function_name.zip"
+        zip -q -r9 $zip_path .
+    elif [ -e "setup.py" ]; then
+        # If you're running this command on macOS and Python3 has been installed using Homebrew, you might see this issue:
+        #    DistutilsOptionError: must supply either home or prefix/exec-prefix
+        # Please follow the workaround suggested on this StackOverflow answer: https://stackoverflow.com/a/44728772
+        python3 setup.py build_pkg --zip-path=$zip_path
     fi
 
     cd ..

deployment/run-unit-tests.sh

@@ -32,5 +32,12 @@ npm test
 cd ../step-functions
 npm test
 
+# If you're running these commands on macOS and Python3 has been installed using Homebrew, you might see this issue:
+#    DistutilsOptionError: must supply either home or prefix/exec-prefix
+# Please follow the workaround suggested on this StackOverflow answer: https://stackoverflow.com/a/44728772
 cd ../mediainfo
-python3 setup.py test
+rm -rf ./pytests && mkdir ./pytests
+cp lambda_function.py ./test*.py ./pytests
+pip3 install boto3 -t ./pytests
+python3 -m unittest discover -s ./pytests -v
+rm -rf ./pytests