Posts in this Series
- Getting Started
- Static Code Analysis
- Unit Testing - What your reading now.
- Functional Testing
Previously we added linting and security checking in the article Static Code Analysis. Today we will look at unit testing our Cloudformation templates using Cloud-Radar.
Unit Testing
Unit testing for infrastructure is not that different from unit testing an application. With an application, we might be testing a small piece of code like a function or a class. With Cloudformation our template is our application. The units will be the AWS Conditions and the resources that use a condition or contains an intrinsic function. I wrote cloud-radar because I didn’t want to have to deal with AWS credentials and actually deploying resources just to verify the logic inside my template.
If you need a template to test you can try out the companion repo and the unit-test-start branch.
Setup
To manage our test runs we will be using pytest. We can install pytest with pip.
pip install pytest
cloud-radar is still very new and may change a lot by the time you read this article. We can make sure it always works by pinning the version.
pip install cloud-radar==0.6.0
Let’s not forget to update requirements.txt.
pip freeze > requirements.txt
Lets create a directory structure to hold our tests.
mkdir -p tests/unit
touch tests/unit/test_log_bucket.py
Writing Tests
cloud-radar works by reading our Cloudformation template and then rendering it the same way that the AWS Cloudformation service would. Our template happens to be very simple. If the parameter KeepBucket is FALSE then the S3 bucket deployed by this template will be deleted when the Cloudformation stack is deleted. If the parameter is TRUE then the bucket will be retained when the stack is deleted.
Let’s create the necessary directory structure.
mkdir -p tests/unit
touch tests/unit/test_log_bucket.py
We will start off with our imports and a pytest fixture.
from pathlib import Path
import pytest
from cloud_radar.cf.unit import Template
@pytest.fixture(scope='session')
def template_path() -> Path:
base_path = Path(__file__).parent
template_path = base_path / Path('../../templates/log-bucket.template.yaml')
return template_path.resolve()
We can now write a test for when KeepBucket is FALSE.
def test_ephemeral_bucket(template_path: Path):
# Create a Template object using the path
# to a Cloudformation template.
template = Template.from_yaml(template_path)
prefix = "hypermodern-cf"
region = "us-west-2"
# Create a dictionary of parameters for our template.
params = {"BucketPrefix": prefix, "KeepBucket": "FALSE"}
# Render the template using our parameters and region.
result = template.render(params, region)
The return of template.render() is a dictionary of the template we are testing but now it has all of its Cloudformation Intrinsic functions and Conditionals solved.
import json
print(json.dumps(result, indent=4, default=str))
{
"AWSTemplateFormatVersion": "2010-09-09",
"Conditions": {
"DeleteBucket": true,
"RetainBucket": false
},
"Description": "Creates an S3 bucket to store logs.",
"Outputs": {
"LogsBucketName": {
"Description": "Name of the logs bucket.",
"Export": {
"Name": "hypermodern-cf-LogsBucket"
},
"Value": "LogsBucket"
}
},
"Parameters": {
"BucketPrefix": {
"AllowedPattern": "[a-z0-9\\-]+",
"ConstraintDescription": "use only lower case letters or numbers",
"Description": "The name of the Application or Project using this bucket.",
"MinLength": 2,
"Type": "String",
"Value": "hypermodern-cf"
},
"KeepBucket": {
"AllowedValues": [
"TRUE",
"FALSE"
],
"Default": "FALSE",
"Description": "Keep the bucket if the stack is deleted.",
"Type": "String",
"Value": "FALSE"
}
},
"Resources": {
"LogsBucket": {
"Condition": "DeleteBucket",
"Metadata": {
"cfn_nag": {
"rules_to_suppress": [
{
"id": "W35",
"reason": "This template will later serve to create a audit bucket with retain on delete."
}
]
}
},
"Properties": {
"BucketEncryption": {
"ServerSideEncryptionConfiguration": [
{
"BucketKeyEnabled": true,
"ServerSideEncryptionByDefault": {
"SSEAlgorithm": "AES256"
}
}
]
},
"BucketName": "hypermodern-cf-logs-us-west-2"
},
"Type": "AWS::S3::Bucket"
},
"LogsBucketPolicy": {
"Condition": "DeleteBucket",
"Properties": {
"Bucket": "LogsBucket",
"PolicyDocument": {
"Statement": [
{
"Action": [
"s3:PutObject"
],
"Effect": "Allow",
"Principal": "arn:aws:iam::555555555555:root",
"Resource": "arn:aws:s3:::LogsBucket/*"
}
]
}
},
"Type": "AWS::S3::BucketPolicy"
}
},
"Metadata": {
"Cloud-Radar": {
"Region": "us-west-2"
}
}
}
We should first start with making sure our template created the correct resources.
# Make sure the correct resources are being created.
assert "RetainLogsBucket" not in result["Resources"]
assert "RetainLogsBucketPolicy" not in result["Resources"]
assert "LogsBucket" in result["Resources"]
assert "LogsBucketPolicy" in result["Resources"]
Now that we know the proper resources were created we can test each one. The bucket only had a single Sub in its BucketName property so its test is very simple.
LogsBucket:
Metadata:
cfn_nag:
rules_to_suppress:
- id: W35
reason: "This template will later serve to create a audit bucket with retain on delete."
Condition: DeleteBucket
Type: AWS::S3::Bucket
Properties:
BucketName:
!Sub ${BucketPrefix}-logs-${AWS::Region}
BucketEncryption:
ServerSideEncryptionConfiguration:
- BucketKeyEnabled: true
ServerSideEncryptionByDefault:
SSEAlgorithm: AES256
# Test Bucket Policy
policy_resource = result["Resources"]['LogsBucketPolicy']
attached_bucket = policy_resource['Properties']['Bucket']
statement = policy_resource['Properties']['PolicyDocument']['Statement'][0]
We can also test that BucketPolicy is attached to the correct bucket and the policy statement has the desired Resource and Principal.
LogsBucketPolicy:
Condition: DeleteBucket
Type: 'AWS::S3::BucketPolicy'
Properties:
Bucket: !Ref LogsBucket
PolicyDocument:
Statement:
- Action:
- 's3:PutObject'
Effect: Allow
Resource: !Join
- ''
- - 'arn:aws:s3:::'
- !Ref LogsBucket
- /*
Principal: !Sub arn:aws:iam::${AWS::AccountId}:root
# Test Bucket Policy
policy_resource = result["Resources"]['LogsBucketPolicy']
attached_bucket = policy_resource['Properties']['Bucket']
statement = policy_resource['Properties']['PolicyDocument']['Statement'][0]
assert attached_bucket == 'LogsBucket'
assert 'LogsBucket/*' in statement['Resource']
assert template.AccountId in statement['Principal']
The last thing to test is the Outputs section.
Outputs:
LogsBucketName:
Description: Name of the logs bucket.
Value: !If [RetainBucket, !Ref RetainLogsBucket, !Ref LogsBucket]
Export:
Name: !Sub ${BucketPrefix}-LogsBucket
# Test outputs
bucket_output = result['Outputs']['LogsBucketName']
assert bucket_output['Value'] == 'LogsBucket'
assert prefix in bucket_output['Export']['Name']
Now that we tested KeepBucket is FALSE, we need to test for TRUE. In this situation, it’s pretty much the same test with the logic inversed. I have gone ahead and done that for us so let’s run pytest now.
[~/repos/dsty/hypermodern-cloudformation] shadycuz->
Precommit
Now that we have working unit tests we just need to modify our .pre-commit-config.yaml to run them for every commit. We will be adding our new hook to the local repo.
- id: pytest
name: pytest
entry: pytest
language: system
pass_filenames: false
types_or: [python, yaml]
We can test out pre-commit with pre-commit run --all-files.
[~/repos/dsty/hypermodern-cloudformation] shadycuz->
I’m going to add a .gitignore to avoid committing a .pyc file. Then we can commit our changes.
[~/repos/dsty/hypermodern-cloudformation] shadycuz->
We can now test Cloudformation specific logic in our template without deploying to AWS!
If you have been following along with the companion repo then your branch should look similar to this. In the next section we will add Functional Tests so that we can deploy our template to multiple regions and ensure it actually works.