- 1. AWS Lambda at a glance
- 2. Why is test coverage within a Lambda function so important?
- 3. Why is test coverage within a Lambda function so important?
- 4. What exactly is PyTest?
- 5. Practical applications of unit testing
- 6. Techniques of testing AWS Lambda functions
- 7. How to start unit testing with the PyTest framework
- 8. Practical application of more advanced unit testing and mocking of AWS services for Lambda
- 9. How to interpret tests
- 10. FAQ
AWS Lambda at a glance
Let’s start with the fact that serverless services are currently some of the most advanced features of cloud computing. Today, every cloud provider has function-as-a-service solutions in their offer. In the project I am involved in, we use AWS Lambda (read more about its capabilities in this article: AWS Lambda Functions).
In a nutshell, AWS Lambda is a serverless service provided by Amazon Web Services that allows developers to run code in the cloud in response to various events. The serverless model means that infrastructure management is on the cloud provider’s side. This makes it just perfect for developing applications requiring rapid scaling and efficient resource management.
Also read:
The testing challenge. Why do you need to test serverless applications?
What do standard applications and serverless applications have in common? Both should pass unit tests. Testing serverless Lambda function code is quite a challenge. Using my project as an example, I would like to present the challenges we face as a team in the context of the unit testing of our application.
Firstly, AWS Lambda functions are often tightly integrated with other cloud resources, such as databases or message queues. This can cause problems if developers want to isolate unit tests, as we need to create tests independent of other components. In any scenario, we need to ensure that our unit tests do not affect the state of other services (such as changing records in a database).
Another challenge is handling asynchronous communication. In many scenarios, our application responds to asynchronous events such as processing an incoming message from a queue. Testing such cases requires appropriate strategies to keep control of the data flow and ensure that the application behaves as we expect.
Why is test coverage within a Lambda function so important?
Before getting into the practical aspects of unit testing, let’s see why it is so important, especially in the context of Lambda functions.
Keeping functions working correctly
Lambda functions are often small pieces of code invoked by specific events. Unit tests help ensure that these functions behave as expected in different scenarios.
Quick error detection
AWS Lambda can respond to various events. If your unit tests are well-written, it can quickly detect errors in the code while the product is still being developed. Unit testing is crucial, as it allows you to avoid introducing errors into the production environment and thus ensures that the application is reliable at this stage.
Shortening the development cycle
Unit testing enables developers to iterate quickly when making changes to code. By shortening the development cycle, the application is developed faster and the value is delivered quicker.
Now that we know why unit testing is important, let’s consider how to do it effectively with the PyTest framework.
What exactly is PyTest?
PyTest is a tool for testing code in Python which allows you to write both simple and complex tests. Among the available tools, Selenium, Cypress, Playright, or Robot Framework are popular (we have already discussed them for you in other articles and on webinars). It’s time to take a closer look at PyTest, which is particularly useful in an AWS Lambda function environment, as it allows local testing before deployment.
What is more, thanks to its integration with mocking libraries such as Moto, you can simulate (“mock”) AWS services directly in your local environment. This will allow you to test your code thoroughly before deploying to the AWS cloud.
Elevate Your Application Development
Our tailored Application Development services meet your unique business needs. Consult with Marek Czachorowski, Head of Data and AI Solutions, for expert guidance.
Schedule a meetingPractical applications of unit testing
1. Handling expected errors
Let’s assume we are developing a Lambda function to process online orders. The unit tests may include error-handling scenarios, such as no connection to the database or a problem with a third-party payment provider. By ensuring that the function handles these situations correctly, we can increase the system’s resilience to failures.
# An example of code for error-handling
def test_error_handling():
# We simulate a database communication error
event = {"order_id": "123"}
result = process_order(event, None)
assert result == {"statusCode": 500, "message": "Internal server error"}
2. Validation of input data
Lambda functions are often called by events, and the input data is crucial for the function to work correctly. Unit tests can cover different instances of input data. You gain confidence that the function validates them correctly and responds as expected.
#Example of a unit test for validating input data
def test_input_validation():
event = {"customer_id": "123", "order_total": 50.0}
result = process_order(event, None)
assert result == {"statusCode": 200, "message": "Order processed successfully"}
# We test invalid input data
event_invalid = {"customer_id": "abc", "order_total": "invalid"}
result_invalid = process_order(event_invalid, None)
assert result_invalid == {"statusCode": 400, "message": "Invalid input data"}
3. Integrations with AWS services
Lambda functions often use other AWS services, such as the storage service S3 or the NoSQL database service DynamoDB. We can use mocking techniques to avoid actual calls to these services while testing. PyTest offers the pytest-mock module, which makes it easy to create mocks for selected services.
import pytest
from my_module import process_s3_event
# An example of a unit test using pytest-mock to mock an AWS service
def test_s3_upload(mocker):
# We create a mock of the S3 client
s3_client_mock = mocker.Mock()
# We update the (“patch”) "boto3.client" function in such a way that it returns our mock
mocker.patch("boto3.client", return_value=s3_client_mock)
# We create an example of an event
event = {"data": "example"}
# We call the function that processes the event
result = process_s3_event(event, None)
# We check if the function has returned the expected result
assert result == {"statusCode": 200, "message": "S3 event processed successfully"}
# We check that the "upload_file" method on the S3 client's mockup was called exactly once
s3_client_mock.upload_file.assert_called_once_with("source_file", "bucket", "destination_key")
4. Multi-layer applications
In a microservice architecture, where Lambda functions act as microservices, unit testing becomes even more complex. A unit testing framework like PyTest allows you to create layered tests that check integrations between various microservices.
import pytest
from my_module import process_microservices
# An example of a multi-layer unit test
def test_microservices_integration(mocker):
# We create a mock for Lambda A function
lambda_a_mock = mocker.Mock()
# We create a mock for Lambda B function
lambda_b_mock = mocker.Mock()
# We define what data will be passed between microservices
event_a_to_b = {"data": "example_data_from_A_to_B"}
event_b_to_a = {"data": "example_data_from_B_to_A"}
# We determine what should be returned by mocks of the Lambda function
lambda_a_mock.return_value = {"statusCode": 200, "message": "Response from Lambda A"}
lambda_b_mock.return_value = {"statusCode": 200, "message": "Response from Lambda B"}
# We patch the imports in the process_microservices function to use our mocks
mocker.patch("your_module.lambda_a_function", lambda_a_mock)
mocker.patch("your_module.lambda_b_function", lambda_b_mock)
# We call the process_microservices function with sample data
result = process_microservices(event_a_to_b, None)
# We check if the function has returned the expected result after the integration of microservices
assert result == {"statusCode": 200, "message": "Microservices processed successfully"}
# We check that Lambda functions A and B were called with the right data
lambda_a_mock.assert_called_once_with(event_a_to_b, None)
lambda_b_mock.assert_called_once_with(event_b_to_a, None)
Techniques of testing AWS Lambda functions
1. Fault tolerance testing
This is a key component of AWS Lambda function testing. Fault tolerance testing covers scenarios in which one of the AWS services working with the function is unavailable or returns errors. An example of the test may look like this:
import pytest
from unittest.mock import patch, Mock
# Example of a fault-tolerance test
def test_error_resilience(mocker):
# We mock a service that returns an error
external_service_mock = mocker.Mock()
external_service_mock.side_effect = Exception("Service unavailable")
mocker.patch("boto3.client", return_value=external_service_mock)
event = {"data": "example"}
result = process_event(event, None)
assert result == {"statusCode": 500, "message": "Internal server error"}
2. Test-Driven Development (TDD) tests
The practice of TDD means writing unit tests before implementing functions. This allows for a more informed code design and ensures that tests cover every line of code.
Below is an example of TDD for a Lambda function:
import pytest
# Example of unit testing with TDD
def test_tdd():
event = {"data": "example"}
result = process_event(event, None)
assert result == {"statusCode": 200, "message": "Event processed successfully"}
# We add a new functionality and write a test for it first
new_feature_result = new_feature(event)
assert new_feature_result == {
"statusCode": 200,
"message": "New feature processed successfully",
}
3. Using Fixture in PyTest
PyTest Fixtures are functions that run before tests (and optionally after them) that can prepare the necessary resources. For example, a fixture can create a fake input or configure a mock of any AWS resource before running tests. This way, each test runs in an isolated and controlled environment.
import boto3
import pytest
from moto import mock_s3
@pytest.fixture()
def mock_s3_resource():
with mock_s3():
# We use Moto library to simulate the S3 service
import boto3
s3 = boto3.resource("s3", region_name="us-east-1")
# Crearing mock – AWS S3 resource
s3.create_bucket(Bucket="mocked-bucket")
yield s3
# Teardown (resource cleansing) is not required, Moto performs automatic cleansing
# We use a decorator to turn on S3 simulation for the test
@mock_s3
def test_s3_operations(mock_s3_resource):
s3 = mock_s3_resource
# Perform requested operations on S3
s3.Object("mocked-bucket", "test.txt").put(Body="Hello, World!")
# Check if the performed operations were successful
obj = s3.Object("mocked-bucket", "test.txt").get()
assert obj["Body"].read().decode("utf-8") == "Hello, World!"
4. Parameterization of tests
Parameterization allows you to run the same test with different input data sets. Using the @pytest.mark.parametrize decorator, you can specify different data sets for a single test function. This way, you can easily verify the performance of Lambda functions in different scenarios without writing multiple similar tests.
@pytest.mark.parametrize(
"input, expected",
[
({"data": "example1"}, {"statusCode": 200, "message": "Success"}),
({"data": "example2"}, {"statusCode": 400, "message": "Error"}),
],
)
def test_lambda_function(input, expected):
result = lambda_handler(input, None)
assert result == expected
How to start unit testing with the PyTest framework
PyTest installation
- Make sure you have Python installed on your system.
- Open a terminal or command line.
- To install PyTest, type the following command:
pip install pytest
3. If you plan to use libraries like Moto for more advanced testing, of course, you will need to install them as well – for example:
pip install moto
Test preparation
- Prepare your unit tests in Python (.py) files that contain test functions. Your tests should be organized following the PyTest convention, in which test function names start with “test_”.
Running tests
- In the terminal, navigate to the directory where your test files are located, or, if they are in the same directory as the terminal, you can skip this step.
- To execute all defined tests from a given file, use the following command, specifying the name of the test file:
pytest file_name.py
For example, if your tests are in the my_tests.py file, the command will look like this:
pytest my_tests.py
If you want to test all files located in the current directory, use the command:
pytest .
Test results analysis
- Once the tests are executed, PyTest will provide detailed information about the test run and its results. You will see which tests succeeded, which failed, and why.
Practical application of more advanced unit testing and mocking of AWS services for Lambda
Now that you know what the PyTest framework is, what unit tests are for, and how they can be used with various techniques, it’s time to move on to a more advanced, practical application of unit testing and AWS service mocking for Lambda functions. When developing serverless event-driven applications using AWS Lambda, the best practice is to validate individual components and services.
Now let’s get into an example of a function for handling notifications in the cloud. Imagine that we have an application that needs to quickly notify users about various events – from a meeting reminder to a sale notification.
That’s where AWS SNS, a service that allows you to send notifications in the form of messages to a wide range of subscribers, comes in. We can think of it as a cloud-based instant messaging system. Here, the AWS Lambda function acts as an intermediary – it accepts specific requests and forwards them to SNS, which then handles the distribution of messages.
How does it work? The process is simple:
- Message preparation: The Lambda function takes two key elements – the identifier of the SNS subject (to whom we want to send the notification) and the message content.
- Sending the message: The function communicates with the SNS using a special API to deliver the message to the specified subject.
You may be wondering: how can we trust that this function works properly? The answer lies in unit testing itself. We run test scenarios to make sure our code does exactly what it’s supposed to do in different situations. We simulate (“mock”) the operation of the SNS service to see how the function behaves when everything goes successfully, and how it handles any errors.
Before writing the first unit test, we will look at an example of the AWS Lambda function we want to test:
main.py:
import boto3
import json
def send_notification(sns_topic_arn, message):
if not message:
raise ValueError("Message cannot be empty")
try:
sns_client = boto3.client("sns")
response = sns_client.publish(TopicArn=sns_topic_arn, Message=message)
return response
except boto3.exceptions.Boto3Error as e:
print(f"Error sending notification: {e}")
raise
def lambda_handler(event, context):
# Parsowanie zdarzenia wejściowego
sns_topic_arn = event.get("sns_topic_arn")
message = event.get("message")
# Validation of input data
if not sns_topic_arn or not message:
return {"statusCode": 400, "body": json.dumps("Missing sns_topic_arn or message")}
try:
# Calling function sending notification
response = send_notification(sns_topic_arn, message)
return {"statusCode": 200, "body": json.dumps("Notification sent successfully")}
except Exception as e:
return {"statusCode": 500, "body": json.dumps(str(e))}
import pytest
import boto3
from moto import mock_sns
from unittest.mock import patch
from main import lambda_handler # Importing from a handler
# Preparing test environment
@mock_sns
class TestLambdaFunction:
# Test to see if we correctly handle the situation when input is missing
def test_missing_data(self):
response = lambda_handler({"sns_topic_arn": "", "message": ""}, None)
assert response["statusCode"] == 400
assert "Missing sns_topic_arn or message" in response["body"]
# Test to see if the function sends the notification correctly
@patch("main.boto3.client")
def test_send_notification_success(self, mock_boto3_client):
mock_sns_client = boto3.client("sns")
mock_boto3_client.return_value = mock_sns_client
mock_sns_client.publish.return_value = {"MessageId": "12345"}
response = lambda_handler(
{
"sns_topic_arn": "arn:aws:sns:region:123456789012:testTopic",
"message": "Test message",
},
None,
)
assert response["statusCode"] == 200
assert "Notification sent successfully" in response["body"]
# Test to check how the functions cope with SNS errors
@patch("main.boto3.client")
def test_sns_error(self, mock_boto3_client):
mock_sns_client = boto3.client("sns")
mock_boto3_client.return_value = mock_sns_client
mock_sns_client.publish.side_effect = boto3.exceptions.Boto3Error
response = lambda_handler(
{
"sns_topic_arn": "arn:aws:sns:region:123456789012:testTopic",
"message": "Test message",
},
None,
)
assert response["statusCode"] == 500
How to interpret tests
- test_missing_data: We check if the function responds correctly when we do not provide the required data. We expect a response with error code 400, meaning the query was invalid.
- test_send_notification_success: Here we simulate a situation in which everything goes successfully (the so-called “happy path”). We use mock_boto3_client to simulate the interaction with SNS, and then check if the function returns a success code of 200 and the corresponding message.
- test_sns_error: We test how the function will behave when the SNS reports an error. Again, we simulate interactions with the SNS and check if the function handles the error correctly, returning a status code of 500.
Remember, these tests are just examples. You may want to test more scenarios to make sure a function works as it should in all possible situations. Unit tests can greatly improve the quality and reliability of your code, so it’s worth paying proper attention to them!
FAQ – frequently asked questions
What and how should we test in AWS Lambda functions?
Testing AWS Lambda functions should cover all key aspects, such as business logic, error handling, interactions with AWS services, and performance. It’s also a good idea to test functions with different inputs to make sure they behave correctly in various scenarios.
Why should you test your AWS Lambda functions?
Testing AWS Lambda functions is key to maintaining high-quality code and application functionality. Through unit testing, developers can quickly identify and eliminate errors, resulting in more reliable and efficient applications.
AWS cloud function testing in Python – summary
Unit testing of AWS Lambda functions with PyTest is more than just a code validation process! It’s also an opportunity to develop more stable, reliable, and efficient applications. Practical usages of unit testing include error handling, input validation, integration with AWS services, performance optimization, and fault tolerance testing. Advanced techniques, such as using PyTest fixtures and test parameterization, allow you to keep even tighter control of code quality. Unit testing is not only a pre-implementation step, but also a key part of continuous application improvement in the dynamic world of the cloud.
I hope this article has helped you understand how we can use unit testing and AWS service mocking to ensure the reliability and correctness of our AWS Lambda functions.
Consult your project directly with a specialist
Book a meeting- 1. AWS Lambda at a glance
- 2. Why is test coverage within a Lambda function so important?
- 3. Why is test coverage within a Lambda function so important?
- 4. What exactly is PyTest?
- 5. Practical applications of unit testing
- 6. Techniques of testing AWS Lambda functions
- 7. How to start unit testing with the PyTest framework
- 8. Practical application of more advanced unit testing and mocking of AWS services for Lambda
- 9. How to interpret tests
- 10. FAQ
"Passionate about cloud technologies, especially AWS, and a Python programming enthusiast. His experience and passion for creating scalable and innovative cloud solutions mean that he constantly desires to take on new challenges. Outside of work, he is interested in sports, personal development, and is also a car enthusiast."