Implementing Automated Smoke Testing with Amazon Nova Act in CI/CD Pipelines

Enhancing CI/CD with Fast, Reliable Testing

Overview of Automated Smoke Testing

Why Smoke Testing Matters in CI/CD

Leveraging AI for Smarter Testing

Implementing Automated Smoke Tests: A Step-by-Step Guide

Prerequisites for Setting Up Your Testing Environment

Project Setup and Dependency Management

Building a Test Runner for Basic Smoke Tests

Validating Login Functionality with Smoke Tests

Expanding Testing to Comprehensive Ecommerce Workflows

Integrating Automated Testing into CI/CD Pipelines

Utilizing Headless Mode for CI Environments

Configuring GitLab CI/CD for Automated Tests

Effective Management of CI/CD Variables

Introducing Parallel Execution for Enhanced Performance

Best Practices for Scaling and Maintaining Tests

Conclusion: Achieving Quality Through Automation

About the Authors: Experts in the Field of Automation and Cloud Solutions

Automated Smoke Testing with Amazon Nova Act in CI/CD Pipelines

In today’s fast-paced development environment, where code is deployed multiple times a day, maintaining application quality is paramount. Automated smoke testing using Amazon Nova Act in headless mode allows development teams to validate core functionality quickly and efficiently within their Continuous Integration and Continuous Delivery (CI/CD) pipelines. Traditional end-to-end testing, often taking hours, can create bottlenecks, making fast and efficient testing crucial.

Understanding Smoke Testing

Smoke testing is a critical subset of testing that focuses on validating the application’s essential functions post-deployment. It emphasizes key workflows—such as user login, core navigation, and critical transactions—over exhaustive feature coverage. Typically, these tests can be executed in mere minutes, providing rapid feedback on code changes crucial for CI/CD.

The Role of Amazon Nova Act

Amazon Nova Act revolutionizes the testing landscape by employing AI-powered UI understanding and natural language processing. This enables interaction with web applications without relying on brittle CSS selectors. Instead, developers can write tests in plain English, adapting seamlessly to UI changes.

This blog post will guide you through implementing automated smoke testing using Amazon Nova Act’s headless mode in CI/CD pipelines, with SauceDemo—a sample ecommerce application—as our demonstration target.

Solution Overview

The proposed solution features:

1. A Python test runner to execute smoke tests.
2. Comprehensive ecommerce workflow validation covering entire user journeys.
3. GitLab CI/CD integration for automated testing.
4. Configuration for parallel execution to enhance testing efficiency.

Headless mode runs browser tests in the background without necessary UI windows, streamlining the automated testing process.

Steps to Implement Automated Smoke Testing

1. Set up Project and Dependencies.
2. Create a Smoke Test for Login Validation.
3. Add E-commerce Workflow Validation.
4. Configure the Automated Testing Pipeline.
5. Implement Parallel Execution.

Prerequisites

Before starting, make sure you have:

• A working Python environment.
• Access to Amazon Nova Act and API keys.
• GitLab account for CI/CD integration.

1. Set Up Project and Dependencies

Create your project and install the required dependencies using the UV package manager:

# Create and navigate to project
uv init nova-act-smoke-tests 
# Open in VS Code 
code nova-act-smoke-tests 
# Install required packages 
uv add nova-act 

2. Create a Test Runner

Create smoke_tests.py and include initial code to verify your set-up:

import os 
from nova_act import NovaAct

# Check API Key
if not os.getenv("NOVA_ACT_API_KEY"): 
    exit("❌ Set NOVA_ACT_API_KEY environment variable")

SAUCEDEMO_URL = "https://www.saucedemo.com/"
with NovaAct(starting_page=SAUCEDEMO_URL) as nova:
    nova.act("Verify you are in the login page")

print("✅ Foundation setup complete!")

3. Test Your Setup

Run your initial setup with:

export NOVA_ACT_API_KEY="your-api-key" 
uv run smoke_tests.py

4. Create Smoke Tests for Login Validation

Expand your tests to include a complete login flow:

def test_login_flow():
    with NovaAct(starting_page=SAUCEDEMO_URL) as nova:
        nova.act("Enter 'standard_user' in the username field")
        nova.act("Enter 'secret_sauce' in the password field")
        nova.act("Click the login button")
        nova.act("Verify Products appear on the page")

5. Configure the E-commerce Workflow Validation

Extend your smoke tests to validate the complete ecommerce journey from login to logout:

def test_ecommerce_workflow():
    with NovaAct(starting_page=SAUCEDEMO_URL) as nova:
        # Complete ecommerce steps...

6. Configure the Automated Testing Pipeline

The next step is to integrate these tests into your CI pipeline. Here we set up GitLab CI/CD to automate smoke tests on every code change.

Create a .gitlab-ci.yml file:

stages:
  - test

smoke-tests:
  stage: test
  image: mcr.microsoft.com/playwright/python:v1.40.0-jammy
  script:
    - uv run python smoke_tests.py
  variables:
    HEADLESS: 'true'

7. Implement Parallel Execution

Maximize efficiency through concurrent test execution. Update your test runner to leverage ThreadPoolExecutor:

from concurrent.futures import ThreadPoolExecutor

def main():
    with ThreadPoolExecutor(max_workers=2) as executor:
        executor.submit(test_login_flow)
        executor.submit(test_ecommerce_workflow)

Best Practices

1. Maintain Test Independence: Ensure tests don’t interfere with each other’s execution.
2. Implement Retry Logic: Handle transient errors gracefully.
3. Configure CI/CD Timeout: Set reasonable timeouts for your testing phases.
4. Regularly Rotate Secrets: Change API keys and monitor their usage.

Conclusion

Through automated smoke testing using Amazon Nova Act’s headless mode, we can significantly enhance our CI/CD pipelines. This solution validates essential functionalities, runs tests rapidly, and incorporates parallel execution for efficiency. As our applications grow, the combination of modern package management and continuous testing ensures rapid feedback and high application quality.

Start implementing automated smoke tests in your applications and optimize your CI/CD processes. For additional resources on browser automation and testing strategies, check out AWS documentation and community forums.

About the Authors

Sakthi Chellapparimanam, Shyam Soundar, and Reena M are AWS Solutions Architects with expertise in cloud infrastructure, automation frameworks, and secure application development. They’re passionate about leveraging technology to solve real-world problems and improve workflows across industries.