Python Framework for CAD Environment Test Automation

I designed and developed a Python-based automated testing framework to validate the reliability of secure CAD design environments used by semiconductor engineers at NXP Semiconductors.

During a large-scale cloud migration involving more than 12 AWS-hosted data center environments, we needed to ensure that engineers could continue using their design tools without disruption, despite major infrastructure changes involving networks, firewalls, proxies, storage, and system configurations.

Previously, environment validation relied on approximately 400 manual test cases maintained in Excel and executed by different IT and CAD engineering teams. I transformed this process into a configurable testing framework built with Python and pytest, expanding the original test coverage into more than 1,000 automated checks covering everything from basic system configurations to complete CAD project initialization workflows.

The framework can be executed on demand through a CLI or automatically through GitLab CI whenever a new VDI profile is deployed. It provides structured test results, identifies regressions, and helps engineering teams detect environment issues before they affect designers.

Project Context - Ensuring Reliable CAD Environments after Cloud Migration

NXP Semiconductors operates globally, with CAD engineering teams relying on complex design environments to develop and validate semiconductor products.

These environments run on Red Hat Enterprise Linux and are accessed through secure Virtual Desktop Infrastructure (VDI). A typical engineering workflow involves numerous dependencies, including internal applications, commercial CAD tools, Unix permissions, shared storage, authentication services, software licenses, and project-specific configurations.

During our migration from on-premises infrastructure to AWS, more than 12 cloud data center environments were created. This introduced significant changes to the underlying infrastructure:

• Network routing, firewall rules, and proxy configurations
• Storage locations and access permissions
• Authentication and Unix group memberships
• Application dependencies and system configurations
• Connectivity between engineering tools and backend services

We therefore needed a reliable way to validate that every new environment was fully operational. Beyond the initial migration, the same challenge remained whenever IT teams deployed patches, upgraded applications, changed infrastructure configurations, or released new VDI profiles. The objective was to make environment validation repeatable, comprehensive, and automated, while detecting regressions before they impacted CAD engineers.

Transforming 400 Manual Tests into 1,000+ Automated Checks

I didn't start completely from scratch. Before the project, different IT engineers and CAD specialists were responsible for manually validating their respective areas of the environment. Their testing procedures were documented in an Excel spreadsheet containing approximately 400 test cases.

I used this existing knowledge as the foundation for the automated testing framework. Working with the different technical teams, I reviewed the manual procedures and translated them into executable Python tests. During this process, I also identified opportunities to improve the test coverage.

Some existing tests covered multiple behaviors in a single manual procedure. I split these into smaller, independent checks to make failures easier to identify and investigate. We also introduced additional tests based on discussions with engineers and newly identified requirements.

The result was a test suite containing more than 1,000 automated checks, covering different tools, infrastructure components, and engineering workflows.

Developing a Configurable Testing Framework with Python and pytest

I developed the framework as a reusable Python package built around pytest. The architecture allows tests to be organized into configurable test sets, each associated with a specific application, infrastructure component, or engineering domain. This makes it possible to execute either targeted validations or a broader environment regression campaign. The automated checks cover several categories. For example :

• System and Command Validation : Execute Linux commands and verify their behavior, including exit codes, command output, generated logs, and expected files.
• Infrastructure and Resource Access : Validate access to shared storage, project directories, network resources, and other infrastructure dependencies.
• Unix Accounts and Permissions : Verify the existence of required Unix accounts, group memberships, and access permissions.
• Application and Dependency Validation : Check that CAD applications, internal tools, software configurations, and required dependencies behave as expected.
• End-to-End Engineering Workflows : Execute more complex scenarios, such as initializing an entire CAD design project, to verify that multiple infrastructure components work together correctly.

Some tests involve several validation steps. For example, after executing a command, a test might need to verify:

• That the command returned the expected exit code.
• That its output contains the expected information.
• That a log file was created without unexpected errors.
• That the expected files or directories were generated.
• That the resulting environment is ready for the next operation.

pytest was particularly useful for structuring these checks and reporting failures clearly.

I also implemented tests covering expected failures. For example, when validating access restrictions, a command executed without the required permissions should fail. In this situation, a correctly rejected operation represents a successful test. This allows the framework to validate security restrictions and negative scenarios, rather than only checking successful operations.

Automated Execution Through CLI and GitLab CI

The next challenge was making the testing framework easy to execute across different environments and deployment scenarios. I provided two execution methods.

• Command-Line Interface The package exposes a CLI that can be launched directly from a RHEL VDI. Engineers can select the relevant test sets and run validations against a specific design environment. This is useful during infrastructure troubleshooting, new data center deployments, and targeted regression investigations.
• GitLab CI Integration I also integrated the framework into GitLab CI pipelines. Whenever the VDI engineering team pushes a new VDI profile, the automated validation suite is executed as part of the build process. This helps detect configuration issues and regressions earlier in the deployment lifecycle.

Some tests require access to specific Unix groups, project accounts, or protected engineering resources. To ensure consistent execution, the framework uses a dedicated service account configured with the required permissions and group memberships. This allows tests to run with the necessary prerequisites without depending on individual engineers' personal accounts.

The Next Challenge: Keeping the Test Suite Relevant

One important lesson from this project is that automating tests is only the first step; maintaining their relevance is an ongoing challenge. CAD environments are constantly evolving. Applications are upgraded, infrastructure is modified, security policies change, and new engineering workflows are introduced. As a result, a test that was valid six months ago may no longer reflect the expected behavior of the environment.

We identified three recurring situations:

• Tests requiring updates
• Obsolete tests
• Missing coverage

Another challenge is distinguishing an actual regression from a test that has become outdated. A failing test doesn't always mean the environment is broken. Sometimes, the environment has changed intentionally and the test expectations need to be updated. Potential improvements include clearer test ownership, regular review sessions with application teams, and better integration between infrastructure changes and test updates.

I was the unique developer on this project, so I've done everything including :

• Analyze existing tests, common issues, current VDI environment and used tools to define a comprehensive test plan,
• Develop a Python/Pytest package as a test framework, allowing the configuration of tousands of technical tests,
• Configured 1000+ unit tests, ensuring a good coverage regarding the NXP environment.

• 1,000 automated environment validation checks instead of 400 manual Excel-based procedures,
• Supported the validation of 12+ AWS cloud data center environments
• Integrated environment validation with GitLab CI so that tests run whenever new VDI profiles are built.
• Provided a configurable Python Framework and CLI, allowing easy evolution in the future.