Intelligent Stimulus Selection to Accelerate Coverage Closure of Simulation-Based Verification for Digital Designs

Abstract

Verification is a critical step in developing digital designs, ensuring that their functionality
complies with specified requirements. However, as digital designs grow increasingly
complex, verification has become more time-consuming and demands greater engineering
efforts. Simulation-based verification, a major functional verification technique, has
become less effective for large-scale designs. The simulation of randomly selected stimuli
yields diminishing functional coverage gains as verification progresses. Consequently,
redundant tests consume verification resources such as computing power and project
time while verifying very few new functionalities. This redundancy slows down the
verification process, compelling engineers to manually create more tests to cover the
remaining intricate functionalities through trial and error.
Recent studies have shown that machine learning (ML) algorithms can identify the
novel tests that are more likely to yield new coverage in small-scale designs. This research
examines the feasibility and efficiency of embedding ML-based novel test selectors into
the functional verification flow of commercial designs to significantly reduce simulation
redundancy compared to random test selection.
The original contributions of this research are:
• The absence of positive tests for coverage holes presents a significant challenge
that hinders supervised ML algorithms from effectively estimating the relationship
between tests and coverage tasks. While this issue has been recognized and explored
in existing literature, there is a lack of evidence that demonstrates the severity of
this obstacle. This thesis addresses this gap by providing experimental evidence
that illustrates how the scarcity of positive tests for coverage holes and rarely-hit
coverage tasks can bias both the training and prediction phases of the supervised
algorithms used to estimate the relationship. Additionally, this thesis demonstrates that supervised neural networks, when deployed as novel test selectors, can still
improve functional coverage progress without being affected by such challenges.
• Introducing several novel ML-based test selectors capable of accelerating functional
coverage closure for commercial designs, outperforming random test selection. For
instance, in the verification of a signal processing unit (SPU), a 46.99% simulation
saving can be achieved to reach 99.5% coverage. In the verification of a bus bridge,
a 26.14% reduction in test simulation is observed to reach 98.5% coverage. Notably,
this study is the first to demonstrate the effectiveness and efficiency of using novel
test selectors to capture the sequential dependencies and correlations between
stimuli.
• Identifying key characteristics related to the performance of neural network-based
novel test selectors, which include the property of neurons, novelty score function
and training data. These characteristics may inspire the development of test
selectors with further enhanced performance. The effectiveness, efficiency, and
performance of a novel test selector are measured by the number of tests required
to reach the specified functional coverage goals and the computational expense of
the test selector.
The findings and methodologies presented in this thesis can guide future efforts in
optimizing verification processes for increasingly complex digital designs.

Date of Award	4 Feb 2025
Original language	English
Awarding Institution	University of Bristol
Supervisor	Kerstin I Eder (Supervisor), Timothy D Blackmore (Supervisor) & James Buckingham (Supervisor)