Five Vital Steps to a Robust Testbench with DesignWare Verification IP and Reference Verification Methodology (RVM)

Introduction

Verification is one of the biggest challenges for System-on-Chip (SoC) designs, and traditional methods have run out of steam. Writing individual tests is impractical for today’s large, complex designs because the state space and number of test conditions is simply too large to code by hand, leading to insufficient test coverage. Limited project resources and time-to-market pressure is making the problem harder. The repercussions are clear--first-pass silicon success is falling.

To address these challenges, design teams are turning to advanced and unified verification methodologies that leverage multiple technologies. Constrained random verification leverages compute resources and functional coverage technology to provide more testing with less test code development. Setting up a constrained random test environment, however, can seem like a difficult task, especially when you consider that environments need to be flexible, scalable, and reusable. The infrastructure for constrained random verification requires more planning and structure, but the benefits in the end are well worth the investment. DesignWare VIP and the Reference Verification Methodology (RVM) provide the tools to rapidly build a well-architected, advanced verification infrastructure. This approach leverages several verification technologies and tools to provide a unified solution:

• Protocol abstraction and functionality (DesignWare VIP models)
• Large portfolio of standard protocol models
• Constrained random verification
• Functional coverage
• Object-oriented programming and Hardware Verification Language features of OpenVera, available with Vera and VCS Native Testbench (NTB)
• Proven verification approach and methodology
• Proven testbench architecture that provides maximum reuse, scalability and modularity

This paper shows how to start performing constrained random verification quickly and easily with DesignWare VIP and RVM. A few benefits of DesignWare VIP with RVM are described here, followed by an introduction to RVM. This supplies a background for the discussion of the five initial steps to coding a complete constrained random testbench. The concepts and techniques used in this paper are explained and demonstrated, and code examples are provided to show real application of the techniques.

This paper is mainly intended for verification engineers who want to use the DesignWare VIP models and RVM. Readers who are new to RVM can use this as a quick start. For those already familiar with RVM, the DesignWare VIP usage examples show how the RVM methodology is applied when using DesignWare VIP. Working knowledge of OpenVera and object-oriented programming is assumed.

Benefits of Using DesignWare VIP with RVM

The DesignWare VIP models offer many benefits by themselves as they provide proven protocol functionality for the verification engineer. When used in an RVM flow, these and many other benefits can be attained as noted below.

• Improved modularity

RVM promotes a layered testbench architecture and provides a standard object-based interface that connects components within a test environment. Coupled with the natural encapsulation abilities of an object-oriented language, improved modularity is a key element in any RVM system. Better modularity simplifies development and reduces maintenance. DesignWare VIP models provide both protocol functionality and control features in a complete, self-contained package that fully supports the modular architecture of RVM and simplifies the development of RVM testbenches. This gives engineers a modular foundation layer over which they can quickly build a robust testbench.

• Efficiency of Abstraction

RVM is based on object oriented programming (OOP). DesignWare VIP abstracts protocol transactions into objects and provides an object-based interface allowing the engineer to work in logical protocol terms without worrying about implementation details. With protocols becoming more complex, this abstraction is a big boost--dealing with the details of a standard protocol is time-consuming and does not add value to the end product. Another benefit of the modular, layered approach allows the stacking of components to create complex systems. For example, a typical webcam transports video data stacked on top of the USB protocol.

• Rapid creation of complex tests

While modularity enables the construction of complex test infrastructures, constrained random verification and efficiency of abstraction allow the easy development of complex tests. Tests that exercise different scenarios within a given set of constraints can easily be created. In encapsulating protocol functionality, DesignWare VIP allows engineers to code with abstracted objects where creating tests for intricate and complex combinations of transactions can be done quickly. These sequences are used to mirror real-world traffic, create stress or corner-case conditions, or simply cover a wide range of conditions. More conditions are created by simply letting the test run for a longer time.

• Increased Reuse

Opportunities for reuse are pervasive when using DesignWare VIP and RVM. DesignWare VIP models are inherently reusable blocks that all have the same look and feel, simplifying the integration of multiple components. The RVM methodology is architected for maximum reuse and it fosters testbench code that maximizes reusable components. RVM even provides a template for a standard testbench flow which can be customized to suit specific needs. The RVM base classes provide reuse through inheritance. Reuse is a central theme for DesignWare VIP with RVM, enabling maximum leverage of this critical design concept.

Introduction to RVM

Now that we have described the ultimate benefits of using DesignWare VIP models with RVM, it is time to dive into a more detailed look. This section gives an overview of RVM and describes a few basic constructs. to give you a conceptual understanding of the basics and get you going on your way to more effective testbenches. The following section will then dive further into the nuts and bolts of building a first testbench using DesignWare VIP and RVM. It shows how the methodology and concepts are used in actual code.

The first question which must be answered is simply “What is RVM?” RVM stands for Reference Verification Methodology. It consists of three things:

• Set of base classes
• Verification methodology
• Modeling approach

The guiding principles of RVM are to:

• Emphasize constrained random verification
• Maximize reuse
• Minimize test-specific code
• Enable more testing with less code

Verification Methodology

RVM provides a template for an advanced verification methodology. Supporting constrained random verification is different than using a directed, sequential flow. RVM provides a blueprint methodology so that testbench code is effectively organized for constrained random verification. And the resulting structure also supports directed testing so it serves all verification needs. The shift to object-oriented programming techniques, the dynamic nature of constrained random verification, and the need to develop code efficiently are all encapsulated in the RVM methodology. To achieve its goals, RVM prescribes an overall testbench organization which impacts the way that testbench code is written. Here are some highlights:

• Most of the code is dedicated to setting up dynamic processes in advance. Everything is already programmed by the time the test is started.
• Test conditions are controlled by constraints instead of procedural code.
• Tests react to significant events dynamically. For example, a testbench must be able to detect the end of the testing sequence since the length of the sequence is not predefined.
• Checking mechanisms are dynamic. Scoreboards or other self-checking mechanisms store information on-the-fly and sometimes perform all checks on-the-fly.
• Objects use the RVM base classes whenever possible to maximize reuse and guarantee interoperability.
• A standard testbench sequence template is used (build-configure-execute-report). Each testbench uses the template but fills in the details according to its needs.
• Programming orientation shifts from procedures to objects. This is not discussed in detail because it is inherent in the adoption of OpenVera and is outside the scope of this paper.

In sections that follow, the effects on testbench coding are demonstrated using actual code.

Base Classes

In an object-oriented programming environment, a set of base classes form the foundation for the entire system. Base classes provide common functionality and structure. Because most objects will be derived from them, a well-architected set of base classes is essential to the end goal of an effective programming environment.

The classes provided by RVM are specifically designed for verification. They provide base functionality needed for simulation (such as logging) and they support any sort of verification task. The DesignWare VIP models are based upon the same RVM classes that are also available for the end user, allowing easy integration of user code and DesignWare VIP in a verification environment.

Further, RVM provides an actual implementation of its base classes, and is not simply a set of guidelines or recommendations. So, instead of writing your own logging routine, you can reuse the rvm_log class. And, with inheritance, extension, and polymorphism, there is plenty of room for customization.

The base classes referenced in this paper are:

• rvm_channel – object-based interface connects elements in a verification environment
• rvm_data – base class for all data objects (transactions, configuration, etc.)
• rvm_xactor – base classes for models
• rvm_log – standard logging object
• rvm_env – base class for the verification environment

The rvm_channel class defines the basic object-oriented interface between components. It fills a fundamental role in building an object oriented system and is discussed separately here.

To achieve modularity, layering, and a scalable environment, the elements within a verification environment need to connect to each other in a consistent yet flexible way. This allows the pieces to be stacked on top of each other or interchanged. Figure 1 shows several elements in a verification environment that need to be connected. For maximum reuse, these pieces should be portable to other applications. An example would be to reuse the lower layers for many tests.

Figure 1: Channel Interfaces Enable Layering, Modularity and Reuse

Figure 2 shows that a standard interconnect can also be used to build up a complex stack out of simpler layers in the same fashion that networks operate. Each layer performs a specific transformation and interfaces only with the layer above and below it through the interface. The layers can be stacked and interchanged.

Figure 2: Stacking with Channel Interfaces

rvm_channel provides a standard interface which transports objects between the components in a testbench. rvm_channel objects are referred to simply as channels, and they natively handle objects of type rvm_data. The base class is extended in order to support any data object derived from rvm_data. The data objects, which typically represent protocol transactions, can be pushed into and pulled out of a channel. So, to interface two components, one component puts objects into the channel and the other pulls them out. A channel, therefore, is unidirectional and specific to a particular data object class. In this way, the interface between components can be standardized.

Given the definition of the data object class, any component that generates data objects can put them into a channel and any component that can operate on the data objects can pull them out of a channel. Modularity is achieved since each side of the channel needs no knowledge of what is on the other side.

The DesignWare VIP models use channels to transport transaction objects to and from the testbench. Notice the efficiency of this approach. A single base interface class is used in multiple applications supporting multiple protocols with very different transaction types. There is only one code base to maintain and all the models will have the same interface.

Modeling Approach

For a DesignWare VIP user, the modeling approach defined by RVM is already incorporated into the models. This means that the models are RVM compliant and they will operate seamlessly within an RVM environment.

Five Vital Steps to Using DesignWare VIP and RVM

In this section, we apply a few basic RVM concepts and techniques to quickly achieve a basic constrained random testbench. The first five steps to using DesignWare VIP and RVM are:

• Create a test environment
• Configure the models
• Connect and use the channel interfaces
• Generate constrained random stimulus
• Control the test

To illustrate these methods in practical use, code snippets are drawn from a complete example testbench. The testbench shows typical DesignWare VIP and RVM usage and highlights the concepts and techniques described. Although the example code uses the PCIE DesignWare VIP, the methods shown are not specific to the protocol. They can be used with any of the DesignWare VIP models. To obtain the example code, please refer to the instructions at the end of this paper.

Step 1: Create a Test Environment (rvm_env)

The rvm_env base class provides a testbench template and serves to organize both the flow and the code associated with a test. A test environment is created by defining a new class extended from rvm_env. Because it provides an overall structure, the environment class contains or affects the entire test environment. As we will see, the vast majority of code in an RVM testbench is contained in the environment where it can be reused by other tests or projects. Because of this, the rvm_env class has probably the largest impact on user coding. So understanding the rvm_env class is central to effectively writing the environment code. A

nother result of the overarching nature of rvm_env is that it is an excellent vehicle for organizing the discussion of RVM and DesignWare VIP techniques and methods. Therefore the subjects presented in this paper are organized and presented in the context of rvm_env.

The rvm_env class contains several methods that correspond to the basic steps all tests follow:

Most of these methods are declared as virtual in the base class and should be extended by the user. To create a user environment, define a new class extended from rvm_env and extend the methods above to add test-specific code. To retain the core functionality of the base class methods, each extended method must call super. as the first line of code. The code snippet below shows the user extension of rvm_env, creating the class user_env. The user_env class instantiates all the components of the verification environment which may include DesignWare VIP models, user designs, generators, scoreboards, etc. At the end of the extended class is a list of the methods that are extended. We discuss each method separately in subsequent sections.

In addition to the methods that have already been mentioned, rvm_env also contains a run_t method which does not require any user extension. This method binds the individual steps into a test sequence by calling the other methods in the following order:

gen_cfg => build => cfg_dut_t => start_t => wait_for_end_t => stop_t cleanup_t => report

So calling this one method launches the entire test sequence. Next, we see how a test executes in this way in the top-level OpenVera program where user_env is instantiated and then env.run_t is called.

Notice how small the program is when you use RVM! Most code is in the environment, which is a reusable component. The test-specific code is minimized and as much code as possible is common so it is not replicated unnecessarily. This yields a smaller code base to maintain.

You may have noticed that there are also calls to env.gen_cfg and env.build in the program. This shows the flexibility built into the RVM architecture in allowing the user to customize the standard test sequence. rvm_env maintains a list of the standard test sequence methods that have already been called. When run_t is executed, it skips the methods that have already been run. So the standard sequence can be modified to suit the needs of each test.

In the example code, calls to set_verbosity are inserted into the standard sequence by first calling gen_cfg and build. After setting the verbosity for logging, run_t is called to continue with the remainder of the test sequence. You can see how all the models have the same interface and usage. In fact, since the testbench itself uses RVM tasks for logging, it is also controlled in exactly the same manner.

The user inherits structure and base functionality from rvm_env and yet can still customize where needed. Furthermore, the customization is under the user’s control. This is a common theme throughout RVM and the DesignWare VIP models.

In the sections that follow, the individual steps in the test sequence are shown in detail.

Step 2: Configure the Models

The DesignWare VIP models are configured using configuration objects which are provided and are ready for use. Configuration objects can be handled just like any other objects so they can be randomized and passed as an argument to a method. The objects come with constraints so they adhere to protocol limits. These can be controlled or extended as desired to create specific test conditions, or used as is to produce a wide range of stimulus. The test configuration is established in gen_cfg. The code below shows that the configuration objects are randomized and then some of the attributes of the vip_cfg object are manually assigned. This is one approach to controlling the values of attributes.

The code snippet below shows that the testbench uses this configuration technique and declares a test-level configuration object to determine testbench behavior.

The next method in the test sequence is build. This is where elements are constructed, configured and connected. The new() task for all the DesignWare VIP models has an optional argument allowing a configuration object to be specified. The code below shows the vip_cfg and dut_cfg objects that were generated in gen_cfg and supplied to the new task for the models. A model can also be configured after construction using the change_xactor_config method.

}

Step 3: Connect and Use the Channel Interfaces

Channels are used by the DesignWare VIP models in three applications. Each model uses these three channel types in different topologies to support the protocol at hand. The arrangement of the channels supports all verification needs, and it is typical that a particular test does not use all the channels.

• Input channels – These channels transport objects from the testbench to the DesignWare VIP models. A typical application would be for a testbench to generate transactions to be driven on a protocol. These transaction objects are put into the channel by the testbench.
• Output channels – Output channels transport objects from the DesignWare VIP models to the testbench. They convey information to the testbench about activity that has occurred. The models put in the channel objects that are constructed from the activity on the protocol.
• Activity channels – Activity channels are provided by monitor models. They are similar to output channels except that they are intended for monitoring purposes only.

The build method in the previous section shows how to connect the testbench to DesignWare VIP models via the rvm_channel interface. For each protocol, the DesignWare VIP models define data objects to represent transactions. Correspondingly, the models also define a channel object to handle the transaction objects. For example, the PCIE models define dw_vip_pcie_tlp_transaction and dw_vip_pcie_tlp_transaction_channel to represent TLP transactions and the interface that accepts TLP transaction objects. From the example, two channel objects are instantiated in the user_env class:

The channels are then connected to the models via the new() task in build():

The gen_out_chan channel is connected to the TLP input channel on the DesignWare VIP Txrx model. This is used by the testbench to send generated transactions to the model. The sink_in_chan channel is connected to the activity channel of the monitor that is watching the RX port of the DUT. This is used to get transactions from the monitor once they are detected on the bus. Figure 3 shows the resulting testbench topology.

Figure 3: Testbench Connections

The number and arrangement of channels differs in each DesignWare VIP model because of protocol needs, but the connection method is common to all models. Typically, there are several channel connections for any given model and it is likely that not all of them are needed for a particular testbench (of the several channels available on the PCIE models, the example testbench only uses two of them). The channels do not have to be used and can be unconnected. In the new() method for each model, the channel arguments are optional. If a channel is not specified, no channel object is either created or connected. This allows the models to run more efficiently.

Step 4: Generate Constrained Random Stimulus

Constrained random generation makes use of the built-in randomize() method that all objects in OpenVera have. The most common use for random generation is to produce a series of random transactions which follow a set of applied constraints. The example shows a simple generator which produces individual (atomic) transactions randomly with no particular relationship between them. This is the simplest form of generator and can be coded as shown next:

The essential parts of the generator are fairly simple. A while loop determines how many objects to generate (as determined by the test_cfg object). Inside the loop, the randomize function is called. Then a copy of the transaction is created and pushed into the channel interface. The ‘randomize with’ construct is used as a convenient way of applying constraints “on-the-spot”. Of note, however, is the fact that the generator does not need to know anything about the object that it is randomizing. In the code above, the definition of randomized_tr does not affect the generator code. The constraints are the only object information included, and they could have been included elsewhere (in an extended class). So the generator code itself can be independent from the testbench code. It simply needs to be given an object to randomize. This is another example of reducing test-specific code and increasing reuse.

The object-based interface provided by DesignWare VIP and RVM creates a subtle yet important shift in thinking. Building on the fact that the models abstract protocols into transaction objects, specifying test conditions is a matter of generating constrained random objects. The shift here is that defining the transactions occurs in protocol terms and uses native language syntax (constraints and assignments), so the verification engineer can think in protocol terms, not model details. This abstraction allows a more natural and efficient thought process.

The object provided to a generator is referred to as a factory object or, simply, a factory. It is a blueprint for randomization and the template for the generated objects. When using DesignWare VIP models, factories are created by extending the provided transaction objects and applying user-defined constraints. When the factory is randomized, the user constraints will be used and, through inheritance rules, the extended objects can be put into a channel. DesignWare VIP models also support the addition of user data members in factories, allowing user customization of the transactions themselves. The details of this are outside the scope of this paper.

As with any programming task, there are several ways to code a random generator. In addition, RVM provides two pre-built generator types: an atomic generator and a scenario generator. These provide a great deal of functionality for specifying and controlling the random sequences that are generated. The details of these generators are outside the scope of this paper.

Step 5: Control the Test

The DesignWare VIP models all share the same features regarding test control. Although there are many features that support test control, this paper highlights two basic features: starting/stopping and logging. The DesignWare VIP models are all extended from the base class rvm_xactor so they all have the same control interface for starting and stopping. Each model implements start_xactor and stop_xactor methods that handle all steps needed to either start or stop a model. Encapsulation of function is one of the key elements in the architecture of DesignWare VIP and RVM. The testbench does not have to know how to start a model, it simply has to call the start_xactor method.

The start_t task in the environment class is where the test is ‘launched’. This generally means that any models or components in the simulation are started. The code snippet below is from the example. This simulation consists of DesignWare VIP models and the start_t code simply calls the start_xactor method of each model.

The only other component of the test that needs to start is the generator code shown in the previous section. Since this is a section of code and not an object, it can be launched by putting the code within a fork-join none block, as shown above.

Another standard function that the DesignWare VIP models provide for controlling tests is logging and messaging. There are many ways to control and configure logging. We examine a few basic ones that will get you going. Messaging is performed by rvm_log objects (logs), which are included in every model. By the way, the user_env that we created also contains a log that it inherits from rvm_env. Inheritance and reuse go hand in hand.

RVM defines message types and severities. This allows two degrees of freedom for sorting and operating on messages. Severities are defined as levels ranging from FATAL to DEBUG. These levels allow the messages to be processed distinctly.

Each log object has a threshold level and will log any messages with a severity equal to or higher than the threshold. The level of logging detail is set with the set_verbosity method, as this code from the top-level program shows:

The rvm_log class has many powerful and flexible features. For example, several of the methods can operate globally on all log objects. A global format can also be specified so that all messages use the same format. These testbench-wide controls are especially useful when integrating multiple models into a single simulation.

Another powerful logging feature is the ability to promote or demote messages. This code is from the build method:

In the example, the monitor produces a WARNING that it is ignoring a configuration parameter. This is a known warning and is not an issue for the simulation. Instead of disabling the message altogether, it can be modified and demoted to a NOTE with NORMAL severity. Messages can also be promoted so that their severity is higher than the default level. This allows tests to be customized to produce more automated and accurate results, and thus simplifying analysis and post-processing tasks. In a large regression, this is potentially a huge benefit.

Symbolics Side Note

In some of the code samples, you may have noticed that method arguments which refer to a symbolic value use a path’ed reference. In the code below, the argument to set_verbosity is env.vip_monitor.log.WARNING_SEV and not simply WARNING_SEV.

The reason for this has to do with proper encapsulation. In order for components to be reusable, they should be self-contained and not reliant on global or external definitions. When an encapsulated object is ported to another application, it remains intact. So the definitions of symbolics are contained in the objects alongside with the methods that use them. Using this architecture also ensures that the correct symbolic is used. Following the example, when rvm_log::set_verbosity is called, you want to make sure that you are using the rvm_log definition of WARNING_SEV and not some other definition.

rvm_env Revisited

In discussing the basic steps of getting to a first testbench with RVM and DesignWare VIP, most of the methods in user_env have been described. But there are a few that remain. We describe them in this section.

cfg_dut_t

cfg_dut_t is a container for any code needed to configure the DUT. Because the DUT in the example is really a DesignWare VIP model, it is convenient to configure it in the build step along with the other models. So the only DUT configuration required is to set the logging level.

A typical usage of cfg_dut_t would be to configure the DUT by writing to internal registers.

wait_for_end_t

This method is where the end of test is determined. Because of the dynamic nature of constrained random verification, this step should be able to adjust to tests of different lengths.

In the build step, the sink_in_chan channel was connected to the activity channel of the monitor. The monitor puts an object in the activity channel whenever a transaction has occurred on the protocol. The get_t method of a channel retrieves objects from the channel. This method does not return until the transaction has completed so that the object retrieved is complete. When the right number of objects has been received, the test is declared to be done.

When using either an activity or output channel, objects must be consumed as soon as they are present in the channel. So the while loop that contains the get_t call must be able to process each transaction in zero time and call get_t again immediately. Otherwise, the channel will back up, which may affect model functionality.

stop_t

The stop_t method is a place to put any code needed to stop the test. Since the generator stops on its own in the example, the only thing that is needed is to indicate the end of the test to the controlling HDL simulator so that it can stop the simulation.

cleanup_t

There is no post-processing in the example so the cleanup_t method has no extended code.

report

The report method in rvm_env collects error and warning metrics from all the log objects and reports a summary of the results. The example reuses the base class code by simply calling super.report().

The resulting output for the example is:

Conclusion

The code shown in this paper is from an example testbench which is a complete and functional test using the PCIE DesignWare VIP models. Looking at the entire example, we see that with the combination of DesignWare VIP and RVM constrained random verification can be performed in a well-architected and efficient manner. The protocol functionality is abstracted by the DesignWare VIP model, allowing the verification engineer to work at a higher abstraction level. A small amount of code (324 lines) can be used to generate a very large set of conditions by controlling constraints and using test execution time instead of code development time. There is a high degree of reuse inherent in the methodology. Test-specific code is minimized with the majority of the code in the environment. Including constraints and configurations, there are only 32 lines of test-specific code!

This paper shows the basic steps to creating a first constrained random testbench with DesignWare VIP and RVM. You can, of course, develop more complex and powerful testbenches using other methods and features of RVM and DesignWare VIP. The concepts described here get you started with solid fundamentals.

The complete example code is available in the following files:

RVM Testbench in OpenVera

Interface definition in OpenVera

For more information about DesignWare VIP, please refer to the following web page: http://www.synopsys.com/products/designware/dwverificationlibrary.html.