Functional Verification and the Ecosystem Argument

I've been involved in the functional verification space for quite some time -- both personally and professionally. On the personal side, I've recently been experimenting with using Python as a functional verification language. The simplest reason? The ecosystem.

I've been involved in many discussions over the years that bring up The Ecosystem Argument. It goes something like this: If we use programming language X for Y (a plug-in language, a tool extension language, an implementation language, etc), we'll benefit from the ecosystem around language X. An ecosystem is a very powerful thing, and encompasses the language tools (editors, compilers, linting tools), the users that know a given language, and the available libraries for the language. So, the argument isn't entirely out of line. Using a popular language rather than an obscure language is often not a bad thing. That said, there's a hole in the generic form of The Ecosystem Argument: a programming language is used for many purposes, and much of the specialized knowledge is domain-specific and not language-specific. To put it another way, just because I know Python doesn't mean that I'm a Machine Learning guru -- despite the fact that Python is heavily used in this space.

All of these elements of a language ecosystem are important to consider. However, depending on the circumstances, some of these factors take on greater importance than others. If users will be using the language to create larger applications, the popularity of the language and availability of relevant libraries may take on increasing importance. If users will be using a domain-specific library (eg SystemC for modeling hardware with C++), the language may be less important than the semantics of the underlying domain because users will be more focused on expressing the semantics supported by the library than the semantics supported by the programming language.  

In my experience, it's not a good assumption that selecting a particular programming language will bring a new group of users into a specific domain. It's possible, but not probable. What's more important is whether a group of users can easily be productive given the ecosystem around a given language.

A key aspect of language ecosystem and productivity is library availability. Libraries are present in all language ecosystems. In some cases (eg Java), the language specifies a rich library that satisfies the requirements of many users. In all cases, there are external libraries that serve more specialized needs. 

Getting access to these external libraries poses another challenge, and it's well worth noting any language whose ecosystem simplifies the process of acquiring and publishing libraries. If we're using C/C++ or Java, there are several steps we need to go through to acquire a new library:

• Locate the project
• Determine whether the project requires other libraries, and go find those
• Determine where to install the library, and how our software will get access to it (eg co-locate it with our project, modify the library path, etc)

This model is pretty workable if we are building an large-scale application or library. The overhead of setting up libraries is typically low compared to the value provided by the application or library. This model becomes much more challenging when we're developing smaller pieces of functionality. In these cases, the overhead of acquiring and setting up the external library may equal or exceed the utility we get from the library. In these situations, it quickly starts to appear less expensive to just build something small that serves our needs rather than try to use an external library.

These situations come up all the time in functional verification. Here's just one example. One of the projects I worked on is a small RISC-V core. My testbench needed to get symbol information from the test files that the core executes as part of the test suite. Ideally, I could have used something like libelf or its successor elftools. However, that would require anyone that wanted to run the test suite for my core to also get and install this library. To avoid adding complication, I simply wrote my own simple ELF-file reader that could be included in the testbench.

Python and other languages address this lost opportunity for reuse by providing mechanisms to easily publish and acquire external libraries. Python has PyPi, the Python Package Index. Javascript has NPM. And, at least with Python, users don't need to have administrator privileges in order to install packages for their own use. 

In a functional verification environment, there are many other more-specialized cases where reuse is desired. The case described above is more generic, since many software projects have reasons to inspect ELF files. What about Bus Functional Models (BFMs)? What about specialized protocol generators? The possibility of being able to easily share and reuse some of these elements by leveraging Python's existing ecosystem is exciting!

 

In my next few blogs, I plan to expand on how I'm using Python for functional verification and the benefits (and challenges) that I've experienced. So, stay tuned!