Lessons Learned from Commercializing a Bug Finding Tool

Slide Note

Coverity, a brand of software tools, transitioned static code analysis to the real world through customer interactions, bug finding laws, and bug handling strategies. Lessons include the impact of lab vs industry settings, customer trials, and understanding bugs in a code base. The importance of accurate analyses and adapting to changing requirements emerged through experiences in the software development landscape.

pay_tho Follow

Uploaded on Oct 02, 2024 | 2 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript

A few Billion Lines of code Later using Static Analysis to find Bugs in the Real World BY Al Bessey, Ken Block, Ben Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles Henri-gros, Asya Kamsky, Scott McPeak and Dawson Engle

Introduction Coverity is a brand of software development products, consisting primarily of static code analysis and dynamic code analysis tools. How static code analysis was commercialized ? What had to be changed for real world use ? How to adapt to the constant change in requirements ? What was the lesson learnt ?

Lab vs Industry The software was able to find bugs in a large code base easily with few simple analyses and tricks in a lab environment. In the real world, hundreds of programmers use the tool to check hundreds of different code bases. The types of errors, number of false positives, type of build are all different from what is predicted by lab results. The programmers do not know how the tool works, unlike testers in the lab who have a knowledge of the tools internal process.

Customer Interaction Two scenarios of customer interaction Initial trial Long term use The trial is a pre-sale demonstration that attempts to show that the tool works well on a potential customer s code. Sales Engineers educate the customers about the tool. Usually happens over a period of 1 day or 2-3 days if code base is very large

Laws of Bug Finding You can t check code you don t see Ability to extract code from build No access to modify build Test Machine You can t check code you can t parse No standard compilers

BUGS Do bugs matter? No, your tool is broken, that is not a bug Misunderstood errors/bugs are considered as false positives How to handle cluelessness ? Do not change the results after an upgrade Myth: More analysis is always good