Bug Tracker for Simulation Models

I am looking at tracking bugs in simulation models and m code. Is there a bug tracker that you would recommend for use with MATLAB/simulink and the integrated SVN/GIT tools?

[Jim Riggs]

I'm not sure what you mean by "bug tracking".

Are you thinking in terms of configuration management where you keep track of when/where a bug or issue is identified (i.e. what software version) and the version where the fix was implemented? This would require implementation of a software version control system, but it can be as simple as establishing (and naming) software benchmark versions and keeping a spreadsheet with the history of issues/bugs and feature modifications.

[Jason Nicholson]

I use both GIT and SVN on a regular basis. A bug tracker helps people have a place to submit bugs. Tracking what commit the bug was identified and when fixed is nice but not a must.

A spreadsheet is reasonable idea. Noted. You should add this idea as an answer.

Answer 1

[Jim Riggs]

In the past, I have used spreadsheets and database programs (MS Access) for issue/bug tracking in simulations. This does require a way to identify the software build that correlates with the issue, so using SVN or GIT is a good way to accomplish this. The database/spreadsheet also allows me to collect ideas or suggestions from group members such as "I think we need to ad this new feature" or "I suspect there is a problem that should be investigated further", etc. This gives me a place to record the issue and it's resolution, even when it doesn't effect the code.

Note that I support new product development, so when we conduct a simulation analysis that will be used to predict product performance, we track the version of the simulation that was used in the analysis. If a bug is discovered, we need to be able to do "forensic analysis" to determine how it influenced the product decision process, so our procedures are designed to provide this ability.

[Jason Nicholson]

Forensic analysis = "Git blame" or "SVN blame" ?

Very nice Jim. This is really quite good and headed towards what I was thinking. This answer is good enough for me. However, I am going to wait a few days to see if anyone chimes in.

[Jim Riggs]

"Forensic Analysis" is identifying how the bug might have effected our decision making process.

You see, my procedures are intended to capture true "configuration management", not simply file version control which is what you get with tools like SVN/GIT. These tools are great for tracking the evolution of each file in the build, but they don't tell you anything about why the changes were made. Configuration management seeks to answer the question "Why is the design the way that it is?" (both the design of the product and the design of the simulation), and it captures the thought and decision-making process. We do this by including outside documents (via hyper-links) in the database. These documents are recorded in categories such as:

• White papers that describe the technical approach to developing simulation models. (how it works)
• Software description documentation (describes how the white paper was implemented)
• Validation products - Data and analysis that validate the models
• Presentations (PowerPoint) that pertain to the model development and/or application
• User instructions and other documentation (any user documentation that exists)
• Product performance analysis reports
• Software bug/issue reports

The database features allow me to search and organize all these documents in different ways, such as documents that pertain to a certain model, or all validation data products, and, of course, bug/issue reports. This is simply a simulation-centric way of organizing a lot of documentation.

[Jason Nicholson]

Jim, it seems you have put a lot of thought into this. I appreciate it. This is great information. Thank you.