Testing is an area of direction lately with my work. It seems that some work was done by a prior employee developing some tests, but it was never integrated into an automated system.
I have generally been impressed by what I saw of the Tinderbox system used by Mozilla Firefox. That system exists from the days of the Netscape browser. The idea is that there are systems the continually build the software throughout the day as people make changes. With each build, some basic functionality tests run. When the software does not build or the tests do not pass, the tree goes red. The tree sherrif does not let anymore changes be made to the product until the problems are fixed with the recent changes. This system appeals to me because it is automated. People aren’t building the software manually. It just happens automatically. There is accountability because the system is rebuilt continuously so when something breaks, you have a small window of changes that caused it.
Then there’s the system most places I’ve worked with use. They manually build the software or perhaps only build once a day. They have few, if any, automated tests. This means things break frequently.
Right now, a continuous integration tool similar to Mozilla’s Tinderbox is used to build the software where I work. It is Hudson. That gets part of the system right. It is an automated system for building the software continuously. Hudson, though, has many plugins for integrating with software test systems as well as issue reporting systems. I’d like to move testing of our product in the direction of using those functionalities. When a change mentions an issue, it gets linked back into our issue tracking system. When each build is made, tests are performed and a report is made of what is passing and what is failing.