There's an interesting article by David Kesmodel in the WSJ on how companies are now having longer and longer beta test cycles for software. The article was also discussed here on Slashdot.
Kesmodel is critical of companies for long beta cycles and for turning the concept of extended testing among real users (beta-testing) into a marketing exercise. I'm sympathetic to that criticism, but I think that since the terms "alpha" and "beta" came into use at IBM in the 60's to refer to test cycles the complexity of software has increased dramatically, and our ability to test that software has not.
Three major drivers of this phenonoman are 1) the number of users, 2) complexity of interaction with other sofwtare, and 3) the desire for early and broad user feedback. First, it is virtually impossible to test major Web applications at scale. If you're Google, how can you possibly replicate the deployment platform (tens of thousands of systems) and test the shear volume of traffic? The only true test is to open it up to the real world.
Second, the number, variety, and configurations of systems that your software must interact with is truely mind-boggling. How amny Linux distributions are there? How many Windows registry variations are there? Even trying to test all variants of Microsoft IE alone can be overwhelming. You have no choice but to open up testing to the world.
Finally, "release early, release often" has become a standard mantra (for good reason) among developers today. Successful Open Source developers practice it religiously. What's the equivalent in the proprietary software world? The extended open beta test. How long was it before Linux 1.0 was released? Three years? Proprietary developers are just doing the same thing as their Open Source counterparts - getting early feedback before all of the features have been implemented.
Comments