Penny: Y'know, just for fun, let's say I'm not.
Leonard: Before an application is released, they give it a trial run. We could do that. And if we hit a rough spot, instead of getting mad, just say, "Hey, we found a bug," and we report it so it can be fixed.
Penny: You mean like a beta test?
Leonard: Well, technically, this would be an alpha test. A beta test requires people that weren't involved in the development of the appl—
Penny: Seriously, do I not get credit for knowing "beta test"?
In the software development, website design and app development world, progress is spread out into various milestones. The Beta Test is the most well-known milestone, which represents a state when software is mostly done, but needs to be tested beyond its developers and internal testers with regular users to catch bugs, glitches and other problems. The idea is to give the software to actual users so that another set of eyes can catch different issues. After all, a developer who's been working on one part knows exactly what to do to "work around" glitches, and they might be only using it the "right way" (as intended), and they might not have thought of (or had time to try) "unorthodox" methods of using the software.
Software Release Cycle
The development stages of software generally go as follows, this being the road map that IBM laid out
in the 1950s:
Pre-Alpha
Pre-Alpha stages of development narrow down the requirements of the software and the design to meet those requirements. It answers the questions of "what should this software do?" and "how should this software do it?" Test plans are also made so that these features can be tested empirically. Sometimes a rough Tech Demo is developed in this phase to give a proof-of-concept to show off at trade shows and meet-ups like Game Jams.If we were developing a game, this might be the early stages of a game engine. We don't know what we'll do with it, but it'll show off some cool things that we could do later.
Alpha
Implementation of the software has started. Requirements can be adjusted to address issues brought up during implementation. For example, a requirement may say "this game must support two players", but in reality the game and hardware can support four players. Software starts out crude with many issues in the mix, but as time goes on, most of the components start meshing well together. Testing is usually done by the developers and only on the components they're working on. They're also at best, spot-checked. This means if the developer is working on, for example, the UI system, they only test the UI part and make sure the correct actions works. They may not discover that clicking the same button 50 times results in the UI breaking, and they may not discover that if the player goes outside the boundaries, the system freezes.In our game development scenario, we've decided on a car racing game. At the start, it may not have many maps, the AI is not there, the physics may be crude and rudimentary, and all the backgrounds are just grey, empty space. But features will continually be added in.
Beta
Implementation of individual pieces slow down and bringing those pieces together, known as integration, starts. Requirements are even more rigid, limited to minor adjustments to avoid major changes. At this point, the software is stable enough for others to test features and find bugs. Graphical assets though may or may not be ready. However, these can to be dropped in since what the development team is worried about at this point is getting the software to work.A product manager recruits beta testers. These are regular consumers who are interested in the product. It's important that the beta testers try the product on their own systems at home. The development team might be having great results on their expensive, top-end computers, but maybe the program doesn't work well on a teenager's $300 laptop or a regular parent's basic home desktop. If the program's demands for processing power or RAM or whatever are too high for regular users, adjustments may be needed.
In a racing game, the AI will at least follow the track, the car's handling behaves believably, but something silly, like driving backwards across the finish line will result in a win may slip through the cracks. The graphics may be glitchy and some buttons may not work properly. Textures may also be basic or not there, models will be boxes, and the sound may as well be one of the audio engineers making engine noises.
Later in the beta phase all major and most or all minor features as well as graphical and audio assets are in the game and it looks like a releasable product. They may still be incomplete or buggy. Requirements are frozen at this point.
The software is handed off to system testers to play with. For games, this could be an internal team of play testers or made public as an open beta. This is intended to find more obscure bugs, balance or tune features, and otherwise ensure the software works as intended. In a racing game, the AI may still be goaded into stupid decisions, a certain part of the wall may not have collision, but almost all tracks exist and the cars (mostly) stay on the track. Major features that are troublesome will be cut or delayed at this point. If a cosmetic feature to decorate the cars in a custom color crashes the game and the schedule is closing in, it may be cut for good or added later in a patch once the bug is figured out.
Note that according to the original IBM test specification, this is the stage at which all features are to be complete with no room to add anything else, although modern software developers are sometimes a bit more flexible with this, particularly when a feature can demonstratively improve the final product. If someone claims a product is in "beta" with major features missing, however (using the previous example, a racing game that's supposed to feature multi-level tracks but only one clipping plane works) then it isn't really in beta. This happens with depressing regularity.
Release Candidate
All of the features have been implemented. This is, theoretically, a product that you could sell immediately, thus the term "release candidate". Testing that goes on in here is system wide and higher level. If there are any new bugs found, only show stopping bugs will be addressed as changing anything in the software means everything will have to be retested. Minor bugs will be left in, either to be fixed with a patch, later release, or a workaround.Certain members of the public may get copies. In games, journalists and some publications, like Prima and Brady Games, will get preview copies. In other software, large corporate clients may receive advance copies so they can ensure their custom software still works with the newer version.
In our racing game, all tracks and cars are in game and look like they will when released. The minor clipping through a barrel will not be fixed, but the game crashing due to rolling over the timer will be. If the crash happens because the car takes a nose dive at a particular out-of-the-way location, it won't be fixed and may be silently patched later.
A trend that started in the late 2000's was publishers putting a stop to the use of the word "Demo", instead changing to the use of "Public Beta". Publishers and creators realized that by changing the name, people who would otherwise have refused to purchase a game with a bad "Demo", would now forgive the developer for a "Beta" under the mistaken impression that the "Beta" would be used to improve the game, or that the player could choose to believe that it was "just a beta" and that it would be magically fixed upon release.
Release to Manufacturer/Gold
The development state of the software is "tagged", or backed up and frozen. Any changes from hereon out has to be made to a new version. Because of the lead time needed to get the software to distributors, this may happen weeks before the actual launch date.A particular note in the game development industry are "day-one patches." These are patches that are released on the same day as the game. One might wonder if there's already a patch out, why wasn't it included in the final game? As part of the process for releasing a game on a platform, a bunch of testing needs to be done to ensure the game doesn't do anything horrible to the platform such as causing crashes. However, if a patch is released instead, the process is more streamlined because as far as the platform maintainers are concerned, the game itself is fine to run on the platform. Trying to get these day-one fixes in would only serve to delay the release of the game even more.
Usage Outside Computer and Software Fields
Both the term and approach are used outside of the computer and software fields, albeit in a looser sense.
- A book author might say she is "beta testing her new novella" by having a local book club read it and give feedback (e.g., on confusing parts or plot holes).
- A fashion designer might "beta test his new sportswear line" by having local athletes wear the clothes in training sessions, and then provide comments on the fit, comfort and style.
See Perpetual Beta and Obvious Beta.
