In the distant past (1950’s or so), project managers and engineers came up with what is known as the project management triangle: fast, cheap, or good; pick two.
While software engineering can be very different from mechanical, it does at least share the same project management setup. Quality software designed cheaply will be late, cheap software released early will be poor in quality, and quality software released on time will be expensive. These differences come from the quality and number (thus cost) of the managers and engineers, the choice of methodologies, scope of features, and internal organizational setups.
What is different is the fact that software engineers aren’t limited by physics the way that our mechanical brethren are. With few exceptions for high performance computing, the limitation of most software projects is the imagination and effort of its engineers, not hard limits in manufacturing technologies or physics. Combine this with a fad-heavy market for programming methods (scrum! extreme! agile! pair!), and it can be very tempting to assume that we can find the perfect balance with the correct management processes and the right methodology.
This is false, of course. Management and methodology is about dealing with the communication overhead when enough people are working on an project. The pipe dream of management and methodology is for a group of N producers to produce N times more than one person alone. This is of course rubbish, as the Mythical Man Month demonstrated handily, it’s simply impossible to manage or process your way to good, cheap, and fast.
So what’s the point of it all then? Why don’t we just go back to waterfall? Because the point of agile, scrum, pair programming and friends is not to get us all the way to good, cheap, and fast. The point is to go from choosing one of three, to choosing two of three. A poorly managed, low discipline team can only choose one of good, cheap, and fast; and this is of course worthless. Cheap software that’s late will probably still run out of VC and be beaten by the competition. Bad and cheap software will struggle to take over the market, and software that’s both bad and late probably shouldn’t be written. However a well managed, well disciplined team can survey the market, measure the competition, and knowingly choose what compromises they wish to make in speed, quality, and cost. Poorly managed teams blunder into one of the choices, usually cheap and bad, and end up having very little control over their own fate.