There are millions of programmers in the world, and plenty of us have been doing it professionally since before Agile was all the rage. It sounds like you have too, but are you really arguing that just because you personally didn't encounter these ideas before Agile advocacy came along, no-one else did either? Or perpetuating the false dichotomy between Agile and Waterfall? I have never worked on a project that followed a strict Waterfall model without any kind of cycle or iteration in the process, not once in my whole career. Now get off my lawn!
Seriously, though, you've cited Spolsky and McConnell, but you seem to be appealing to their authority on the basis that if the techniques we're discussing had been in common usage, they would surely have been mentioned in those authors' writing in the late '90s. I would point out that the first edition of McConnell's Code Complete was published in 1993 and didn't say much about OOP, yet OOP was already in use by many professional developers, and books about languages supporting it had been published many years earlier: the first edition of The C++ Programming Language was published in 1985, for example. Similarly, the second edition of Code Complete was published in 2004, yet contains no index entry for the word "concurrent" or "parallel" (though there is a passing reference to multithreading on page 337). So perhaps it is unwise to infer too much from any absence of references in the kinds of source you mentioned.
I have never argued that "no-one" encountered those ideas, and your repeated exaggeration of my position is frustrating.
Nobody in the Agile movement claimed novelty for individual pieces. What was different was the way the pieces went together, and the shift in values and cultural norms that they required. That wasn't purely novel either; the fact that several people invented similar processes at the same time suggests that, as with most inventions, change was in the air. But what the Agile people undeniably did was to define and promote that difference in a way that gave people something to rally around. At the time, waterfall and a waterfall-ish RUP were the big processes people were discussing, and the notion that one would focus on people over process, or working code over documentation was heretical.
Also, I'm having a hard time taking your approach to evidence seriously. Rapid Development was a major survey of current software project management and a catalog of best practices put out by a large tech publisher. Splosky was also writing clearly about current practices. Your counter-evidence is the (unnamed) projects you've (anonymously) been on. Plus that one of them wrote a different book that you can pick nits on.
And honestly, I don't get why you're arguing the nits and ignoring my main points. You clearly have some big issue with your version of Agile and it's imagined history. But I honestly don't care what process you follow as long as it doesn't suck for you, so arguing it with me isn't really helping anybody.
I'm sorry you're feeling frustrated with this discussion. I'm afraid I just don't understand what your "main points" are supposed to be, if you think I'm only picking nits with the details.
I started this subthread with some fairly black-and-white statements about the relative timing of good practices being adopted, Agile taking off, and less proven practices entering the picture:
Since then, you have accused me, repeatedly, of being wrong about the history of Agile. Is your objection that early versions of a few Agile processes already existed, albeit in relative obscurity, before the Manifesto was written, but I talked about how 'concrete examples of "Agile" processes started to appear' after the Manifesto was written?
If so, I'll happily concede that point. I don't think it really matters whether you define Agile as starting from the evolution of the general approach/abstract philosophy (which had evidently started by the time of those initial processes) or the writing of the concrete Manifesto and popularisation of the Agile moniker. Nor do I think it really matters whether you define processes like XP and Scrum as appearing as "Agile" processes when they were popularised and given that label (around the time the Manifesto also became widely known) or from when the original processes that became what we now call XP and Scrum were first used anywhere. Any way you cut it, we're talking about the same groups of people, the same general period of time, and the same end results.
My main point is simply that many of the good practices that were adopted by those early Agile processes and are often considered to fall under the Agile umbrella today did not actually originate during that period of time or with the people who worked on those early processes and the Manifesto. However, many of the Agile practices I consider unproven did originate during that period of time and with those people.
While you may wish that no-one in the Agile community claims otherwise, unfortunately that is not always true. While I was researching my post with sources, I found a comment on Martin Fowler's personal web site claiming that Kent Beck coined the term "refactoring" in the mid-'90s, for example, and numerous comments from various posters on other forums suggesting that Fowler's book was the definitive original reference. I think it is important to debunk such claims and any attempt to paint Agile and Waterfall as some sort of dichotomy. That just isn't how software development works, and being "Not Waterfall" is not an argument for switching to any given Agile process.
> I found a comment on Martin Fowler's personal web site claiming that Kent Beck coined the term "refactoring" in the mid-'90s, for example
No, I didn't. Sorry. I had read something similar in several other places, and when I did a search (since Fowler's site itself was blacked out) the resulting excerpt was truncated in a critical place that made it look like the above was true. Still, there are plenty of people who have claimed that the Agile movement originated ideas like refactoring in on-line forums and the like, so aside from setting the record straight regarding Fowler's site, my original point stands.
I didn't have anything specific in mind; I'm going by my own experience, discussions with past colleagues, and conferences over the years more than anything. That said, it's not hard to turn up examples with a few minutes looking through my bookshelf and Google Scholar.
For automated testing, just stick search terms like "automated software testing" into Scholar and set the latest year to, say, 2000. You'll turn up dozens of papers going back decades that talk about techniques similar to what we would call unit testing today.
Continuous integration is another obvious one. The likes of Grady Booch were writing about the advantages of frequent integration builds by the early '90s. The term "continuous integration" appears in Object-Oriented Analysis and Design with Applications (1993), for example.
The prior existence of refactoring techniques is kind of obvious, as people had been discussing simple transformations like moving functions and data up and down a class hierarchy since the dawn of OOP. If you want a specific citation, Design Patterns (1995) mentions the specific term "refactoring" a few times, in turn citing academic papers on the subject cowritten by one of the authors as early as 1990, around a decade before Fowler's book and several years before Beck coined the term according to some Agile advocacy.
That's all I've got time for now, but I'm sure you could find eqally authoritative sources for the other things I talked about if you genuinely want to know.
I'm not claiming TDD as something widespread before Agile. I'm claiming TDD as something unproven that the Agile consultants started mixing in later. (Please see the list in my second post to this thread.)
For the third time: you are incorrect about the history. Test-first development was created by working programmers as part of Extreme Programming before the term "Agile" even existed.
I'm not necessarily disputing that, though I note that you have changed from TDD to merely test-first development, which I don't think is the same thing.
My point is that TDD, like several of the other practices in XP that do seem to have originated with those people during that time period, is to this day not proven as any better practice than the alternatives.
Seriously, though, you've cited Spolsky and McConnell, but you seem to be appealing to their authority on the basis that if the techniques we're discussing had been in common usage, they would surely have been mentioned in those authors' writing in the late '90s. I would point out that the first edition of McConnell's Code Complete was published in 1993 and didn't say much about OOP, yet OOP was already in use by many professional developers, and books about languages supporting it had been published many years earlier: the first edition of The C++ Programming Language was published in 1985, for example. Similarly, the second edition of Code Complete was published in 2004, yet contains no index entry for the word "concurrent" or "parallel" (though there is a passing reference to multithreading on page 337). So perhaps it is unwise to infer too much from any absence of references in the kinds of source you mentioned.