Greetings,

I want to start by making it clear that I know why testing is good, and that it’s really important, but I think that the TDD proponents are glossing over the most difficult part of a project.

I would very much like someone to address the issue of modifying code that is not new, and not already perfectly tested (or even completely specified!).  That is to say, the vast majority of actual code out there.

TDD is intensely focused on the early development phase (or at least TDD proponents are), and on writing new code, as opposed to what the majority of software developers actually do; maintain and update existing code.

It’s really straightforward (and fun!) to write entirely new code in the TDD fashion.  I’ve done it for about 3 decent sized projects now (one Java and two Rails), and it can be really pleasant, and a great focusing tool.  No arguments there; when you do it from the start, it’s really wonderful.

On the other hand, when you’re making incremental changes here and there throughout a very large, pre-existing, only partially tested codebase, it’s vastly less pleasant to try and do it test-first.

The Ruby Autotest tool is not such a pleasant tool at that point.  You stop wanting to write failing tests, because fixing it means autotest is going to try to do a full retest, which sucks for developer flow…  Even if your tests take ‘only’ 5 minutes to run, breaking a test makes you wince, and writing a failing test and then fixing it is for masochists only (and ones who want to miss the project milestones at that).

The focus of every presentation (this one included) I’ve seen on TDD being on the start of a project makes me wonder why nobody’s talking about later in projects…

Nearly every developer out there is going to face a large codebase with poor testing coverage, and will have to make changes that aren’t entirely new code, to existing code that isn’t entirely tested.  Does TDD have a solution for the ‘large, crufty codebase’, or is it suited only for 1.0 versions, small projects, and projects that were TDD from the start?

This isn’t really a rhetorical question for me.  I really want to get my organization’s culture more oriented towards testing.  I’ve got buy-in from lots of people that when they’re writing new modules and services, they’ll do it test-first (or at least ‘test around the same time as the code’, which is all I can ask for at this point), and that’s great.  But has anybody developed any tools to make TDD better suited to maintenance and improving existing code?

—  Morgan

p.s.  I’m skipping BDD entirely, because BDD is so hardcore in the ‘only for already very well specified solutions’ camp, that it’s meaningless for this question.  I’m also using ‘TDD’ and ‘test-first’ interchangeably, and I probably shouldn’t be.

p.p.s.  The title refers to how in Logans Run, everybody was destroyed at 30, so there weren’t any old people. In the world of TDD (or at least TDD presentations), there are no old projects, every one is fresh and new, so the issues that come with an old code base are never addressed.


12 Comments

  1. On Arnaud M Says:

    ( WOW, sorry for the truncated post, my browser submitted when I unpaused the media player :S )

    Hi, you express here a point I have felt since the early days of “test-first XXXX”.

    I do not think that the Agile universe has anything to propose yet to address the ‘large, crufty codebase’, apart from the obvious:

    – Struggle to identify functionality suitable for modularization (by leveraging protocol boundaries, file format boundaries), this is probably the most painful step
    – Mock modules in the “modern zone” of the code
    – Identify a candidate module well surrounded by mocked modules
    – Write tests for the surrounded module
    – Migrate the candidate module to the “modern zone”
    – Iterate

    I too have yet to read someone about using BDD in the long run but I am pretty confident that the general direction it shows us is the right one, even if the road ahead is still under construction.

    I feel that Gregg has a good point when he says that we have to assimilate that the test part of the code is integral to the code, not just a lateral artifact. Yes there is an increased cost in test infrastructure with BDD, but we must get rid of the depressing sensation that it is a test cost, to embrace the idea that it is THE (normal) cost of developping the code.

    Software agility is a virtuous circle.

    P.S. I love the title of your post !


  2. On Dan Croak Says:

    Hey Morgan,

    I think the only solution is really just identifying where test coverage is poor (rcov or otherwise), and making sure your client allows you to carve a significant portion of your time systematically improving the test coverage one small piece at a time (in the real world, you’ll almost always have to be continuing to build new features – with tests – at the same time).

    I definitely leave autotest way before I’m seeing 5 minute test runs (I’ll focus by hand on a test file or test method such as ‘ruby test/unit/user_test.rb’).

    I can’t think of any tools that are going to help you fill the gap any quicker than what you’d use regularly … test/unit, shoulda, heckle, rcov, etc.


  3. On Cyberfox Says:

    Greetings,
    Yeah, I’m using rcov as much as I can (had to hack up custom rcov scripts to get it to work), trying to use Heckle but having some issues… Using flog to find bad code sections… I don’t know what ‘shoulda’ is, I’ll look it up.

    It’s not so much my clients, but my company, as I’ve never done the contract thing. I wish I could carve out time explicitly, but instead I have to pad my estimates and use the extra time to improve the test coverage, and do my best to encourage a testing culture in the other devs so we don’t fall too far behind. 🙂

    I’ve been given a lot of freedom to do deep refactorings that both improve the ability to build future features, _and_ provides points that can be mocked to improve testability.

    But doing TDD at this point in the development cycle is just painful, unfortunately. I get my TDD fix working on my own projects. 🙂

    — Morgan


  4. On Giles Bowkett Says:

    I’ve seen this too, and I’m not sure what the answer is. However, I have to tell you, working on legacy apps that don’t have test coverage is a heinous pain in the ass – things break and you have no idea why. What I usually do is just isolate my tests in a separate file. I used a blog post about it as my site, so you should be able to click my name and see it. Basically you just throw the test in one file which only tests for the things you need. The Achilles heel for that approach is complicated objects that require shit-tons of association classes to exist before they can do anything. It kicks ass for many use cases, though. There’s a lot you can do to stay TDD this way, and fast as hell, and it’s usually very easy to move the tests from their unique files into the larger existing test files. And sometimes it’s better to have separate files anyway, because you’ve got your tests separated by topic and not just by the class they test.


  5. On Mike Says:

    I think TDD proponents would tell you that one of the best ways to introduce tdd, or automated testing at all, on an existing project is to try to write tests for every bug-fix that you do, or every new enhancement that you have to make.

    Bug 1003323849 comes in. Write a test, somehow, that fails for what’s in the bug. Fix the bug. Make sure the test passes.

    It’s not always easy to work an automated test into the guts of an existing application, but if you make the effort to do so then soon you’ll have a pretty big set of tests that test against known issues instead of lots of “theoretical” bugs that haven’t happened.


  6. On Keith Braithwaite Says:

    Leading TDD-er Michael Feathers has published a comprehensive and authoritative work on the topic: “Working Effectively with Legacy Code” http://www.amazon.com/Working-Effectively-Legacy-Robert-Martin/dp/0131177052/


  7. On Hamlet D'Arcy Says:

    You, sir, need to read Michael Feather’s “Working Effectively with Legacy Code”. The book was published in 2004, so it’s something the Agile folks have been grappling with for at least 5 years. There was a rousing session at CITCON North America on the subject two weeks ago, but alas you weren’t there!


  8. On orangepips Says:

    Read “Working Effectively with Legacy Code” by Michael C. Feathers. This book provides heuristics and strategies that focus on writing tests around what’s being changed. It does not advocate writing unit tests for the whole system before making changes, but rather writing tests around the areas that are being changed. Such that you write tests to capture how the area that’s going to be changed behaves. Note, this may involve making changes to that code to make it testable, which the book provides ideas on how to approach. Then make changes and unit test.


  9. On John Bender Says:

    Cucumber provides the tags for focusing on certain tests:

    http://railscasts.com/episodes/159-more-on-cucumber


  10. On Ian Mayo Says:

    Hi there Morgan,
    from memory the intended role of TDD in the maintenance of legacy codebases is as a refactoring safety belt.

    So, before you make ‘risky’ changes to a portion of legacy code, write some white-box tests to verify the outputs of that module – here you’re encoding an API reference for that module. Then hack away with your machete to tidy/refactor it. Then re-run the tests to verify that:
    a) your expectations of that module were correct
    b) you haven’t changed the outside appearance of that module (or that any changes you made were intentional).

    The tests then serve as verified, auto-updating API documentation for that module.

    Yes, it’s easier to preach this than to practice it. Yes, I could adhere to it more than I do.

    Ian


  11. On Carl Says:

    Michael Feathers wrote the book “working effectively with legacy code” on exactly this area. It’s well worth looking at. Not exclusively Java either…


  12. On Cyberfox Says:

    Greetings,

    I evidently need to tweak my WordPress setup; I didn’t know this old post even had new comments on it. 🙁

    I like the idea of segregating tests; so when doing specific fixes (i.e. maintenance), you don’t even try to run your whole unit test suite until you feel you’re done and integrating. There should be a way to do that easily…

    Many of you suggested Michael Feathers book on legacy code. I recall skimming it in a store well before I ever wrote that post (which was back in late 2007), and for some reason I’ve mentally binned it with ‘Code Reading’ by Diomidis Spinellis which I thought was a singularly poor treatment of the subject. I may need to re-examine ‘Working Effectively’, and see if I merely confused the two, or if I had a legitimate disagreement with it.

    I’ve since been through Yet Another company who had extremely poorly tested code (in fact, zero tests or specs when I came on board; outsourced code FAIL), and it definitely frustrates me. I’ll have to see if there’s a way to tune autotest to switch into a ‘legacy fixes’ mode, which only runs a single test suite as you’re making changes.

    Anyhow, I appreciate the comments. There’s some good ideas, and the impetus to take a second look at Michael Feathers’ book.

    — Morgan