Thoughts on Software Development

I have been in the Software development business for 42 years in nine different companies. I've seen some projects succeed and far too many flounder. Here are my observations on what went right and what went wrong.

  1. Why projects succeed
    1. Continuing contact with the ultimate customer; lots of feedback on prototypes
    2. Excellent requirement specifications
    3. Automated Unit and System testing
    4. Constant emphasis on quality
    5. Good design
    6. Good team communication
    7. A real business case for the project
  2. Why projects fail
    1. Bad Requirements
      1. Wrong Requirements
      2. Creeping Requirements

        These are new requirements that enter the project once it has started. These take a lot of time to retrofit into an existing design. New requirements are always needed in a project, but they should only be accepted if absolutely necessary and the schedule and resources updated.

    2. Going for the big kill

      This is trying to get everything in the first release. This delays the project and prevents initial user feedback. Time is so critical in the software business.

    3. Slavishly following a programming methodology.

      Objects are good. Object Oriented Design is better. Patterns are wonderful. Use these to the maximum extend that it makes sense, but no further.

    4. Optimizing too early

      Trying to make the code really fast and efficient before its working with the essential features.

    5. Managers putting in features

      Putting in cool little features that are not a felt need by the client. Maintenance, documentation, and testing on these features can drag your schedule down and jeopardize your project. I've worked on several projects where cool features were put in early on, and we wasted lots of time maintaining and debugging them.

    6. Missing one level of indirection

      Hard coding the components together instead of using an interface layer. The guts of the components cannot evolve without affecting the other components.

    7. Not enough internal marketing

      Don't assume an obviously better product will prevail. It takes internal marketing and personal initiative to show people.

    8. "Broken Windows"

      A few problems with quality will doom an otherwise good project.

  3. Why projects don't fail.
    1. Lazy programmers.

      Almost all software developers I have had the pleasure to work with are hard working dedicated individuals.

  4. Four Variables of Product Development

    The customer can pick any three:

    1. Cost

      Early in the development cycle, you can spend too much money. You can spend not enough later and damage the project. You need a happy medium.

    2. Time

      How long will it take to finish the product

    3. Quality

      Don't sacrifice this one. Programmers really want to produce high quality code. This leads to higher job satisfaction. Oddly enough, a project can often be done in less time, if quality specifications and testing are higher.

    4. Scope

      Scope is the most important variable for software development. Sometimes you cannot control the Cost,Time, or Quality, but if you can control scope, you can increase the odds of a good product.

    5. Caveat

      In addition to these you need to consider your resources. Do you have enough testers, programmers, money, machines, documenters, tools? How can you maximize your resources?. Often you can trade one for another with careful planning.

  5. Miscellaneous Tips

    Test rigorously during the development cycle with representative data. I've been burned by testing with small sample sets that worked great, but real life size data sets were too slow. Test constantly with many data sets representing the range of real data using automated tools.

    1. Philippe Kruchten, author of The Rational Unified Process graphics:
      Architecture is not just about building the right UML models, with three, five, or seven views; it is also about people. As I used to tell the architecture team on an air-traffic control project I worked on: Fifty percent of the job of an architect is communications -- communications with other teams, internally, with the customers, with suppliers, with other parts of the company -- twenty percent is planning; and the rest is technical stuff.

    2. "There are only two hard problems in computer science: cache invalidation, naming things, and off-by-one errors."

    3. Anders Hejlsberg, designer of C#:
      I'm a strong believer in being minimalistic. Unless you actually are going to solve the general problem, don't try and put in place a framework for solving a specific one, because you don't know what that framework should look like.

    4. Steve McConnell, author of Code Completegraphics.:
      You'd probably want to tar and feather a teacher who gave you a programming assignment, then changed the assignment as soon as you finished the design, and then changed it again just as you were about to turn in the completed program. Burt that very process is an everyday reality in professional programming.
      page 75 of Code Complete

      "All in all, I think test-first programming is one of the most beneficial software practices to emerge during the past decade..."
      page 504 of Code Complete

    5. Hal Fulton

      "I've enhanced the gui until it no longer works. This is called 'rebugging'". -from the Austin Ruby Mailing list

    6. Juval Lowy

      Modern Software Engineering is the ongoing refinement of ever increasing degreees of decoupling.
    7. Amit Asaravala
      Management isn't about popularity; it's about respect. Staffs respect leaders who show fairness and justice within a disciplined organization. Great managers are prepared to make unpopular decisions. They're truly great managers when they know an unpopular decision now will lead to success later. They're prepared to wait for the respect born out of success whilst bearing the anxiety of unpopularity in the short term.
      "SD PEOPLE & PROJECTS" Jan. 2005: Managing at Microsoft By Amit Asaravala