Why did we lose the XP Practices?

Nowadays everyone is agile. Or at least they say. Most organisations use Scrum. Or they claim to do it, while still doing waterfall. But why is the industry so focused on Scrum? What happened to eXtreme Programming? Why did we loose the XP practices? Are they too extreme?

The first agile methodology I heard of was eXtreme Programming back in 2001 when I was a CS student and read the eXtreme Programming Explained book by Kent Beck (published 1999). I remember reading about the ideas of pair programming, refactoring, test driven development and simple/evolutionary design and thinking that this is the future. The planning game and user story for requirements parts was nothing I payed that much attention to – I was completely focused on the code writing at that time.

Scrum is a few years older than XP, being launched in 1995 by Jeff Sutherland and Ken Schwaber. If you’re a regular reader of my blog you know that I have thought and written quite a lot about Scrum but very little about XP. The reason is that I’ve been quite focused on the management view of agile projects in order to get the customer interaction working. While XP is a bit vague (and too extreme with a customer full time on the team), Scrum offers a set of rules that is easy to get started with and easy to explain to the customer.

I think that I’m not the only one being dragged to Scrum because of the relative ease of getting started combined with the focus on the relation with the customer. Add to that the fact that most managers that are involved in methodology selection understand the core Scrum process, but without a strong foundation in development most XP practices make no sense. From a “marketing to management” perspective Scrum is easier to argue for and has become the de facto standard agile methodology.

But I think that we lost something along the way. We lost the technical XP practices that are an important foundation for the ability to be agile. Without those technical practices, can a Scrum team really become truly agile?

Technical XP Practices

eXtreme Programming is based on a number of values, principles and practices. Most of these practices are technical (source):

  • Pair Programming
  • Test-Driven Development (TDD)
  • Continous Integration
  • Refactoring
  • Small releases
  • Coding standard
  • Collective code ownership
  • Simple (Precisely enough) design
  • Metaphor(standardized naming schemes)

Then there are some non-technical practices: User stories and the planning game, 40-hour workweek, and on-site customer.

Compare that with the Scrum rules. Those rules talk about the team, the events and the artifacts. There’s nothing in scrum about how the actual programming should be performed.

Mike Cohn thinks that it is good to leave those practices out of the rules:

I love the XP engineering practices, particularly things like test-driven development, the focus on automated testing, pair programming, simple design, refactoring, and so on. However, I think it’s a mistake to say to the team “you’re self-organizing, we trust you, but you must do these specific engineering practices….” This sends a mixed message to the team that causes confusion. I love the XP practices but don’t like mandating them. I want teams to discover the value on their own.

I can see the point of letting the teams discover the value on their own. But most teams never do!. Or rather, most teams do see the value, but not for themselves. Most teams think that they are different, that their legacy code base makes it impossible to get started or that their user interface technology makes automated testing impossible to introduce.

Losing the Foundation

Giving up on the technical practices (often even before having tried them) is in my opinion a big loss. The refactoring, covering tests, continuous integration (and nowadays continuous delivery too) are not optional practices for a truly agile team. They are the foundation that enables agile. Without them a team will never be able to be truly agile.

People are giving up because taking that step is hard. It is harder for developers to change the technical practices than to change the interaction with management. Because the interaction with management is not the core work of the developers. But the interaction is the core work of management, which means they understand it and can drive the change. That’s why Scrum succeeds: It is understood and can be worked on by management while it doesn’t require changes to the core work of the developers.

We lost the XP practices because we as developers let management once more take the lead.

But that is now changing.

XP is Coming

With Scrum in place, the business and the developers are getting closer. A cross functional scrum team taking full responsibility can do that in cooperation with the business. The managers’ supervising and coordinating efforts are no longer needed. Ironically, by working hard to introduce Scrum the managers make themselves obsolete. The teams now work directly with the business.

The teams are now ready to take the next step, where they discover that they need technical practices to continue their agile journey. They need the XP practices.

We are no ready to take the next step and embrace the change to our way of working as developers. Just look at the all the buzz about Continuous Delivery, Devops, BDD.

The XP practices were never lost, but we and our organisations were not ready for them. We had to start with Scrum to get the agile mindset going.

Now the time has come for eXtreme Programming.

5 comments

  1. Sorry, but I don’t think XP will ever make come-back, it has been dead and buried since the early 2000s when it became obvious it doesn’t live up to its promises.

    The original CCC project failed miserably (well, many of the involved people got a book deal out of it) and the only thing that XP still has is a cool name. If it would have been named “Methodology That May Offer Some Advantages Under Very Limited Circumstances”, no one would have taken a second look at it.

    * Pair Programming – complete failure in every way.
    * Test-Driven Development (TDD) – not XP-specific, advantages may have been overstated. Use BDD instead.
    * Continuous Integration – not XP-specific
    * Refactoring – not XP-specific
    * Small releases – not XP-specific
    * Coding standard – not even agile methodology-specific
    * Collective code ownership – not even agile methodology-specific
    * Simple (Precisely enough) design – largely the reason XP failed; not XP-specific
    * Metaphor(standardized naming schemes) – not even agile methodology-specific

    See, for example, McBreen, P. (2003). “Questioning Extreme Programming”, Matt Stephens, Doug Rosenberg (2003), “Extreme Programming Refactored: The Case Against XP”

  2. I most seriously hope you are right. XP is the mac of agile!*

    However, my experience from being part of an agile movement in a larger organization suggests that there are some serious obstacles of which I don’t know if all can be overcome even with time and hard work. Those obstacles are not organizational, but personal.

    Most agile practices work well for extroverted people with a solid self-confidence, but not so well for people who are introverted, shy or who get stressed out by having people watching them think/work. My impression is that that XP was thought out by top performers, for top performers.** Unfortunately there are not only extroverted top performers in most organizations.

    Around me at my workplace I see a lot of different attitudes towards agile practices, not only the more controversial ones such as pair programming. Some people find it really, really hard to share their work before its “done”. Some people get totally stressed out by working in short cycles. Many people find it very hard to break down requirements into slices thin enough to implement in one sprint. Some people cannot leave their perfectionistic fingers away from finishing features which are not yet needed. Many people are embarrassed by speaking publicly.

    I’m sure that some of these reluctancies towards agile practices are a matter of habit and might be overcome by habit and by increasing trust levels within the teams. But I’m also sure that there will always be a certain percentage of my colleagues who would prefer to be given a strict specification by their boss and be left alone in their office for a month to finish their implementation. I just hope that that percentage can be brought down to some very small figure.

    As the Mike Cohn quote above suggests, I believe that the benefits of the XP practices must be discovered by each team member by him or herself. So how does one make people make that discovery? My experience is that the best way is a viral approach. I personally caught the infection just a few month ago from some colleagues who were assigned to my team. Seeing pair programming, TDD and the other stuff actually _work_, _for us_, _in our specific environment_ is a very powerful and convincing experience. By making agile role models out of ourselves we can continue to spread the infection. I know I will, and I will continue to hope for the Promised Land of XP at the end of my journey.

    * http://agilewarrior.wordpress.com/2014/02/03/xp-is-the-mac-of-agile/

    ** Disclaimer: I never read any serious literature on XP; my impressions and views are based upon rumours.

  3. I agree with Sander, and Mike that you refer to. You need to seperate practices and Way of Working. XP was a term and practices used by geeks, that management had hard to understand. It was something good that code monkeys used, but it wasn’t anything they could apply in their daily work. Then came Agile and it used words and term that management could understand, just as you write.

    I am convinced that tomorrow we will se a new WoW but we, the code monkeys, will still use pair programming, refactoring, and so one.

  4. Your right that the word agile has lost its meaning.

    Maybe people stopped using XP because it doesn’t work. There are no metrics (real metrics not subjective “studies”) that show that XP or any other agile process works better than something else.

    The original XP project, the C3 project at Chrylser, was a failure. In spite of that several team members (from “the best software development team in history”) went on to become consultants selling XP with nothing more behind it than one failed project.

    If XP worked better than the alternatives I doubt it would have fallen into disuse.

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.