May. 21, 2008
Interviewers: Scott Swigart and Sean Campbell
Interviewees: Diana Larson and James Shore.
In this interview we talk with Diana Larson and James Shore about Agile Development. In specific, we talk about:
- Value in Agile development training
- Philosophy of training and instructional design
- How Agile benefits organizations
- Adoption of Agile methods by a small part of a large organization
- The limitations of Agile development
- Where Agile works and where it doesn’t
Sean Campbell: To get us started, Diana, please introduce yourself.
Diana Larson: Hi. I’m a partner at Future Works Consulting here in Portland, although I work all over the world, so I don’t actually spend a lot of time here.
I’ve been working with high tech and software companies for about 15 years–the last seven or eight years almost exclusively on Agile projects. I help with improving product performance from the human organizational systems side, as opposed to the technical systems side.
Currently, I’m chair of the board of the Agile Alliance and active in a number of different conferences and things that go on around the world.
Sean: Thanks. And you, Jim?
Jim Short: I’m a consultant and author and speaker. I’ve run a consulting company since 2000, with a focus on operational excellence for software teams, which really means answering the question, “What does it take to produce great software?” Specifically, “How can we be more productive, and how can we have fewer defects? How can we improve our time to market? And how can we make and meet commitments to anybody that we need to make and meet commitments with?”
That’s my focus, and the way I’ve found works best to achieve all of that is with Agile development. I’ve been working with Agile teams in an operational capacity since 1999. I started out as a team lead on an Agile project and I’ve continued that.
I have a background as a programmer, so I have a lot of technical knowledge, but also I bring a lot of business knowledge to projects, which is important, because you need to bring everything together.
Diana: I would like to mention that Jim is the co-author of a book that came out last November called “The Art of Agile Development.”
Sean: Thanks–and you’re also an author yourself, right?
Diana: I co authored a book called “Agile Retrospectives: Making Good Teams Great.”
Sean: I understand that the two of you have trained together numerous times in the past. Roughly how many times have you worked together in a training environment?
Diana: Maybe half a dozen times.
Jim: We’ve known each other for years. Like Diana, I work around the world and almost never get to work in Portland, but when I do, I tend to call Diana.
Sean: That’s a strong endorsement, to say the least.
Just to talk a little about training in general, Scott and I have some experience as technical trainers, and I have noticed that the company owners and managers who send people to training typically want to know what benefits it will give them three to six months in the future.
That issue seems to be especially relevant for topics that have been as passionately evangelized as Agile has been. How do you answer when a manager asks you what the lasting impact will be of training like the event you’re planning now?
Diana: There are two one-day events. One will be one focused more on the business side of Agile and the other will be focused more on the technical side. These training workshops are really tailored to give people the experience of what it means to be in an Agile environment. That’s something that people can carry away with them.
Specific skills aren’t really the focus of these events; they’re about giving people the capacity to understand what it means to work in an Agile environment and how that might give a greater effectiveness to the software development process. We also help prepare people who aren’t in the Agile space work with partners who are, because we know that that’s a rub point right now.
Jim: I want to add to that by pointing out that training on its own isn’t going to have the desired effect–people have to take the knowledge back and do something with it.
There’s lot to Agile, and you’re not going to get it all in a one-day session. What we hope to do is to get people excited about it and show them where to go next, so that they can go back and do the hard work and pay attention and be passionate about it.
Scott Swigart: Tell us in a little more detail what each of these one-day workshops covers.
Diana: I’m going to take the lead on the first day. First, we’re going to look at what we’re really talking about when we talk about Agile. There are a lot of misconceptions and a lot of miscommunication out there about it.
It’s become a marketing buzzword, as everyone knows, and a lot of people are saying they’re doing Agile when that’s not what’s really going on. So we really want to communicate the truth to people, including the potential costs and benefits. Agile isn’t for every project, and it’s not for every company, so we want to give people some sense about how they can judge whether it’s really going to benefit them.
We also want to inform people how, if they decide it’s a good fit, they would go about adopting Agile in their own workplace. What are some of the steps and how do you go through that? What does it mean when you make that decision to adopt Agile–not just for the development teams, but for the broader organization?
We also want to talk with them about what makes Agile projects successful. What kinds of conditions do they need to put in place to ensure success? Because that’s really my interest, and certainly the Agile Alliance’s interest–we want to see successful Agile projects, so we support that in whatever way we can.
Finally, the last piece is how best to exploit Agile. How do you really make sure that your teams and your organization are doing all the things that are going to give you not just good development and good teams and good projects, but really great ones?
So that’s we’re focusing on for the first day. I’ll let Jim talk about the second day.
Jim: The first day is aimed at managers–people who are making the decision about whether or not to bring in Agile, and helping them understand the resources to apply there. The second day is aimed at contributors–people who are on the teams who are going to be executing Agile projects.
The key question that we’re going to try to answer is, “What do you need to know and what is it like to be on an Agile team?” Again, we’re going to talk about those Agile myths. People hear about Agile and one of the things the think of is, “Are we going to have to sit next each other and pair program? That’s a crazy idea, and we don’t want to do that.”
We’re going to help people answer questions like, “Is that what Agile is really about? Do we have to do that or do we not have to do that?”
We’ll talk about how planning works in Agile, because we have a very iterative approach. We’ll talk about Agile engineering, how you support the process, and about Agile’s focus on teamwork and people working together. We call it “individuals in interaction.”
Sean: What in both of your backgrounds do you think allows you to deliver a class that has a really high impact? If somebody’s trying to analyze the investment, what should drive people to take the class?
Diana: I draw on a background of instructional design and training, as well as the consulting practice that makes up the bulk of my working life these days. Both of us take a very consultative approach to training, so this is not a day of sitting and watching PowerPoint slides.
It’s about talking to people about what their real situations are, bringing in their experiences as much as possible, and having them really do work during the day. They will have simulated experiences, interacting with the other people there.
That’s one major benefit of this training–the people who come are going to network and interact with other people who are also interested. They’ll build a cohort of folks who can go forward and support each other. People they can call on the phone later and say, “I tried this strategy that I learned in the course. Have you tried that yet? How’s it going for you? I’ve bumped into these things.”
By the end of the day, they’ll know each other. They’ll be interacting and experiencing some of what it means to be Agile, and they’ll walk away with not just an intellectual understanding, but a deeper understanding of what this is really going to feel like for them.
Jim: One of the things I bring to the course is really an intense dislike of boring training courses.
[laughter]
I can’t stand sitting there and watching a talking head talk, talk, talk, and read slides, and I know Diana agrees with this. We both believe strongly that training needs to be experiential–something that you can actually take back and use, preferably from a workshop in the training course that addresses some of your real-world problems and situations. The more of that the better.
When I do course design, I always focus on activities and simulations that are very hands-on and as close to the real world as possible.
Diana: It needs to be engaging enough and relevant enough to the people in the room that they pay attention through the whole day and walk away confident that they know something they didn’t know before. When we’re putting these days together, we drive for people to walk away feeling like, “That was worth my time.”
Scott: What are some of the common problems that drive organizations to look at Agile development, and what are the benefits that they typically realize over time?
Jim: I see very consistent problems that teams face out in the field, and it’s really not about driving to Agile, it’s about driving to anything to solve those problems.
There are too many defects, and they’re not able to deliver when they said they would, or they’re not sure what they’re going to be able to deliver. They want to go faster or be more productive, because software development takes a long time, and it always takes longer than the people who are paying for it think it should.
People are interested in Agile because they’ve heard that it’s working. They’re saying, “We’ve heard about Yahoo trying it, we’ve heard about Google trying it, we’ve heard about this other company trying it. Let’s try it, too.”
And just like anything, you can’t pay it lip service and expect to succeed. People who are really making the effort are investing willpower and passion behind it. They’re creating what we call “high performance teams,” which is where people work together, take ownership of their work and their problems, and they start powering through things like a freight train.
I often see that these Agile teams overwhelm the business’s ability to keep up with them because they’re so rapidly solving problems. The ones that adopt the Agile engineering practices have very, very low bug counts–I’ve seen reports of one or two new defects per month. And although Agile really emphasizes changing directions to take advantage of business opportunities, we’re also seeing a lot more predictability in terms of making commitments and then meeting them.
Diana: One of my friends and colleagues, Jeff Sutherland, says that if you’re not getting four times your performance and productivity, you need to adjust your Agile process, because it’s capable of that. It’s capable of giving you that kind of increase in performance.
Sean: When somebody leaves the class, what’s the most immediate takeaway?
Diana: One of the things that happens when you start adopting an Agile mindset is that it starts showing you your own culture. People don’t understand the culture they’re in until they take a step outside and look at it in a new way.
Agile is extraordinarily good at surfacing the impediments in your organization to real productivity, and so right away, many people start noticing some of those things. At that point, they can choose to live with the impediments, or they can make the adjustments to fix them, but either way, their better awareness will help them be more efficient.
Jim: Awareness of the possibilities and the opportunity to take the next step makes a huge difference, especially in the second day, where we’re going to be talking more to the engineers and the technical people. There’s a lot of distrust in the technical community of anything that’s seen a lot of hype, and unfortunately, Agile development has seen a lot of hype.
In reality, Agile was a grassroots movement. I may be a little biased here, but I started out as a programmer, and I think the reason Agile development is being hyped is because it actually has seen a lot of success. But as a result of that success and that hype, I think a lot of people are distrustful of it, even if they don’t really know what it is or how it can fit in with their work.
Those same people are going to come out of our training knowing what Agile is, whether they want to pursue it, and if so, what their next step is to get started.
Sean: What expectations do you think people should have coming in to the class, to get the most out of it?
Jim: They should expect to have a good time, they should expect to interact a lot with a lot of people, and they should be prepared to share their thoughts and experiences, including the negative ones, and really be prepared to learn things that are new and different from their existing experience.
Scott: If an organization can adopt Agile techniques at a corporate level, that’s obviously ideal, if you’ve got executive support. Talk a little bit, though, about a small department in a massive organization. It may have a certain amount of autonomy, but no illusions that they’re going to get the whole 30,000 person company to buy into Agile.
Jim: It’s absolutely possible to implement it at that level. It’s actually the most challenging to have one person in a team or a group of people trying to do it on his or her own. If you can get a whole group of people who have a common purpose, even if that’s only an eight person group working with Agile, that’s enough to succeed.
In that situation, you will run into problems interfacing with the rest of the organization, and you need specifically to be aware of that interface. And since Agile is a different way of thinking about things than most people are used to, people get excited about it, and they sometimes get a little evangelistic. You’ve got to be aware of that, and not try to take over the organization–maybe just gently push.
Diana: A couple of the Agile practices come in very critically at that point. One is the daily stand-up meeting, where people have the opportunity to surface impediments. Those issues get logged, and someone is designated to go off and figure out where those impediments are arising from and what can be done about them. They bring that information back to the team, and the team makes choices about how they’re going to deal with it.
Another practice is the retrospective, which happens in a heartbeat kind of way at the end of every iteration, where the team has the opportunity to reflect on what’s been going on and really do continuous improvement. That also has implications for the interface between the team and the rest of the organization, and it gives them an opportunity to examine that relationship, how it’s working, and how it can be improved.
Those practices embedded in Agile really do help that eight-person team within a larger organization do what they need to do.
Also bear in mind that part of the reason that we’re having the first day on the business side of Agile is to alert the management layers of the organization that this is not something that just happens to the development teams. There also needs to be an awareness that while managers can use their same skills, they’re going to have some different roles and some different ways to apply those skills.
That has to be a piece of the puzzle, because if we try to manage business as usual while we’re doing this radically different thing at the development-team level, that creates a mismatch. And it’s one of the issues that has to be worked through for an Agile adoption to be successful.
Sean: That’s interesting, and it brings up the notion that developers by their nature are critical, in the better sense of the word, and they are only going to feel comfortable if a concept isn’t sugar-coated. So, what is the dark side of Agile when it gets applied?
Diana: One of the things that we’ve already talked about is that if you really are applying Agile in a good way, you start seeing the deficits in the rest of your organization and in your previous processes and so on. That means you have to make a choice about whether or not to deal with those issues, and those are sometimes hard choices. That’s a decision point at which sometimes folks say, “That’s all too hard. We just don’t want to go there.”
Sean: One thing I’ve heard is that Agile can tend to kill projects quicker and make people a little more visible. Maybe a project isn’t executed as well as it should be, and normally they’d throw a million dollars at it over the course of a year, and then only on Day 364 would they figure out that it’s not going to ship. Agile tends to illuminate those failures quicker.
Diana: Right–the whole idea of failing fast. Of course, the benefit is that it avoids spending money on a dead-end project. [laughs]
Sean: But it also creates a certain visibility of failure early on that, at least initially, creates a bit of a pain point.
Jim: That’s a really good point, and it also gets back to your question about whether a little department can adopt Agile on its own. If that little department discovers that they’re part of something that’s not really functional, who gets blamed?
I appreciate you asking this question about the dark side of Agile, because I think before people will believe or listen to me they have to feel like I’m not just trying to promote it.
Another piece is that, if you’re doing an Agile project, you need to work together. That is a social and cultural change that can be difficult for people.
Let’s say that you have a group that’s all in the same physical office, so you have the opportunity for everybody to sit together. One of the things I have found is that sitting together in the same room, even if you do nothing else, increases and improves communication so much that you really see a big improvement in productivity.
But asking people to move out of their offices is a real challenge, because offices are a status symbol, so there’s often a lot of resistance to sitting together. Even though some of these things are really effective, they’re different from the way people are used to working, and that can be hard to accept.
Diana: There’s another piece to the question of status. Very often, someone has been the star performer in a previous methodology, whether it’s Homegrown or Waterfall or whatever. All of a sudden, Agile doesn’t really have space for star performers because we’re looking at the team as a whole.
Not everyone fits in that environment, although there are certainly star performers who really want to bring everybody else up to their own level by helping them learn the stuff they know. Those folks are great.
Jim: The question is whether that star performer is a leader or they’re a hero who puts on the cape, runs in and rescues the project, and then runs out. If they wear the cape, they might not like Agile.
Diana: Many times, when star performers move off of the team when the team goes Agile, we discover that that star performer was actually depressing everyone else’s performance.
Jim: The reality is that you can only hire so many people with capes, anyway, so if you’re trying to optimize for that, you’re going to have a challenge, too.
Diana: That’s one of the myths–that Agile can only work if you’ve got only master-level developers. We’ve seen lots and lots of instances where a team that had been considered average-level performers has done absolutely fabulous work.
Sean: Agile seems to be applied only to software development. Do you think it has applicability beyond that?
Diana: I think there are practices in Agile that can work a lot of places, but it’s really tailored to environments with a lot of uncertainty, where the requirements are changing over time. Someone came and talked to me about other places in their organization that had gone Agile. They were feeling the pressure to go Agile, too, and they were in a call center with people sitting in cubes individually talking to customers.
They might benefit from a daily stand up meeting, but that’s a very predictable kind of work, so Agile is not a good fit there. In software development, where things are changing all the time and we’re learning new things, we can never know as much about the project in the beginning as we know at the end. It’s that kind of volatile, often mission-critical environment that Agile is well suited for, but that’s not every environment.
Jim: The Agile folks seem to have independently invented many of the principles that underlie lean manufacturing. There’s a lot of similarity between lean manufacturing ideas and Agile development.
But the way you actually do the work is completely different, because despite how people like to talk about software as being manufacturing or being like construction, it’s not at all. It’s a completely different world.
Scott: I’ve had many, many people say, “Look, I can tell you exactly how long it’ll take to lay down a mile of pavement. Why can’t you tell me how long it’ll take to build these features?” And the answer is that software development is invention.
Ask me how it will take to invent a new type of asphalt that will last twice as long, and you’re likely to understand it when I say I can’t predict that. Software development is like that–you’re always creating something that didn’t previously exist, otherwise you wouldn’t be writing it.
Diana: Right.
Scott: This has been a great conversation. Thanks.
