Agile won’t make you faster

Posted by Luke Walter on December 31, 2011

A common misperception of what agile is about is speed – specifically, that it makes development faster.  I hear this a lot when asking folks what they’ve heard about agile development – “Agile will make us faster.”  I suppose this is unsurprising given the chronic lateness and cost overrun of the typical software project.  Slowness is the disease, and agile is the cure. Or are they?

Merriam-Webster defines agile as:

1: marked by ready ability to move with quick easy grace <an agile dancer>

2: having a quick resourceful and adaptable character <an agile mind>

The Cambridge Dictionary defines it – in British (not American) English – as:

able to move your body quickly and easily <Monkeys are very agile climbers.> <You need to have agile fingers to do this kind of work.>

Notice that the key word here isn’t ‘fast’, but ‘quick’.  ‘Fast’ is a fraught word, and seems to be what most think will be the new nature of an otherwise traditional project – exhaustive, ill-informed, up-front specifications and all – once we just get agile.  But do we really want the whole train wreck to simply increase speed and arrive sooner?

Agile’s primary benefit isn’t speed but, rather, feedback – between development team members, the development team and the business, and the business and the market.  The most noted effect is greater understanding and insight into what’s working and desired (and what isn’t).  This allows course correction that wouldn’t be possible if you were only hurtling toward a mistaken destination ever faster.  Admittedly, over time and with experience, this difference does lead to a speed-related effect:  more frequent delivery of only the highest-value items.

But what’s delivered to the business and the market isn’t faster in the same sense as what’s expected; You won’t get a whole year’s worth of pre-agile production of code in only three months post-agile. And you wouldn’t want to, would you?  What you might get – with learning, practice, and experience – is a traditional year’s delivery of value in half or a third or even a quarter of the time.  But first you’ll have to get really good at listening to the feedback you elicit, involving your customers, and changing your mind and your plan.  But will you be faster? Which is more important?

Category Categories: Uncategorized  |  Comments View Comments

Where do we credit work done on unfinished backlog items?

Posted by Luke Walter on December 19, 2011

One of the most common questions asked in Scrum classes and coaching is how to handle backlog items that don’t get finished in a sprint – where do we credit the ‘work done’ for unfinished backlog items?

While it’s understandable to want to allocate credit for the effort spent to the sprint to which it ‘belongs’, it’s important to remember why there’s no partial credit in Scrum: Because working software is the primary measure of progress (#7 of the principles behind the Agile Manifesto: http://agilemanifesto.org/principles.html). The metric of record is therefore how much a team can complete to potentially-shippable quality each sprint, not how much they can complete to 90% finished status. This is a common occurrence with new Scrum teams. By the end of Sprint, they have a whole bunch of backlog items “90% complete” (whatever that means), which means they’ve got nothing done to potentially-shippable standard, which means they’ve delivered zero business value. In other words, they’re 0% done.

Attempting to ‘give credit where it’s due’ by tracking how much work was performed in which sprint skews the emphasis toward measuring how hard people tried to reach a goal, which isn’t what we want. We want to measure the goals reached: backlog item story points competed per sprint, or business value delivered if you’re using that metric.

The most common recommendation is that an unfinished backlog item gets returned to the product backlog, with its original estimate intact. When it gets committed to a new sprint and completed, the backlog item’s full effort estimate gets credited to the Velocity of the new sprint. While this may appear to skew the numbers, the numbers average out over time – and it’s this average velocity that a PO uses for forecasting and planning. Moreover, this practice adds an incentive for the team to do their best to discuss, negotiate, estimate, and commit to finishing backlog items to full completion in a sprint.

If management and finance want to track costs for purposes outside of managing a team (one that doesn’t need managing because it’s self-organizing) – say, for hourly client billing – then a task-level metric might be used. A carried-over backlog item’s constituent tasks done in the first sprint will remain counted as tasks completed (or task hours completed and remaining if you’re using those metrics) as part of graphing the sprint burndown in any good agile project management software such as ScrumWorks Pro.

This is a not-uncommon business need, although the practice is easily abused to “manage” agile teams’ “productivity”. The danger here is that it leads to a mentality (to use a relay-race analogy) of tracking the runner rather than tracking the baton. In Scrum it’s the baton that’s important – that is, story points of backlog items done to potentially-shippable completion each sprint (Velocity). Since ultimately that’s what we care about, then that’s what we should spend our expensive, limited time encouraging and focusing on.

Category Categories: Uncategorized  |  Comments View Comments

Technical Debt Webinar – Q&A

Posted by David Parker on December 10, 2011

Earlier this week, we ran a webinar entitled “Technical Debt – the High Cost of Future Change”. The topic was of course, technical debt in Agile projects. Although we left what we thought was ample time for questions, as it turned out there were many more than we had time for. So, as promised, we are posting te questions and our responses here. I hope these are helpful!

Q: I’ve found that when asked, people come up with a very ‘full’ definition of done, but in the end they don’t follow it. How do we walk our own walk?
A: I could answer this in a number of ways, but I’ll take a more hard-edged approach here. It is about accountability. An important aspect of Scrum is the idea of the “single wringable neck”. At the end of the day, it’s the product owner who determines whether a story has met the agreed upon definition of done. If he or she chooses to ignore aspects of that definition for whatever reason (expediency, effort to be a liked, business pressure), ultimately he is the one who bears the responsibility for the impact.

Q: Isn’t it likely that someone writing poor code might also write poor and ineffective tests?
A: Absolutely. Agile practices are designed to uncover and display issues like this so that they can be addressed. Practices such as Pair Programming, code review, daily stand-ups, retrospectives, and sprint review meetings are so important. There is no magical fix for poor test writing, but when we know the problem exists we can work to fix it.

Q: How was “”easyness”" vs “”difficulty”" of changing the code measured? What metrics exist for what seems like a semi-subjective value?
A: There are many variables to estimating the difficulty of a code changes. We could talk about things like the complexity of the code (Cyclomatic Complexity), the experience of the programmers, their familiarity with the topic and/or the specific module, the quality of documentation, etc. It ends up being a fairly subjective estimate. In Scrum teams, story sizes are estimated in relative terms in terms of story points.

The primary benefit to using a technique involving Relative Estimates is that you are asking the team to give you an estimate of difficulty relative to other work that has already been completed. This means that a team can easily give judgments like “This will be twice as hard as that” and come up with functional estimations for predictions without spending a great deal of time coming up with them. Estimates are just subjective guesses anyway, understanding that can be a valuable way to put more time into building something and less time into trying to guess how much time it will be to build it.

Planning Poker, also called Scrum poker, is one technique for building relative estimates and for coming to consensus on the effort or relative size of the stories.

Q: Technical debt always meant: those things that you postpone doing that you know must be done, that become more expensive to do as time goes on — e.g. the technical “”debt”" has compounding “”interest”" applied.
A. I like the extension of the metaphor to include the interest on the “debt” because it is quite valid. That said, I would also point out that there are often valid and justifiable reasons for incurring technical debt. As long as teams incurring that debt know what they are doing, have justified it, and have the means to pay it off in the future then that’s fine. (Perhaps there should be some kind of Consumer Protection Agency for developers? Never mind…)

Q: How often does pair programming fail as the result of pairing developers where one has a dominant and one has a passive personality — and how can this be detected and treated?
A: In pair programming you have a two programmers work together at one workstation – a driver and an observer. The driver is the person who types in code while the observer reviews each line of code as it is typed in. The idea is that the driver is focused on completing the tactical aspects of the task at hand. The observer takes a strategic view, looking for ways to improve the code and at the how it fits into the overall system framework.
Generally, you know there is a problem when you see lack of engagement between the two. Simply put, they are not talking. A good way to try to encourage communication is to swap their roles frequently – at least once per day.

Q: If the automated tests exist – but are as old as the code that it’s testing – how would that help?
A: They help to prevent regressions, where changes in other portions of the code may break something that was untouched.

Q: Should refactoring be included as a user story in the sprint?
A: It works to have refactoring stories in sprints, or even to devote an entire sprint to refactoring. Whichever way you choose, the stories must be compliant with INVEST – independent, Negotiable, Valuable, Estimable, Small, Testable. (Bill Wake).

Q: what happened to defensive programming?
A: I think it is an issue of semantics. The principles of defensive coding are well represented and addressed in Agile development techniques. These include reducing source code, complexity, engaging in formal source code reviews, software testing (especially in the context of Continuous Integration practices), re-use, and so on. I would posit that an experienced and mature Agile team is practicing defensive programing.

Q: In the case of a legacy system with existing technical debt, is it advisable to dedicate one or more sprints to reducing the technical debt?
A: This isn’t a technical decision, of course. The decision to devote a sprint to paying down the technical debt of a legacy application should be made based on the business merits of doing so. Look at how much that technical debt is costing you in terms of delivering new functionality and addressing defects. Look also at the impact of moving your developers off enhancements and defects, and onto a project whose impact will not been seen for months. You strategy will be informed by the current maturity and projected lifespan of your application.

You might be interested in a recent webinar we did that looked at integrating Agile Project management and Project Portfolio management. See the link here.

Category Categories: Uncategorized  |  Tag Tags: , , , , , , , , ,  |  Comments View Comments

A Marriage Made in Heaven: Agile and Project Portfolio Management

Posted by David Parker on November 1, 2011

On October 27th, I co-presented the webinar, “A Marriage Made in Heaven: Agile and Project Portfolio Management,” with Russ King, Vice President, Product Development, Results Positive, Inc. and Caleb Brown, Systems Engineer at CollabNet. We explored the benefits of marrying Agile Project Management and PPM and we did a live demo showing this using HP’s PPM solution and CollabNet’s ScrumWorks Pro to demonstrate the powerful capabilities of managing a resource constrained project portfolio.

Judging by the number of questions, the audience was clearly heavily engaged.  We had a 15 minute Q&A and did not come close to answering all of the questions, so I’ve listed most of them here and provided a response. The ones pertaining to HP PPM will be in a separate blog post written by Russ King of Result Positive.

If you missed the session live, you can watch the recording and download my slides. I also encourage you to try SrumWorks Pro for free and find out why ScrumWorks Pro is the best Agile project management solution on the market today. Also, don’t forget to register for our upcoming webinars. CollabNet offers multiple webinars a month that cover a variety of topics related to software development.

As promised, here’s the follow up to the live audience questions.

Q: How feasible is Agile on Projects & Programs?
A: Agile is typically thought of in the context of individual projects. Companies sometimes fail to scale that paradigm to a program level, where the program is a superset of multiple projects, each running its own lifecycle and release plan. The trick is to weave those separate lines of development (projects) into a coherent and seamless deliverable (program). The complexity comes in gathering meaningful metrics and planning releases that thread the elements together. This is exceedingly difficult to do manually.   CollabNet’s ScrumWorks Pro is a tool that can make this manageable. It supports the planning of complex releases that weave in multiple development threads.

Q: Will this process will be feasible for maintenance related projects (Incident handling, less than 8 hours development works, etc.,)?
A: From the PPM perspective, an individual defect is not in and of itself a project and as such, would not be tracked.  What might be tracked is a larger group of maintenance items in the form of an Epic. From an Agile perspective, a bug report or defect is just another piece of deliverable business value, like a User Story or any other Product Backlog Item. From a bug report, the product owner and team would create a Product Backlog Item (PBI), along with success criteria (definition of done). It is prioritized against all of the other Product Backlog Item by the product owner. Again, multiple bugs/defects are often grouped in an Epic.

Q: It seems the PPM is geared toward a waterfall process.  It appears there is only visibility into the Development phase, but with agile, you could potentially address all phases within a single sprint.  Is that just because of the way this implementation was set up or is it there isn’t a true marriage of the agile within PPM?
A: PPM in this scenario is focused on evaluating the ROI of different projects and deciding where to make investments. Agile is focused on execution of the projects that are chosen. That said, the scenario we propose makes the entire organization more Agile, in that the feedback loop is instantaneous.  This allows those that are making the investment decisions to adapt and make course corrections that are indicated by that feedback loop. The integration gives all team members the ability to work in a more Agile fashion, and gives Stakeholders and Project managers the ability to benefit from the faster feedback and data generated by the team working this way.

Q: Can the tasks in Scrum WorksPro be connected to tasks, timelines in Source forge?
A: Not with Sourceforge. However this is possible with CollabNet TeamForge, the current commercial version of Sourceforge.

Q: Can you clarify what part of Agile PPM can be done in scrum works pro without need for HP PPM?
A: ScrumWorks Pro is focused on project execution and project management. As such, ScrumWorks does a number of things not accomplished in HP PPM. These include PBI tracking and prioritization, Task management sprint planning, release planning, team velocity, forecasting, and many other functions related to the management of an Agile project.

Q: So are you proposing (in the demo) to combine a phase/waterfall planning and design phase, but then execute in an agile framework?
A: Combining HP PPM and ScrumWorks Pro adds to the agility of the entire organization. Feedback loops between the development team and the PMO are enhanced allowing the PMO to make course corrections required. I would not say that as a result the entire enterprise has become agile – only that they’ve become more agile.  Generally, we do not see many organizations that practice a pure version of ANY methodology –be it Agile or otherwise. The reality is that organizations have a mix of methodologies, like Scrum, Kanban, Waterfall, hybrids, etc. Different teams in large organizations will often build software differently, so the challenge is to roll up the data from those disparate teams. Despite their differences, there are a number of common metrics you can track regardless of project type.  These include actual cost versus budgeted cost, scope change, personnel/resource change, delivery dates, and others. Tools like ScrumWorks and HP PPM do a good job in tracking these kinds of numbers.

Q: Continuing from the first question, from a portfolio perspective, having “”open-ended”" project budgets within the Agile/SDLC process is not in the best interest of my customer.  How does budget planning and Agile development work together while still having some control over costs?
A: Project prioritization and the associated budgeting/funding are is under the purview of the PPM tools.  The agile project management tool tracks the amount of time individuals spend on the project. The integration between the HP PPM tool and the Agile Project Management tool, allows you to easily compare budgets against actuals.

Q: For the Forecast report in SWP, if new backlog items are added during the sprint, does that add to the top or bottom of the bar? Also, how does the Project Portfolio Management tool fit into the larger Enterprise Architecture discipline?
A: It depends on what report you are looking at. In the forecast report, added backlog item appears on the bottom of the report and impacts the forecasted delivery date.

The “Burn-Up” Report shows the relationship between work completed per iteration (sprint velocity) and project scope change. The forecast feature extrapolates the rate work gets done against the rate of scope change to provide an empirical release completion forecast for more accurate release planning.

Q: In agile, what are the differences between being adaptive to late changes in requirements within a sprint and scope change?
A: Scope change refers to any added or subtracted scope, typically measured in some form of relative effort unit like Story Points. As such, scope may be added as a team discovers more about an existing requirement. In other words, if the team finds out that a requirement is more complex than was originally envisioned, they may re-estimate the number of story points and this might add scope to a sprint. The opposite could also be true. Whether this occurs because of a discovery inside a sprint or outside of it doesn’t change the nature of how it is tracked or reported upon.

Q: When a committed backlog item could not be completed in a sprint, naturally it holds the top most priority in the following sprint. How does ScrumWorks helps in tracking this item from the beginning to end?
A: An unfinished PBI may or may not be a high enough priority in a future sprint. The determination is made by the product owners. In any event, any activity against that PBI is tracked. Tasks completed that relate to that PBI are tracked, as are those that were uncompleted.

Q: What certification do CollabNet-trained scrum masters receive?
A: Those who attend one of our Certified ScrumMaster or Certified Product Owner training are eligible to take the exam deliver by the Scrum Alliance. It should be noted that CollabNet is one of the leaders in ScrumMaster product Owner training.  We have more  Certified Scrum Trainers on staff than any other vendor, and we’ve trained more than 12,000 ScrumMasters.

Q: If an organization wants to be able to report a metric of time to resolution for individual PBIs, what settings are available in this integration to include/exclude a PBI from the current active lists so that a countdown starts appropriately?
A: Forecast reports in ScrumWorks can be filtered on any number of aspects, allowing a user to deliver estimates on individual tasks, Stories, Epics or Themes. By the way, you can try out ScrumWorks Pro either in a hosted environment or as a free download.

Q: We are HP PPM users. Who owns the IP for the ScrumWorks integration technology? Cost?
A: Results Positive and CollabNet worked closely together to build the integration. Results Positive however owns the integration IP and can provide any required assistance you might need in getting the integration set up.

Category Categories: Uncategorized  |  Tag Tags: , , , , , , ,  |  Comments View Comments

Technical Debt for PMs

Posted by Victor Szalvay on October 5, 2011

Projects@Work has published one of my articles: Technical Debt for Project Managers.

Article summary:

Technical debt describes the cumulative consequences of cutting corners in software development, but it escapes the attention of many project managers as they focus on scope and schedule. That’s a mistake because it impacts both. Here are questions to help you ascertain the real state of technical affairs.

Category Categories: Uncategorized  |  Comments View Comments

The upside to getting your neck wrung

Posted by Luke Walter on September 30, 2011

A common characterization of the Product Owner role is as the “single wringable neck” – the one person the business can go to when the product fails to deliver on expectations or market success. This saying encapsulates the difficulty and responsibility of the position; Few people would sign up for this, and in fact, when organizations are new to agile and particularly to Scrum, the role of Product Owner is often the most difficult to isolate and instill in only one person. Most organizations – attempting to be agile or not – fractionalize product decision making between multiple individuals, and thereby spread not only authority and ownership, but also responsibility. None of this is much good for anyone, of course; Spreading authority and ownership makes it extremely difficult to come to a solid decision, give development teams clear direction, or achieve anything like a cohesive vision – frustrating for the team and counterproductive for the organization. Spreading responsibility is even stickier: Whose fault is it when the product fails? It’s not the team’s fault – they were just following confused and ever-shifting instructions. It’s not any one product owner’s fault, either – they’re just one of many. Perversely, though, this arrangement can often be a cozy one for someone with authority.

I once had this put to me bluntly by a would-be product owner when we were discussing how much information and detail should be given the team when defining requirements. I was trying to impart the value of the more is-better approach, when he stated he wanted to provide only “Just enough detail to hold others accountable [in the event of failure], but not so much that I can be held responsible.” He appeared to be joking, but there was no denying the element of morbid truth.

If you look closely, however, it’s obvious this arrangement is an attempt to come to grips with the intrinsic unpredictability of software development, by reserving the ability to change requirements or direction mid-course. A dysfunctional and chaotically wasteful method, for sure, but such things happen when laboring under false assumptions like those embedded in waterfall.

Ironically, this ability to change course is the very raison d’être of agile development – fortified with clearly defined roles and transparency to prevent the perverse situation above. The team has the authority to say how much work it can do in a given time, and the responsibility to meet the commitments it makes. The product owner has the authority to determine what does or doesn’t get into the product (and set the priority order of what does), and the responsibility for the success or failure of those decisions. With power comes responsibility. And there’s nothing like your neck on the line to provide the motivation to perform at your best.

Category Categories: Uncategorized  |  Comments View Comments