… [a] research study, funded by Hewlett-Packard and conducted by the Institute of Psychiatry at the University of London, that found, “Workers distracted by e-mail and phone calls suffer a fall in IQ more than twice that found in marijuana smokers.”

Stanford study: Cognitive control in media multitaskers:

Results showed that heavy media multitaskers are more susceptible to interference from irrelevant environmental stimuli and from irrelevant representations in memory. This led to the surprising result that heavy media multitaskers performed worse on a test of task-switching ability, likely due to reduced ability to filter out interference from the irrelevant task set.

Meet the Life Hackers – New York Times Magazine (10/16/05) (based on research by Gloria Mark, University of California)

Research shows that 40 percent of the time, workers ramble along a different tangent when an interruption ends because their short-term memory has been disrupted.

Ho do you manage your interruptions?

It presents some information that is different from how the Standish Group defines success in their often refered to CHAOS report. The Standish Group assumes that success is the traditional “on time, on budget and on scope”. In many ways this does not make sense as an estimate of time and budget are made when the least information is available (early in or before a project starts).

In Scott’s survey the following success factor priorities emerged:

• “Schedule: 61.3 percent of respondents said that it is more important to deliver a system when it is ready to be shipped than to deliver it on time.
• Scope: 87.3 percent said that meeting the actual needs of stakeholders is more important than building the system to specification.
• Money: 79.6 percent said that providing the best return on investment (ROI) is more important than delivering a system under budget.
• Quality: 87.3 percent said that delivering high quality is more important than delivering on time and on budget.
• Staff: 75.8 percent said that having a healthy, both mentally and physically, workplace is more important than delivering on time and on budget.”

Comparing Agile with other development methods

Based on these success criteria the survey investigated the success rate of different development methods:

A small step, but apparently Agile methods have the upper hand if the study is correct.

The missing pieces

What I still think is missing is a study that actually measures the outcome of a project after it was put into production and compares this with the desired effects that were envisioned. Neither the Standish Group nor Scott’s study on success factors seem to cover this.

• How quick was the return on the investment (if the goal was to increase revenue and/or reduce cost)?
• Were all of the desired effects realised?
• Were all the developed features used in a way so that they contributed to realising the desired effects? (think “reduce waste” if you are into Lean or “usability” if you are a human factors kind of person).

Measuring the above items would give more insight into the true success rate of a project rather than measuring the highly subjective “do you think is was successful with regard to factor X”? The only problem, I guess, is that few projects detail the desired effects in a way that is measurable in isolation from other factors in the environment (e.g. “the economy”).

The above items would also make it clear that the usability perspecive should be included from the start of a project.

Coupled with the extremely short feedback cycle realized by using dynamically typed languages such as Ruby, Rails may have a deeper impact on software development practices than I previously thought.

At a recent Scrum training session Ken Schwaber said “it’s all bout telling the truth”. Philip’s post contains some interesting quotes related to this topic:

“When a vice president in the Windows [Vista project] asks you whether your team will ship on time, they might well have asked you whether they look fat in their new Armani suit.

“[...]the intrepid managers finally understood how to get past the dilemma. They simply stopped telling the truth. -’Sure, everything fits. We cut and cut, and here we are. Vista by August or bust. You got it, boss.’ Every once in a while, Truth still pipes up in meetings. When this happens, more often than not, Truth is simply bent over an authoritative knee and soundly spanked into silence.”

Didn’t Microsoft adopt Scrum a year go? Maybe they skipped the part about transparency. Granted, the Windows Vista project may be one of the more complex software projects to date, but how many VP:s do you really need? And even if you do the waterfall dance, there must be some insight into real progress, even for pointy haired bosses?

Update: Apparently too many readers turned his blog post into the standard Windows/Linux shootout which made him remove the interesting parts. Google cache currently has the original.