We've come a long way since 2004. That year, one tech forecaster made the bold prediction that "eventually up to one third" of smartphones would include a camera. He could hardly have imagined the popularity of selfie sticks today. That same year, the agile movement was emerging, prompting some members of the UX community to wonder about its impact. Were agile and UX going to form a dream team or an odd couple? To answer this question, I interviewed UX specialists about their initial experience with agile and shared the results in an Interactions article in 2005 .
Fast forward to 2016, when I received a notification from ResearchGate that this article had been cited in 50 subsequent articles. At first, I was astounded that so many articles had been written on this topic. Then I thought back to 12 years ago and marveled at how things had evolved in both the agile and UX communities. Even the terms agile and UX have been joined, and possibly superseded, by new terms like lean and design thinking. Many people I work with have only known an agile lifecycle. Finally, I wondered: What became of the people I interviewed in 2004? Had their perspective changed after 12 years? I decided to find out and share the results in this article.
When I reconnected with the people interviewed in the original article (Thank you, LinkedIn!), they were all still working in UX roles and were happy to share an update. Here's what they've been up to since 2004:
- MG has worked for multiple organizations in generalist UX roles and in both agile and non-agile teams.
- PV continues to work as an independent UX consultant and usually works remotely on two or three projects concurrently. PV works with both agile and non-agile teams.
- TB has worked for a wide variety of organizations (including startups, agencies, and enterprises). Most recently TB worked in a dual role as product owner and UX lead on software for investment firms. Since 2004, TB has worked exclusively on agile teams.
This article reports on their experiences and reflections in their own words using the same categories as the original article, namely: incorporating UX, understanding users, design, and evaluating design usability. Readers should be aware of some caveats when they see interviewees using the term agile:
- The interviewees are referring to the popular variants, such as Scrum, that incorporate concepts like sprints and product owners. Some interviewee comments might not apply equally well to all variants of agile.
- Some quotes refer to published recommended agile practices while others refer to what they actually experienced in their project teams—which can be quite different.
Early formulations of agile did not identify UX as a distinct aspect of software development. Today, UX is more widely recognized, especially since the publication of Lean UX  in 2013. In the 2004 interviews, all interviewees reported a positive (or at least somewhat positive) reception to their UX role. As well, they reported a blurring of roles, as advocated by agile, such that the UX specialist participated in non-UX tasks and vice versa.
The current interviews portray a more nuanced and less positive picture. These UX specialists usually need to define a suitable role on an agile team and advocate for UX in project discussions of resources and trade-offs. Here's what they said:
- At my current employer, [agile] never really took hold, despite several efforts, including training. You end up with the practices (e.g., standup, sprints) without any passion or commitment, kind of like people who go to church out of habit. —TB
- Many developers immediately start cutting scope [i.e., discarding parts of the design so there is less to implement] from a wireframe, even though that wireframe is already the result of considerable de-scoping. While I've worked with developers who find ways to build great user experiences, unfortunately, I find this mentality rare indeed. —TB
- Conceptually, I agree with the agile view that teams should avoid excessive role specialization. However, while I don't mind doing things like writing code on occasion, doing this on a regular basis detracts from the time I can devote to design and results in context-switching overhead.—MG
- While agile gives developers an opening to do design, that might not always be best for the team. In a healthy team, work goes to the person best suited to do it. In an unhealthy team, it can go to whoever wants to argue about it. This has resulted in hours spent arguing about design with certain developers.—MG
- I like the agile practice of working closely with developers—I get to understand what's difficult to implement and why. This understanding informs my designs for subsequent iterations.—PV
The points above lead me to suggest some useful distinctions for understanding UX-agile integration:
- Coders vs. leaders. When discussing the "development team," interviewees distinguished between working with coders (who implement designs) and leaders (any other person responsible for supervising coders or setting product direction, schedule, resources, etc.)
- Individual differences among coders. Interviewees reported large individual differences among coders in how effectively they work with UX on initial designs and design changes.
- Agile in name vs. agile in practice. Interviewees encountered teams that called themselves agile but followed few agile practices. So, we need to distinguish between published agile practices and what UX staff actually encounter.
Early formulations of some popular agile methods (e.g., Scrum, XP) eschewed the practice of preceding design with a phase of user research to develop a deep understanding of users and their needs. Since then, some practices like personas have become part of the agile mainstream. In 2004, interviewees reported being able to extend agile practices with traditional UX approaches like personas and sometimes significant user research prior to the project kick-off (e.g., the first sprint).
In the current interviews, these UX specialists reported less opportunity for what they consider adequate up-front user research. Here's what they said:
- There is quite a range. In some projects, no user research was conducted; the product visionary was the primary source of insight. In my current project, we are interviewing many end users to create personas and other artifacts.—TB
- Agile [as I've experienced it] does not have space (e.g., sprints) for understanding users in the UX sense. Business people think they know what users want. Technologists have an idea of what they want to build. Designers are expected to start wireframing right away.—MG
- Conducting research with actual users is rare. Speaking with user proxies is more common.—MG
- In one project, we conducted about 50 hours of interviews. We did not synthesize the data but a couple of people understood it well and answered questions from the team.—MG
- User research is typically not conducted because the intent of agile is to get customers involved during development. However, teams I work with usually consider it sufficient to only collect data from user proxies or domain experts.—PV
In summary, interviewees reported that the common practice either does not include any research or the research is only with user proxies.
The agile literature has always suggested that UI design practices used in waterfall projects need significant adjustment to stage work into multiple short iterations. The 2004 interviews confirmed this. Interviewees felt the resulting UI designs were, at a minimum, no worse than what they had achieved in waterfall projects.
The current interviews dwelled on the nature of collaborating with developers. Here's what they said:
- Here's some ways I've adapted my UI design work to agile: (1) Try to work on designs two or three iterations ahead of the agile team, which allows more high-level design work. (2) Sometimes have a "UI spike" iteration that focuses exclusively on UI enhancements. (3) Complete and review designs with user proxies in one iteration, then give the designs to the development team to build in the following iteration.—PV
- It's hard to juggle supporting the implementation of the sprint in progress with designing for upcoming sprints. It's hard both logistically and mentally. I wonder whether these should be separate jobs.—TB
- In one case that worked well, I involved the developers in fleshing out the design details. We succeeded by communicating frequently and by their willingness to respond rapidly to my feedback.—TB
- In one example of things not working well, the designer was not informed of the latest team discussions, resulting in them continuing to work on wireframes that were then discarded.—TB
- An advantage of agile is that we can adjust the design of a feature as the developer works on it. When details are specified too far in advance, with a different set of technical assumptions, the developer may not be able to deliver on the design.—MG
- Developers' attitudes have a large impact on the design outcome. As a positive example, I worked with a developer who really wanted to understand the requirements and was sensitive to UX fit and finish [attention to detail]. We worked together to design the functionality in a generalizable way to enable its anticipated use in other parts of the application.—MG
- A negative example is a developer who wanted to develop the basic functionality and then stop before the experience was smooth.—MG
Some themes in these quotes include:
- A split in attention between designing for the current iteration versus future ones.
- Collaborating with developers: This aspect can have a large positive or negative impact on the design implementation, depending on the developer.
Agile practices for evaluating design usability can seem lightweight from a UX perspective, where entire books have been devoted to just a single evaluation topic, for example, inspection methods or evaluation metrics. In 2004, our practitioners were sometimes able to employ UX methods for evaluating usability. When success was limited, barriers seemed due to longstanding factors, like schedule impact and difficulties making design changes late in the cycle, rather than to anything specific to agile.
The current interviews indicate that opportunities for usability evaluation remain limited. Here's what they said:
- In one project, we conducted quick usability tests and discussed design options with a small number of users. We were able to make minor design changes but any major change was not usually feasible.—MG
- In my projects, I generally don't get the opportunity for formal usability evaluations.—PV
- One practice teams use is presenting a dog and pony show [i.e., a promotional-style presentation] to the customer for their reactions.—PV
- In one company, periodic usability testing was feasible while in another company it was not feasible due to lack of access to customers. Often the closest approximation of design evaluation has been agile sprint reviews held for stakeholder feedback.—TB
These quotes highlight two barriers to usability evaluation: not having an opportunity to do an evaluation and not being able to make changes based on evaluation results.
So, have things changed since 2004? In that year, all interviewees reported a positive initial encounter with agile. Certainly, job satisfaction was positive—they felt actively engaged in a team pursuing a common goal. But did they feel the resulting designs were better than those from their prior waterfall projects? Not necessarily—but at least they were no worse.
Today, these same people feel the same or a little worse compared to 12 years ago. They still prefer agile over waterfall as a vision, especially as it enables them to quickly see designs implemented and to collaborate with developers to fine-tune the designs.
So, why aren't they more positive overall than in 2004? One reason is they were already quite positive so there wasn't much room for improvement. The other reason is they were dissatisfied with how some teams failed to follow agile best practices, as illustrated by these quotes:
- One team followed so few agile practices that I'd hesitate to even call this an agile project. They were agile in name only.—TB
- The agile practice of lightweight documentation is great—but NO documentation is not.—MG
Feelings of satisfaction and project success did not seem heavily determined by whether the project used agile or waterfall or by the particular agile methods. Rather, the important factors were:
- How well the project was managed. Was there effective implementation of development methods, clear project direction and decision-making practices, adequate team resources and skills?
- How receptive developers were to UX. Success stories often involved working with individuals who were especially attuned to UX.
The conclusion seems to be that outcomes are determined more by perennial project success factors (people and process) rather than by technical agile method characteristics. To confirm this conclusion, I posed a thought experiment to the interviewees:
Which project described below would you prefer to work on?
- Project 1: You can specify your favorite agile practices but you can't choose the staff.
- Project 2: You can choose your favorite staff from past projects, but you can't specify what agile practices will be used.
Everyone chose Project 2.
While this article is based on the experiences of only three people, they are typical UX professionals who have experienced agile throughout its entire history and who can compare it to non-agile projects. So, that's how things look in 2016. Maybe I'll check back in seven years—just like the Up Series documentaries on British television.
Thanks to the people who shared their experiences: Tom Bellman, Mahitha Gokulachandra, and Pawan Vora.
Paul McInerney works at IBM and has more than 20 years of experience in UX roles on software product development teams. He has written or presented guidance on several topics of UX design practice, including agile UX, UX specifications, scenarios, and design patterns. email@example.com
Copyright held by author. Publication rights licensed to ACM.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2017 ACM, Inc.