XXVI.3 May - June 2019
Page: 70
Digital Citation

Creative Next: AI, automation, and the practice of user experience design

Dirk Knemeyer, Jonathan Follett

back to top 

The word automation conjures an image of a factory full of robots, a modern marvel symbolizing both technological progress and the regression of working-class opportunities and lifestyles. But our notion of automation generally remains ossified in this physical, machine-replaces-labor frame. We don't think of automation in the realm of knowledge work beyond the most mundane and mindlessly repeatable tasks. But automation, powered by machine-learning advances in artificial intelligence (AI), is coming. It's actually already been here for decades, going back to relatively primitive software innovations that eluded our ability to connect the dots back to industrial robotics before it. Perhaps surprisingly, modern AI automation has been making original art for years [1] and has collaborated with a human team on an original painting that sold at Christie's for $432,500 [2]. Beyond art making, AI automation can also write procedural content such as stock blurbs and minor league sports stories [3].

back to top  Insights


Automation will eventually transform the professional tasks and responsibilities of UX professionals—designers most quickly of all. The consequence of these changes will be the elimination or specialization of more rote tasks or those that involve the use of software to prepare deliverables. As a result, UX designers will see their roles flatten. On the one hand, craft and creation will be replaced with strategy and creative direction. On the other, designers will become more specialized craftspeople who are part of the important but diminishing number of makers. Into the foreseeable future, this will not result in fewer total jobs or work opportunities; it will, though, change the knowledge and skills that we need to be effective in what is sure to be a mercurial work environment.

back to top  Automation in User Experience Design Practice Today

User experience design is already heavily automated, even though we may not think of it that way. Software, from older core tools like the Adobe Creative Suite to the latest and greatest such as Figma, Sketch, and Framer, has transformed laborious physical design tasks into fast and easy ones. While it may be hard to imagine today, in as recently as the 1990s designers would spend more time using rubber cement and Exacto knives than sitting in front of a computer. Layouts progressed from sketches to physical mockups to camera-ready art. The toolkit of a designer relied heavily on arts and crafts skills. It was a physical activity, not a digital one. The metaphors used in the Adobe Creative Suite were developed with these analog creators in mind, translating their tools and terms into a digital package that reduced the time their work would take by a significant percentage. Over the years, this has only accelerated. Whereas Adobe's InDesign, created for print publications, was once bent to be a core digital design tool, more modern apps offer better automation thanks to being natively designed for their specific purpose.

This evolution has provided a number of benefits. First, it enabled user experience designers to expand the scope of their knowledge and responsibilities to keep pace with the environments they were creating for. Platform innovations such as mobile computing and the increasingly complex and interactive nature of software require different knowledge, skills, and focus from what similar professionals would have needed 10, much less 20, years ago. So, as software automates some things, we are able to evolve with the environment thanks to the time afforded by automation. This has enabled user experience writ large to mature into a better defined and strategically important profession. While research and design have long circled the nucleus of UX, much of the work required in the dotcom and Web 2.0 days boiled down to what we often referred to as "design monkey" work. Research as an integrated part of the design process was rare, and development as part of the UX was almost nonexistent. Today, we take for granted that research is a pillar of UX and that front-end development is an integrated part of a UX team and effort—to say nothing of strategy and the integration into—not merely taking orders from—product, marketing, and engineering organizations.

Much of this is possible only because of automation. Otherwise, the combination of UXers not having enough time and organizations not having enough budget would have delayed or prevented the evolution of the discipline. Automation, without our realizing it, improved our lives, our field, and the world of computing that we helped to create, all at once. Over the past decade, UX has been a high-growth profession. While this growth is not the result of automation—the mobile revolution created an explosion of needs for more UX—automation did not prevent the job market for human professionals from exploding. The growth of UX became the proverbial tide that lifts all ships, even as the ocean itself got bigger to accommodate still more of us. That is a product of the evolutionary state of the field.

back to top  Automation and User Experience Design Practice, Tomorrow

Automation promises to impact our work in new and remarkable ways. AI enables software to behave in ways more akin to a human brain, albeit an extraordinarily obtuse one at this point. It excels at doing huge quantities of grunt work so humans no longer have to do it themselves. It is also increasingly able to affect actual workflows with solutions that replace our thinking work, not just our hand and production work. For the most part, the more your work requires you to physically make things, the larger the impact AI automation will have. People at the executive and strategic layers should not experience much impact on their day-to-day, because the AI won't be smart enough to solve strategy in the complex, highly customized ways and with the specific context required at a corporate executive level. Someday it will, but we are talking decades, not years.

As time passes, we should expect our software to automatically generate assets for us. Consider icons. Today, the typical way to source icons is to select them from an online library. They may need some manual styling to match their company or client brand, but the work of conceptualizing what should be depicted on the icon, and the general depiction of that choice, merely leverages what others have created before and made available on the Internet. In the future, our software will generate these for us, including matching the exact style—thickness, personality, color, shading—for your particular product. Rather than search and hunt manually among the tsunami of options available on the Internet, your design software will be able to take one icon of a specific and correct design and automagically produce icons for any other number of words you tell it you need.


Sure, there will be concepts you need iconified that are uncommon or that require you to manually determine what to depict to successfully communicate it. But your work as a designer will involve less hunting for examples to borrow, steal, or be inspired by from the Internet and less actual drawing and fabrication. While icons are an easy and concrete example to illustrate this, it will extend to everything that requires actual drawing. So long as there is a template to start from—which in the shorter term will be human-created and in the longer term will be machine-created—the machine can reproduce it in different ways to save us from having to actually make the thing. As with CNC routers in physical fabrication contexts, the work becomes one of editing and tweaking the machine's handiwork and of taking care of the special cases.

This is not just speculation and theory. A prominent example is Wu Chunsong, who runs Alibaba's artificial intelligence design lab. Their intelligent design platform, called Lu Ban, created millions of banner designs in 2017 for Alibaba customers. At the 2018 UCAN Conference, attended by Interactions' own Dan Rosenberg, Wu suggested that his group would soon reduce their 2,400 person UX department by more than half [4]. True, these reductions are primarily in what we might think of as production people on the fringe of UX design jobs. But it is just the beginning.

In the future the machine will do the making. The designer of the future will be more of a creative director. At some point, the software they use to make a website, a mobile app, and some proportion of heavier software will not even enable them to fabricate assets. That will be done by different software and, instead of being operated by designers, will likely be used by specialists whose job is to create style templates for companies or projects. Let's call them stylists. It will be their templates that form the design basis for the software to put together a prototype built from the designer's creative direction.

The reason this work will shift from a designer to a stylist? The amount of fabrication needed for any specific project will be tiny. The stylist will be providing a small but hugely important piece that is then scaled out by automation. To train people to do those tasks, and use the software to support it, will be inefficient. It will also add required skills to the role of a designer that might not suit people who would otherwise be exceptional at creative direction and strategy but not the fabrication of graphics. It will take time for that evolution to fully happen, as those of us already trained to fabricate will continue to do so. But it will further flatten roles and responsibilities. What we think of historically as interaction design will become even more the center of a designer's craft, as visual designers select into either more of a creative director path or a stylist path.

back to top  What Can I Do to Be Ready?

This is just one example of how AI automation will impact the user experience profession in the decade ahead. The trends to be aware of to future-proof yourself are:

  • Tasks that involve making will be increasingly automated. If you spend a meaningful proportion of your time making today, you should think about whether you want to move more toward the inventing and strategic aspects of your job or specialize in the making parts. At some point down the road, your all-purpose approach will be made redundant.
  • Embrace continuous learning. We've already experienced this as the Internet revolution was rapidly followed by a mobile revolution, each of which accelerated and spread progress even while changing the professional rules and requirements to succeed in UX. It will be common that untrained people are enabled by smart-ware to do specialized things, such as writing and UX research. We will eventually all be highly competent researchers and writers, even if we did not receive formal training. You can start your learning by checking out some of the existing consumer software for writing like Grammarly and Hemingway. Get familiar with today's consumer-grade solutions so you can be ahead of where their evolution and future versions will go.
  • Pure thinking work will give your career the longest life expectancy. It will be a while before machines can effectively do strategy, whether it be at the market, company, or product and service level. The people doing this kind of work are generally in upper and executive management; more senior people are typically privileged with this type of work. Traditionally considered higher value, it will increasingly become so as other things are automated. Unless you want to be a specialist, explore ways that you can make your work more strategic and purely problem solving—based sooner rather than later. It will get you ahead of a wave that hasn't crested yet.

Regardless of your training and role, there is nothing to be afraid of here. These things will take time: starting as novelties, then becoming essential tools, before methodically rendering different types of work obsolete. In the process there will be opportunities to go in a different direction, evolving with the changing needs of the workplace. The most important skills are the ability to tolerate ambiguity and the spirit to embrace change and reinvention. If you continue to learn, grow, and invite new directions, these AI-powered advances should make both your life and career better.

Creative Next is a podcast exploring AI automation's impact on creative fields. The show recently completed season 1, about learning. Season 2 starts soon and will explore professions related to communication. Visit to learn more.

back to top  References

1. Nudd, T. Inside 'the next Rembrandt': How JWT got a computer to paint like the old master. Adweek. June 27, 2016;

2. Cohn, G. AI Art at Christie's Sells for $432,500. The New York Times. Oct. 25, 2018;

3. Liberatore, S. Your days could be numbered if you're a sports writer: The Associated Press is using AI to write Minor League Baseball articles. The Daily Mail. June 30, 2016;

4. Mallya, H. 5 technologies that were key to Alibaba clocking $25.3 B GMV in 24 hours. Yourstory. Nov. 14, 2017;

back to top  Authors

Dirk Knemeyer is a social futurist, helping to imagine and implement social systems that better humanity. Previously a designer and entrepreneur, he created successful technology companies in Silicon Valley, Boston, and his native Ohio.

Jonathan Follett is a writer, electronic musician, and emerging tech researcher. He is a principal of Golnvo, a digital design studio in Boston crafting the future of healthcare through strategy, creativity, and vision. He is the author of four books, including Designing for Emerging Technologies (O'Reilly Media, 2014).

back to top 

Copyright held by authors. Publication rights licensed to ACM.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2019 ACM, Inc.

Post Comment

No Comments Found