Forums

XXVII.6 November - December 2020
Page: 72
Digital Citation

Designing AI systems that make organizational knowledge actionable


Authors:
Denise Wilkins, Siân Lindley, Max Meijer, Richard Banks, Britta Burlin

back to top 

Knowledge management describes processes involved in creating, sharing, using, and maintaining an organization's knowledge. In computer science, knowledge management is traditionally associated with artificial intelligence (AI), as researchers developed algorithms to manage large datasets [1]. Although not a new AI technique, machine learning (ML) offers interesting opportunities for designing knowledge-management systems because it allows people to create systems based on sample data rather than explicit descriptions of computational procedures in code. This makes it possible to design interfaces that depend on difficult-to-code and dynamic user behavior, such as the utilization of knowledge. It also enables end users to contribute to ML models by providing sample data. In this article, we begin by presenting one view on organizational knowledge and how it relates to meaning, context, and action. We then move on to present three tensions that need to be addressed when designing AI systems to make organizational knowledge actionable.

back to top  Insights

ins01.gif

Organizational knowledge is a large research area and longstanding focus for HCI. Communication and collaboration tools such as Workplace and Slack support knowledge by facilitating information flow. Likewise, knowledge-management systems such as Document360 and Confluence provide spaces for creating, organizing, and publishing knowledge articles. In our own research, we spoke to 35 professionals and found that sharing knowledge is integral to collaborative work: People share knowledge to create content with their team. Consequently, much knowledge exists in the enterprise. However, the tools that are designed to facilitate collaborative work struggle to enable seamless knowledge experiences: Users face time and effort costs when capturing knowledge in information systems, externalizing tacit knowledge is cognitively challenging, and it is difficult to apply information about best practices to one's own work. Knowledge management has struggled to transform the workplace: When our participants needed information, they typically asked a coworker for help. Nonetheless, as work becomes more globalized, remote, and diverse, workers need better solutions to reduce the burden of knowledge management and provide equally accessible forms of support.

ML offers the potential to address many of these challenges by creating a new paradigm for knowledge management that automatically constructs knowledge bottom-up by mining the text and unstructured data that people naturally produce during work (e.g., text and clicks in email, chat, and documents) [2]. This technology could enable tools like Slack to automatically detect and surface who knows what about a topic. Similarly, a "team intelligence layer" in Google Cloud Search could eliminate knowledge silos by integrating knowledge across different apps and recommending relevant content wherever users are working [3]. This bottom-up approach is radically different from traditional knowledge-management solutions, which rely on users to actively build and curate knowledge repositories top-down. So, when combined with online information storage, ML systems have the potential to unlock knowledge for the entire organization. There are, however, unanswered questions about how to ensure that knowledge mining is beneficial organization-wide.

First, there are questions about how mined knowledge can have a tangible impact in helping users get work done. For example, ML systems could autodetect patterns in data and make recommendations based on insights, such as the next steps in a process based on what the system infers from many similar processes. Or systems could detect task goals from documents and chats, and proactively recommend knowledge (e.g., graphs, figures, people) that could progress work. Second, researchers and practitioners must consider how these tools impact worker power: ML applications create new questions around privacy, ownership, and control. As a business-to-business data-mining service that workers are obliged to use, these tools risk disempowering workers by treating them as a resource to be mined for knowledge. Nevertheless, prior work can shape our community's response to this issue. In the context of work and beyond, the UX community has designed tools that help empower users by enabling them to work collectively to advance their own interests, for example in the case of platform cooperativism and social media for digitally networked activism [4,5]. Here, our goal is to suggest a way forward to address these challenges. We begin by asking, What is knowledge? Then we explore three tensions introduced by ML for knowledge management, which have implications for designing ethical, sustainable, and socially just systems.

back to top  Knowledge is About Meaning, Action, and Context

Knowledge has a fundamental connection with people that manifests in three main ways. First, there is the knowledge that is internal to a person and those with whom they interact; in other words, the things that a person knows and is familiar with. Second, knowledge represents a person's capacity to act, or their ability to work skillfully. Third, knowledge can be captured in artifacts, such as the text and pictures in documents [6]. Given the personal nature of knowledge, its diversity, and the various places it lives, users spend countless time trying to find and apply relevant knowledge to their work. They also struggle with unidentified gaps in their knowledge ("unknown unknowns"). However, designers can help users make better use of knowledge by designing interactions that allow them to take action on the multifaceted knowledge that exists in the organization.

ins02.gif

Create explicit connections between digital content and the broader context. We can make this more concrete by exploring how systems articulate the relationship between digital data and its meaning and role in the workplace. Knowledge has a powerful semantic layer that often remains unarticulated within information systems. Semantics represent the link between digital data and the broader context—including physical things, people, and organizing principles—with much historical research. For example, the Semantic Web (https://www.w3.org/) attempts to embed semantics into Internet data to allow content to be meaningfully processed by machines. An explicit representation of semantics in a knowledge system could mean, for example, understanding that a document is in fact a budget that is part of a marketing campaign. This would enable a greater level of intelligence that better supports collaboration between humans and machines.

Bring together know-about and know-how. We can also think about how knowledge is bound up with action. On the one hand, there is a body of declarative knowledge, or know-about, which entails descriptions of things. Knowledge also has an important procedural component, know-how, that encompasses working practice and skill [6]. Existing systems tend to separate know-about from know-how. For example, authoring frequently entails recording know-about, which makes declarative knowledge readily available in communication and authoring apps. In contrast, the know-how and skill that went into producing that knowledge remain elsewhere: embodied in the author, in a separate how-to document, or in project-management tools (e.g., Trello, Monday.com, Hive). ML techniques could help designers recouple know-about and know-how to represent knowledge in a much more coherent and multidimensional way.

Bring knowledge to hand. Specifically, ML techniques enable systems to extract data and represent relationships in a contextually sensitive way, which offers several design opportunities. First, knowledge could be captured, surfaced, and used in context, in the applications in which users are active. This would be a marked change from traditional solutions that require users to interrupt focus by switching to dedicated knowledge-management systems. Second, richer knowledge about the users' social, historical, and physical situations could be used to surface relevant knowledge proactively, rather than asking users to search for it. Third, timely and meaningful knowledge experiences, which center around progressing the work that a user is actually completing, could be designed. For instance, when starting a budget for a marketing campaign, a novice might want to know, "How have experts in my company previously gone about creating a budget? What did they do in practice?" At the moment, answers to such questions aren't readily available to the wider organization.

back to top  Important Tensions Around ML For Knowledge Management

Here we have suggested how ML techniques and UX practice can be combined for a new model for knowledge management. Nevertheless, in order to create acceptable, useful, and ethical systems there are unresolved tensions that need to be explored. We outline three tensions here.


ML techniques enable systems to extract data and represent relationships in a contextually sensitive way.


Tension 1: Maintaining privacy and ownership in a context of openness and sharing. ML systems for knowledge management rely on people making their data available for modeling. However, privacy, ownership, and control are important concerns around knowledge. Workers make active and cognizant decisions about the knowledge they do (and don't) want to share, and how they want to share it. Workers similarly take steps to make knowledge ready for sharing. For example, people can be hesitant about sharing in-progress work; similarly, norms indicate that published work shouldn't be edited or expanded. Privacy and ownership become increasingly challenging when we start to deal with knowledge that has been automatically constructed through mining users' data rather than through users' deliberate contributions. First, opaque ML models make it difficult for users to understand, track, and control what data has been collected and how data has been used to construct knowledge. Second, knowledge mining risks eroding worker autonomy and control over knowledge contributions.

Tension 2: Aligning different concerns around ML for knowledge management. Resolving this tension becomes increasingly complex because different organizations and different types of workers are likely to have different concerns. For example, an organization might have a collaborative approach to work, but workers might be in competition with each other, such as in a sales team. Similarly, due to different system access, gig workers might not benefit from knowledge management in the same way as employees. Likewise, frontline workers and knowledge workers face different types of surveillance and have different access to digital tools, so their subjective cost-benefit calculations are likely to be different. Although cooperation is a general challenge for knowledge management, when organizational knowledge is mined universally rather than created through an individual's deliberate choice, there is a danger of creating a one-size-fits all approach that glosses over differences between workers. It also introduces challenges relevant to the broader field of human-centered AI, such as concerns about ubiquitous surveillance and the explainability of ML models, which different stakeholders might feel differently about.

Tension 3: Productivity versus empowerment. ML for knowledge management also introduces a tension between productivity and power—specifically, whether workers are resources to be mined for knowledge. For example, if a contractor works for a company and draws on their own network when performing a task, should that become a learning point for the ML? This has implications for the ownership and governance of both the raw data and the knowledge that has been constructed by ML. Although, in practice, it is unlikely that workers will have full control. For instance, the use of a knowledge-mining service might not be apparent when signing a work contract, or it could become part-and-parcel of giving up intellectual property when joining a company. So, although knowledge mining might support productivity and collaborative work, it may unintentionally function to disempower workers by automatically transferring ownership of a resource that is integral to their value.

A way forward. In order to address these tensions, as a minimum, the UX community needs to provide tools that are interpretable, explainable, and transparent about the data and knowledge that has been mined and how data has been (and could be) used. Similarly, our community needs to design meaningful controls to help users transport their data and knowledge, restrict how others can interact with their knowledge, and stop knowledge mining completely if required. However, there are unanswered UX challenges about how this could be achieved in practice. For example, is it sufficient to list knowledge contributors, or should each step in the knowledge-generation process contain "how did this happen?" information? Both examples raise questions about how to make this large amount of supplemental information digestible for users.

However, we call on our community to do more than this by designing tools that empower workers to come together as a collective to negotiate and self-determine how their data is used. Although this would involve practical UX challenges, one approach could be to draw on insights from platform cooperativism, where the UX community has already helped users have collective ownership and governance of their data and work. For example, MIDATA (https://www.midata.coop/en/home/) allows users to share personal data to medical-research studies of their choosing; in turn, users receive benefits such as self-learning and free health-monitoring tools. Similarly, Up&Go (https://www.upandgo.coop/), by and for gig workers, is cooperatively owned; it enables gig workers to receive a living wage, and for service prices to reflect true costs. In the context of ML for knowledge management, designers could enable workers to identify their own knowledge challenges, determine how knowledge should be shared among workers to benefit work, and direct how mined knowledge should be transferred to the organization.

Although this is just the start of the discussion, we have sought to shed light on some of the opportunities of ML for knowledge management. We advocate utilizing the multifaceted data that exists in the organization to drive meaningful, contextualized action. Simultaneously, we emphasize the need to take a human-centered approach: by designing systems that empower users to have control over their knowledge. In this way, our community can take steps to ensure that ML for knowledge management is worker-driven and beneficial for workers across the whole organization.

back to top  References

1. DeJong, G. Artificial intelligence implications for information retrieval. Proc. of the 6th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, 1983, 10–17.

2. Winn, J., Guiver, J., Webster, S., Zaykov, Y., Kukla, M., and Fabian, D. Alexandria: Unsupervised high-precision knowledge base construction using a probabilistic program. 2018.

3. Lin, J. Future of work #4: Enterprise knowledge management 2.0 + AI-powered search & discovery = team intelligence. Medium. May 21, 2018; https://medium.com/work-bench/future-of-work-4-enterprise-knowledge-management-2-0-61ca08fd2a5d

4. Wilkins, D.J., Chitchyan, R., and Levine, M. Peer-to-peer energy markets: Understanding the values of collective and community trading. Proc. of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020, 1–14.

5. Wilkins, D.J., Livingstone, A.G., and Levine, M. Whose tweets? The rhetorical functions of social media in developing the Black Lives Matter movement. British Journal of Social Psychology 58, 4 (2019), 786–805.

6. Nickols, F. The knowledge in knowledge management. The Knowledge Management Yearbook 2000-2001 740 (2000), 12–21.

back to top  Authors

Denise Wilkins is a social scientist working in the area of human experience and design at Microsoft Research Cambridge. Working within the Future of Work theme, she currently researches enterprise knowledge. Day to day she studies how technology can be used to positively transform work. Her research takes a mixed-methods, human-centered approach. t-dewilk@microsoft.com

Siân Lindley is a social scientist at Microsoft Research. She is currently researching organizational memory, content reuse and remix, and cross-application workflows in the context of machine learning in the workplace. She works in interdisciplinary teams to produce user insights, envisionments, and prototypes. sianl@microsoft.com

Max Meijer is an associate researcher at Microsoft Research in its lab in Cambridge, U.K. Working within the Future of Work theme, he currently focuses on generating compelling and innovative designs for new user experiences that make use of automatically extracted business knowledge. t-mamei@microsoft.com

Richard Banks is principal design manager for Microsoft Research in its lab in Cambridge, U.K. He works in partnership with social and computer science on one of the lab's key themes: the Future of Work. rbanks@microsoft.com

Britta Burlin is a principal design manager at Microsoft Research Cambridge. Her work within the Future of Work theme focuses on enterprise knowledge and AI. Prior to joining Outlook Design in 2011, she held the position of interaction design manager for Whirlpool Europe and Electrolux Srl Europe. brburlin@microsoft.com

back to top 

Copyright held by authors. Publication rights licensed to ACM.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2020 ACM, Inc.

Post Comment


No Comments Found