NIST AI Risk Management Framework Playbook

Socio-technical AI risks emerge from the interplay between technical development decisions and how a system is used, who operates it, and the social context into which it is deployed. Addressing these risks is complex and requires a commitment to understanding how contextual factors may interact with AI lifecycle actions. One such contextual factor is how organizational mission and identified system purpose create incentives within AI system design, development, and deployment tasks that may result in positive and negative impacts. By establishing comprehensive and explicit enumeration of AI systems’ context of of business use and expectations, organizations can identify and manage these types of risks.

• Document business value or context of business use
• Reconcile documented concerns about the system’s purpose within the business context of use compared to the organization’s stated values, mission statements, social responsibility commitments, and AI principles.
• Reconsider the design, implementation strategy, or deployment of AI systems with potential impacts that do not reflect institutional values.

Organizations can document the following:

• What goals and objectives does the entity expect to achieve by designing, developing, and/or deploying the AI system?
• To what extent are the system outputs consistent with the entity’s values and principles to foster public trust and equity?
• To what extent are the metrics consistent with system goals, objectives, and constraints, including ethical and compliance considerations?

AI Transparency Resources:

• GAO-21-519SP: AI Accountability Framework for Federal Agencies & Other Entities. URL
• Intel.gov: AI Ethics Framework for Intelligence Community - 2020. URL
• WEF Model AI Governance Framework Assessment 2020. URL

Algorithm Watch. AI Ethics Guidelines Global Inventory. URL

Ethical OS toolkit. URL

Emanuel Moss and Jacob Metcalf. 2020. Ethics Owners: A New Model of Organizational Responsibility in Data-Driven Technology Companies. Data & Society Research Institute. URL

Future of Life Institute. Asilomar AI Principles. URL

Leonard Haas, Sebastian Gießler, and Veronika Thiel. 2020. In the realm of paper tigers – exploring the failings of AI ethics guidelines. (April 28, 2020). URL