The risk management system
Article 9 requires providers of high-risk AI systems to establish and maintain a continuous, iterative risk management system throughout the entire lifecycle of the product. This system must identify and analyze potential risks to health, safety, and fundamental rights, implementing mitigation measures to reduce those risks to an acceptable level.
What it means
The intent is to enforce a "safety-by-design" approach. Rather than treating risk assessment as a one-time checkbox before release, the EU AI Act mandates a living process that begins during the design phase and continues through deployment and post-market monitoring.
In practice, this means you must systematically account for not only how the system is intended to work but also how it might be reasonably misused or how it might interact with other systems in a real-world environment. The focus is specifically on preventing or minimizing harms to individuals' physical safety and their fundamental rights (e.g., privacy, non-discrimination).
How to meet it
- Establish a formal risk management framework that defines the methodology for identifying, analyzing, and mitigating risks across the AI lifecycle.
- Conduct comprehensive risk identification workshops to map out potential hazards stemming from intended use and foreseeable misuse.
- Perform an analysis of each identified risk to estimate its probability of occurrence and the severity of the resulting harm.
- Implement technical mitigations (such as hard constraints or safeguards) and non-technical mitigations (such as detailed user instructions and training).
- Define a clear threshold for "acceptable residual risk" based on relevant harmonized standards or industry benchmarks.
- Integrate a feedback loop where data from post-market monitoring is used to update the risk assessment and trigger new mitigation measures.
Evidence an auditor asks for
- A Risk Management Plan detailing the governance, roles, and methodology used to manage AI risks.
- A Risk Register or Matrix documenting identified risks, their initial severity/probability, the mitigations applied, and the final residual risk level.
- Technical documentation (e.g., architecture diagrams or code specifications) proving that specific mitigation measures were actually built into the system.
- User manuals and "Instructions for Use" that explicitly communicate known risks and required safety behaviors to the end-user.
- Version-controlled logs showing when the risk assessment was reviewed and updated in response to system changes or field performance data.
Common pitfalls
- Treating the risk assessment as a static document completed once during development rather than an iterative process.
- Focusing exclusively on technical failures (e.g., system crashes) while neglecting socio-technical risks like algorithmic bias or infringement of fundamental rights.
- Failing to provide documented justification for why certain residual risks were deemed "acceptable" despite not being fully eliminated.