Technical documentation and record-keeping
High-risk AI providers must create and maintain detailed technical documentation that demonstrates compliance with the Act's requirements before the system is deployed. Additionally, high-risk systems must be designed to automatically generate logs (record-keeping) throughout their lifetime to ensure traceability of the system's functioning.
What it means
The intent is to provide national competent authorities and notified bodies with a "technical file" that allows them to assess whether a high-risk AI system is safe, transparent, and compliant without needing access to proprietary source code unless necessary. Documentation must be comprehensive enough for an external expert to understand how the system was built, how it behaves, and how risks are managed.
Record-keeping focuses on the operational phase. It requires that the AI system automatically captures events (logs) as they happen. This ensures that if a failure or incident occurs, there is an immutable trail of evidence to determine why the system made a specific decision or where the process failed.
How to meet it
- Develop a comprehensive Technical File containing the general description of the system, its intended purpose, and the hardware/software requirements.
- Document the design specifications, including architecture, algorithmic choices, and data flow diagrams.
- Detail the data governance processes used for training, validation, and testing datasets (e.g., provenance, preparation, and bias mitigation).
- Integrate a risk management system into the documentation that maps identified risks to specific mitigation measures implemented in the software.
- Configure the AI system to automatically log critical events, such as input data triggers, output decisions, and system errors, with precise timestamps.
- Establish a version control process to ensure technical documentation is updated whenever the system undergoes a substantial modification.
Evidence an auditor asks for
- The complete Technical Documentation folder (the "Technical File") structured according to the Act's requirements.
- Data sheets and reports detailing the characteristics of training, validation, and testing datasets.
- A Risk Management Plan and corresponding risk register showing how risks were identified and reduced.
- Sample log files or a live demonstration of the automatic logging functionality in the production environment.
- An audit trail or change log demonstrating that documentation has been updated alongside system version updates.
Common pitfalls
- Treating technical documentation as a one-time "launch" activity rather than a living document that evolves with the model.
- Providing high-level marketing descriptions of the AI's capabilities instead of specific, quantitative technical specifications and performance metrics.
- Relying on manual logs or retrospective reports instead of implementing the required *automatic* logging capability within the system architecture.