What is AI Bias?
AI bias refers to systematic distortions in AI systems that lead to unfair or discriminatory outcomes. Bias can arise in every phase of the ML lifecycle.
Regulatory Relevance
The EU AI Act explicitly requires the assessment and minimisation of bias for high-risk AI systems. The GDPR also prohibits discriminatory automated decisions.
Types of Bias
1. Data Bias
- Selection Bias: Non-representative samples
- Historical Bias: Past discrimination in the data
- Measurement Bias: Faulty or inconsistent data collection
- Label Bias: Biased annotations due to human prejudices
2. Algorithm Bias
- Aggregation Bias: Different groups are treated equally even though they differ
- Evaluation Bias: Evaluation only on non-representative benchmarks
- Deployment Bias: System is used in a different context than planned
3. Interaction Bias
- Feedback Loop: Biased output reinforces itself through user behaviour
- Confirmation Bias: Users accept AI results that confirm their prejudices
- Automation Bias: Excessive trust in automated decisions
Methods for Bias Detection
Statistical Fairness Metrics
| Metric | Description | Formula |
|---|---|---|
| Demographic Parity | Equal prediction rates across groups | P(Ŷ=1|A=0) = P(Ŷ=1|A=1) |
| Equalized Odds | Equal TP and FP rates across groups | P(Ŷ=1|Y=y,A=0) = P(Ŷ=1|Y=y,A=1) |
| Calibration | Equal calibration across groups | P(Y=1|Ŷ=p,A=a) = p |
| Counterfactual Fairness | Result does not change when protected attributes change | Y(A←a) = Y(A←a') |
Assessment Procedures
- Disaggregated Evaluation: Measure model performance separately by demographic groups
- Dataset Analysis: Review representativeness and distribution of training data
- Red Teaming: Targeted testing for discriminatory outputs
- Counterfactual Testing: Systematically vary inputs and compare outputs
Tools
Open-source tools such as AI Fairness 360 (IBM), Fairlearn (Microsoft) or What-If Tool (Google) can help with systematic bias analysis.
Bias Mitigation
Before Training (Pre-Processing)
- Data balancing through oversampling/undersampling
- Reweighting of data points
- Removal or transformation of sensitive attributes
During Training (In-Processing)
- Fairness constraints in the objective function
- Adversarial debiasing
- Regularisation for fairness
After Training (Post-Processing)
- Threshold adjustment per group
- Calibration of prediction probabilities
- Reject option for uncertain decisions
Bias Assessment Checklist
- Protected attributes identified (gender, age, ethnicity, etc.)
- Training data reviewed for representativeness
- Fairness metrics defined and measured
- Disaggregated evaluation carried out
- Red teaming sessions completed
- Bias mitigation measures implemented
- Production monitoring for bias established
- Results documented
Conclusion
Bias detection and mitigation is a continuous process, not a one-off project. Establish regular review cycles and clear responsibilities within your organisation.
Bias-Erkennung fuer Hochrisiko-Systeme?
Arbeiten Sie mit Creativate AI Studio und spezialisierten Forschern zusammen, um fortgeschrittene Fairness-Audits, Red-Teaming-Prozesse und Bias-Mitigation-Strategien fuer Ihre KI-Systeme zu entwickeln.
Fairness-Monitoring in der Produktion aufbauen?
Von disaggregierter Evaluation bis zu kontinuierlichem Bias-Monitoring — wir helfen Ihnen, Fairness-Pruefungen systematisch in Ihren ML-Lebenszyklus zu integrieren.