Biased evaluations aren’t a deviation from expected behavior. They are expected. How did they initially confirm this AI was safe to deploy? It's misleading to label this an incident.
"It can take weeks or even months to fully understand the source of an AI incident. One client had no such plan when it discovered that an AI-based HR screening tool used to assess job applicants was giving preferential treatment to a specific demographic group.”
https://hbr.org/2024/12/how-to-prepare-your-company-for-ai-incidents
Mastodon Source 🐘
"Shortly after it had deployed a generative AI system, one client learned that the system had been trained on huge amounts of problematic data — including copyrighted materials and personal data that had been collected without users’ consent.”
These are organizational learning opportunities surfaced by shipping software. The specific type of software is incidental.