Mo McElaney- Digital Discrimination: Cognitive Bias in Machine Learning (and LLMs!) | PyData Vermont

Cognitive bias in machine learning reflects human biases, as AI systems are trained on data created by humans and algorithms designed by humans

Notable examples of AI bias include:

• Lensa app generating hypersexualized and racialized images, particularly affecting Asian women
• Healthcare algorithms perpetuating systemic discrimination through biased training data
• Gender Shades Project revealing significant accuracy gaps in facial recognition systems across gender and skin color

Key solutions for combating AI bias:

• Implementing “train then mask” approaches to handle sensitive data
• Requiring computer science students to take ethics courses
• Using open source tools for transparency and accountability
• Following established guidelines and frameworks (EU AI Act, US AI Safety Institute)

Organizations working on AI bias include:

• Linux Foundation AI (LFAI)
• National Institute for Standards and Technology (NIST)
• Various university research programs
• IBM’s Granite project

Important principles for responsible AI development:

• Transparency in model training and data sources
• Accountability for outcomes
• Regular evaluation and monitoring
• Community engagement
• Protection of privacy and individual rights

Regulatory landscape:

• Over 37 countries have proposed AI regulation frameworks
• EU focuses on ethics, transparency, and accountability
• US emphasizes innovation and competitive leadership
• Vermont implementing state-level AI guidelines and ethics

Bias mitigation requires intervention at all stages:

• Data collection and selection
• Algorithm development
• Training and validation
• Deployment
• Ongoing monitoring and adjustment