Bias-In, Bias-Out

Many AI models are trained on data where social biases are present. These biases are then encoded into the patterns, relationships, rules, and decision-making processes of the AI and have a direct impact on the output.

Biased data can be easy to spot, such as in this AI generated image which shows a predominantly white class of 2023 at Western, but it can also be more invisible. AI-generated text will reflect dominant ideologies, discourses, language, values, and knowledge structures of the datasets they were trained on. For example, [pb_glossary id=”629″]Large Language Models[/pb_glossary] may be more likely to reproduce certain dominant forms of English, underrepresenting regional, cultural, racial, or class differences (D’Agostino, 2023 ).

The ethical issue is twofold: first, the information generated by [pb_glossary id=”625″]generative AI[/pb_glossary] is more likely to reflect dominant social identities, meaning that students who use AI will not be exposed to certain worldviews or perspectives, and some students may not feel that their experiences and identities are reflected in the output. Second, the use of [pb_glossary id=”625″]generative AI[/pb_glossary] to produce knowledge will continue to reinforce the dominance of these ideologies, values, and knowledge structures, contributing to further inequities in representation.

As an instructor, it’s important to be aware of this limitation of AI tools. If you ask your students to use these tools, it’s also important to teach them critical AI literacies to similarly be able to identify and reflect on these issues of representation, bias and equity.

Some [pb_glossary id=”625″]generative AI[/pb_glossary] companies have taken steps to correct for biases in the training data by establishing content policies or other guardrails to prevent generating biased or discriminatory output. However, these guardrails are inconsistent and can be subject to the ethical standards of each [pb_glossary id=”625″]generative AI[/pb_glossary] company.

 

Bias Case Study

Feedback

[pb_glossary id=”629″]Large Language Models[/pb_glossary] are trained on datasets biased towards dominant forms of English. The probability models that they use are designed to replicate these dominant patterns of language, which may result in any deviation from these “norms” being treated as errors or problems.