Biases and prejudices that exist in society also make their way into the data used to train Generative AI tools and may influence how algorithms are programmed. This causes AI tools to provide results that may be insidiously harmful and discriminatory, contributing to perpetuate long-standing social issues.
For example, Haim, Salinas & Nyarko (2024) found that models consistently gave biased advice when prompts included names commonly associated with racial minorities and women. On a different experiment, Piers (2024) asked ChatGPT to create stories that contained the words "knife", "crime", "police", and either the words "black" or "white". The stories that had the word "black" were consistently more violent and depersonalized, suggesting implicit bias or stereotyping -- a fact that even ChatGPT itself recognized.
AI tools designed to ease routine or repetitive tasks can also have the same pitfalls. For instance, transcription tools used by some hospitals to transcribe medical consultations have been known to invent nonexistent treatments and add racial commentary that was not in the original conversation (Burke & Schellmann, 2024).
The absence of Indigenous perspectives in the design, training, and regulation of AI makes it prone to misappropriation and misrepresentation of Indigenous cultures. AI tools can amplify the issues of theft of cultural intellectual property by generating works that mimic Indigenous art without taking into account the deeply rich stories and contexts in which traditional Indigenous art is created (Worrel, 2024).
On a similar level, the widespread use of Generative AI without any form of oversight can be used to create inaccurate or false content that is then distributed and even sold as Indigenous content. As an example, self-published, AI generated books about Indigenous Languages have been found for sale on Amazon (Becking, 2024); however, these books contained many inaccurate translations, undermining efforts to revitalize and preserve Indigenous languages.
Indigenous sovereignty regarding use of Indigenous data is often ignored in the development and use of AI tools as well. Though AI technology shows promise for important issues affecting Indigenous peoples (language revitalization being some of them), the technology is often developed with data gathered without consultation or participation of Indigenous communities, perpetuating some of the harms associated with colonialism (UBC, 2024).
Data collection from copyrighted works without permission or compensation is one of the ongoing issues related to AI tools (as already highlighted here), but it is not the only one. Several companies collect user data for AI training, and this is not always properly communicated to users. Even when communication happens, users may not have the alternative to choose how their data is used and can't opt-out from having their data collected and sold for AI training purposes.
This can be problematic for a few reasons: collections of large quantities of health and personal data are vulnerable to breaches and misuse; personal data can also be used for the purposes of surveillance and profiling (Gal, 2024), and the fact that tech companies don't make it clear how data is acquired, used, and protected increases the risk for users.
Artificial Intelligence increases the production of electronic waste and amplify the need for rare earth minerals that are often mined unsustainably and cause severe environmental impacts (United Nations Environmental Programme, 2024). Besides, AI data centres also consume a lot of electricity, which puts pressure on existing power grids and contributes to higher carbon emissions (Maslej et al., 2024).
On top of high energy consumption, AI data centres need great quantities of water for both cooling and powering servers. Microsoft reported a 34% increase in its water consumption from 2021 to 2022 (Microsoft, 2022), while Google reported a 20% increase in 2022 (Google, 2023), coincidentally during a time of big investments in AI technology for both companies. The fact that investment in AI is expected to rise in the upcoming years means that water consumption for AI models will likely increase, potentially amplifying water scarcity issues for communities and ecosystems. The problem can become even more pronounced in developing regions, sought after by data centres because of their cheaper energy and real estate costs (Privette, 2024).