Week 12 Ā· Unit 9.2: My childrenās digital world Ā· 3 lessons
2
When AI goes wrong
Digital Literacy 9: Unit 9.2, objectives 9DW.06, 9TC.04
GoalExplain the main risks of AI and weigh them against benefits for different people.
New words
bias Ā· an unfair leaning towards some people or answers, for example copied from unbalanced training datamade-up answer Ā· a confident reply that is false or inventedfake media Ā· images, sound or video made or changed to look realprivacy Ā· a personās right to keep their own information to themselves
Explanation
AI can go wrong in several ways. If its training data is unbalanced, it copies that bias: a speech tool trained mostly on one accent may misunderstand others. A chatbot can give a made-up answer that sounds confident, such as a book that does not exist. Systems that learn from photos, voices and messages can threaten privacy if the data is collected or kept carelessly. Some jobs will change as tasks are automated, so people may need new skills and support. AI can also make fake media, such as a voice that sounds like a real person, so seeing or hearing is no longer proof. None of this means AI should be banned. It means we weigh benefits and risks, ask who is affected, and check what matters.
Risk
Example
What helps
bias
one accent misunderstood
varied training data, testing with many users
made-up answer
an invented source
check with trusted sources
fake media
a cloned voice
check where it came from; ask the person
Worked examples
Pupil group
Words understood out of 100
accent like training recordings
94
other accents
71
Junās school tests a speech tool. It understands the pupils who speak like the recordings it learned from, and it fails more often for a pupil with a different accent.
Priya
Give me three books about bees.
Chatbot
Of course! The third is āThe Bee Archiveā by Dr L. Harmon.
A chatbot is asked for books on bees and lists one title with a real-sounding author. The book does not exist.
Odd voice message asks for moneyāStop, do not replyāPhone the person on a known numberāTell a trusted adult
A voice message seems to come from a pupilās uncle, asking for a gift card to be bought. The family phones him on his usual number and finds that he sent nothing.
Class activity
Weigh it up (groups of four, on paper). Invented case: the town of Brookvale clinic wants an AI program to decide the order in which patients are seen. Draw a table with three columns: Benefit, Risk, Who is affected. Fill in at least three rows, covering speed, bias (what if the training data came from one age group?), privacy and who checks. Then decide: should the clinic use it, use it with conditions or not use it? Write one sentence of conditions, for example āa nurse can change the orderā. Compare with another group.
Practice
1
A hiring tool for a bakery learned from the last ten years of hires, and nearly all were young men. What risk is this and what could the bakery do?
Past hires
Share
young men
9 in 10
everyone else
1 in 10
Bias: the tool may prefer people who look like past hires. The bakery should check the tool with varied examples, let people review the results and not rely on it alone.
2
A pupil copies a chatbotās āfactā into a project. What should they have done?
Checked it with at least two trusted sources and named them. A chatbot can be wrong while sounding sure.
3
Why is āit sounds just like my teacherā no longer enough proof that a voice message is real?
It is me. Send me the code from your phone now.
AI can copy voices. Check with a known contact method or ask the person directly, and tell an adult if the message asks for money or secrets.
4
Give one benefit and one risk of AI for a worker whose job changes. Who could help?
Benefit: dull tasks are done faster. Risk: the workerās skills may no longer be needed. Employers, schools and training centres can help with new skills.