A job applicant once sent a US online tutoring company two applications that were identical except for one detail: her date of birth. The first, showing she was over 55, was rejected automatically. The second, with a younger birth date, got an interview. The company's screening software had been set up to reject women aged 55 or older and men aged 60 or older, and in 2023 it paid US$365,000 to settle the case with the US Equal Employment Opportunity Commission.
Nobody at that company sat down and decided to reject a specific grandmother. The software did it, hundreds of times, without anyone looking. That is the uncomfortable thing about AI bias: it rarely looks like prejudice. It looks like efficiency. This guide explains what AI bias is, where it hides inside ordinary small-business tools, and how a Hong Kong owner can check for it without a data science degree.
What is AI bias?
AI bias is when an AI system treats people or situations unfairly in a consistent, repeatable way, usually because of the data it learned from or the rules it was given. The result is a pattern: certain names, ages, genders, accents or postcodes get worse answers, lower scores or fewer offers, even though nobody intended it.
Think of a new waiter trained for six months by one manager who always gave the best tables to regular customers in suits. Nobody told the waiter to judge people by clothes. He simply copied what he saw. AI learns the same way, except it serves thousands of customers a day and never questions its training.
The key word is consistent. A single wrong answer is an error. When the same kind of person keeps getting the same kind of wrong answer, that is bias.
How does an AI system become biased?
AI becomes biased in four main ways: it learns from unbalanced history, it latches onto hidden clues that stand in for things like age or gender, someone writes an unfair rule into it, or its own decisions feed back into future training. None of these requires bad intent, which is why bias survives in well-meaning businesses.
1. Unbalanced history
Reuters reported in 2018 that Amazon scrapped an experimental hiring tool after finding it downgraded résumés containing the word "women's", as in "women's chess club captain". The tool had learned from ten years of past résumés, most of which came from men. It concluded that male-looking résumés were what success looked like.
2. Hidden clues, also called proxies
Removing the "gender" or "age" field does not remove the information. Graduation year hints at age. A school name, a district or a gap in employment can hint at gender, ethnicity or family status. AI is very good at finding these patterns, which is exactly the problem.
3. Rules written by people
The tutoring case above was not even sophisticated AI. Someone typed an age cut-off into automated screening software. Automation simply applied the unfair rule faster and more quietly than a person would.
4. Feedback loops
If a system shows a promotion mostly to one group, that group buys more, and the system concludes it should show the promotion to that group even more. Over months, a small tilt becomes a large one.
Where can AI bias show up in a small business?
For a Hong Kong SME, AI bias most often appears in five places: CV screening, customer service chatbots, marketing images and ad targeting, pricing or credit decisions, and AI summaries of staff or customer feedback. Each one makes decisions about people at speed, which is where unfair patterns multiply before anyone notices.
--- Hiring: an AI shortlisting tool ranks candidates lower because of a long career gap, an older graduation year or a non-local university.
--- Customer service: a chatbot understands standard English and Cantonese well but gives poorer answers to customers who write in mixed languages, simplified characters or with spelling mistakes.
--- Marketing: an image generator asked for "a successful business owner" keeps producing the same age and gender, and your brochure quietly signals who your brand is for.
--- Pricing and credit: a tool that adjusts discounts or payment terms by postcode can end up treating whole neighbourhoods differently.
--- Staff evaluation: an AI that summarises performance from chat logs may rate confident, fluent writers above quieter staff who deliver more.
A useful rule of thumb: if the AI output affects whether a person gets a job, a price, a service or a second chance, it deserves a bias check.
What do real AI bias cases teach business owners?
Real cases show three lessons. The business using the tool stays responsible, the software vendor may also be held responsible, and most bias is found by simple comparison tests rather than by technical audits. A US$365,000 settlement and a lawsuit that survived dismissal in June 2026 both started with someone noticing a pattern.
The employer is still the employer. When the EEOC settled the tutoring case in 2023, its chair stated that an employer remains responsible even when technology automates the discrimination. Buying software does not transfer your legal duties to the vendor.
The vendor can be in the frame too. In Mobley v. Workday, a job seeker alleges that AI screening tools used by many employers rejected him and others because of race, age and disability. In June 2026, US District Judge Rita Lin refused to dismiss the core claims, according to GPA's summary of HR Executive reporting. Earlier rulings accepted that a vendor could plausibly act as the employer's "agent" when customers delegate rejections to it.
Large language models carry bias as well. In 2024, Bloomberg tested GPT-3.5 by asking it to rank otherwise identical résumés with different names. The rankings shifted depending on the race and gender the names suggested. The same chat assistant your staff use to "quickly shortlist" candidates can tilt the result.
Is your business responsible if the AI tool is biased?
Yes. In Hong Kong, the four anti-discrimination ordinances apply to the outcome, not the method. If an AI tool you use treats people less favourably because of sex, pregnancy, disability, family status or race, your business is exposed in the same way as if a staff member had made that decision.
The four laws are the Sex Discrimination Ordinance, the Disability Discrimination Ordinance, the Family Status Discrimination Ordinance and the Race Discrimination Ordinance. They cover employment and, in many cases, the provision of goods and services. Complaints go to the Equal Opportunities Commission.
Hong Kong does not currently have a law against age discrimination in employment, only a voluntary code of practice. That does not make age filters safe: they damage your reputation, shrink your talent pool and can still collide with disability or family status rules.
On data, the Privacy Commissioner's Model Personal Data Protection Framework for AI, published in June 2024, asks organisations that buy AI to use a risk-based approach, with fairness among its principles and human oversight for higher-risk uses. It is guidance rather than law, but it is the yardstick a regulator would reach for.
How can you check your AI tools for bias?
You can check most SME tools in one afternoon with five steps: run a swap test, ask the vendor three questions, remove obvious hidden clues, have a person review every rejection, and sample results once a month. None of these needs technical skills, only a spreadsheet and some discipline.
Step 1: Run a swap test. Take two identical inputs and change one detail, such as the name, age, gender or district. Feed both into the tool. If the score, answer or price changes, you have found something worth investigating. This is the same test that exposed the tutoring company.
Step 2: Ask the vendor three questions. What data was the model trained or tuned on? Has the tool been tested for unequal outcomes, and can you see the results? Who is responsible if a customer or candidate complains? A vendor who cannot answer clearly is telling you something.
Step 3: Remove hidden clues. Before AI screens a CV, strip out photos, dates of birth, graduation years and marital status. In customer tools, avoid using postcode or name as a factor in prices or offers.
Step 4: Keep a person on every "no". Let AI sort, summarise and suggest. Let a human make the final call on rejections, refusals and price exceptions. This one rule removes most of the legal risk.
Step 5: Sample monthly. Pull 20 recent AI decisions and look at who got which result. If one group keeps landing at the bottom, pause and dig.
These checks sit naturally alongside the other AI guardrails every business should set, such as data rules and approval steps.
What do people get wrong about AI bias?
The three most common mistakes are believing machines are neutral, believing that deleting sensitive fields fixes the problem, and believing bias only matters to big companies. Each belief leads owners to trust AI decisions they should be checking, especially in hiring and customer pricing.
"Computers are objective." AI reflects the data and rules behind it. It is consistent, which is not the same as fair. A consistently unfair system is worse than an occasionally unfair person, because it never gets tired of being wrong.
"We removed age and gender, so we are safe." As the Amazon case showed, proxies such as words, schools and dates carry the same signal. Removing fields helps, but testing is still needed.
"We are too small to worry." A ten-person company that uses AI to screen 300 applicants for one role makes 300 decisions. Small businesses also have smaller reputations to lose: one viral post about an unfair chatbot answer can hurt more than any fine.
"Bias and hallucination are the same thing." They are related but different. A hallucination is a made-up fact. Bias is a tilted pattern, and it can appear even when every individual fact is correct.
Frequently asked questions about AI bias
These are the questions Hong Kong business owners most often ask about AI bias, from whether ChatGPT is biased to whether a free tool carries the same risks as paid software. The short answers below are designed to be used as a quick reference before you buy or roll out any AI tool that judges people.
Is ChatGPT biased?
All large language models can show bias because they learn from human writing. Vendors reduce it with training and safety rules, but tests such as Bloomberg's 2024 résumé experiment show it is not eliminated. Treat its judgements about people as suggestions, never verdicts.
Can AI bias ever be fully removed?
No. It can be reduced and monitored. The realistic goal is to catch unfair patterns early, correct them, and keep a person accountable for decisions that matter.
Are free AI tools riskier than paid ones?
Not necessarily, but paid business tools are more likely to offer audit logs, admin controls and a contract that says who is responsible. With free tools, you usually carry all the risk.
Do I need a bias audit before using AI for hiring?
Hong Kong has no specific requirement, unlike New York City, which has required bias audits for automated hiring tools since 2023. A simple swap test and human review of every rejection is a sensible minimum.
The bottom line on AI bias
AI bias is not a science-fiction problem or a big-tech problem. It is what happens when a fast, tireless system copies yesterday's patterns into today's decisions. The fix is not to avoid AI but to supervise it: test with swaps, ask vendors hard questions, strip out hidden clues and keep a human on every decision that affects a person's job, price or service.
Owners who do this get the speed of AI without inheriting its blind spots. AI should make your business fairer and faster, not just faster.
We understand AI. UD stands with you.
Reviewed by the UD AI team.
Not sure whether the AI tools your team already uses are making fair decisions about candidates and customers? Take the free AI Ready Check and we will walk you through it step by step, from listing the tools that judge people to setting the swap tests and human reviews that keep them fair.
