When the algorithm discriminates, the employer is liable

“We didn’t design it, we don’t know how it works, and we certainly didn’t intend that outcome” is a description of the problem, not a defence to it.

How it happens without anyone deciding it

Nobody sets out to build a tool that screens out disabled applicants. It happens anyway, and usually like this.

A model is trained on the CVs of people who were hired and did well. Those people are the output of years of human decisions, with whatever preferences and blind spots those decisions carried. The model learns the pattern, including the parts nobody would defend out loud.

Or it learns a proxy. Not a protected characteristic — a feature that stands in for one. Postcode carries information about ethnicity. A gap in employment history carries information about disability, caring responsibilities and pregnancy. Years since graduation carries information about age. Speech or video analysis penalises an accent, a stammer, or a face that moves differently. Remove the protected characteristic from the data and the proxies remain, which is why “we don’t collect that information” is not the reassurance it sounds like.

Then it scales. A human recruiter with a blind spot affects the applications they personally read. A model with the same blind spot affects every application, consistently, for as long as it runs.

Why the law is harder on this than employers expect

Intention is irrelevant. Indirect discrimination arises where a provision, criterion or practice puts people sharing a protected characteristic at a particular disadvantage and cannot be objectively justified as a proportionate means of achieving a legitimate aim. Nobody has to have meant anything. An automated screening rule is a provision, criterion or practice.

Not understanding the model makes it worse, not better. Justification requires showing that the practice is a proportionate means of achieving a legitimate aim. That is difficult to establish about a process you cannot explain. “We don’t know why it rejects those candidates” is close to conceding that you cannot justify it.

The employer is the respondent. The claim is brought against the employer, not the vendor. Whether you can pass any of it on depends on your contract, and standard AI vendor terms are not generous.

The duty to make reasonable adjustments is a positive obligation. It is not enough to avoid disadvantaging a disabled applicant. If your process puts them at a substantial disadvantage, you must take reasonable steps to remove it — including in an automated sift. A timed online assessment with no adjustment route is the clearest example.

Compensation is uncapped. No ceiling on financial loss, plus a separate award for injury to feelings, currently £1,300 to £62,900 and beyond in exceptional cases. And a tool applied to every applicant does not produce one claim.

Where the exposure concentrates

High-volume recruitment above all — the more applications a model touches, the larger the population affected and the easier the statistical pattern is to demonstrate.

Then: automated sifting on criteria nobody has reviewed; video or speech assessment, where disability effects are pronounced; performance monitoring and productivity scoring, which reaches disabled workers and those with caring responsibilities; and tools used to rank for promotion or redundancy selection, where the consequence is immediate and the affected group is identifiable.

If you are an employer

The position is manageable, and it is much more manageable before a claim than after one.

Know what you are running. Including AI features enabled inside platforms you already licence, which is where most unknown exposure sits.

Test outcomes, not intentions. Compare selection rates across protected groups at each stage. If a group is progressing at a materially lower rate, you need to know now rather than in a tribunal bundle. Take advice on doing this properly — the analysis itself involves processing sensitive data.

Make the human review real. Competent, empowered, before the decision takes effect. If your reviewer cannot in practice overturn the system’s output, you have an automated decision with a signature on it.

Build in an adjustments route that a candidate can use without having to disclose a diagnosis to a machine.

Read the vendor contract. Training data lawfulness, bias testing evidence, audit rights, indemnities. Ask what testing the vendor has done and for the results. A vendor who will not tell you is telling you something.

Record the reasoning. Why you chose the tool, what you tested, what you found, what you did about it. Justification is evidenced, not asserted.

If you are a candidate or an employee

You are entitled to know when a significant decision about you is made by automated means, to meaningful information about the logic involved, to human review, and to contest the outcome.

In practice, if you were rejected by a process you suspect was automated, the useful steps are: ask whether AI was used and request the information you are entitled to; make a data subject access request; keep the correspondence; and note the timing.

These claims are newer than most, but the underlying law is not — it is the Equality Act, and it works the way it always has. You do not have to prove what the model did. You have to show facts from which a tribunal could properly conclude that discrimination occurred, and the burden then shifts to the employer to prove otherwise. Explaining an unexplainable model is their problem, not yours.

Time limits are short. Take advice early.

How we can help

For employers: reviewing tools before deployment, auditing those already running, vendor contract terms, defending claims.

For individuals: assessing whether a rejection or a decision gives rise to a claim, and running it.

We do not act for both sides in the same matter, and we run conflict checks before taking instructions.

The hub

AI law →

Also in this area

AI in recruitment and automated decisions →

Also in this area

Does the EU AI Act apply to your UK business? →

Start with a conversation

A free 20-minute call. Tell us what has happened and we will tell you whether we can help, what it would involve and roughly what it would cost.

No charge

A free 20-minute call

Tell us what has happened and we will tell you whether we can help, what it would involve and roughly what it would cost. No advice is given on this call and there is no charge for it.

£350 plus VAT

A paid strategy session

One hour with a partner, followed by a written summary of your position and options. For people who want proper advice without instructing a firm yet. Credited in full against your fees if you go on to instruct us.

Or reach us directly

0207 112 8818

WhatsApp +44 7476 045967

[email protected]

We answer enquiries the same working day.