My Africa ObserverBUSINESS. CAPITAL. AFRICA.
InvestigationsTHE BIG READNetherlands

EQUAL TREATMENT | PART 3 OF 7 Who Decides You’re Suspicious

Part three of MMO’s Equal Treatment investigation examines how algorithms, intelligence, confirmation bias and institutional feedback loops can turn a weak signal into a conclusion nearly impossible to reverse.

Share
LinkedInWhatsApp
Reading options
Reading tools
100%
Read article
Original cover: My Morocco Observer · Equal Treatment?, Part 3. Photo source ↗
In this article19 sections
Originally published by My Morocco Observer

Part 3 of 7 in the Equal Treatment? series. Original publication: 11 Jul 2026. Read the original article ↗

How Algorithms, Intelligence and Human Bias Can Turn a Weak Signal Into an Institutional Conclusion

Executive Summary

A person is rarely labelled suspicious because one piece of evidence proves wrongdoing. More often, suspicion begins with something smaller: a transaction alert, an anonymous tip, a postcode, a database match, an association with another person or a risk score generated by a system the citizen cannot see.

Each signal may justify further examination. The danger begins when uncertain information travels through institutions, gains authority through repetition and is interpreted by people already expecting to find a problem.

Part Three of My Morocco Observer’s Equal Treatment? investigation examines how algorithms, intelligence, profiling, confirmation bias and institutional feedback loops can transform a weak indication into a conclusion that becomes increasingly difficult to challenge.

Who decides you’re suspicious? Modern institutions rarely arrive at that answer through a single decision. It may begin with an algorithm, an intelligence report, a database entry or a professional interpreting an uncertain signal.

A name enters a database. A transaction generates an alert. An anonymous report reaches an investigator. A postcode raises a score. A telephone number appears next to another number already under examination. At first, none of these proves wrongdoing. They are signals. Reasons to look more closely.

But once a signal enters an institutional system, it begins to travel. A compliance department records a concern. A public authority selects a file for review. An investigator adds the information to a dossier. A database preserves the classification. Another professional later sees that several systems have associated the same person with risk. What may have begun as one uncertain indication now appears to have been confirmed by multiple sources.

The citizen sees only the consequences. More questions. Longer reviews. Additional controls. Delayed payments. A fraud investigation. A police visit. A rejected application. A demand to explain events from years earlier. The institution sees a pattern. The citizen may never know where that pattern began.

This is the central problem of modern suspicion. Institutions must identify risk before harm occurs. Banks cannot wait until money laundering has been proven. Police cannot always wait until a planned offence has been completed. Fraud agencies cannot examine every file manually. Border authorities cannot treat every traveller as presenting the same risk. Selection is unavoidable. But the methods used to select people can create their own reality.

A signal can become a profile. A profile can trigger scrutiny. Scrutiny can produce more recorded anomalies. Those anomalies can then be used to confirm the profile that caused the scrutiny in the first place. The system begins to validate itself. The question is no longer simply who is suspicious. The deeper question is who decided, using which information, and how can that decision ever be corrected?

The First Flag

A single institutional flag marks the starting point where suspicion begins to travel through official systems

Suspicion rarely begins with certainty. It begins with a departure from what an institution considers normal. A payment differs from previous transactions. A customer’s declared income does not appear to match their spending. A student’s address resembles patterns found in earlier fraud cases. A person is linked to a vehicle, telephone, address or individual already known to investigators. A traveller’s route, nationality or booking behaviour triggers additional screening. A company operates in a sector associated statistically with cash, fraud or corruption.

None of these facts necessarily proves misconduct. But institutions do not require proof before beginning an investigation. They require a reason. That distinction is essential. A signal may legitimately justify a question. It should not silently become an answer.

Modern enforcement systems increasingly depend on the ability to identify weak signals across enormous quantities of information. That can improve public administration. It can uncover relationships that a human reviewer would miss. It can direct scarce resources towards cases deserving closer attention. It can also create an illusion of objectivity. A risk score may look precise because it is expressed as a number. A database match may appear authoritative because a computer produced it. An intelligence report may gain credibility because it is marked confidential.

But precision is not the same as truth. A score reflects choices about which data matter, how variables are weighted and what outcome the system was designed to predict. A database is only as reliable as the information entered into it. Intelligence may be incomplete, mistaken, outdated or deliberately misleading. The first flag is therefore not merely a technical event. It is a human decision translated into institutional form.

A Signal Is Not Evidence

Institutions use several kinds of information that are often treated as though they were interchangeable. They are not. Intelligence may identify a possible threat, relationship or direction for investigation. An alert indicates that an event matched predefined rules or statistical patterns. A risk score estimates the likelihood of a specified outcome based on selected variables. An indication supports a hypothesis but may remain open to alternative explanations. Evidence is information capable of supporting or undermining a factual claim. Proof is the legal or institutional threshold at which the available evidence is considered sufficient for a decision.

Problems arise when the boundaries become blurred. An alert is repeated in a report. The report is referenced in another system. A professional reads that multiple records associate the person with risk. The records may appear independent. In reality, they may all originate from the same initial signal. This is sometimes described as information laundering. Uncertain information gains apparent credibility as it passes through official channels.

The wording may also change. A possible link becomes a known association. An unverified report becomes intelligence information. A transaction requiring explanation becomes suspicious financial behaviour. No individual professional may intend to exaggerate the information. The transformation can occur gradually through summaries, classifications and institutional language. By the time the citizen encounters the consequences, the original uncertainty may have disappeared from view.

When Databases Begin Talking to Each Other

Government and financial databases exchanging records can multiply a single unverified signal into apparent confirmation

Governments and financial institutions increasingly exchange or combine information. The logic is understandable. Crime, fraud and money laundering do not remain inside one department. A person may interact with tax authorities, banks, municipalities, social-security agencies, border services and police. A fragmented state can miss important connections. Data sharing promises a more complete picture. But a complete picture built from incomplete or inaccurate parts can become more dangerous than the original fragments.

Suppose an authority records a person as presenting increased fraud risk. That classification may influence later checks. Those checks may generate new records, even when no violation is established. Another authority may then see that the person has been subject to repeated scrutiny and interpret the volume of attention as evidence that the risk was well founded. The number of records grows. The quality of the underlying information does not necessarily improve. The person can become institutionally suspicious because institutions have repeatedly treated them as suspicious.

This creates a duplication problem. Ten records do not equal ten independent findings when all ten originate from the same unverified assumption. Yet the professionals seeing the later records may not know their common origin. They see institutional density. The citizen sees a door that keeps closing.

The Algorithm Does Not Know Why

Algorithms are often introduced because human decision-making is considered inconsistent, slow or expensive. A machine can analyse thousands of files. It can apply the same rules to every case. It does not become tired. It does not dislike a person. It does not consciously prefer one ethnic group over another. These are genuine advantages. But an algorithm does not understand the moral or social meaning of its output. It identifies patterns in data according to the task it has been given.

If historical data reflect unequal enforcement, the system may learn those inequalities. If previous investigators focused heavily on certain neighbourhoods, more recorded violations will exist there. A model trained on those records may classify those neighbourhoods as higher risk. That produces more scrutiny. More scrutiny produces more recorded incidents. The next version of the model receives apparently stronger evidence that the area deserved attention. The bias can amplify over time.

The European Union Agency for Fundamental Rights has warned that algorithmic bias may develop or become stronger when the outputs of earlier systems become the training data for later ones. It has called for assessment before deployment and regular testing afterwards, particularly when algorithmic decisions affect people’s rights. The system does not know that its prediction may partly reflect where institutions previously chose to look. It only knows that the pattern exists in the data.

When Neutral Variables Become Proxies

Postcode, surname and foreign transfers can quietly act as proxies for origin inside a formally neutral system

An institution does not need to record ethnicity explicitly to produce unequal outcomes. Other characteristics can operate as proxies: postcode, nationality, country of birth, language, surname, family composition, occupation, income pattern, business sector, travel history, type of housing, foreign bank accounts and international relatives. Each variable may have a legitimate administrative or investigative purpose. The risk emerges from how they are combined.

A postcode can correlate with ethnicity and income. A surname can correlate with national origin. Foreign transfers can correlate with migration history. A sector associated with cash can disproportionately contain minority-owned businesses. The model may never contain a field labelled race. It can still reconstruct much of the same information indirectly. This is why formally neutral systems can create indirect discrimination.

The Dutch childcare benefits scandal demonstrated the danger when nationality and group-based risk selection become entangled with aggressive fraud enforcement. The Dutch Data Protection Authority later concluded that the student-finance agency DUO used a fraud-control approach that discriminated against students with a migration background. The regulator found that the selection method was insufficiently substantiated and involved both direct and indirect discrimination.

In May 2026, the Dutch State Commission against Discrimination and Racism warned that data-driven profiling by public authorities was expanding faster than legal safeguards and democratic oversight. It also questioned whether the effectiveness of such profiling had been sufficiently demonstrated. The issue is not whether governments may assess risk. They must. The issue is whether risk factors describe relevant behaviour or quietly reproduce assumptions about groups.

The Human Mind Inside the Machine

Algorithms do not replace human judgment. They shape it. A professional receiving a high-risk alert does not begin from a neutral position. The system has already framed the case. The person is not simply a customer, student, traveller or citizen. They are a customer whose transaction generated an alert. A student selected by a fraud model. A traveller flagged for additional screening. A suspect named in intelligence. That framing affects how later information is interpreted.

Psychological research identifies several mechanisms relevant to institutional decision-making. Anchoring occurs when an initial piece of information exerts disproportionate influence over later judgment. Confirmation bias leads people to notice, search for or give greater weight to information supporting an existing belief. Belief perseverance describes the tendency to maintain an initial conclusion even after contradictory information emerges. Expectation bias can shape how ambiguous behaviour is interpreted. Groupthink discourages alternative explanations within teams that have converged on one narrative. Automation bias occurs when people place excessive confidence in computer-generated recommendations or fail to challenge them.

These are not defects limited to dishonest or incompetent professionals. They are ordinary features of human cognition. Judges, investigators, doctors, scientists, auditors and compliance officers are all vulnerable. Dutch judicial research has described how confirmation bias can lead decision-makers to emphasise information supporting an existing view while giving less attention to information that contradicts it. It has also examined belief perseverance, in which an established conclusion survives despite contrary information. The point is not that professionals cannot be trusted. It is that trustworthy systems must be designed around the fact that professionals are human.

When the Hypothesis Starts Selecting the Evidence

Tunnel vision emerges when an investigative hypothesis begins selecting evidence instead of testing explanations

Every investigation requires a hypothesis. A bank may suspect that funds lack a legitimate origin. A fraud agency may suspect that a person’s declared situation is inaccurate. Police may suspect that an individual uses a particular telephone or vehicle. An inspector may suspect that a company is concealing activity. The hypothesis determines what investigators look for. That is necessary. But once the hypothesis becomes dominant, the investigation can begin selecting evidence instead of testing explanations.

Information consistent with the theory appears meaningful. A payment becomes concealment. A contradiction becomes deception. Silence becomes suspicious. An international contact becomes a criminal connection. A refusal to answer becomes evidence of guilt. An answer becomes rehearsed. Evidence that does not fit may be interpreted as insignificant, mistaken or deliberately misleading.

This is tunnel vision. The danger is not merely that investigators ignore one document. The danger is that every fact becomes capable of supporting the same conclusion. A hypothesis that can explain everything is extremely difficult to disprove. If the person cooperates, they may be said to be managing appearances. If they resist, they may be described as obstructive. If records exist, the records may be considered manufactured. If records are missing, the absence may be treated as suspicious. The person enters a system in which innocence no longer has a clear form.

Institutional Momentum

Suspicion also creates organisational momentum. An investigation requires time, staff and authority. Once substantial resources have been committed, changing direction becomes psychologically and institutionally difficult. A team may have briefed senior management. A bank may have escalated a file. A prosecutor may have authorised intrusive measures. A public authority may have imposed a sanction. Reversing course can mean acknowledging that earlier decisions were mistaken. That creates a subtle incentive to interpret new information in ways that preserve the original direction.

No conspiracy is required. Nobody needs to falsify evidence. Professionals may sincerely believe they are following the facts while the structure of the process makes alternative interpretations increasingly difficult to accept. The longer a classification remains in place, the more natural it appears. Time itself begins to function as validation. The person has been under investigation for a year. Surely there must be a reason. The account has been reviewed repeatedly. Surely the concern cannot be baseless. Several departments have examined the file. Surely they cannot all be wrong.

But duration is not evidence. Repetition is not independence. Institutional agreement is not proof when institutions rely on the same underlying information.

The Feedback Loop

The most dangerous systems do not merely identify risk. They create the conditions that later appear to confirm it. A person is flagged because the system considers them risky. Because they are flagged, they receive more scrutiny. More scrutiny produces more questions, inconsistencies and recorded events. Those events are added to the file. The expanded file is used to confirm the original classification. The person then receives even more scrutiny.

The cycle can occur in many contexts. A neighbourhood receives more police attention, producing more recorded offences. A customer receives more transaction reviews, producing more documented anomalies. A benefit recipient faces more checks, producing more opportunities for administrative errors. A traveller is questioned repeatedly, creating a history of border interventions. The system later interprets the history of intervention as evidence that the person has long been associated with risk. But that history may partly be a history of the system’s own attention. This is the institutional version of a self-fulfilling prophecy.

The Toeslagenaffaire Was Not Only About an Algorithm

The Dutch childcare benefits scandal shows how political pressure and institutional culture, not software alone, drove injustice

Public discussion often presents institutional scandals as technological failures. A bad algorithm. An incorrect model. A flawed database. That explanation is incomplete. The childcare benefits scandal was not caused by software alone. It involved political pressure, harsh legislation, fraud narratives, administrative culture, limited proportionality, inadequate information sharing, weak legal protection and institutions that repeatedly failed to reconsider their assumptions.

Technology made the system faster and broader. Human and institutional choices gave it direction. The parliamentary inquiry found that fundamental principles of the rule of law had been violated and that the executive, legislature and administrative judiciary had all failed affected parents. The lesson is therefore larger than the claim that algorithms can be biased. It is that technology can strengthen institutional certainty at precisely the moment when doubt is most necessary. A system may process thousands of cases consistently. If the underlying assumption is wrong, consistency scales the injustice.

When Secrecy Protects the System

Institutions cannot reveal everything. Police must protect methods, informants and ongoing investigations. Banks cannot publish every transaction-monitoring threshold. Fraud agencies cannot disclose rules in a way that permits deliberate evasion. Intelligence services depend on confidentiality. These are legitimate concerns. But secrecy creates an accountability problem.

A citizen may be unable to see what data were used, where those data came from, whether the information was current, which variables influenced the score, whether an algorithm or a human made the decisive choice, whether multiple records originated from one source, how to challenge the classification, or when the classification will be removed. The institution can defend opacity by invoking security. The citizen experiences consequences without meaningful access to the reasoning.

This creates a procedural imbalance. The person may need evidence to challenge the decision. The evidence is controlled by the institution whose decision is being challenged. In its 2020 SyRI judgment, the District Court of The Hague emphasised the importance of transparency and the protection of private life when the state combines large quantities of personal data to identify possible fraud risks. Transparency does not require the publication of a manual for evasion. It requires enough explanation for independent review and effective legal protection.

The Citizen’s Impossible Position

A person classified as risky may not know that the classification exists. They may notice only its effects. Repeated bank questions. Unexpected inspections. Delayed decisions. Additional identification checks. Applications receiving more scrutiny. A growing file whose contents remain inaccessible.

Even when the person suspects that inaccurate information exists, correction can be difficult. Which institution created the record? Which institution currently relies on it? Is the classification a formal decision that can be appealed? Is it internal intelligence? Is it an automated recommendation? Was the person selected because of one variable or several? Has the information spread elsewhere?

A citizen may successfully correct one database while the same information remains in another. An institution may remove a formal label while the earlier history continues to influence professional judgment. The right to correction means little when the person cannot identify what requires correction. The right to appeal means little when the system claims no formal decision has been made. The right to explanation means little when the explanation consists only of a risk-based assessment.

What Institutions Are Entitled to Say

Fair journalism must acknowledge the strongest institutional case. Governments, banks and enforcement agencies face real threats. Fraud harms public services and public trust. Money laundering supports organised crime. Security agencies must identify danger before it becomes an attack. Police must sometimes act on incomplete information. Public bodies cannot manually review every citizen with equal intensity. Algorithms can reduce arbitrary individual decision-making. Data analysis can identify patterns humans would overlook. Intelligence cannot always be disclosed without compromising sources or methods. Criminal organisations deliberately exploit transparency and adapt to controls. Not every person selected for review has been accused of wrongdoing. Not every difference in treatment is discrimination.

All of these arguments are legitimate. But they do not resolve the accountability question. The need to select does not justify every selection method. The existence of risk does not prove the reliability of every signal. Confidentiality does not eliminate the need for independent review. Efficiency does not excuse disproportionate harm. A system is not fair merely because every employee follows the same procedure. If the procedure is built on distorted data or untested assumptions, consistent application can produce consistent injustice.

What Institutions Must Be Able to Demonstrate

An accountable institution should be able to answer fundamental questions. What triggered the initial flag? Was the information verified? How old was it? Did multiple records originate from the same source? Which variables influenced the classification? Were any variables proxies for protected characteristics? How often does the system produce false positives? Which groups bear those false positives? Can employees override automated recommendations? Are they encouraged to do so? Is exculpatory information actively sought? What would cause the institution to abandon its original hypothesis? How can a citizen correct inaccurate information? How long is risk information retained? When does a person return to an ordinary category? Who audits the system? Are the audit results public?

These questions do not prevent institutions from protecting society. They determine whether that protection remains compatible with the rule of law.

Who Decides You’re Suspicious?

The core question of accountability: who has authority to review, explain and reverse a classification of suspicion

The central danger is not that institutions sometimes become suspicious. Suspicion is necessary. The danger is that suspicion becomes difficult to reverse. A weak signal enters a system. The system gives it structure. Professionals give it interpretation. Databases give it permanence. Repeated scrutiny gives it apparent confirmation. Secrecy protects it from challenge. Time gives it authority.

Eventually, the original question disappears. The person is no longer being investigated because they may present a risk. They present a risk because they have been investigated. That reversal should concern every democratic institution. A person should never be required to prove innocence against a classification whose origin, logic and evidence remain hidden.

Algorithms can support judgment. They cannot replace accountability. Intelligence can guide investigation. It cannot silently become proof. Professional experience can inform decisions. It cannot be immune from cognitive bias. Institutions can act before certainty exists. But they must remain capable of admitting when certainty never arrived.

Ultimately, the question who decides you’re suspicious cannot be separated from a second question: who has the authority to review, explain and reverse that decision? Because the strength of an institution is not measured only by its ability to detect danger. It is measured by its ability to recognise when it was wrong.

This is Part Three of MMO’s Equal Treatment? investigation.

Sources & Methodology

This investigation is based on official regulatory publications, parliamentary reports, judicial decisions and European fundamental-rights research concerning data-driven profiling, algorithmic discrimination, risk selection, cognitive bias and institutional accountability.

The principal sources include the Dutch State Commission against Discrimination and Racism’s reporting on data-driven profiling, which concluded that public-sector profiling can conflict with the rule of law, democratic legitimacy and effective governance, and identified weak legal boundaries, limited transparency, inadequate legal protection and the risk that historical discrimination may be reproduced through data; the Dutch Data Protection Authority’s findings concerning DUO, which concluded that the agency’s fraud-control approach discriminated against students with a migration background and was unlawful; the Dutch Data Protection Authority’s Algorithmic Risks Report Netherlands, which examines the risks created by algorithmic systems, including opacity, discrimination and inadequate governance; the European Union Agency for Fundamental Rights report Bias in Algorithms, which found that algorithmic bias can develop and amplify over time, especially when previous system outputs become the basis for later models, and recommends assessment before deployment and regular testing afterwards; the District Court of The Hague SyRI judgment, which assessed large-scale public-sector data analysis against privacy and human-rights safeguards and emphasised transparency and effective legal protection; Dutch judicial research on decision-making, which discusses confirmation bias and belief perseverance; and parliamentary investigations into the childcare benefits scandal, which examined how fraud policy, political pressure, strict enforcement, weak proportionality and institutional failure harmed affected families.

An algorithmic flag, intelligence report or risk score does not automatically establish wrongdoing or unlawful discrimination. MMO distinguishes between signals, risk assessments, verified evidence, judicial findings and its own institutional analysis. My Morocco Observer received no funding or compensation from any government authority, financial institution, technology provider, political party, campaign group or other party mentioned in this investigation. All principal sources are publicly available. Any institution facing substantive criticism will be offered a full right of reply before publication.

Call for Evidence

My Morocco Observer is seeking documented experiences from people who believe inaccurate data, algorithmic risk scores, intelligence records or institutional tunnel vision affected how they were treated. MMO also invites current and former public officials, investigators, compliance professionals, data scientists, lawyers, judges, academics, auditors and whistle-blowers to contribute evidence or expertise.

MMO is particularly interested in material concerning unexplained risk classifications; inaccurate or outdated database records; algorithmic fraud selection; repeated scrutiny originating from one initial signal; decisions based on anonymous or unverified information; cases in which exculpatory evidence was overlooked; automated recommendations that employees could not challenge; successful correction or appeal procedures; internal audits revealing false positives or unequal impact; and safeguards that prevented tunnel vision or discriminatory profiling.

Submission does not guarantee publication and does not imply that MMO accepts an allegation as fact. Claims will be independently assessed. Supporting documents may be requested. Confidentiality can be discussed with sources. Any individual or institution facing substantive allegations will be offered a full opportunity to respond before publication.

About the Investigation

Equal Treatment? is an ongoing My Morocco Observer investigative series examining institutional decision-making, proportionality and public trust in Europe. MMO bases its reporting on official documents, public data, legal decisions, expert analysis and verified personal accounts. The purpose is not to presume guilt or innocence. It is to establish whether the promise of equal treatment survives contact with institutional practice.

Sources & further reading 1

Continue the investigation

View the series
My Africa ObserverBack to top

Explore this story

Investigations
The wider perspective

Continue reading

Explore investigations
Free newsletter · The Africa Brief

The five business developments that matter across Africa.

A concise briefing on companies, capital, markets and technology across the continent.

Launching soon · A weekday edition is planned.

Explore The Africa Brief