A colleague once sent me a spreadsheet of survey responses with the name column deleted and the subject line said anonymised. It had a postcode, a date of birth and a job title. Within about four minutes I had worked out which row belonged to my own manager, and I had not tried very hard. Nothing in that file was secret on its own. The combination was.

That is most of what this part is about. Ethics in analysis is not a lecture bolted onto the end of a course. It is a set of concrete checks that sit between the model you built in Part 20 and the number you are about to put on a slide.

A dataset with no names in it

Analysts inherit a comfortable myth: that identity lives in a small number of obvious columns, and that if you strip those out the rest is harmless. Names, email addresses, phone numbers, account IDs. These are direct identifiers, and yes, they need to go. The problem is what is left.

A quasi-identifier is a column that identifies nobody on its own but narrows the field sharply in combination with others. Postcode. Date of birth. Gender. Job title. Employer. The date somebody joined. None of those is sensitive. Together they are a fingerprint.

The classic demonstration came from Latanya Sweeney at MIT in the 1990s. She took a de-identified set of state employee medical records, bought a copy of the public voter roll, and matched the two on date of birth, gender and five digit postcode. That was enough to pull out the medical records of the state governor by name. Her accompanying estimate, that the same three fields uniquely identify 87 percent of the United States population, became one of the most quoted statistics in privacy.

It is also, as usually quoted, out of date, and I think it matters that an analyst notices that rather than repeating the number. Philippe Golle re-ran the calculation on the 2000 census and got 63 percent for the same three fields. Still a majority of the population. Still far too high to call a file anonymous. But 63, not 87, and the person who cites 87 in 2026 has copied it from a blog post rather than a source. Holding the citation habit you learned about correlation claims in Part 12 applies just as much to privacy statistics as to your own findings.

The practical lesson survives either number. Uniqueness is a property of combinations, not of columns, and it rises fast as you add fields. You cannot judge whether a file is safe by looking at the column names one at a time.

How a linkage attack works

The mechanism is simple enough to draw. You have a file with sensitive information and no names. Somewhere else there is a file with names and no sensitive information: a voter roll, a company directory, a public LinkedIn profile, a leaked customer list. Both files contain the same handful of boring columns. Join on those, and the sensitive column now has a name attached to it.

Nobody has to hack anything. It is a join, using exactly the skills from Part 7. That is what makes it an analyst problem rather than a security team problem: you are the person with both the ability to do it and the ability to see when your own output makes it possible.

What the law actually asks of you

Data protection law is large, but the parts that touch a working analyst reduce to a handful of principles. The European GDPR, which set the template most other regimes now follow, lists them in Article 5. Two of them will shape your daily work more than the rest.

Purpose limitation says personal data must be collected for specified, explicit and legitimate purposes, and not processed later in a way incompatible with those purposes. In plain terms: the fact that a column exists in the warehouse does not mean you are allowed to use it for your question. Support ticket transcripts collected to resolve complaints are not automatically available to score employee performance.

Data minimisation says the data must be adequate, relevant and limited to what is necessary for the purpose. This is the one analysts break most often, usually with the best intentions, by writing SELECT star and pulling every column into a notebook because it might be useful later. Necessary is judged against the question you are answering now.

You will also meet two words that are constantly confused. Pseudonymisation replaces or separates the identifying fields but keeps a route back to the individual, usually a key held elsewhere. The data is still personal data and the law still applies. Anonymisation means no realistic route back exists at all, and only then does the data fall outside the rules. The United Kingdom regulator published detailed guidance on the distinction in March 2025, and its position is that the test is contextual: what matters is whether re-identification is reasonably likely given the means available to whoever holds the file.

That contextual test is why the same file can be safe on your laptop behind access controls and unsafe as a public download. Nothing about the data changed. The people who can reach it did.

Illustrative figures from a 12,000 response staff survey. Note the last two rows: coarsening two columns removed nearly all the risk and cost almost none of the analysis.

Where bias enters an analysis

Bias in everyday speech means prejudice. In analysis it means something narrower and more useful: a systematic difference between what your numbers say and what is true, one that does not shrink as you collect more rows. That last clause is the important one. Random error averages out with volume. Bias does not.

The United States National Institute of Standards and Technology sorted the sources into three groups in its 2022 publication on managing bias, and the grouping travels well beyond machine learning. Systemic bias sits in institutions and history, and shows up in your data because the world that produced the data was already uneven. Statistical or computational bias comes from how the data was sampled, measured or processed. Human bias comes from the people choosing the question, the cut and the chart.

The one that catches careful people is the proxy. You want to measure something you cannot observe, so you substitute something you can. Health need becomes healthcare spending. Job performance becomes manager rating. Customer satisfaction becomes survey response. Each substitution is reasonable and each carries an assumption: that the gap between the proxy and the real thing is roughly the same for everybody.

When that assumption fails, the analysis is wrong in a way no amount of technical rigour will catch, because every query ran correctly. The measurement was the error, not the code. This is the same class of mistake as the metric definition problems in Part 17, but with consequences that land on people rather than on a forecast.

flowchart LR
  Q[Question you were asked] --> D[Data that happens to exist]
  D --> S[Who never made it into the data]
  S --> P[Proxy you pick for the thing you cannot measure]
  P --> M[Metric or model]
  M --> A[Decision taken about people]
  A --> D

A proxy that failed at scale

The clearest published example of proxy bias I know sits in a 2019 paper in Science by Obermeyer and colleagues. A widely used algorithm helped United States health systems decide which patients should be enrolled in extra care programmes. It predicted future healthcare costs and treated that prediction as a stand in for how sick a patient was.

Cost is a defensible proxy for need. Sicker people generally cost more. But historically less money was spent on Black patients at the same level of illness, for reasons that have nothing to do with how ill they were. The algorithm read lower spending as lower need, so at any given risk score, Black patients were in fact sicker than White patients with the same score.

The authors calculated that fixing the target variable, so it predicted illness rather than cost, would raise the share of Black patients flagged for additional help from 17.7 percent to 46.5 percent. The model was not badly built. It answered its stated question well. The stated question was the wrong one.

Read that story as a warning about target variables, not about healthcare. Every time you write a query that stands in for something you cannot measure directly, and I do this weekly, you are making the same move on a smaller scale. Write down what the proxy is standing in for, and ask who the gap between them is largest for.

How do you check for bias you did not expect?

You cannot audit for every possible unfairness, and pretending otherwise leads to a checklist nobody completes. What you can do is run three cheap checks on anything that will inform a decision about people. Each takes minutes.

First, count who is missing. Compare the composition of your dataset against the population it claims to describe. If your customer satisfaction data comes only from people who answered an email survey, you have measured the satisfaction of people who answer email surveys. Sampling and non response are where most bias enters, long before anyone writes a model.

Second, break the headline number out by group before you publish it. Not to publish the breakdown, necessarily, but so that you have seen it. If the overall figure is 42 percent and one group sits at 12 percent, you now know something the headline was hiding, and you get to decide deliberately whether it belongs in the story rather than discovering it when someone else finds it.

Third, write down the proxy chain. One line per substitution: I am using X to stand for Y, because Y is not measured. Then read the list and ask, for each line, whether the gap between X and Y is plausibly bigger for some group than another. Most of the time the answer is no and you move on in thirty seconds. When the answer is yes, you have found the thing that would have embarrassed you in six months.

The three categories described in the NIST publication on bias, translated into analyst work. The third row is the one people resist and it is the most common.

Deciding what to hand over

Most of the ethical decisions you will actually face arrive as a small request. Someone wants an extract. A partner team wants the raw table instead of the summary. A vendor wants a sample to demonstrate their tool. These are the moments that matter, and they usually come with a deadline attached and no obvious person to ask.

Two questions resolve nearly all of them. Does this describe identifiable people, and does the requester need row level detail or an answer? The second question is the one nobody asks. A surprising share of requests for raw data are really requests for a number, and the person asking will happily take the number if you offer it. Sending the aggregate instead of the extract is not obstruction. It is doing the job properly.

When you do have to send row level data, coarsen before you send rather than after. Replace full dates with months, full postcodes with regions, exact ages with bands. Then check the smallest resulting group, because a category with two people in it undoes all of it. And put an expiry on the file, because extracts survive far longer than the reason they were created.

flowchart TD
  A[Someone asks you for a dataset] --> B{Does it describe identifiable people}
  B -->|No| P[Share it and move on]
  B -->|Yes| C{Do they need rows or an answer}
  C -->|An answer| G[Send the aggregate and suppress small groups]
  C -->|Rows| D{Is this the purpose it was collected for}
  D -->|No| E[Stop and ask whoever owns the data]
  D -->|Yes| F[Drop direct identifiers and coarsen the quasi identifiers]
  F --> H[Send with access control and an expiry date]
  G --> H

Coarsen two columns and name your proxies

If you do two things from this part, do these. Before any dataset leaves your machine, coarsen the two columns that carry the most identifying power, which in almost every file are the date field and the location field. And before any analysis about people leaves your machine, write one line per proxy naming what it stands in for. Those two habits are cheap, they take under ten minutes together, and between them they cover the large majority of the harm a junior analyst is realistically in a position to cause.

What I would avoid is treating this as a compliance exercise you complete once. The privacy risk in a file changes when a new public dataset appears that shares its columns, and the bias in a metric changes when the population it measures changes. Both are properties of the situation, not of the file, so they need looking at again when the situation moves.

Part 22 closes the series with the portfolio and the job hunt, and this part feeds directly into it: a portfolio project built on a public dataset you coarsened sensibly, with a written note about the proxies you chose, reads as far more professional than one more model with an accuracy score. Keep the Data Analyst guide as your map, and revisit Part 10 if the idea of checking who is missing from a dataset is still abstract.

This week, take the last extract you sent anyone and ask two questions of it: could it be joined to something public, and what is the smallest group in it. If either answer worries you, send a corrected version today rather than waiting to be asked.

References

• Article 5, principles relating to processing of personal data, GDPR
• Pseudonymisation guidance, Information Commissioner’s Office
• Towards a Standard for Identifying and Managing Bias in Artificial Intelligence, NIST SP 1270
• Revisiting the Uniqueness of Simple Demographics in the US Population, Golle 2006

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts