Copilot: humans may see your prompts and the photos you send

When you send a sentence or a photo to an artificial intelligence assistant, the interface gives the impression of a closed exchange between a person and a machine. In some cases, however, a third presence can enter the conversation: a human evaluator.

Internal documents and accounts from contractors indicate that hundreds of people recruited to improve Microsoft's consumer Copilot may receive real prompts, images uploaded by users and the outputs generated by the system. Part of their job is to judge whether the AI's response actually matches the request.

Microsoft does not entirely hide the principle. Its documentation on Copilot privacy states that trained specialists may review conversations in order to build, evaluate and improve the accuracy and safety of the models. The affair nonetheless reveals what that abstract wording can mean behind a screen: real photographs, real people and sometimes content the evaluators were not prepared to see.

Quality evaluators, not just moderators

The most important element is the nature of the work being asked. The contractors are reportedly not only tasked with spotting prohibited content or handling a report. They must also determine whether Copilot correctly carried out the instruction it received.

That may seem harmless when it comes to correcting the colour of a sky or moving an object. The task changes in nature when the request concerns the body of a real person, a sexualised transformation, an image obtained without obvious consent or content that may amount to exploitation. The evaluators interviewed say they encountered explicit images, prompts related to eating disorders and potentially illegal material.

According to material reviewed by 404 Media, instructions sometimes asked workers to rely on their intuition to decide whether the output met the user's wishes. Several of them say they were not warned in advance about how often distressing content would appear.

Microsoft responded that customer data is used in accordance with its terms, in particular to improve its products and enforce its code of conduct. The recruitment company Prolific, named in the documents relating to the evaluators, had not provided public comment at the time of the first revelations.

This set-up raises two different security questions. The first concerns the user whose data is being reviewed. The second concerns the worker who sometimes receives, without sufficient context, a violent, sexual or psychologically difficult image. Protecting one does not automatically guarantee protection of the other.

"Anonymised" does not always mean unrecognisable

Microsoft says it takes steps to remove information that could identify a person before using conversations to train its models: names, contact details, identifiers, addresses and certain metadata. Its documentation also states that faces appearing in files intended for training may be blurred.

Yet the accounts of the evaluation work describe photographs in which faces remain visible. This is not necessarily the same processing chain. Data used to train a model, samples selected to evaluate a response and content reviewed after an alert may follow different routes.

That is precisely the problem for the user: the word "anonymisation" sums up several technical operations that do not provide the same protection. Removing an account name does not make a sharp portrait anonymous. Stripping a photo's metadata does not erase a face, a uniform, a home or a readable document inside the image.

A privacy policy can therefore be accurate within its own scope and still be insufficient for understanding a concrete situation. To know what an evaluator actually sees, you need to know the pipeline concerned: selection, automated detection, image transformation, presentation to the human and retention of their decision.

Turning off training does not block all human review

The confusion often stems from three different uses of the same data:

1. training a model in order to change its future behaviour;
2. evaluating the quality of a response that has already been produced;
3. investigating possible abuse or a breach of the rules.

For the consumer version of Copilot, a signed-in user can decide that their future conversations will not be used to train the models. They can also delete conversations from their history. Microsoft says it retains conversation activity for 18 months by default; an uploaded file may be kept for up to 18 months unless the user deletes it earlier.

But the training option is not a universal switch. Microsoft states that some conversations may undergo automated and human analysis to improve the product or digital safety. When a conversation is suspected of violating the code of conduct, limited human review may still be necessary, and no opt-out is offered for that step.

In other words, "do not use my conversations to train the AI" and "never show my content to a person" are two different requests.

Consumer Copilot is not Microsoft 365 Copilot

The Copilot brand covers several products, and their rules are not identical. The revelations concern the consumer service and its chat and image features. They do not support the conclusion that every exchange in Word, Excel, Teams or a work environment follows the same route.

For Microsoft 365 Copilot used with a work or school account, Microsoft states that prompts, responses and data drawn from Microsoft Graph are not used to train the large foundation models. The company also says these services have disabled the Azure OpenAI abuse monitoring mechanism that includes human review of content.

This protection does not exempt an organisation from checking its own settings, its third-party agents, its retention rules and the permissions granted in SharePoint or Teams. It simply means that a personal account in the Copilot app and a managed account in a Microsoft 365 environment should not be treated as one and the same product.

What is better not to send to a consumer assistant

The most useful rule is not to treat every chatbot as public. It is to treat it as a cloud service entrusted to a third party, with logs, security checks and the possibility of human intervention.

Before pressing "send", a few precautions greatly reduce the risk:

• do not upload an identity card, medical records, an unpublished contract or a document containing credentials;
• crop photographs and mask faces, number plates, badges, addresses and unnecessary background details yourself;
• do not send a photo of another person without their consent, especially to alter their body or appearance;
• replace names, case numbers and contact details with fictitious values before asking for a rewrite;
• check the training, personalisation and history settings separately: they do not control the same thing;
• for work purposes, only use the tool and account approved by your organisation.

Deleting a conversation afterwards is still useful, but the best protection comes before the transfer. Data that has not been sent does not need to be filtered, anonymised or entrusted to a contractor's discipline.

The real "human in the loop"

The industry often presents human presence as a safeguard: a person watches the machine, corrects its mistakes and prevents abuse. That is true, but incomplete. That human also becomes a new possible destination for the data.

Serious oversight should therefore protect both sides of the screen. Users must know in which circumstances their content may be viewed, in what form and for how long. Evaluators must receive training, a warning about the nature of the material, tools to decline a task and appropriate support for difficult content.

The existence of human review is not, in itself, a scandal. It can improve the safety of a service. The lack of clarity arises when the interface looks like a private conversation while the actual organisation looks like an industrial evaluation line.