Dr Kevin Shepherdson, CEO and Founder of Straits Interactive
I recently attended the IAPP Asia Forum 2026 in Singapore, where Professor Simon Chesterman moderated a keynote discussion involving data protection regulators from Singapore, Japan and the United Kingdom.
During the session, Professor Chesterman turned to the audience and raised a deceptively simple question: how many of us actually read a privacy notice before clicking the “I agree” button?
Very few hands appeared to go up.
His question exposed an uncomfortable truth. Even in the traditional digital environment, most people do not carefully read privacy notices. We scroll past pages of legal language, select the checkbox and continue using the service.
In legal terms, the organisation may have obtained a record of consent. But did the individual genuinely understand what they had agreed to?
Professor Chesterman then connected this problem to artificial intelligence. Our relationship with information has changed. We are no longer merely submitting information into a static website or database. We are interacting continuously with AI systems and, increasingly, with AI agents.
That raises a much harder question: Can someone meaningfully consent at the beginning of an AI conversation when neither the user nor the organisation can fully predict where that conversation will lead?
The old problem: Nobody reads the privacy notice
The weakness of the notice-and-consent model predates generative AI. It rests on the assumption that individuals will read a privacy notice, understand how their personal data will be processed, assess the consequences and then make an informed decision.
In reality, privacy notices are often:
1. long and legally complex;
2. presented when users are trying to complete another task;
3. written broadly to cover multiple possible uses;
4. accepted on a take-it-or-leave-it basis; and
5. treated as an administrative hurdle rather than a meaningful decision.
Most users simply want access to the service. They click “I agree” because declining may prevent them from proceeding.
This creates a difference between formal consent and meaningful consent. Formal consent means that the organisation can demonstrate that the user clicked a button, selected a checkbox or otherwise indicated agreement. Meaningful consent means that the individual understood the material purposes, had a genuine choice and could reasonably anticipate what would happen to their information.
Generative AI widens the gap between the two.
A chatbot is not an ordinary online form
A conventional online form normally asks predetermined questions: name, email address, date of birth, employment details or payment information.
The organisation can identify the fields it intends to collect and explain why it needs them. A generative AI chatbot operates differently. It invites the user into an open-ended conversation.
The user may begin with an innocent request: “Please help me improve my résumé.”
To provide a better answer, the chatbot may ask about the user’s career, work history, strengths, weaknesses and preferred roles. During the conversation, the user may disclose:
1. their approximate age;
2. periods of unemployment;
3. health conditions;
4. family responsibilities;
5. nationality or immigration status;
6. previous salary;
7. workplace disputes;
8. educational background; or
9. dissatisfaction with a current employer.
The user may not think of the conversation as a collection of personal data. It feels more like speaking to an assistant. But the chatbot is processing the information, placing it in context and generating responses based on the combined conversation.
More importantly, the system may derive conclusions the user never expressly stated. It might infer that the user is approaching retirement age, has a health-related employment gap, is financially vulnerable, lacks confidence or is likely to leave their employer.
These inferences may themselves constitute personal data where they relate to an identifiable individual.
The European Union’s General Data Protection Regulation expressly recognises profiling as automated processing used to evaluate or predict matters such as a person’s work performance, economic situation, health, preferences, interests, reliability or behaviour. (EUR-Lex)
This reveals the fundamental difference:
A GenAI system does not merely process what a user tells it. It may also derive new information from what the user tells it.
A simple structure would be The 3Ps of Personal Data in GenAI: Process, Personalise and Profile. Within Process, use the 5Rs to explain how the information moves through the system.
What Can Happen to Your Personal Information? The 3Ps of GenAI
A simple way to understand how a GenAI system may use personal information is through the 3Ps:
1. Process
The AI first processes the information to perform the requested task. However, processing may extend beyond generating an immediate answer. The 5Rs of AI processing make this easier to remember:
Respond
The system analyses the user’s prompt, uploaded files and relevant conversation history to generate an immediate response.
Retain
Prompts, outputs and interaction records may be stored for purposes such as maintaining conversation history, troubleshooting, security monitoring, audits or legal compliance.
Refine
Depending on the provider, account type, settings and contractual terms, information may be reviewed or used to improve the service, evaluate performance or develop future models.
Relay
The information may pass through several parties in the AI supply chain, including the application provider, cloud platform, model provider, knowledge system, plug-ins, external APIs and monitoring services.
Recommend
The information may later influence recommendations, rankings, classifications, scores or decisions—whether within the same conversation or in a downstream business process.
The important question is therefore not merely, “Did the chatbot answer my question?” It is also, “What else happened to my information while that answer was being produced?”
2. Personalise
The system may use personal data, previous conversations, preferences or account information to tailor the user’s experience.
Personalisation may affect:
1. the tone and level of detail of responses;
2. the topics or products recommended;
3. the examples selected;
4. the learning content presented;
5. the next questions asked; and
6. the actions or options prioritised.
For example, an AI tutor may adapt its explanations based on what it believes the learner already understands. A career assistant may recommend roles based on a user’s work history, interests and previous questions.
Personalisation can improve usefulness and relevance. However, it can also create risks when information disclosed in one context is used unexpectedly in another.
Users may not realise that a previous conversation has shaped the response they are now receiving.
3. Profile
Profiling occurs when an AI system evaluates or predicts characteristics about an individual.
The system may infer information concerning a person’s:
1. interests and preferences;
2. knowledge or competency;
3. emotional or psychological state;
4. health or personal circumstances;
5. financial position;
6. reliability or risk;
7. likely behaviour; or
8. suitability for a role, service or opportunity.
Some of these conclusions may be drawn even when the individual did not explicitly provide the information.
For example, a résumé conversation might allow the system to infer a person’s approximate age, seniority, salary range, employment vulnerability or likelihood of leaving their current employer.
An AI tutor might infer that a learner is weak in a particular area and adjust the learning pathway accordingly. If the same inference is later used to grade, rank or certify the learner, the consequences become more significant.
This is why personal data in GenAI should not be understood only as the information a person knowingly enters.
It may also include the information the system processes, personalises and profiles from the interaction.
The 3Ps at a glance
Process: What happens to the information within the AI system and its supply chain?
Personalise: How is the information used to tailor the user’s experience?
Profile: What new conclusions or predictions are made about the individual?
The 3Ps help explain why consent is so important—but also why consent alone is insufficient. A user may understand that their prompt will be processed to generate an answer, yet remain unaware that it could be retained, relayed, used for personalisation or converted into a profile that influences later recommendations and decisions.
Consent is a starting point, not the entire solution
Consent is important because it may provide the legal basis on which an organisation collects, uses or discloses personal data.
But consent answers only one question: Are we permitted to process this information for the stated purpose?
It does not answer every governance question. For example:
1. Was collecting the information necessary?
2. Was the purpose reasonable and appropriate?
3. Did the user understand that inferences could be generated?
4. Could the system produce an inaccurate or harmful profile?
5. Who can access the conversation?
6. How long will it be retained?
7. Can the user correct or delete the information?
8. Is the data being reused for another purpose?
9. Who is accountable if the processing causes harm?
Consent does not automatically make every subsequent use fair, proportionate or safe. Nor is consent always the applicable legal basis. Privacy laws may permit processing under other legal grounds or statutory exceptions.
This is why organisations should not treat the “I agree” button as a transfer of risk from the organisation to the individual.
A more accurate principle is:
Consent may permit processing, but it does not excuse poor system design or remove organisational accountability.
Singapore: More than obtaining consent
Under Singapore’s Personal Data Protection Act, organisations are generally required to notify individuals of the purposes for collecting, using or disclosing their personal data and obtaining consent unless an exception applies.
The purposes must also be ones that a reasonable person would consider appropriate in the circumstances.
This means an organisation should not assume that broad wording in a privacy policy gives it unlimited permission to use information for any future AI-related purpose.
Singapore’s Personal Data Protection Commission has specifically addressed AI systems used to make recommendations or decisions. Its guidance recognises both systems that make decisions autonomously and systems that assist human decision-makers through recommendations and predictions. It also explains that organisations may rely on meaningful consent or, where appropriate, exceptions such as business improvement or research. (Personal Data Protection Commission)
More recently, on 20 July 2026, the PDPC published its Advisory Guidelines on the Use of Personal Data in Generative AI. The guidelines address:
1. how personal data may be collected and used to develop GenAI models;
2. how data-protection responsibilities should be allocated across the GenAI lifecycle; and
3. how organisations should handle individuals’ requests concerning the processing of their personal data in GenAI. (Personal Data Protection Commission)
The Singapore position should therefore not be reduced to “obtain consent and proceed.” The broader expectation is that organisations use personal data for appropriate purposes, communicate those purposes, allocate responsibilities and remain accountable throughout the system’s lifecycle.
The European Union: Consent plus stronger individual rights
The European Union adopts a more explicitly rights-based approach. Under the GDPR, consent must generally be:
1. freely given;
2. specific;
3. informed;
4. unambiguous; and
5. capable of being withdrawn.
Consent may be questionable where there is a significant imbalance of power or where the provision of a service is made unnecessarily conditional on agreeing to unrelated processing.
The GDPR also gives individuals rights relating to access, correction, erasure, restriction and objection. It contains specific protections concerning profiling and decisions based solely on automated processing that produce legal or similarly significant effects.
The European Data Protection Board has confirmed that GDPR principles apply to personal data processed during both the development and deployment of AI models. Its opinion examines, among other things, when an AI model may be considered anonymous, whether legitimate interests may provide a lawful basis, and the consequences of unlawfully processed personal data during model development. (European Data Protection Board)
The distinction is not that Europe always requires consent while Singapore does not.
Rather:
1. the EU places stronger emphasis on fundamental data-protection rights and enforceable individual remedies;
2. Singapore places significant emphasis on reasonable purposes, accountability and practical organisational responsibility.
Neither framework allows an organisation to treat a single click as unlimited authorisation.
What organisations should do
The solution is not to produce an even longer privacy notice. Organisations need to move from one-time disclosure to continuing transparency and accountability.
Before deploying a GenAI system, the DPO, GRC professionals, product owners and developers should work together to:
1. Map the personal data the system may collect, receive, generate, and infer.
2. Identify the legal basis or applicable exception for each processing purpose.
3. Separate data required to answer the user from data retained for analytics, security, or model improvement.
4. Conduct a data protection impact assessment (DPIA) and an AI risk assessment.
5. Examine the complete supply chain, including model providers, cloud services, retrieval systems, and external APIs.
6. Define retention, deletion, and access control requirements.
7. Determine whether conversations may be used for model improvement.
8. Assess how requests for access, correction, withdrawal, or deletion will be handled.
9. Test for unexpected disclosures, harmful inferences, and foreseeable misuse.
At the point of interaction, organisations should use concise and layered notices. A user should be told, at an appropriate time:
1. that they are interacting with AI;
2. what information is being collected;
3. why it is needed;
4. whether the conversation will be stored;
5. whether it may be reviewed or used to improve models;
6. whether third-party providers are involved;
7. what information should not be entered; and
8. what controls or rights are available.
Where the purpose changes during the interaction, a just-in-time notice may be more meaningful than relying on a privacy policy accepted months earlier.
For example, an AI wellness assistant should not wait until the user has disclosed sensitive health information before explaining how such information will be handled.
What professional users should do
Users also need to become more careful about what they enter into GenAI systems. Before uploading a document or sharing personal information, ask:
1. Is this information genuinely necessary?
2. Am I disclosing information about a colleague, customer or another person?
3. Is this a public AI service or an approved enterprise system?
4. Will the conversation be retained?
5. Can it be used to improve the model?
6. Can I delete it?
7. Are external providers involved?
8. Can I remove names and identifiers first?
9. Would I be comfortable if this information appeared in a system log, security investigation or vendor review?
However, user responsibility has limits. An organisation cannot simply display “Do not enter sensitive information” and assume that it has discharged its duties.
Warnings cannot compensate for poor access controls, excessive retention, hidden secondary purposes or unsafe system design.
From blind consent to continuing accountability
Professor Chesterman’s audience poll captured the weakness of traditional privacy notices: most people do not read them before clicking “I agree.”
But GenAI presents a deeper problem.
Even users who carefully read the notice may not be able to anticipate everything that an AI system could infer, how an open-ended conversation could evolve or how information might move through a complex chain of providers.
1. Meaningful consent therefore requires more than a checkbox. It requires:
2. understandable and contextual transparency;
3. careful choices by users;
4. privacy-conscious system design;
5. collaboration between DPOs, GRC professionals and developers; and
6. continuing accountability throughout the AI lifecycle.
Consent remains important. But it is only the beginning. And an even more consequential question follows. Once the AI has processed and inferred information about an individual, can it use that information to recommend, rank, shortlist, score—or act?
That is where consent gives way to the next major issue: automated decision-making in the age of agentic AI.
