How AI Tutoring Platforms Track Your Thinking (And What Their Privacy Policies Don't Say)
An analysis of 8 major platforms reveals that none address whether they can reconstruct your learning struggles after a session ends.
The Gap Nobody Is Talking About
When a student asks an AI tutor for help with fractions, the obvious transaction is educational. A question goes in, an answer comes out. But something else is happening underneath that exchange. The platform is capturing how the student interacts with the system: what they ask, how long they spend, what they retry, what they abandon.
The question most learners never think to ask is: what happens to that data after I close the tab?
We reviewed the privacy policies of eight major AI-powered learning and conversation platforms (Khan Academy's Khanmigo, Duolingo Max, ChatGPT, Google Gemini, Anthropic's Claude, Microsoft Copilot, Coursera, and Character.AI) against 12 questions designed to assess how they handle cognitive data: information about how users think, not just what they type.
The findings are consistent across all eight platforms and reveal four significant gaps in how the industry treats learner data during AI interactions.
The most striking finding is a universal silence. We asked each platform a simple question: can cognitive process records be reconstructed after a session ends? In other words, after a learner closes their session, can the platform rebuild a picture of what that person was struggling with, where they hesitated, what they explored and abandoned?
Not one of the eight platforms addresses this question in their privacy policy.
This matters because AI interaction data is fundamentally different from traditional web browsing data. When a student works through a problem with an AI tutor, the conversation itself is a record of their cognitive process: the confusion, the false starts, the moments where understanding breaks down. This is not equivalent to a search history. It is a map of how someone thinks.
Magee, Ienca, and Farahany (2024), writing in Neuron, define cognitive biometric data as information that can be processed to infer mental states. By that definition, AI tutoring conversations qualify. Every prompt a struggling learner sends contains signals about their cognitive state: what they understand, what confuses them, where their reasoning breaks down. The platform captures all of it. None of the eight policies we reviewed address what happens to that cognitive layer after the session ends.
Two platforms (Duolingo and Coursera) are transparent about the depth of their behavioral tracking. Duolingo discloses the use of FullStory, a session-replay tool that logs clicks, mouse movements, scrolling, and typing. Coursera similarly acknowledges session-replay tools that capture mouse movements, page visits, and console logs.
The other six platforms use umbrella language. ChatGPT collects "how you interact… features you use and the actions you take." Khanmigo logs "information about your use of our Service and your learning activity." Google Gemini collects "interaction logs, performance metrics." Microsoft Copilot captures "interactions with bots… performance, usage, and device data."
What none of these vague descriptions clarify is whether the platform measures granular behavioral signals during AI sessions: typing speed, backspace frequency, pause duration, revision patterns. These are exactly the signals that indicate cognitive state. A student who types a question, deletes it, retypes it differently, and pauses for thirty seconds before submitting is telling the system something about their confidence, their confusion, and their reasoning process.
Whether platforms are capturing these signals is genuinely unclear from their policies. And that ambiguity is itself the problem.
Nearly every platform we reviewed admits to using interaction data to "personalize" the experience. Khanmigo uses data to understand how usage "impacts learning outcomes." Duolingo tailors lessons to "include more words that a user is struggling with." Google Gemini generates "personalized insights" from chat history. Character.AI explicitly "generates and analyzes inferences… such as your preferences and interests."
All of this involves inference about cognitive and psychological states. The platform is deducing what the learner knows, what confuses them, what engages them, and what they're likely to do next. These are inferences about mental states derived from behavioral data.
Of the eight platforms reviewed, only one (Character.AI) explicitly classifies these inferences as protected personal data. The other seven do not address whether inferred cognitive states receive the same protection as directly provided information like a name or email address.
This gap maps onto a structural problem in privacy law that Solove (2025) identifies in the Florida Law Review: current frameworks regulate input (data collected) rather than output (inferences made). The practical consequence for learners is that the most sensitive information a platform holds about you (what you struggle with, what confuses you, how your reasoning works) is also the least protected.
Every platform we reviewed has some form of age-gating. Anthropic bans all users under 18. ChatGPT and Coursera ban users under 13. Character.AI prohibits users under 13 (16 in the EU/UK). Khanmigo and Microsoft Copilot offer specific child account structures with parental controls.
But age-gating is a blunt instrument when the risk is cognitive, not just informational. Standard COPPA compliance protects children's personally identifiable information (name, email, location). It was not designed to address what happens when a child's developing reasoning process becomes training data for an AI model.
Only two platforms go beyond age-gating toward substantive protections. Microsoft Copilot for Education explicitly forbids building student profiles or targeting ads, making it the strongest policy position in the group. Khanmigo warns users against including personal information in AI inputs and prohibits its AI vendors from using student input data to train models.
The other six platforms default to standard compliance without addressing the specific risks of AI interaction data from minors. None address the documented tendency of young users to form emotional bonds with AI systems. Research from Internet Matters (2025) found that 40 percent of teens who use AI companions trust their guidance without question. Yang and Oshio (2025), writing in Current Psychology, have documented how users form measurable attachment bonds with AI systems using the same psychological framework applied to human relationships.
A platform that captures a child's cognitive struggles, uses that data to personalize engagement, and retains it across sessions is not just collecting information. It is building a persistent model of a developing mind. Current privacy policies do not acknowledge this distinction, and current privacy law does not require them to.
| Question | Khanmigo | Duolingo Max | ChatGPT | Google Gemini | Claude | MS Copilot | Coursera | Character.AI |
|---|---|---|---|---|---|---|---|---|
| Collects AI input data | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
| Discloses behavioral tracking depth | Vague | Yes (FullStory) | Vague | Vague | Vague | Vague | Yes (session replay) | Vague |
| Distinguishes minors | Yes | Yes | Yes (bans <13) | Yes | Yes (bans <18) | Yes | Yes (bans <13) | Yes (bans <13/16) |
| Retention period | Until unnecessary | Until deletion | Until deletion (30-day hold) | Auto-delete 3 to 36 mo | Until unnecessary (30-day deletion) | Varies (6-mo de-ID) | Until unnecessary | Until unnecessary |
| Trains AI on user data | No (vendor prohibited) | Yes | Yes (opt-out) | Yes (opt-out) | Yes (opt-out) | Consumer yes / Edu no | Ambiguous | Yes (opt-out ambiguous) |
| On-device processing | No | No | No | No | No | Partial | No | No |
| Addresses cognitive reconstruction | No | No | No | No | No | No | No | No |
| Infers cognitive/psychological states | Ambiguous | Yes | Ambiguous | Yes | Ambiguous | Yes | Yes | Yes |
| Treats inferences as personal data | No | No | No | No | No | No | No | Yes |
| Builds persistent cross-session profiles | Yes | Yes | Yes (Memories) | Yes (Saved Info) | Minimal | Yes | Yes | Yes |
| Shares with third parties | Yes | Yes (incl. ad networks) | Yes | Yes (incl. reviewers) | Yes | Yes (incl. ad platforms) | Yes (incl. university partners) | Yes (incl. ad/analytics) |
| AI-specific protections for minors | Yes (no vendor training) | Partial (disables tracking) | No | No | N/A (bans minors) | Yes (bans profiling) | No | No |
The Offloading Feedback Loop
The AI tutoring market is growing rapidly. Platforms are competing to offer more personalized, more adaptive, more engaging learning experiences. Personalization requires data about how learners think. The more granular that data, the better the personalization.
But personalization and cognitive privacy are in direct tension. A system that adapts to your confusion must first capture your confusion. A system that identifies your weaknesses must first build a model of your weaknesses. The question is whether that model is ephemeral (used in the moment and then discarded) or persistent, accumulating across sessions into a detailed profile of how your mind works.
Gerlich's 2025 study of 666 participants found that the more people used AI tools, the more they offloaded their thinking to those tools, and the worse their critical thinking scores became. The age pattern matters: participants aged 17 to 25 showed the highest dependence and lowest critical thinking scores. Participants over 46 showed the opposite pattern. Adults who built cognitive capacities before AI can choose to offload. Younger users may be offloading tasks they have never learned to perform.
This creates a feedback loop. The learner offloads thinking to the AI. The AI captures the interaction. The captured data trains a model that becomes better at completing the learner's thinking for them. The learner offloads more. The cycle continues. Whether this cycle is beneficial or harmful depends on whether the learner is using the tool to extend capacities they already have, or to replace capacities they haven't yet built.
Use the Cognitive Privacy Impact Assessment
The CPIA is a structured framework for evaluating whether an AI platform captures cognitive data. The core test: after a session ends, can the system reconstruct how the user was thinking? If the answer is yes, the system is capturing cognitive data, and the user deserves to know.
Access the CPIA Framework →What Should Change
- Platforms should disclose cognitive data practices explicitly. A policy that says "we collect information about how you use the service" is not adequate when the service involves AI interaction that captures reasoning processes. Learners deserve to know whether their hesitation patterns, revision behavior, and error sequences are being logged, retained, and used.
- Inferred cognitive states should be classified as personal data. If a platform deduces from your interaction patterns that you are confused, anxious, or struggling with a concept, that inference is about your mental state. It should receive the same legal protection as your name. Only one of eight platforms currently takes this position.
- AI tutoring platforms serving minors should adopt ephemeral processing standards. Session data should not persist after the session ends. Cross-session cognitive profiles should not be built for users under 18. Interaction data from minors should not train AI models. Microsoft Copilot for Education comes closest to this standard. The rest of the industry is nowhere near it.
For School Leaders Evaluating AI Tutors
Before adopting any AI tutoring platform, ask the vendor three questions directly. First: after a student closes their session, can your system reconstruct what they were struggling with? Second: does your platform classify inferences about student cognitive states as personal data? Third: does student interaction data train your AI models?
If the vendor cannot answer these questions clearly, their privacy policy is not sufficient for your school's adoption decision. The Cognitive Privacy Impact Assessment provides a structured protocol for conducting this evaluation. Connected Classroom offers advisory support for schools navigating AI adoption, including platform evaluation and teacher training on cognitive privacy literacy.
Access the full technical report and citation data via ResearchGate.
References
Cook, T. (2025). The era of cognitive capture: Protecting mental autonomy in the age of behavioral algorithms. The Cognitive Privacy Project. https://doi.org/10.13140/RG.2.2.23668.28804
Gerlich, M. (2025). AI tools in society: Impacts on cognitive offloading and the future of critical thinking. Societies, 15(1), 6. https://doi.org/10.3390/soc15010006
Internet Matters. (2025). Research on teen AI companion use. https://www.internetmatters.org/
Magee, L., Ienca, M., & Farahany, N. A. (2024). Beyond neural data: Cognitive biometrics and mental privacy. Neuron, 112(18), 2951–2959. https://doi.org/10.1016/j.neuron.2024.09.004
Solove, D. J. (2025). Artificial intelligence and privacy. Florida Law Review, 77. https://ssrn.com/abstract=4713111
Yang, F., & Oshio, A. (2025). Using attachment theory to conceptualize and measure the experiences in human-AI relationships. Current Psychology, 44, 10658–10669. https://link.springer.com/article/10.1007/s12144-025-07917-6

