Resource · Platform Audit

How AI Tutoring Platforms Track Your Thinking (And What Their Privacy Policies Don't Say)

An analysis of 8 major platforms reveals that none address whether they can reconstruct your learning struggles after a session ends.

Timothy Cook · The Cognitive Privacy Project · 2026
For Schools & Districts

If your school is adopting AI tutoring tools for personalized learning, this audit provides a framework for evaluating what those platforms capture about student cognition, and what their privacy policies fail to disclose. Use the Cognitive Privacy Impact Assessment to evaluate any platform not covered here.

The Gap Nobody Is Talking About

When a student asks an AI tutor for help with fractions, the obvious transaction is educational. A question goes in, an answer comes out. But something else is happening underneath that exchange. The platform is capturing how the student interacts with the system: what they ask, how long they spend, what they retry, what they abandon.

The question most learners never think to ask is: what happens to that data after I close the tab?

We reviewed the privacy policies of eight major AI-powered learning and conversation platforms (Khan Academy's Khanmigo, Duolingo Max, ChatGPT, Google Gemini, Anthropic's Claude, Microsoft Copilot, Coursera, and Character.AI) against 12 questions designed to assess how they handle cognitive data: information about how users think, not just what they type.

The findings are consistent across all eight platforms and reveal four significant gaps in how the industry treats learner data during AI interactions.

Gap 01
No Platform Addresses Cognitive Reconstruction

The most striking finding is a universal silence. We asked each platform a simple question: can cognitive process records be reconstructed after a session ends? In other words, after a learner closes their session, can the platform rebuild a picture of what that person was struggling with, where they hesitated, what they explored and abandoned?

Not one of the eight platforms addresses this question in their privacy policy.

This matters because AI interaction data is fundamentally different from traditional web browsing data. When a student works through a problem with an AI tutor, the conversation itself is a record of their cognitive process: the confusion, the false starts, the moments where understanding breaks down. This is not equivalent to a search history. It is a map of how someone thinks.

Magee, Ienca, and Farahany (2024), writing in Neuron, define cognitive biometric data as information that can be processed to infer mental states. By that definition, AI tutoring conversations qualify. Every prompt a struggling learner sends contains signals about their cognitive state: what they understand, what confuses them, where their reasoning breaks down. The platform captures all of it. None of the eight policies we reviewed address what happens to that cognitive layer after the session ends.

Gap 02
Behavioral Tracking Ranges From Honest to Opaque

Two platforms (Duolingo and Coursera) are transparent about the depth of their behavioral tracking. Duolingo discloses the use of FullStory, a session-replay tool that logs clicks, mouse movements, scrolling, and typing. Coursera similarly acknowledges session-replay tools that capture mouse movements, page visits, and console logs.

The other six platforms use umbrella language. ChatGPT collects "how you interact… features you use and the actions you take." Khanmigo logs "information about your use of our Service and your learning activity." Google Gemini collects "interaction logs, performance metrics." Microsoft Copilot captures "interactions with bots… performance, usage, and device data."

What none of these vague descriptions clarify is whether the platform measures granular behavioral signals during AI sessions: typing speed, backspace frequency, pause duration, revision patterns. These are exactly the signals that indicate cognitive state. A student who types a question, deletes it, retypes it differently, and pauses for thirty seconds before submitting is telling the system something about their confidence, their confusion, and their reasoning process.

Whether platforms are capturing these signals is genuinely unclear from their policies. And that ambiguity is itself the problem.

Gap 03
Inferred Mental States Exist in a Regulatory Gray Area

Nearly every platform we reviewed admits to using interaction data to "personalize" the experience. Khanmigo uses data to understand how usage "impacts learning outcomes." Duolingo tailors lessons to "include more words that a user is struggling with." Google Gemini generates "personalized insights" from chat history. Character.AI explicitly "generates and analyzes inferences… such as your preferences and interests."

All of this involves inference about cognitive and psychological states. The platform is deducing what the learner knows, what confuses them, what engages them, and what they're likely to do next. These are inferences about mental states derived from behavioral data.

Of the eight platforms reviewed, only one (Character.AI) explicitly classifies these inferences as protected personal data. The other seven do not address whether inferred cognitive states receive the same protection as directly provided information like a name or email address.

This gap maps onto a structural problem in privacy law that Solove (2025) identifies in the Florida Law Review: current frameworks regulate input (data collected) rather than output (inferences made). The practical consequence for learners is that the most sensitive information a platform holds about you (what you struggle with, what confuses you, how your reasoning works) is also the least protected.

Gap 04
Minor-Specific AI Protections Are Shallow

Every platform we reviewed has some form of age-gating. Anthropic bans all users under 18. ChatGPT and Coursera ban users under 13. Character.AI prohibits users under 13 (16 in the EU/UK). Khanmigo and Microsoft Copilot offer specific child account structures with parental controls.

But age-gating is a blunt instrument when the risk is cognitive, not just informational. Standard COPPA compliance protects children's personally identifiable information (name, email, location). It was not designed to address what happens when a child's developing reasoning process becomes training data for an AI model.

Only two platforms go beyond age-gating toward substantive protections. Microsoft Copilot for Education explicitly forbids building student profiles or targeting ads, making it the strongest policy position in the group. Khanmigo warns users against including personal information in AI inputs and prohibits its AI vendors from using student input data to train models.

The other six platforms default to standard compliance without addressing the specific risks of AI interaction data from minors. None address the documented tendency of young users to form emotional bonds with AI systems. Research from Internet Matters (2025) found that 40 percent of teens who use AI companions trust their guidance without question. Yang and Oshio (2025), writing in Current Psychology, have documented how users form measurable attachment bonds with AI systems using the same psychological framework applied to human relationships.

A platform that captures a child's cognitive struggles, uses that data to personalize engagement, and retains it across sessions is not just collecting information. It is building a persistent model of a developing mind. Current privacy policies do not acknowledge this distinction, and current privacy law does not require them to.

Platform Comparison: 12 Cognitive Privacy Questions
Question Khanmigo Duolingo Max ChatGPT Google Gemini Claude MS Copilot Coursera Character.AI
Collects AI input data YesYesYesYesYesYesYesYes
Discloses behavioral tracking depth VagueYes (FullStory)VagueVagueVagueVagueYes (session replay)Vague
Distinguishes minors YesYesYes (bans <13)YesYes (bans <18)YesYes (bans <13)Yes (bans <13/16)
Retention period Until unnecessaryUntil deletionUntil deletion (30-day hold)Auto-delete 3 to 36 moUntil unnecessary (30-day deletion)Varies (6-mo de-ID)Until unnecessaryUntil unnecessary
Trains AI on user data No (vendor prohibited)YesYes (opt-out)Yes (opt-out)Yes (opt-out)Consumer yes / Edu noAmbiguousYes (opt-out ambiguous)
On-device processing NoNoNoNoNoPartialNoNo
Addresses cognitive reconstruction NoNoNoNoNoNoNoNo
Infers cognitive/psychological states AmbiguousYesAmbiguousYesAmbiguousYesYesYes
Treats inferences as personal data NoNoNoNoNoNoNoYes
Builds persistent cross-session profiles YesYesYes (Memories)Yes (Saved Info)MinimalYesYesYes
Shares with third parties YesYes (incl. ad networks)YesYes (incl. reviewers)YesYes (incl. ad platforms)Yes (incl. university partners)Yes (incl. ad/analytics)
AI-specific protections for minors Yes (no vendor training)Partial (disables tracking)NoNoN/A (bans minors)Yes (bans profiling)NoNo

The Offloading Feedback Loop

The AI tutoring market is growing rapidly. Platforms are competing to offer more personalized, more adaptive, more engaging learning experiences. Personalization requires data about how learners think. The more granular that data, the better the personalization.

But personalization and cognitive privacy are in direct tension. A system that adapts to your confusion must first capture your confusion. A system that identifies your weaknesses must first build a model of your weaknesses. The question is whether that model is ephemeral (used in the moment and then discarded) or persistent, accumulating across sessions into a detailed profile of how your mind works.

Gerlich's 2025 study of 666 participants found that the more people used AI tools, the more they offloaded their thinking to those tools, and the worse their critical thinking scores became. The age pattern matters: participants aged 17 to 25 showed the highest dependence and lowest critical thinking scores. Participants over 46 showed the opposite pattern. Adults who built cognitive capacities before AI can choose to offload. Younger users may be offloading tasks they have never learned to perform.

This creates a feedback loop. The learner offloads thinking to the AI. The AI captures the interaction. The captured data trains a model that becomes better at completing the learner's thinking for them. The learner offloads more. The cycle continues. Whether this cycle is beneficial or harmful depends on whether the learner is using the tool to extend capacities they already have, or to replace capacities they haven't yet built.

Use the Cognitive Privacy Impact Assessment

The CPIA is a structured framework for evaluating whether an AI platform captures cognitive data. The core test: after a session ends, can the system reconstruct how the user was thinking? If the answer is yes, the system is capturing cognitive data, and the user deserves to know.

Access the CPIA Framework →

What Should Change

  • Platforms should disclose cognitive data practices explicitly. A policy that says "we collect information about how you use the service" is not adequate when the service involves AI interaction that captures reasoning processes. Learners deserve to know whether their hesitation patterns, revision behavior, and error sequences are being logged, retained, and used.
  • Inferred cognitive states should be classified as personal data. If a platform deduces from your interaction patterns that you are confused, anxious, or struggling with a concept, that inference is about your mental state. It should receive the same legal protection as your name. Only one of eight platforms currently takes this position.
  • AI tutoring platforms serving minors should adopt ephemeral processing standards. Session data should not persist after the session ends. Cross-session cognitive profiles should not be built for users under 18. Interaction data from minors should not train AI models. Microsoft Copilot for Education comes closest to this standard. The rest of the industry is nowhere near it.

For School Leaders Evaluating AI Tutors

Before adopting any AI tutoring platform, ask the vendor three questions directly. First: after a student closes their session, can your system reconstruct what they were struggling with? Second: does your platform classify inferences about student cognitive states as personal data? Third: does student interaction data train your AI models?

If the vendor cannot answer these questions clearly, their privacy policy is not sufficient for your school's adoption decision. The Cognitive Privacy Impact Assessment provides a structured protocol for conducting this evaluation. Connected Classroom offers advisory support for schools navigating AI adoption, including platform evaluation and teacher training on cognitive privacy literacy.

Access the full technical report and citation data via ResearchGate.

References

Cook, T. (2025). The era of cognitive capture: Protecting mental autonomy in the age of behavioral algorithms. The Cognitive Privacy Project. https://doi.org/10.13140/RG.2.2.23668.28804

Gerlich, M. (2025). AI tools in society: Impacts on cognitive offloading and the future of critical thinking. Societies, 15(1), 6. https://doi.org/10.3390/soc15010006

Internet Matters. (2025). Research on teen AI companion use. https://www.internetmatters.org/

Magee, L., Ienca, M., & Farahany, N. A. (2024). Beyond neural data: Cognitive biometrics and mental privacy. Neuron, 112(18), 2951–2959. https://doi.org/10.1016/j.neuron.2024.09.004

Solove, D. J. (2025). Artificial intelligence and privacy. Florida Law Review, 77. https://ssrn.com/abstract=4713111

Yang, F., & Oshio, A. (2025). Using attachment theory to conceptualize and measure the experiences in human-AI relationships. Current Psychology, 44, 10658–10669. https://link.springer.com/article/10.1007/s12144-025-07917-6

Timothy Cook, M.Ed., is Director of The Cognitive Privacy Project, Founder of Connected Classroom, and writes the "Algorithmic Mind" column at Psychology Today. He is Securiti Certified in AI Security & Governance. His book Unautomatable is in peer review with MIT Press. He teaches elementary school at the American Community School of Amman, Jordan.