Anthropic’s New AI Index Shows What Sets Top AI Users Apart

The central finding from Anthropic’s new AI Fluency Index is that the better the output looks, the less people question it. Published this week, the AI Fluency Index is the first large-scale attempt to measure not just how people use AI, but how well they use it.

Researchers at the AI company analyzed 9,830 conversations on Claude AI during a single week in January 2026, tracking 11 specific behaviors that represent effective human-AI collaboration. What they found should make us all think about AI literacy.

Researchers at the AI company analyzed 9,830 conversations on Claude during a single week in January 2026. In their research, they tracked 11 specific behaviors representing effective human AI collaboration.

The framework they used was created by professors Rick Dakan and Joseph Feller in collaboration with Anthropic. The tracked behaviors included clarifying goals, providing examples of good output, and questioning AI reasoning.

Most People Already Iterate

They found that the most common behaviors exhibited were iteration and refinement. 85.7 percent of conversations showed users building on previous exchanges rather than accepting the first response. This means most people are not treating AI as a vending machine where they write a prompt and accept whatever comes out. They are treating it as a work in progress. A session that is collaborative between human and machine.

Conversations with iteration showed roughly double the rate of other fluency behaviors compared to those without. Users who iterated were 5.6 times more likely to question Claude’s reasoning and four times more likely to flag missing content. This matters because people who push back get better results.

The Artifact Effect

In about 12 percent of the conversations, Claude produced artefacts. Artefacts are products created by AI, such as code, a document, an interactive tool, or an app. These tangible outputs are designed to look and feel like finished products.

The research found that in artefact conversations, users were more skilled and more direct upfront. Before the AI produced the final output, people clarified goals more often, increasing by 14.7 percent. Specifying format increased by 14.5 percent. Providing examples of what they wanted increased by 13.4 percent.

The problem is that once the polished output was presented, once the artefact was created, users were less likely to identify missing content. That behavior dropped by 5.2 percent. People were 3.7 percent less likely to check facts and 3.1 percent less likely to question the model’s reasoning. Every discernment behavior dropped when the AI created an artefact.

The researchers describe this as a tension between description and delegation versus discernment. Users became better at telling the AI what to build, but worse at evaluating what it built. This gap should concern anyone who works in education or has children using artificial intelligence. It reflects the pattern seen when people learn to use AI tools without learning to think critically about the outputs.

Why Education Should Pay Attention

So far, the conversation around AI in schools has been dominated by access. Do students have AI? Should they use it? What are the rules? Anthropic’s data suggests the more urgent question is whether students are developing the skills to use it well.

The Fluency Index identifies three competency areas. First, description, telling an AI what you need. Second, delegation, structuring the collaboration with AI. Third, discernment, evaluating what comes back.

The first two competency areas can be categorized under effective prompting. The third, discernment, shows the biggest gap and may be the most important skill. In only 8.7 percent of conversations did users check facts and claims. In just 15.8 percent did they question the reasoning of the artificial intelligence. These are foundational elements of critical thinking.

When AI can produce convincing pieces of work like essays or a working application in seconds, the ability to ask whether it is correct becomes the differentiating skill.

What Fluent AI Users Do Differently

The report explores three high-impact behaviors that were underused. First, directing the interaction. Only 30 percent of users told the AI how they wanted it to behave, for example, asking it to walk through its reasoning or push back on assumptions. Setting these terms turns AI into a collaborator rather than an answer machine.

Second, staying with the conversation through iteration. Iteration is the gateway behavior and is strongly correlated with other effective practices. It means treating the first AI response not as a finished product but as a draft. This may be the single highest leverage intervention a user can make.

Third, questioning polished outputs. The artefact effect suggests that the more capable AI becomes, the more effort humans need to invest in evaluation. The output may look better, but discernment becomes more important.

The Limits Of What This Study Can Tell Us

There are limits to this study, and Anthropic is transparent about them. The sample comes from a single week of Claude users. Behaviors are classified as present or absent, which removes nuance. The findings are correlational, not causal. It is unclear whether iteration causes better evaluation or whether both reflect a more thoughtful user.

The framework identifies 24 fluency behaviors, but only 11 are observable through chat conversations. The other 13, including being transparent about AI’s role in work or considering the consequences of sharing AI generated content, happen outside the platform. These unobservable behaviors may be among the most important for educators.

The artefact effect also has multiple possible explanations. Users may evaluate the output on another platform or share drafts with colleagues for review. Those actions would not appear in this observational data. The researchers acknowledge this and caution against concluding that polished outputs automatically switch off critical thinking.

From AI Literacy To AI Fluency

Some may ask whether AI fluency is different from what many call AI literacy. The distinction matters. Literacy implies a threshold, while fluency suggests a spectrum that develops through deliberate effort and practice. Anthropic plans to track this spectrum over time, comparing new and experienced users and exploring whether specific interventions can improve discernment behaviors.

The practical takeaway for educators and parents is concrete. The skill of iteration is vital. As is the ability to take control of the conversation upfront. Most critically, the need to develop the skills to discern AI outputs, especially when those outputs look perfect, is most important.