To measure how well people use AI at work, separate usage from proficiency. Usage, such as logins and seats in use, shows who opens the tool. Proficiency shows in decisions: which job someone hands to AI, what they show it, and whether they catch its mistake in a situation they haven’t seen before. Measure each habit separately, count first tries, and treat the result as a guide to where to help, not proof of productivity.

Usage is not proficiency

Most AI dashboards report usage: how many seats are active, how many chats people start, how often they come back. These numbers are useful. They tell you whether people have started, and where adoption is stalled.

They don’t tell you whether the use is any good. One person opens AI ten times a day and forwards every answer unchecked. Another opens it twice and catches the wrong figure before it reaches a client. Usage counts both the same.

Course completions have the same gap. Finishing a module tells you someone was there, not what they will do next.

What to measure instead: decisions

Proficiency is a set of habits, and each one shows in a decision you can judge:

Judge the decision, not the wording, and judge it by fixed rules, so the same answer gets the same result for everyone.

Count the first try at a new situation

A right answer to a case someone has seen before may be memory. The honest signal is the first answer to a situation that is new to them. Two more rules help:

A level per habit, not one score

Someone can brief AI well and check its work badly. One overall score hides that. Give each person a level on each habit. One way to name the levels, which Skill Tree uses, runs Aware, Practicing, Capable, Independent and Leading: from recognizing the situation, through doing it with guidance and adjusting it when things change, to doing it unprompted and building it into how a team works. What AI fluency is describes each level.

What lesson results can show, and what they can’t

Decisions in lessons can show that a person made the right call in a realistic situation, on a first try, across more than one setting. That is real evidence of a habit.

They can’t show:

If you need to show a return, measure real work too. Pick one recurring job, such as a weekly report, and compare before and after: how long it takes and how often it comes back for corrections. Ask the people who receive the work.

Report by team, and protect people’s answers

People hide what they don’t know when they think a manager is reading every answer. Report what people can do and where they are stuck, by team. Keep their answers, what they typed and their conversations with AI out of the report. Tell people what admins can see before they start.

Follow up on what the results show

An example: Skill Tree’s dashboard

Skill Tree’s lessons grade each decision by fixed rules in its own code. Only a first answer to a new situation counts toward a level, and a level counts only once the habits it builds on are in place. Admins see each person’s level on each habit, where they are in the course, which lessons they missed and whether they have gotten them right since. When two or more people on a team are stuck on the same thing, the dashboard offers a talking point for the next team meeting. Admins never see anyone’s answers. See security and privacy for teams for the full list.