Not everything that can be measured should be.
Remember, the goal of monitoring AI usage is not total visibility into every interaction every employee has with a language model. Rather, it is specific, purposeful visibility into the things that carry genuine organizational risk.
Here's a starting point. The first list covers the things that answer real business questions: risk, spend, adoption, compliance. The second covers the things most organizations have no legitimate reason to be asking about. Monitoring shouldn't creep from one into the other.
Monitor
- Adoption rates for approved AI tools
- License and seat utilization
- Sensitive-data events — PII or source code detected in an LLM prompt
- Overall AI spend
- Unapproved tool usage on company networks
- Data volume sent to external LLM endpoints
Leave alone
- Full prompt text from personal accounts
- Personal device activity
- Emotion- or affect-inference from webcams
- Individual AI "productivity scores" meant to rank workers
- Private communications on personal accounts
- Individual browsing history unrelated to AI tools
The first list answers real business questions related to risk, spend, adoption, and compliance.
On the other hand, the second list column answers questions that most organizations don’t really have a legitimate reason to be asking. Monitoring should not creep from the left column into the right.