In-Ear Insights: Measuring and Improving AI Proficiency

In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss how to measure AI proficiency impact beyond speed. You’ll discover why quality matters more than volume when AI accelerates work. You’ll learn a six‑level framework that lets you map your AI skill growth. You’ll see practical steps to protect your role in fast‑moving companies. 00:00 – Introduction 02:45 – The speed‑only trap 05:30 – Introducing the six‑level AI proficiency model 09:10 – Quality vs quantity in AI output 12:40 – Managing AI access and fairness 16:20 – Actionable steps for managers and individuals 20:00 – Call to action Watch the full episode to level up your AI leadership. Can’t see anything? Watch it on YouTube here . Listen to the audio here: https://traffic.libsyn.com/inearinsights/tipodcast-ai-proficiency-measuring-ai-performance.mp3 Download the MP3 audio here . Need help with your company’s data and analytics? Let us know! Join our free Slack group for marketers interested in analytics! [podcastsponsor] Machine-Generated Transcript What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode. Christopher S. Penn: In this week’s In Ear Insights, let’s talk about AI and the way the things that we are measuring in business to measure AIs, the productivity, the benefits that you’re getting out of it. One of my favorite apps, Katie, is called Blind. This is an anonymous confessions app for the business world where people who work at companies—mostly in big business and big tech—share anonymous confessions. They have to say what company they’re with, but that’s it. There were three posts that really caught my eye over the weekend. The first was from a person who works at Capital One bank who said, “Hi, I’m a junior software engineer.” Three years into my career, my co‑workers are pumping out so many poll requests with Claude code and blitzing through jobs that used to take three to five days in less than an hour. I feel like every day at the office is a race to see who can generate more poll requests and complete them than anyone else. The second one was from JP Morgan Chase saying, “I just downloaded Claude coat and wtf. I don’t know what to think. Either we are cooked or saved.” The third was from an engineer at Tesla who said, “I joined recently as a contractor and don’t have access to Claude. I’m slower than the others on my team and it stresses me out.” So my question to you is this, Katie: Obviously people are using generative AI to move very fast. However, I don’t know if fast is the metric that we should be looking at here, particularly since a lot of people who manage coders don’t necessarily manage them well. They don’t. For example, very famously, Elon Musk, when he took over Twitter, fired people who didn’t write enough code. He measured people’s productivity solely on lines of code written. Anyone who’s actually written code for a living knows you want less code written rather than more because there’s a certain amount of elegance to writing less code. So my question to you is, as we talk about AI proficiency—sort of AI proficiency week here at Trust Insights—what would you tell people who are managing people using AI about measuring their proficiency and measuring the results that they’re getting? Katie Robbert: So first, let me answer your question. No, I do not frequent—was it Blind? Yeah. Anyone who knows me knows that I am honest and direct to a fault. So no, that would annoy me more than anything—just say it to my face. But that aside, I understand why apps like that exist. Not every company builds a culture where an open‑door policy is actually true. The policy is: the door is open only if you have positive things to share; the door is closed if you have complaints. I sympathize with people who feel the need to turn to those kinds of apps to express concern, frustration, fear. It seems, Chris, that a lot of the fear over the past couple

Listen