Vanity Metrics vs Decision Metrics

A vanity metric is any number that makes you feel good without telling you what to do. A decision metric is a number that, when it moves, changes your next action. That is the entire distinction, and it is worth more than any list of "good" and "bad" metrics, because the same number can be either one depending on what you do with it. A million video views is the classic example: impressive, satisfying, and completely hollow if it leads to no visits, no sign-ups, no sales. It feels like success. It informs nothing. The skill is not memorising which metrics are vanity. It is learning to ask, of any number in front of you, "if this changed, would I do anything differently?"

This is the companion to the marketing dashboard piece in the performance marketing pillar, and where that one is about building the dashboard, this is about the single judgment that should govern every number you report, track, or celebrate. I will give you the test that separates the two kinds of metric, the nuance that "vanity" is about context not the metric itself, the uncomfortable truth that even your favourite performance numbers can be vanity in disguise, and the three specific traps that dress up vanity as insight. No jargon, just a way of thinking about numbers that will save you from a lot of confident bad decisions.

The test: would you act differently?

The whole distinction collapses into one question you can ask of any metric: if this number moved significantly, up or down, would I do something different? If the honest answer is no, it is a vanity metric, however impressive it looks. If the answer is yes, and you can say what you would do, it is a decision metric, and it has earned your attention.

Run a few through it. Your follower count jumps 20%, would you change anything about how you run the business? For most e-commerce stores, no, so it is vanity. Your cost-per-acquisition jumps 20%, now you are pausing campaigns, checking what changed, shifting budget. That is a decision metric. Total impressions doubled, and? Probably nothing. Conversion rate dropped a point, and you are auditing the funnel. The pattern is clear: vanity metrics are usually totals and counts that go up and to the right and feel like progress, while decision metrics are usually rates, costs, and ratios that tell you something is working or breaking and point at what to do about it.

The reason this matters so much is that vanity metrics are not neutral. They actively crowd out judgment. A report full of rising vanity numbers creates a feeling of success that makes it harder to notice the decision metrics quietly going the wrong way. You can have record traffic, record impressions, record followers, and a business that is slowly dying on conversion rate and acquisition cost, and the vanity numbers will reassure you right up until the cash runs out. Feeling informed is not the same as being informed, and vanity metrics are very good at the first and useless at the second.

The nuance: vanity is about context, not the metric

Here is where most "vanity metrics" advice goes wrong: it hands you a fixed list of bad metrics to avoid, as if a number were vanity by its nature. It is not. The same metric can be vanity in one context and a genuine decision metric in another, and treating the lists as gospel will make you ignore numbers that actually matter for your specific business.

Follower count is the favourite punching bag, and for most e-commerce stores it deserves it, more followers rarely changes what you do. But for a creator whose business is their audience, or a brand whose sponsorship deals are priced on reach, follower count is a direct decision metric. Email list size looks like vanity until you are a business whose revenue scales directly with list size and you would invest differently as it grows. Even page views, the textbook vanity metric, can be a real signal for a content business monetising through advertising. The metric is not the point. What the number does in your specific business is the point.

So the practical move is not to download someone's list of vanity metrics and ban them. It is to look at each number you report and ask the test honestly for your model: does a change in this drive an action in my business? Sometimes the answer surprises you, a number you dismissed as vanity turns out to trigger real decisions, or a number you have been proudly reporting turns out to change nothing you do. The discipline is contextual, not a checklist, which is harder but actually correct.

The uncomfortable part: even ROAS can be a vanity metric

Now the bit that stings, because it applies to numbers you probably trust. Even a "performance" metric, one that looks rigorous and revenue-linked, can function as a vanity metric if you act on it without understanding what it hides. The clearest example is return on ad spend. ROAS feels like the opposite of vanity: it is about money, it drives budget decisions, it sits at the heart of every ad dashboard. But a high ROAS can be hollow in exactly the way a million views is, it can look like success while the underlying business loses money, because ROAS counts revenue not profit and credits sales the ads did not actually cause. A team optimising hard toward a ROAS number, feeling rigorous and data-driven the whole time, can be making the business worse. That is vanity wearing a lab coat.

This is the deeper version of the lesson, and it is the thread running through this whole pillar: a metric becomes vanity the moment you trust it without understanding what it actually measures. Views are obvious vanity because everyone knows they are shallow. ROAS is dangerous vanity precisely because it looks deep. The same goes for any number you have stopped interrogating, attribution-credited conversions you take at face value (when attribution does not prove causation), platform-reported sales you sum without deduplicating, analytics revenue you treat as your real revenue. The protection is the same in every case: know what the number counts, know what it leaves out, and tie it to an action. A decision metric you have stopped thinking critically about is just a vanity metric you have not noticed yet.

Pull it together and the whole thing is one habit of mind. Of every number you report or celebrate, ask: would a change in this make me act, and do I actually understand what it measures? If a number passes both, it earns its place and your trust. If it fails either, it is decoration at best and a confident liar at worst. Vanity metrics are not the harmless numbers people think they are, ignored in the corner of a report. They are the numbers that make you feel successful while the ones that matter slip. The job is not to collect more numbers. It is to know which ones, when they move, should move you, and to keep interrogating even the ones you trust, because the most dangerous vanity metric is always the one you have decided is safe.

A few common questions

What is a vanity metric? A vanity metric is any number that makes you feel good without telling you what to do. The classic example is a large view or follower count, impressive and satisfying, but if it leads to no visits, sign-ups, or sales, it informs no decision. Vanity metrics are usually totals and counts that go up and feel like progress, as opposed to rates, costs, and ratios that signal whether something is actually working.

What's the difference between a vanity metric and a decision metric? A decision metric is one that, when it moves, changes your next action, and you can say what that action is. The test for any number is: "if this changed significantly, would I do anything differently?" If no, it's vanity (e.g. impressions, followers for most stores); if yes, it's a decision metric (e.g. conversion rate, cost-per-acquisition, LTV:CAC ratio). The danger of vanity metrics isn't that they're useless, it's that they crowd out judgment, creating a feeling of success while the metrics that matter quietly go wrong.

Is follower count always a vanity metric? No, vanity is about context, not the metric itself. For most e-commerce stores, follower count rarely changes what you do, so it's vanity. But for a creator whose business is their audience, or a brand whose sponsorship deals are priced on reach, follower count is a genuine decision metric. The same applies to email list size, page views, and other "classic vanity" numbers, they can be real signals in the right business model. Don't ban a fixed list of metrics; ask the test for your specific business.

Can ROAS be a vanity metric? Yes, and that's what makes it dangerous. ROAS looks like the opposite of vanity, it's revenue-linked and drives budget decisions, but a high ROAS can be hollow if you act on it without understanding what it hides: it counts revenue not profit and credits sales the ads didn't necessarily cause, so a team optimising toward it can make the business worse while feeling rigorous. The deeper rule: any metric becomes vanity the moment you trust it without understanding what it actually measures. Views are obvious vanity; ROAS is dangerous vanity, because it looks deep.