How to Evaluate Your Meta Ads Agency: The KPIs Ecommerce Brands Must Track

Your Meta ads agency sends you a monthly report. It has nice charts. ROAS looks acceptable. Spend is up. There might even be a slide titled "Key Wins This Month."

But here's what most founders tell us when they actually book a strategy call: I don't know if they're doing a good job.

That's a problem. Not because agencies are inherently bad — but because most monthly reports are designed to look good, not to tell you the truth. And if you don't know which metrics actually signal agency performance versus which ones are just noise, you'll either fire a good agency too early or keep paying a bad one too long.

This guide gives you the 7 KPIs to evaluate your Meta ads agency — and what each one tells you about whether they're actually earning their retainer.

Key Takeaways

→ ROAS alone is not a performance metric — it's an input to a decision, not an answer.
→ The most important agency KPI most brands never track: creative win rate.
→ A good Meta ads agency improves your numbers over time. Flat performance = coasting.
→ Attribution transparency is non-negotiable. If your agency can't explain how they're tracking conversions, nothing else they report is reliable.
→ Across Dash Activate Online's managed ecommerce accounts, the average creative win rate sits at 8% — with top-performing accounts reaching 14–29%. If your agency can't tell you theirs, ask why.

Why Most Agency Reviews Miss the Point

Most brand-side agency reviews focus on one thing: ROAS. If ROAS went up, the agency is good. If it dropped, there's a problem.

That's an incomplete picture — and agencies know it.

ROAS can look great while your business deteriorates. You can manufacture high ROAS by cutting prospecting spend and leaning into retargeting an existing audience. You can inflate ROAS by excluding low-intent campaigns from reporting. You can hold ROAS flat while CPMs climb and your reach shrinks month over month.

The real question isn't "is ROAS up?" It's: are they making decisions that build performance over time? That requires a different set of metrics.

Here are the seven.

KPI 1: Blended ROAS vs. Channel ROAS — Are They Reporting the Right Number?

What it is: Blended ROAS = total revenue ÷ total Meta ad spend. Channel ROAS = what Meta's dashboard reports, which includes view-through attribution and may overlap with other channels.

Why it matters: Meta's in-platform ROAS is always higher than your real business return. It overcounts — especially after iOS privacy changes. An agency that reports only in-platform ROAS without reconciling to your actual revenue is hiding attribution inflation.

What to look for:
→ Your agency should report both numbers and explain the gap.
→ If they only ever show you in-platform ROAS, push for the blended view (total revenue from Shopify/your backend ÷ Meta spend).
→ A widening gap between the two numbers over time = a signal something is off with tracking.

Green flag: Agency proactively flags attribution discrepancies and ties reporting to your backend revenue source.

Red flag: Agency only references Ads Manager ROAS and gets defensive when you ask how it compares to actual revenue.

KPI 2: New Customer CAC — Are They Growing Your Customer Base or Just Retargeting It?

What it is: Customer acquisition cost for new customers only — not retargeting existing buyers or warm audiences.

Why it matters: Retargeting your existing customers is easy and looks great in a dashboard. Prospecting — finding and converting cold audiences who have never heard of you — is where agencies earn their money. If new customer CAC is not being tracked separately, you don't know if growth is actually happening.

What to look for:
→ Your agency should segment reporting between prospecting (cold) and retargeting (warm) campaigns.
→ New customer CAC should be trending toward a number that works with your contribution margins and LTV.
→ If you can't get a clean new-customer CAC out of your agency, ask them to pull it from your backend data (Shopify has this natively).

Green flag: Monthly reporting includes a dedicated new customer CAC line, tracked against your target CAC based on margins and LTV.

Red flag: Agency reports "cost per purchase" without distinguishing new vs. returning customers — or says they can't separate it.

Related: How to Lower CAC on Meta Ads for Ecommerce (Without Cutting Scale)

KPI 3: Creative Win Rate — Are They Actually Improving Your Creative Output?

What it is: The percentage of ad creatives tested that reach winner status — defined as clear performance above your CPA or ROAS threshold over a statistically meaningful spend window.

Why it matters: This is the KPI most brands never ask for — and the one that reveals the most about an agency's actual capability. Creative is the primary lever in Meta ads post-iOS. The algorithm is largely fixed; your creative is what differentiates performance. If your agency is running the same creative for 60+ days without testing, they're not doing their job.

The DAO benchmark: Across Dash Activate Online's active ecommerce accounts, the average creative win rate — total winning creatives divided by total creatives that went live — sits at 8%, with best-performing accounts reaching 14–29%. That means roughly 1 in 12 creatives hits winner status on average, and up to 1 in 4 at the high end. Both numbers are valuable only when an agency is actually running structured tests to generate them.

What to look for:
→ Your agency should be testing a minimum of 4–6 new creative concepts per month per account.
→ They should be tracking win/fail/inconclusive outcomes — not just "running ads."
→ Win rate by itself isn't the full picture. Volume × win rate = winners per month. You want both.

Green flag: Agency maintains a creative testing log, reports win rate by month, and uses winner insights to brief the next round.

Red flag: Agency can't tell you their creative win rate. Or they measure "good performance" by feel rather than a defined threshold.

KPI 4: Creative Velocity — How Many New Concepts Are They Testing Each Month?

What it is: The number of meaningfully different creative concepts (not just variations of the same ad) tested per month.

Why it matters: Creative fatigue is real and it's fast. On Meta, a winning ad typically runs 3–6 weeks before performance degrades — sometimes faster in fashion where audiences are smaller. Without constant creative replenishment, ROAS degrades quietly while the report still looks clean.

What to look for:
→ For accounts spending $10K–$50K/month: 4–8 new creative concepts per month is a reasonable floor.
→ "Variations" (different copy on the same visual) count less than genuinely different concepts (different hook, format, mechanism, or emotional angle).
→ Ask your agency how many net new concepts went live last month, not how many ads.

Green flag: Agency has a structured creative brief process and can show you what was tested, what the hypothesis was, and what they learned.

Red flag: Agency reuses the same 3–4 creatives for months. Or counts every copy test as "a new creative."

Related: How to Scale Meta Ads Budget Profitably for Ecommerce: The Decision Framework

KPI 5: CPP Trend — Is Cost Per Purchase Moving in the Right Direction Over Time?

What it is: Cost per purchase (CPP), tracked month-over-month and adjusted for seasonality.

Why it matters: ROAS is a ratio — it moves with AOV, not just ad efficiency. CPP is cleaner. If you lower your prices or shift product mix, ROAS drops even if the agency is doing everything right. CPP strips that out and shows whether your cost to acquire a buyer is actually improving.

What to look for:
→ Look at a rolling 90-day trend, not month-over-month (too noisy).
→ Separate CPP by campaign type — prospecting CPP and retargeting CPP should be tracked independently.
→ Industry context matters: fashion ecommerce CPPs behave differently from health/supplement CPPs based on AOV and margin structure.

Green flag: Agency tracks CPP trend separately from ROAS and can explain what's driving changes (creative, audience, seasonality, offer).

Red flag: Agency can't explain why CPP moved. Or they only report ROAS without a CPP view.

KPI 6: Frequency and Audience Health — Are They Burning Out Your Market?

What it is: Ad frequency (average number of times a user saw your ad in a given period) plus audience overlap and reach metrics.

Why it matters: High frequency is a silent ROAS killer. When the same people see the same ad too many times, click-through rates fall, CPMs rise, and the algorithm reads declining engagement as a quality signal — which compounds the problem. Agencies running limited creative rotations on small audiences often drive frequency up without flagging it.

Thresholds to watch:
→ Prospecting: Frequency above 3.0 in a 7-day window is worth investigating.
→ Retargeting: Frequency is naturally higher, but above 8–10 in a 30-day window suggests creative exhaustion or an over-narrow audience.
→ Reach should be expanding month-over-month for prospecting campaigns — if reach is flat or shrinking while spend holds, you're running in a compressed pool.

Green flag: Agency proactively monitors frequency and rotates creative before performance degrades rather than after.

Red flag: You've been running the same ad for 60+ days and frequency is climbing. Your agency mentions it only after you ask.

KPI 7: Attribution Transparency — Can They Explain Exactly What They're Tracking?

What it is: A clear, documented explanation of the attribution window being used, what a "conversion" actually counts, and how in-platform results compare to backend data.

Why it matters: Attribution is where most agencies hide underperformance — not through lies, but through opacity. Meta's default attribution settings (7-day click + 1-day view) count conversions that happen naturally, regardless of whether your ad caused them. If your agency has never had a direct conversation with you about attribution settings, you don't know what you're paying for.

What to look for:
→ Ask: "What attribution window are our campaigns running on and why?"
→ Ask: "How does our Meta-reported revenue compare to Shopify revenue in the same period?"
→ Ask: "Is Conversion API (CAPI) live on our account, and how are we verifying signal quality?"

A good agency has clean answers to all three. A mediocre agency gives you a vague explanation about how "Meta reporting works differently."

Green flag: Agency set up CAPI, uses 7-day click attribution for conversion campaigns, and reconciles Meta revenue to Shopify monthly.

Red flag: Agency uses 7-day view attribution. Or they don't know if CAPI is set up. Or they can't explain the gap between Meta conversions and Shopify orders.

Related: How to Brief a Meta Ads Agency: What Ecommerce Brands Must Prepare Before Onboarding

The Agency Accountability Scorecard: Red Flags vs. Green Flags

Use this scorecard in your next agency review. If you're scoring 4+ red flags, you have the data you need to have a direct conversation — or to make a change.

What Good Reporting Actually Looks Like

A good Meta ads agency doesn't just send you data — they send you interpretation.

The difference between a capable agency and an expensive one: narrative. "ROAS was 3.2x this month" tells you nothing. "ROAS was 3.2x, down from 3.7x — CPM increased 18% due to early Q4 auction pressure, we rotated 5 new creatives to counter frequency buildup, and prospecting CPP held flat at $41" tells you they're watching the account.

Good reporting should answer:
→ What happened?
→ Why did it happen?
→ What did we do about it?
→ What's the plan for next month?

If your monthly report doesn't answer those four questions, you're getting a dashboard, not agency management.

Frequently Asked Questions

How often should I evaluate my Meta ads agency?
A formal review every 90 days is a reasonable cadence. Track KPIs monthly but give campaigns enough runway — a single bad month doesn't make a bad agency, and a great month doesn't make a good one. Look for trends, not snapshots.

What ROAS should my Meta ads agency be delivering?
There's no universal number — ROAS depends on your margins, AOV, and attribution setup. A fashion brand with a $60 AOV and 40% margins needs different ROAS floors than a supplement brand at $80 AOV with 65% margins. A good agency will set a ROAS floor specific to your unit economics, not an industry average.

How long should I give a new agency before expecting results?
60–90 days is reasonable for prospecting campaigns to find their footing, provided creative testing is running and attribution is clean. If an agency is still "in learning phase" at month four without improvement, that's not learning — that's stagnation.

What is a creative win rate and what's a good number?
Creative win rate is the percentage of tested ads that exceed your performance threshold (usually CPA or ROAS floor) over a defined spend window. Across Dash Activate Online's managed ecommerce accounts, the agency average is 8%. Best-performing accounts reach 14–29%. The right benchmark for you depends on your testing volume — but the number you need to ask your agency for is their actual tracked rate, not a claim.

Should I use agency ROAS or Shopify revenue to evaluate performance?
Both, but Shopify (or your backend) is the truth. Meta's in-platform reporting overcounts due to attribution overlap and view-through conversions. Always reconcile to backend revenue. A good agency helps you build that view.

The Bottom Line

Evaluating your Meta ads agency isn't about finding reasons to fire them. It's about holding them to the same standard they agreed to when they sold you on the engagement.

The brands that get the most out of agency relationships are the ones who know exactly what to ask — and read the answers clearly.

If you're running through this list and realizing you don't have visibility into most of these metrics, that's worth a direct conversation with your agency. And if that conversation doesn't go well, you now have a framework to know why.

If you want to see how your current setup stacks up — or if you're evaluating agencies and want a second opinion — book a strategy call with Dash Activate Online. No pitch, no fluff. Just a clear read on where you are and what the path forward looks like.

Related reading:
→ When to Hire a Meta Ads Agency for Your Ecommerce Brand
→ How to Brief a Meta Ads Agency: What Ecommerce Brands Must Prepare
→ How to Scale Meta Ads Budget Profitably for Ecommerce