May 27, 2025

Case Study: How Quvy Outperformed ChatGPT in Predicting Winning Instagram Ads

Case Study: How Quvy Outperformed ChatGPT in Predicting Winning Instagram Ads

Why General AI Isn’t Enough for Smarter Ad Testing

You can use ChatGPT to help write ads. It’s fast, creative, and good at coming up with variations. But when it comes to predicting which ad will perform best, that’s a different story.

We tested ChatGPT against Quvy, the AI we built specifically to predict ad outcomes. The results weren’t even close. And before we dive into the data, let’s look at two key reasons why:

1. Randomness vs. Consistency

ChatGPT is stochastic. That means if you ask it the same question multiple times, it might give you a different answer each time. That’s fine for brainstorming. But if you’re trying to make data-driven decisions, randomness is a problem.

Quvy is deterministic. Same input, same output. Every time. This level of consistency is essential for running fair, reliable tests on your ad creatives.

2. Accuracy You Can Measure

We ran a benchmark comparing both tools’ predictions to actual ad performance in the real world. ChatGPT’s Spearman score was 0.34, a weak correlation with actual outcomes. Quvy scored 0.78, strong alignment with what actually happened.

That’s not just a win. It’s proof that Quvy doesn’t just guess, it knows.

The Problem

Marketers today are under pressure to launch ads that work, not just look good on paper. Tools like ChatGPT help you come up with multiple versions fast, but they can’t tell you which one is going to convert. And every wrong guess costs you money.

That’s why we built Quvy with radical candor in mind. While ChatGPT says everything looks “promising,” Quvy cuts through the noise and gives you clarity.

The Head-to-Head Test

To prove the point, we ran a direct comparison between Quvy and ChatGPT using a real ad campaign for a mobile game.

Test Setup:

  • Goal: Predict which Instagram ads would perform best, before going live
  • Ad Set: 10 video ads created for the same mobile game
  • Tools Tested: ChatGPT vs. Quvy

Step 1: Using ChatGPT

We asked ChatGPT to analyze the 10 ads. It gave feedback on tone, visuals, emotional appeal, and CTA strength. The responses were helpful from a creative brainstorming perspective.

But here’s the issue:
It didn’t simulate user behavior. It didn’t rank the ads based on outcomes. And it gave every ad some version of “this looks good.”

Step 2: Using Quvy

Then we ran the same 10 ads through Quvy.

Quvy simulated over 10,000 impressions per ad using its predictive models, trained on real-world ad performance data, historical trends, and account-level patterns. It didn’t just comment on the ads. It ranked them by predicted performance with CTR estimates for each one.

Here’s what Quvy predicted:

  1. Fire Knight – 2.12% CTR
  2. Open World – 2.09% CTR
  3. Game On – 1.56% CTR

These winners were clear standouts, with Fire Knight barely edging out the second-place contender.

The Real-World Test

We launched those same ads live on Instagram, same budget, same audience, same conditions.

🔥 The result? The top 3 winning ads were the exact same, in the exact same order.

Real-world CTRs:

  • Fire Knight had the highest CTR on Instagram at 1.67%
  • Open World followed at 1.56%
  • Game On came in third with 1.42%

That’s not just intuition. That’s performance prediction at scale, and radical candor in action.

Why This Matters

You’ve got infinite ad ideas at your fingertips, especially with tools like ChatGPT.

But your budget is finite.

That means you need to choose wisely. Every dollar you spend on a low-performing ad is money you could’ve spent better elsewhere. GPT will tell you your ad looks great. Quvy will tell you if it’s going to work.

Accuracy That’s Predictable, Not Random

Where ChatGPT’s feedback can vary from one prompt to the next (even with the same input), Quvy is deterministic. Run the same ad through it ten times, and you’ll get the same result every time.

That consistency is a huge advantage for A/B testing, optimization, and making data-backed decisions when real money’s on the line.

(Note: Quvy’s core scoring model is deterministic. Components involving targeting or real-time simulation may include some randomness.)

The Takeaway

Where ChatGPT gave ideas, Quvy gave answers.

And when the predictions matched real-world performance? That sealed it.

With Quvy, you can:
✅ Eliminate guesswork
✅ Prioritize high-performing creatives
✅ Reduce wasted spend
✅ Launch smarter and faster

General AI is great for coming up with ads. Quvy is built to help you pick the right ones.

Run your ads through Quvy and see the results before they go live.

👉 Run a Free Simulation Now!

Stop guessing. Start testing.

Consent Preferences