How I work

How I run CRO

CRO gets a bad name when it turns into testing button colors because someone in a meeting had a hunch. At Quantum Fiber I’ve moved the program to the other end: every test has to say where the idea came from, who it’s for, and why it should change what they do.

From hunches to a program

Quantum Fiber had no CRO function when I arrived. I built it from scratch, hired and led a four-person team of UX designers and analysts, and set the testing framework. Then I wrote down how the program should grow up, from one-off tests to research-driven experimentation, and took that to the leadership team. The framework below is what came out of it.

Every idea names its evidence

We tag every test idea to one of eleven research pillars, so we can see what kind of evidence it rests on and notice when we’re leaning too hard on one source:

  • Technical analysis
  • Heuristic analysis
  • Digital analytics
  • Behavioral analysis
  • Voice of customer
  • User testing
  • Competitive and market research
  • Product and merchandising data
  • Campaign and channel insights
  • Exploratory and stakeholder input
  • Integrated and hybrid insights

Behavioral analysis is where a lot of the value hides. I review the on-page errors customers hit in Quantum Metric as a regular practice, and it feeds a steady list of fixes to development. The biggest so far was a recurring error sequence worth $6M in lost customer lifetime value.

Every hypothesis names a principle

A test should change how someone decides, so each hypothesis names the behavioral principle it’s betting on. We work from 49 principles in six groups: cognitive ease, motivation and risk, social influence, value and loss, attention and perception, and choices and memory. Naming the principle makes a result teach us something, win or lose, and it keeps the backlog from filling up with generic advice.

The same thinking explains some of my older wins. Shane Co. renamed “diamond solitaire earrings” to “diamond studs” because that’s what customers call them, and sales rose 40%. The product page redesign broke each jewelry choice into steps, required ones first, and conversion rose more than a point on a base of about 4.5%.

Every test is for someone

We write tests for a specific customer and the decision in front of them. Someone comparing their current provider’s bill against ours needs different proof than someone who has to get internet set up before a move date. Personas built from research keep a test pointed at a real person’s question.

A backlog anyone can read

Every idea in the backlog carries the same fields: the problem, the hypothesis, the recommended change, the primary metric, the research pillar and the psychological principle. Anyone in the business can pick one up and see why it’s there and how we’ll know if it worked.

Measure past the click

A conversion rate can go up while the business gets worse. On the subscription signup funnel we measure lead quality, activation and revenue per visitor alongside conversion, and review them every week, so a win has to hold up after the sale.

AI built on the framework

I built a Copilot agent on top of this framework: the principles, the research pillars and the leadership deck. It reviews a page experience, looks at what past experiments did, and generates test ideas driven by a named principle, then helps prioritize them. Building AI into the testing lifecycle this way took test velocity up 40% and conversion up 50% year over year, with no added headcount. More on how I use AI.