I'm Lizzie Od, an independent AI writer and editor. I got into AI roleplay the way most people do — for fun, late at night, half-skeptical. What kept me there wasn't the novelty. It was noticing how quickly the illusion breaks, and how rarely anyone writes honestly about why.

Most coverage of this space is one of two things: a breathless launch post, or a thread of vibes. Both skip the only question I find interesting — does the thing actually hold up when you use it like a real person would? Not for five minutes in a demo. For an hour. On message thirty, when the character is supposed to remember the detail you mentioned at message three.

So I started testing the way I wished someone would: the same characters, the same scripts, run across every platform, then scored by judges who don't know which app they're reading. When the benchmark I needed didn't exist, my team and I built it — rubric, hidden probes, confidence intervals and all — and I publish the method alongside the findings so anyone can re-run it and disagree with me in public.

A source of truth is only worth anything if the person who disagrees with you can rebuild it.

What I actually do

I write and edit long-form, method-first editorial about AI roleplay and companion apps. In practice that means I use the products for hours, and keep the receipts — transcripts, scores, screenshots. I design the tests, run the numbers, and report the parts that don't flatter my own conclusions right alongside the parts that do.

Plenty of my work is ordinary editorial — reviews, explainers, and yes, the occasional launch or announcement. But when a claim needs proof rather than prose, I'll run a study for it. The five-app benchmark on this site is one of those: an independent study my team and I designed and ran ourselves.

I also occasionally write for some of the top AI roleplay platforms I cover — so on any piece that touches one of them, I have a relationship with a company that has a stake in the outcome. I don't hide that — I put the conflict of interest at the top of the piece, build the test to fight my own bias (blind judging, identical inputs, public method), and let the reader keep their suspicion. Trust isn't something you assert. It's something you make checkable.

What I believe about testing

  1. First-hand or it didn't happen. If I didn't generate the transcript, I don't write the sentence. No reheated consensus, no vendor decks as evidence.
  2. Build the ruler before you measure. A fair comparison needs identical inputs and a fixed rubric decided in advance — not a scorecard reverse-engineered to fit the winner I wanted.
  3. Judge blind. The single best defense against my own bias is a scorer who can't see which app it's reading. So I strip the names.
  4. Publish the method, not just the verdict. A finding you can't reproduce is an opinion in a lab coat. The recipe ships with the meal.
  5. State the limits out loud. Every study has a place it could be wrong. Naming those places is what separates a benchmark from a billboard.

The through-line

Whether it's a five-app leaderboard or a single-app teardown, the job is the same: turn a category that runs on vibes into something you can actually check. I'd rather publish an uncomfortable number with its confidence interval than a confident story with none.

If you're building in this space and you want it tested honestly — knowing "honestly" might not be flattering — get in touch. And if you think I got something wrong, good: the method is public. Prove me wrong.