Does a rewrite keep a formal voice formal? We tested it, and production did not

A client who writes formally does not want to see their brand voice come back chatty, and a client who writes in lowercase with emoji does not want a polished press release. We had never measured whether Sealis does this. We built a test, found that it mostly did not, changed the pipeline, and measured again. The result is better but not solved, and this note says where it still falls short.

01The test

We invented nine writers, from very formal to very casual: a luxury skincare brand, a clinical skincare brand, a friendly skincare brand, a Gen Z skincare brand, a law firm, a listed company's chief executive, an academic writing her own email, a wealth manager who contracts twice in about 450 words, and a start-up founder. Each has five or six short samples. We wrote eight drafts of the kind AI produces (three full of stock phrases, three plain, two chatty and one short line) and rewrote every draft in every voice: 72 rewrites per run, through the production pipeline on gpt-6-luna.

A second model, gemini-3.6-flash, judged each rewrite blind. It rated formality from 1 to 7, said whether the rewrite was more casual or more formal than the writer's own samples, rated the voice match from 1 to 5, and guessed which of the nine writers had written it (chance is 11%). We checked the judge by giving it each writer's own held-out sample: it called the formal writers' register the same 100% of the time, so it does not see "more casual" where there is none.

02What we found

Production, before the change:

• Formal voices came back more casual 85% of the time. "More casual" includes "slightly"; clearly more casual was 13 of 48 rewrites.
• Casual voices came back more formal 83% of the time.
• Voice match was 2.31 out of 5, and the judge named the right writer 28% of the time.
• Facts kept: 100%. The pipeline was not losing content. It was flattening every voice toward one middling register.

Sentence length showed it. The law firm's sentences shrank from 18.8 to 14.1 words, and the chief executive's from 18.3 to 12.9. The wealth manager's contraction rate went from 0.6 to 3.0 per 100 words. The Gen Z brand was given "so sorry for any inconvenience caused by the delay".

03Why it happened

We read the outputs and the retry notes and found five causes.

• The contraction rule. One contraction anywhere in a writer's samples produced "use contractions wherever natural". The wealth manager's "We are holding" came back "We're holding".
• The meaning check treated tone as content. It flagged a dropped "tirelessly" or "sincerely", and the retry it triggered pulled the draft's own words back in.
• The digest cut content. Reasons for a price rise and calls to action were deleted, and because the meaning check compares against the digest, the loss was never seen.
• The stock-draft voice pass had no measured rules for the writer's habits.
• Edits kept the draft's sentences. On plain and chatty drafts the shape of the draft leaked into the rewrite.

04What we changed

• Every draft is written from its points once there are 80 or more words of samples, with the points grouped by paragraph so an email's layout survives.
• Measured register rules go into the prompts: how often the writer contracts, exclamation marks and emoji, British or American spelling, sentence length.
• The meaning check now tells tone from content. On 20 real content errors it caught all 20. On 14 rewrites that changed tone only, it flagged 9 before and 2 after.
• The digest keeps reasons and calls to action.
• For a writer who never contracts, exclaims or uses emoji, drift is also repaired in code afterwards, and guarded so it can never cost a fact.

05What it did

On the same 72 rewrites, before and after:

• Formal voices made more casual: 85% to 75%.
• Casual voices made more formal: 83% to 75%.
• Voice match (1 to 5): 2.31 to 2.68. Run-to-run noise is about ±0.1.
• Right writer named: 28% to 39%.
• Facts kept: 100% before and after. A separate 12-email fact test lost 0 of 122 facts.
• Cost: about 3.4 calls per rewrite, up from 3.28.

Against a second, different judge (deepseek-v4-pro) the direction held, but the voice match gain was smaller and close to noise (2.24 to 2.42). The right-writer gain held on both judges. In a blind A/B against the old output, the new one was preferred 32 to 21 by one judge and 22 to 16 by the other, with the rest split.

Here is one apology, in the Gen Z brand's voice. Before:

so sorry for any inconvenience caused by the delay to your recent order. we know how frustrating this must be, and our team is working hard to sort it as quickly as possible.

After:

ok, so sorry about the delay with your order 😭 we know waiting is frustrating, and our team's working to sort it as quickly as possible.

Both keep the order number, the date and the discount code. Only the second sounds like the brand.

06What it did not fix

• Formal voices still come back a notch plainer. The judge still called 75% of formal rewrites slightly more casual. In the delay apology, two of the formal voices read almost the same before and after: they were already close, and the change moved little. Part of this is the topic: a sale reminder from a law firm cannot be as formal as a legal opinion. But the judge named customer-service phrasing ("We are sorry", "discount code") where the writer would put the same point in their own terms.
• Three things we tried did not close it. An instruction to say each point in the writer's own terms made no difference. A voice check that reads the rewrite beside the samples and retries was not clearly better for one to one and a half more calls a rewrite, so it is built but switched off. Turning model reasoning on measured no better, at about twice the time.
• Old letters. On the earlier benchmark of period letters, the change made no measurable difference to voice distance or the right author, which is what we wanted: it should not regress there.

07Limits

• The voices are invented. We wrote them, and they are very different from each other, which makes the test easier than real life.
• The judge is a model. No human has rated these rewrites. We used two judges from different families and they agreed on direction, not size.
• 72 rewrites per run. Differences under about two standard errors should be read as noise.
• One writing model. Everything ran on gpt-6-luna.
• A real client's voice may behave differently. That is the next test.