Four prompt changes that did not help

We tried to get better voice matching out of the model Sealis uses (gpt-6-luna) by changing how it is asked, not which model it is. It didn't work. None of the four changes beat the prompt we already use, so we kept the current one. This note records what we tried so nobody, including us, repeats it.

01What we tried

All four ran on the same 12 AI-written drafts and four voices as the second test in the voice transfer report: 24 rewrites per run, the same checks, the same scoring.

• Current prompt. The one in production, as the baseline.
• Lower temperature (0.6 instead of 0.9). Less random wording.
• Notes first. The model first writes short notes on how the person writes, then the rewrite.
• Closest sample re-shown. The saved sample most like the draft is repeated right before the text to rewrite.

02What happened

Share of rewrites classified as the intended voice, and the share that passed every check:

• Current prompt: 46% in the intended voice, 33% passed everything, no facts lost.
• Lower temperature: 42%, 38%, no facts lost.
• Notes first: 46%, 29%, one rewrite lost a fact.
• Closest sample re-shown: 54%, 33%, one rewrite lost a fact.

Every variant kept about 99% of the draft's points and cleared 94 to 97% of the stock phrases, the same as the current prompt.

03How we read it

The closest-sample run looks best on voice (54% against 46%), but 24 rewrites cannot support that gap. On this test, differences under about 8 points are noise. It also lost a fact where the current prompt lost none, and we ship nothing that loses facts. The other three were level with the current prompt or worse.

The honest conclusion is that on this model and this test, rewording the prompt and changing the temperature did not move voice matching. Modern email voices, which are short and plain, stayed the hardest case.

04What we did not do

• We ran the development drafts only. We did not run the hold-out set, because nothing on the development set was worth confirming.
• A fifth run, combining notes first with the closest sample, stopped after 8 rewrites when an outside service rate-limited us. We have no result for it and report none.
• We tested one model and four changes. This is not evidence that no prompt could help.

05What changes

Nothing in the product. The current prompt stays. The limits of the benchmark are on the voice transfer report. If a later idea beats the current version by more than noise without losing a fact, it will go in the product and we will say so here.