Does a style-matched rewrite identify its sender better than a plain rewrite? A preliminary test

This is a preliminary test, run on a small set, and it should be read that way. We asked a narrow question: when a model rewrites an email, can a reader tell whose email it was more often if the rewrite was matched to that sender's style than if it was a plain rewrite? We have a first answer, and good reasons to hold it loosely.

01What we did

We used a rebuilt set of work emails from the public Enron collection, assembled from 10 CSV files. It is not the original sample file we meant to use. We picked 6 senders, labelled Sender A to Sender F here, and held out 3 emails from each, giving 18 planned emails.

Each held-out email was rewritten twice by Gemini 2.5 Flash, reached through Google Cloud Vertex AI. One rewrite was plain. The other was style-matched, meaning the model was shown the sender's other emails first. This is not the model the Sealis product uses, so the test says nothing direct about the product.

A judge was then asked to pick the sender of each rewrite from the six. Guessing would be right 1 time in 6, or 16.7%.

02What we found

4 of the 18 emails were dropped because the style-matched calls failed, which left 14 emails with both versions.

• Style-matched rewrites were attributed to the right sender 9 times out of 14 (64.3%, 95% interval 39 to 84%).
• Plain rewrites were attributed correctly 3 times out of 14 (21.4%, interval 8 to 48%).
• The difference is +42.9 percentage points. Chance is 16.7%.

On six emails the style-matched rewrite was identified and the plain one was not. On none was it the other way round. A sign test on that split gives p = 0.03.

Facts survived equally in both arms: 95.1% of 61 facts were kept in each.

03Why this is only preliminary

The sample is tiny. The intervals are wide, and the plain arm's interval overlaps chance.

The dropped emails matter. If the 4 dropped emails are counted as misses for the style-matched arm, it scores 9 of 18, or 50%.

The model service was unreliable during the run. 26 of 203 model calls returned server errors (502). We retried where we could, but that is how the 4 emails were lost.

One sender carries a lot of the gap. Leaving out Sender A cuts the difference to +27 points. Leaving out any of the other five senders keeps it between +42 and +50.

The data are a rebuilt set, not the original sample file, so we cannot be sure it matches what we meant to test.

04What this does and does not show

It suggests that a style-matched rewrite can keep more of a sender's identity than a plain one, on this set, with this model. It does not show how large the effect is in general. It does not test the Sealis rewrite pipeline, and it says nothing about how readers judge the result.

05Next step

Rerun on the original sample file, with more held-out emails per sender and the server errors handled so that no emails are lost. After that, test the product's own pipeline.