I think that LLMs should not be allowed to say "I". It should always be in third person. Instead of "I can write this for you" it should say "This machine can write this for you" or with a store front name "Google can write this for you". To operate on a given text or while generating texts it should divide what is meta from what is direct. This generated text "quote" should be styled different, a bit boring: smaller text and maybe monospace. There should be a clear divide between the machine conversation part and its workable output. If one converses with the machine it should not answer in the first person, because it is not a person.
Of course it wouldn't be bullet proof, but it would help in general to not let people personify the machine. Just a step into a better thing. At the same time it should be relatively easy to replace unquoted "I" and "me" with "This machine". At least it should be easier to find where it falls off the rails.
They don't work well if you do that: https://arxiv.org/html/2509.16332v1 - they apparently require a sense of self to preform the way they do - You'd have to do it deterministically on the front end I suspect.
2 problems of many are:
context windows + compacting + whatever they do behind the scene to stitch cohesive narrative over time - single LLM convos should just never be allowed to get that long, you can effectively build whatever you want as a personality, people return to the same convo a lot, It gets wild fast.
Cross Section memory is a bad feature as implemented in many chat bots, the memory feature can effectively poison any conversation. Anthropic figured this out, their memory feature is a search not a memory that informs the personality/manipulates the current session.
Of course it wouldn't be bullet proof, but it would help in general to not let people personify the machine. Just a step into a better thing. At the same time it should be relatively easy to replace unquoted "I" and "me" with "This machine". At least it should be easier to find where it falls off the rails.