But it's conceivable it could be related to you. "I'm sorry, this content is not viewable by non-<insert group of choice>" or "You are on the Anonymous watch list". That makes it a client error.
In the end, many times the response codes are murky. I'm very comfortable with it being a 4xx; you'd put it somewhere else. The working group will hash it out and we'll use what they say. shrug
400 is about an error the client may be able to do something about (eg log in, move to a new country).
500 is about something going wrong on the server that you have no control over at all.
This belongs in th 400 range and, as such, using the symbolic 451 is not only OK, it is a great idea.