Definitely an active ban hammer is a must. They really dropped the ball on RDR.
Also, GTA5 was very clunky. You couldn't switch shoulders or roll. I generally want the same mechanics as RDR. Heavy emphasis and reliance on head shots, same shoulder/camera controls, rolling, etc. I know a lot of people liked how GTA 5 worked, but this isn't GTA. Lets stick with the concepts that made RDR great.
Smaller posses. When you can make an entire 8 man team a single posse in PVP, it leads to a lot of problems. People stack the odds and it becomes very unsportsmanlIke. A 4 man posse is big enough.
In the same grain as that, I'd like to see a clan match mode. Ultimately because that sort of thing really jammed up public sessions. People getting kicked to make room for a clan match, other clans not being able to get a sizable posse because the session was full, etc. In my mind a clan match mode is much like BF in way that it is customizable. You can adjust rules, adjust team sizes, weapons, etc. And this mode would use your social club clans and go towards your clan social club leader board. This makes everyone happy all around.
Also, when we get DLC maps, I would like for them to find a way to not divide them. The DLC maps were useless after a while because no one played them. Mix them into the regular rotation like other games do, but make sure the matchmaking works so that people can still find a session.
And I hope to God they don't end up using anything from GTA in that regard, because you could never get into a session in GTA. Eternal cloud loading got old.