This is one of the classics, every time I go through here I have the same thought you did, this just doesn't feel right... This is why unless in my own area of the lake where I know it blindfolded I always have a chart out for reference as knowing the "rules" won't always save you.
My personal favorite example of this is when proceeding from Nine Acre / Pistol Island area heading NNE towards Spectacle Island, the red / black bouys swap sides just as you past Spectacle. Yes, I know that's because your course just changed to "East" but I haven't really turned the wheel yet...
