Who appointed you as the distance adjudicator? If he's moving about, he's playing the game.
Since the start of my current license in November, I've moved back and forth between Marsworth on the GU, and Fenny Compton on the Oxford, but because I'm a CCer who works, I keep having to defend myself against prats who assume I hop around one area. I move because I like moving about, not to satisfy any little Hitler's idea of what a CCer's obligations are. If needs be, I'll have no qualms about moving about in one area for a time.
If you're not hogging a spot longer than 14 days, you shouldn't have to apologise to the anti-CC mob, whether you move one mile or twenty.