Port Royal Golf and Racquet Club, Barony