Thom Browne, Tom Ford, and Serena Williams were among the Palm Beach contingent