Many families of Long Island were effected by the Revolutionary War.  It seems there were few fence sitters in those days.  And at the end of the American Revolution, two-thirds of it's population would leave with the British for Canada and remain loyal to the King.

"When smoldering sparks broke out into flame, the Neck population was hopelessly divided: even families split into opposing camps; The family names of Smith, Townsend... Kissam and Cornwell... were all well represented on both sides. The Dodge, Onderdonck, Schenck and Sands families were overwhelmingly or entirely Whig, while the Hewletts, Motts, Pearsalls, Ludlows, Clowes and Dentons were largely Tory." - Hick's Neck, the Story of Baldwin, Long Island.

It can be noted that all the families listed above as primarily "Tory" are primarily those that are linked with the Carman family name more than any of the others with the exception of the Smiths and Sands (and the Smith's comprise many different families, not all Whig).