GENE SCHERMERHORN'S New York was 23d Street, unpaved, in the 1840's and 50's, a farm-surrounded neighborhood of Manhattan quite ''far out of town.'' In fact, the Manhattan of Gene Schermerhorn's boyhood, the city of new gaslights, water newly gushing in from the Croton reservoir, of heavy horse-drawn traffic, just about ended at Eighth Street. There was no Central Park uptown, nor a Brooklyn Bridge downtown, and Fifth Avenue was a muddy, rutted stagecoach route. But there was lots to do for a boy of 10 in that New York of 1852, such as running with the volunteer firemen, sleigh riding on Broadway, lassoing loose pigs along Sixth Avenue, gaping at the Crystal Palace, which opened on Fifth Avenue and 42d Street in 1853, fishing in any of the network of ponds around Kip's Bay.

Gene Schermerhorn was one of the long line of New York Schermerhorns, Dutch settlers who arrived in the 1600's, quickly established themselves as members of the mercantile elite and remained a force in the city's financial life for centuries. The family name still graces Schermerhorn Street in Brooklyn, Schermerhorn Hall at Columbia University, Schermerhorn Row at the South Street Seaport.

Gene's letters to his nephew Phil somehow became separated from the Schermerhorn family and were discovered in a thrift shop in the 1970's. Eventually they were purchased by Barbara Cohen, owner of the New York Bound Bookshop, who published them.

In them, Gene recalls having had such a good time as a boy growing up in that youthful, somewhat undefined city of 1850, that, in 1886, in his mid-40's, as his city began to swell, sprawl and change, he felt a need to communicate those happy boyhood memories to his nephew.

''I propose to write for you some things which I can remember about New York when I was a boy; for I think that some day - when you are a little older perhaps, you will like to know something about it,'' he wrote. ''I doubt if you care for such things now, but I will try both to amuse you at present with what I and other boys did and to interest you when you are older and can appreciate the great changes that have taken place. It is not so very long ago and I don't mean to tell you any 'old man's yarns' for I am not quite a grandfather but only Your Uncle Gene.''

And so, looking backward from his perspective of 44 years, Gene Schermerhorn began on Dec. 8, 1886, a series of 10 letters that spanned a little more than two years. They are nostalgic, wonderfully descriptive, straightforward, occasionally rambling letters, always warm and remarkably free of the varnish that occasionally coats youthful memory.

They are filled with the minutiae of a long-gone New York, the sort of detail that even the most meticulous of historians might deem superfluous. Gene Schermerhorn writes to his nephew that a number of country homes in the area which now houses Carl Schurz Park and Gracie Mansion, ''among them your Great Great Grandfather's'' were ''torn down last spring and the site is now covered with stores and flats.'' That these country homes vanished is perhaps not terribly significant to the evolving tableau of New York, but the account is an example of what makes this little book so memorable.

Next door to the Schermerhorn house - a three-story frame version of later brownstones - on 23d Street just west of Sixth Avenue was a garden, and next door to that ''were the stables of the Sixth Avenue Omnibus line; the stages starting here and going down Sixth Ave, Eighth Street and Broadway to the Battery.'' The streets were unpaved and there ''certainly was plenty of room, plenty of dirt (clean dirt) and plenty of boys; what more could be desired!'' Kite flying, marbles and ''something we called Base Ball were the great games.''

Gene was fascinated by buildings and took care to describe them, their grounds laid out in gardens, their ornate iron railings, the stone walls that enclosed these then country houses. He took summer stagecoach rides up Bloomingdale Avenue (which was what Broadway from 23d to 152d Streets was then known as) to the villages of Yorkville and Harlem, and winter rides in horse-drawn sleighs. He spied on the squatters living in the rough and rocky region now known as Central Park and ran breathlessly to fires with the local volunteer engine company.

Gene's letters are truly extraordinary memoirs utterly packed with tiny oddments of history, and his drawings which accompany them, meticulous and fine-lined, full of the smallest detail, underscore his love for these small, but significant, memories.

-- Excerpted from the January 4, 1983 issue of The New York Times' Books Of The Times, review by Fred Ferretti.