In my second novel there is a character who, on her first trip to London, cannot afford a guidebook, so instead just uses an old A-Z, choosing to visit those streets that sound enticing: Lamb’s Heart Yard, Golden Square, Old Seacoal Lane – anywhere with a name that suggests that William Blake might once have walked there. She is often disappointed, discovering that however romantic the street name might be, the reality now is less Dickensian gables and cobbled streets and more Costa Coffees and NCPs.

This might seem to be a salutary lesson in “what’s in a name-ism”, based on the realisation that what a place is called is unlikely to have any bearing, at least with the passing of time, on its pleasantness. However, that salutary lesson seems not to have been learnt by me, suggesting as I did last autumn that our family went on holiday to Martha’s Vineyard: a place I knew next to nothing about, but have always thought sounded very nice.

You have to admit, it does. Martha – a lovely name, used in a great song by Tom Waits, and also in another very beautiful song by All About Eve, Martha’s Harbour, which I had always vaguely connected in my mind with Martha’s Vineyard, knowing the one thing I did know about the place, which is that it is an island (and therefore probably has harbours). And then Vineyard, conjuring up images of green, grape-filled vistas, long satisfying harvests, and, of course, wine. Add that to some vague notion that James Taylor and Carly Simon live there, constantly playing their music, and Martha’s Vineyard sounds like paradise.

In a way, I would quite like now to be able to tell you it isn’t at all, to prove that all that sounds glittery is not gold. However, in fact, Martha’s Vineyard is very close to paradise: a strange, slightly Truman Show-ish, ultra-American-posh paradise, but paradise nonetheless.

First things first: Martha’s Vineyard is an island near Nantucket in Massachusetts, off the Cape Cod coast. We flew into Boston, and set off the next day for Cape Cod. Although it’s a great summer destination, the journey down the coast is a good reason to go to Massachusetts in the autumn – I’m going to say the Fall, even though that can present a rather spoiling image of going on holiday with a shouting Mark E. Smith. Go via the 3 rather than the big Freeway 93, because you get to see them properly that way: the colours, that is. I know nothing about trees – I couldn’t tell you the difference between an oak, a poplar and a Christmas – but I know that the leaves they produce in this part of America at this time span the spectrum of the rust rainbow.

Lush maple reds, soft yellows, and brown split into more shades than you would have thought such an ostensibly dull colour capable of – it’s no wonder that Massachusetts inspires its own specific brand of tourists during this period, known locally as “leaf-peepers”. If you get lucky and have bright, cold, clear weather the leaves really put on a show – it’s like a giant visual poem of autumn, using only the palette of decomposition.

To get to Martha’s Vineyard itself, we had to board the Steamship Authority ferry, the only boat that will allow cars on (you have to book in advance). The journey across the Nantucket Sound takes about 45 minutes, but the first impression you have upon alighting in the Vineyard is that you haven’t in fact reached dry land. To get to where we were staying, we turned left out of the harbour, and immediately on to a lengthy, narrow causeway, with the Nantucket Sound on one side and an enormous lagoon on the other.

It’s a good introduction to the remarkable wateriness of Martha’s Vineyard in general, surrounded obviously by sea, but also fissured by the sea all across the face of its small land-mass. All types of wetlands are here, ponds, lagoons, rivers, natural reservoirs, marshes – the island is really a collection of smaller islands linked by bridges, ferries and causeways. In sunny weather it is a place of supreme beauty.

The other element that cements Martha’s Vineyard’s beauty is the architecture. There are no great buildings or modernist museums, but virtually every house corresponds to a movie ideal of a 19th-century American cottage, all clapboard, white-picket fences, cedar-shingle walls, and pristine gardens. We stayed in one of these, Mermaid’s Boathouse, a particularly cute example of a Vineyard house overlooking a lagoon, and from there out to sea. We rented it, through the net, from a very fine American lady called Jill Katz, who has also given the cottage an Interiorslevel of all-white, maritime-influenced decor.

But you could spend your entire time on Martha’s Vineyard entranced by the houses, especially down the road from Mermaid, in the town of Oak Bluffs, a section of which is completely given over to fairy-tale “gingerbread” cottages. These are pink, green, yellow, ornate, absurdly pretty, and real – real people live in them – which is hard to believe as you walk through the back streets of Oak Bluffs: it just seems like a film set, especially when you get to the Tabernacle, an enormous circle of cottages, with a filligreed pagoda in its centre. This was one of the first times I felt that The Truman Show – or perhaps, The Prisoner– element of the island: everything is almost surreally pretty and clean.

Unlike The Truman Show, however, it’s not all the same. The capital, Edgartown, is grander, less cottagey than elsewhere – it contains the island’s museum, and a series of mansions built by wealthy whaling captains in the Greek Revival style. Vineyard Haven is the hub of the island, where the boats come in, and feels the least fairy-tale of the towns, although it’s still ridiculously pretty, and it’s got Martha’s Vineyard’s most famous place to eat, the Black Dog, a great place to go for sea-watching and clam chowder.

Up-island, as it’s called, is the most secretive part, full of private beaches and celebrity enclaves, although it’s worth driving to its Cotswold-like rolling hills, and the fishing town of Menemsha. At the eastern end are the wilds of Chappaquiddick, home to a wildlife refuge, the Mytoi Japanese gardens, endless empty beaches, and of course, the Dyke Bridge where Teddy Kennedy’s car crashed into the water on that fateful night in 1969. It is, I have to say, very Martha’s Vineyard that you won’t find any reference to this scandal anywhere.

I had gone looking for it one day, assuming that it would be marked, but eventually gave up, deciding we should have a family picnic instead. It was only after we walked back to our car that I realised that our picnic spot had been in fact just below Dyke Bridge, and my kids had been munching their way through bags of Cheesy Wotsits about four feet away from where Mary Jo Kopechne drowned.

Ah, well: for me, it all added to the wonderful weirdness of Martha’s Vineyard. So my advice is, overall: if a place sounds really alluring – if it has an exotic or romantic sounding name – it probably is really alluring. Therefore, next stop: Baghdad.

Need to know

David Baddiel and family flew to Boston with Virgin Atlantic (0870 5747747, www.virginatlantic.com) which flies from Heathrow from £425 return.

Ferry: book in advance via www.steamshipauthority.com.

Staying: Mermaid’s Boathouse sleeps six and costs £2,000 a week through www.aerodrome.com/mermaid. Nine Zero (001 617 772 5800, www.ninezero.com) in Boston has rooms from about £190.

Tour operators: Black Tomato (020-7610 9008, www.blacktomato.co.uk) offers sevennight trips including five nights in a cottage sleeping four on Martha’s Vineyard and two nights at Nine Zero in Boston from £1,499pp, including flights and car hire. America As You Like It (020-8742 8299, www.americaasyoulikeit.com) has a week at Colonial Inn, Edgartown, from £948pp. The price includes B&B, car hire and flights.

Reading: New England Insight Guide (£14.99).

Further information: www.massusa.com; www.mvy.com.