The instinct to stay home in times of trouble is a powerful one, and suddenly decamping with the children to some sleepy rural backwater in this country rather than venturing into the big bad world abroad seems very attractive.

With children aged nine, five and three, we usually try to find an affordable holiday by renting a house in the middle of a field, as far away from irritable neighbours and troublesome traffic as possible, but still within reach of the sea. But the eldest is now beginning to rebel against total seclusion and so we decided to try staying in a small town instead.

Rye, perched on top of a hill, with its steep, cobbled lanes radiating from the beautiful Norman church at its centre, is an architectural historian's fantasy. It is also every child's dream, being within sight of Camber, one of the best sandcastle and dune beaches in Britain, and home port to Captain Pugwash.

So it was that we found ourselves scrambling to leave London in the weekend rush-hour. You know family holidays are meant to be relaxing, it's just that at the point of departure the empirical evidence is all against. Despite previous resolutions, you didn't take a day off to pack, you are loading the car hours later than intended in the driving rain, the children are playing up, and you are hoping to get round the metropolis and out to Sussex on a Friday night. Have you taken leave of your senses?

The drive though the sad sprawl of suburban high streets finally gives way to the Kent Weald and then over the South Downs to the East Sussex coast. This is not exactly dramatic country, with its ribbon development and dank Dickensian marshes, but when you finally enter Rye, you can suspend all modern disbelief.

The town became one of the Cinque ports in the late 12th century - it was free from royal taxes and in return provided the King with ships and seamen to keep watch against France. Its prosperity, based on shipbuilding and fishing, grew over the centuries, until the harbour silted up and the sea retreated, leaving it stranded on its bluff. Today, it is a perfectly preserved small town, with twisting medieval alleys, timbered Tudor houses and red-brick Georgian residences.

We stayed in Mermaid Street, one of the town's most picturesque lanes, crowded with houses from every period between the 15th and late 18th centuries. Our billet was Christmas Cottage, an exquisite doll's house of an 18th-century cottage right in the centre, whose scale had a miraculous effect on the children and turned them, temporarily, in to paragons of neatness.

The first treat was to venture out under a clear sky and full moon for some serious snooping. Curtains seem in short supply and because cars have been banished from many of the narrow streets, it is possible to gaze at leisure into any number of fine houses. Up Mermaid Street and along West Street to peer into Jane Austen drawing rooms full of elegant antiques. Past the Tudor parlours of the Mermaid Inn, once graced by Elizabeth I, where pirates used to plot by its roaring fire. On past the designer kitchen with its African grey parrot and into Church Square, where ancient flying buttresses support the distinctly drunken walls of St Mary's Church.

Sneak a look into the medieval hall with its galleries hung with animal skins, and wander back past half-timbered houses down Watchbell Street to the Lookout. From here on a fine day you can see nearby Winchelsea, rebuilt on a hill in the late 13th century after storms had breached the old harbour's shingle banks and washed it away, and Camber Castle, with its 16th-century towers set against the marauding French.

The Heritage Centre, run by English Tourism, has a beautiful scale model of Rye, a labour of love completed over several years by a local couple. The 15-minute light-and-sound show takes you on an excellent scamper through the town's history and is a good place to start. You can also climb the tower of St Mary's Church, god willing and weather permitting, to get a stunning view of the whole town. The children, of course, were not impressed with our relentlessly educational efforts to explore the neighbourhood, and demanded a trip to the beach.

Camber Sands is a glorious two-mile sweep of beach, a quarter-of-a-mile wide at low tide, and backed by spectacular dunes. Climb these and look the wrong way and you'll see the flat landscape of Romney Marsh pock-marked by pylons and Pontins. But turn your eyes to sea, and you can drink in the big skies and limpid light, with the shingle ridges of Rye Harbour at one end and the shadow of Dungeness at the other.

We first saw Camber Sands on a day of scudding black cloud and churning brown sea. The pebble beach at Broomhill along the bay was blasted by the wind and the currents sucked away at the shore. Easy to see why the coast had moved so much over the centuries. This was great beachcombing weather and we went home with a happy trawl of whelks and crabs, mermaid's purses, old shoes and urchins' shells.

In fact, most of the week was spent on the beach presenting that peculiarly English scene: adults wrapped and huddled against the chill while small, half-naked children chatter away - heads down, bottoms up - filling their buckets with treasures.

We went with a long list of places to visit if the weather was too foul for the beach. The romantic ruin of Bodiam Castle and its moat is nearby and makes a good day trip if combined with Christopher Lloyd's legendary garden at Great Dixter. Garden lovers can also pay homage at Derek Jarman's plot at Dungeness. Herstmonceux Science Centre at Herstmonceux Castle near Pevensey is the former home of the Royal Greenwich observatory and has lots of family activities.

In the event we didn't need them. We were more than occupied with the literary trails around Rye itself. The Pugwash trail at the Rye museum and a visit to Lamb House, home to Henry James and now owned by the National Trust provided happy afternoons of entertainment. And we would not have missed the 1066 experience at Battle for anything. Here you can do a guided tramp round the fields of the Battle of Hastings as either a Norman knight or mad Saxon axeman. These days, there are genteel English Heritage paths with wood chips to take you down from the half-ruined abbey to the woods and back via an imaginatively-created playground. But the determined can still find plenty of mud to wallow in. And lots of ye olde tea shoppes to water at after the fray.

Eating out can be a problem with small children, and we tend to rate places by the quality of their gardens first, their food second. The Bell Inn at Iden combines ace climbing frames with excellent fresh local fish. In Rye itself, the Ypres Castle pub just below the gun garden has a grassy area for children and serious cooking for grown-ups. The Landgate Bistro, just outside the 14th-century town gate, is an adult retreat, but more than lives up to its Good Food Guide reputation. The old tea shop in Winchelsea has heroically tolerant owners and excellent cake.

Way to go

Getting there: English Country Cottages has properties available all over the UK and in Ireland and France. The brochure request line is: 0870 5851111, and the reservations number is 0870 5851155. Christmas Cottage sleeps six and is priced from £359 for a week, or from £180 for a two-night break.

Other attractions: Battle Abbey and the site of the Battle of Hastings are run by English Heritage, open daily, tel: 01424 773792, battletown.co.uk.

Lamb House, former home of Henry James, a National Trust property, is open Weds and Sat afternoons, tel: 01892 890651, nationaltrust.org.uk.

Great Dixter, Northiam, is open Tues-Sun afternoons, tel: 01797 252878.

Further information: Rye Tourist Information on 01797 226696.