A ship is pulling past the lighthouse on Inchkeith Island, making its way up the dark sliver of the Firth of Forth. To the north-west, the Ochil hills stand black beneath the lucent edges of autumn's pale sky. I am at my desk, at home, in the centre of the capital of Scotland.

Beauty will have something to do with Edinburgh being voted our favourite UK city for the sixth year in a row. Yet there are other lovely cities. And Edinburgh has never been particularly fawning to its guests. Remember the scene in Irvine Welsh's Trainspotting when some poor festival-goer gets mugged by Renton and Begbie? This is an improvement. In the 18th century, the city's ladies used a nasty local law to trap wealthy English lads into marrying them; the poor man only had to joke about how good a wife a woman might make before a pastor could, and would, be called.

Yet, even then, Edinburgh was seen as a superb destination. 'What a wonderful city Edinburgh is,' cried Coleridge, 'striking you with a sort of bastard Sublimity from the enormity and infinity of its littleness ...' Given that the contents of bedpans were still being chucked from the windows at the time, this explosion of love suggests the city's broader reach.

My guess is that Observer readers have good taste, and see the way Edinburgh reflects our nature within nature. I grew up in the Scottish Highlands but moved to Edinburgh after a stint in south London. It was 1991, and my first experience of the city was poetic, if discouraging. The haar was in, that cold fog that rises so thick from the Forth that, given a hammer, you could beat the sound of doom on its skin. I walked out to get a paper, and out of the mist a funeral parlour formed, an old man weeping on its steps.

Such darkness is inherent in Edinburgh's claim as the 'Athens of the North'. North, according to the wonderful Bill Duncan, author of the Wee Book of Calvin, is something we should love, along with 'black clothes, silence, November, and loud solemn whistling employing a tremulous vibrato'.

Back in the Nineties, Edinburgh was a city that suited a youth with pretentious and artistic aspirations. Those tremendous Georgian tenement flats could be rented for a pittance. Life passed in rooms with 15-foot high ceilings and exquisite cornicing.

I discovered the Baillie Bar, and Old Reekie could have been the St Petersburg of Crime and Punishment. 'Two drunks came walking out the door and, supporting and cursing each other, climbed up to the street.' The beautiful city has always had a seamier side. It is still one of the few places where no problems face those wanting a drink at 3am.

Over the years, the city has changed; improving, while losing little of its grit. The food that always came in from the farmlands of Angus and Perthshire or the black-edged seas around Scotland's west coast is now turned into fabulous meals by Michelin-starred chefs like Martin Wishart. Also, in the last few years, rooftop bars have opened at Oloroso, the Tower and Harvey Nichols. They take advantage of the city's heights, where ornate wind-vanes tell of our history, of plague, religion and, in the youth carrying the flame of knowledge above Robert Adams's Old College, the thirst for education.

The youth holds the light in the darkness in this city that the poet Kathleen Jamie calls, 'a place of wind and Northern sunlight'. In the 19th century Boswell staggered drunkenly into the dark closes, but he would always emerge to write. These days Alexander McCall Smith divides his time between laughing, writing bestselling novels, playing the bassoon and being a world authority on the law and medical ethics.

And Edinburgh's modern thinkers are approachable. This is a city with liberal instincts. Attempts have been made to make Edinburgh exclusive - a couple of private members' clubs and some silly secret societies - but for the most part such ill-charmed circles are eschewed. The openness is annually reiterated through the city's famous festivals.

If I happen across a lost-looking visitor while walking, I find they are often merely exploring, enjoying the calm beauty of a city where the lights burn warmly behind neo-classical and medieval facades.

Here a new type of city is growing, one that chooses not to expand into vast metropolises like London or Los Angeles but instead remains compact, while aspiring to all the sophistication of larger places.

An open-top tour bus is passing beneath my window. 'This is Stockbridge,' I hear the guide say. 'Edinburgh's bohemian quarter.' I laugh, but, why not?

Soon it will be winter, and I will stand at my window, gazing down at the serpent's curve of the road below, the cobbles slick and black in the cold. Beyond will lie the Botanic Gardens, with the lights of the contemporary art gallery in Inverleith House burning in the clear air, while out in the Forth, the lighthouse on Inchkeith Island will wink at me. Edinburgh: a number one destination, and not a bad city to return to either.

· Ruaridh Nicoll's latest novel Wide Eyed is published by Black Swan