I beg to differ re: extending the tubes. What you say is true to some extent, but only when they are the sole access between two points.
The ideal arrangement is found on the Met line, where the trains stop for a few stations then run fast. The next best thing to this is to have interchanges between the slow tubes and faster services, as the Jubilee line has at Wembley Park and Finchley Road (West Hampstead would be better, but I digress).
So re: the Waterloo & City, Chessington passengers would only be travelling on the tube stock to Raynes Park, then they'd be on fast services to Waterloo, which could be increased as they wouldn't be stopping at the intermediate stations to Clapham Junction, nor those that could then be added up to Waterloo. If they wanted Bank, they could change back to the W&C at Waterloo - it'd still be quicker than staying on the tube, but for those wanting intermediate stations, the service will be much better.
You're missing the point:
nobody wants to go to Waterloo. It's nowhere near the City. So what you're suggesting is a downgraded user experience where what was once just one train + change at Waterloo becomes two
or more changes.
(And, to be honest, the South West Main Line actually has some decent infrastructure and isn't really struggling all that much. Its only real pinch-point is Waterloo itself. This line is
platinum compared to the ex-SE&CR network, which barely rates a "slightly polished turd".)
The tubes are essentially high-capacity express buses that run underground. They're not designed to be comfortable for long-distance travel, hence the longitudinal seating for capacity.
Explain the "A" Stock, then. And why do most buses in London have so much transverse seating instead of the longitudinal variety seen on the new Overground stock?
Yes, trains solve similar problems to buses, but so do planes. It's the scale of the journey which define which solution fits best. Buses are primarily for short journeys of only a few miles. Trams and light rail cater for slightly longer journeys—providing an "express bus" option. Metros are designed for even longer journeys. Inter-city / HSR caters for journeys that are longer still... and so on up the scale.
Or so the theory goes. In reality, if you live near the terminus of a particular mode of transport, it's not uncommon to see people getting on there, making themselves comfortable and just staying on the vehicle all the way in to their place of work. Sure, it might take longer, but
convenience also counts for something.
Unfortunately, the UK's rail networks were built in an era when
nobody really understood how people would use the new technology, so it's no great surprise that exceptions and oddities abound.
Four-tracking the Catford Loop or the Chatham main line via Kent House isn't a trivial engineering project. And there's no reason to do so anyway: building new tunnels
with very few intermediate stations is arguably a much better option for south London's rail networks than any amount of buggering about on the surface. Fast services from Victoria to Kent could run fast, underground, all the way to the M25, where four-tracking a line running through open countryside is cheaper. One or two intermediate stations, at strategic interchanges can be built along these new tunnels, but it's difficult to justify having more than one per tunnel. Lewisham might be justifiable; Bromley another. The tunnel would end before, or just after, the termini of suburban metro line(s), where there'd be another interchange. From that point on, the fast lines return to the original alignment—though some of that could do with having a bunch of kinks ironed out.
This releases a lot of capacity for metro services, but still leaves the question of where the fast tunnels end. Do they dead-end at stations like Victoria and Charing Cross? Or do they continue through London to add more RER-style cross-London connections? Should such connections link to express services, or metros? The devil is in the details.
The comfortable journey length is probably about 5-30 minutes, and whilst a bus might get you x miles, a tube will get you 2.5-3x miles in the same time, so as long as your interchange to fast rail or your destination is within that, tubes are fine out as far as you like. Clearly, a single line poking it's head out into South London is going to be swamped, but as long as it interchanges with the big rail every 6 stations or so, there will be no problem, and then big rail doesn't have to worry about serving a branch or bunch of stations any more.
It's not that simple: Commuters travel into London from as far afield as Canterbury and
Brighton, not just Orpington or Sevenoaks. And there are a lot of stops before these trains have even reached the M25. By the time these trains rock up at Chislehurst or East Croydon,
they're already packed full to bursting!
Remember, these "fast"—and I use the term very loosely—services also have to compete for space on the London approaches with the stopping services. The ex-SE&CR network's timetable is one gargantuan compromise. If it were ideal, there'd be no demand
at all for the HS1 Domestic trains. As it is, the stopping service from Orpington to London via Kent House can barely manage 3 tph. each way. Any more and the fast services won't have enough paths into London.
Commuters on these routes don't
expect their trains to be quick. And why would they care if a train takes 15 minutes longer to get them to work if the alternative is to get out onto a windswept platform at Lewisham, run down the stairs and through the subway to the other, reaching the other up platform just as their connection pulls in. (Or, worst still, just as it
pulls out.) Commuters like to read newspapers, muck about with their Blackberry phones and so forth. If the rain is sheeting down, why in blazes would they get out of the train before it gets into London, where the terminus stations usually have large platform canopies and overall roofs?
You need to think in terms of
user interfaces. People like convenience. The more faff and hassle your product involves, the less popular it will be. Take a good hard look at the Apple iPhone's success. When it first appeared, it didn't even support 3G or MMS, yet it still sold like hot cakes. Why? Because it made the basics
easy to use—something Apple's competitors had singularly failed at for years. The same rules of interface design implemented by Apple apply to
all interfaces—digital or analog, electronic or mechanical, what interface is made of doesn't change this.
There's a very good reason why many still choose to
drive into London, and it has nothing at all to do with speed. Rail needs to provide not only a quicker journey, but a competitive level of
comfort and convenience. Trains don't do the "door-to-door" thing, but they
can let you work on your laptop in safety during your journey, removing the stress of driving. But if the typical passenger's experience is being crammed into a glorified, overheated sardine tin on rails, it had
better be a bloody fast journey!