Arsenal generally fill their stadium well though it's rarely completely full to capacity. They count tickets sold rather than bums on seats so official attendances can be deceiving. But they're now in a stadium that befits the size of the club, no doubt about that.
Spurs are the London club in most dire need of a new stadium, with a huge waiting list for season tickets - hence the 25k extra they're looking to accommodate with a new stadium build. They will want to emulate what Arsenal have done with the Emirates and as soon as they can, because without a Sugar Daddy increasing revenue and cashing in on their large fanbase is vital.
With West Ham it remains to be seen whether they can draw 60k regularly, but the Olympic stadium looks like a good financial opportunity for them, or at least for their owners. As I understand it, there's not much of a waiting list for tickets at their current ground, so it's questionable whether they'd be looking at 60k had the opportunity of the Olympic Stadium not arisen.
Chelsea don't fill their stadium as easily as you'd think and often have to advertise tickets on the radio or on Tube posters. They might be a huge global brand now but the fanbase isn't what some would imagine. They're a big club, but not a giant, and supply does outstrip demand at times. In the fuiture with success behind them and a new generation of fans that success generates, 60k+ might be good for them. At the moment, it might be a bit too much too soon, though it would help them with UEFA's FFP rules which will start biting soon. It's telling that a new stadium was never a huge priority for Roman Abramovich until now.
I think the smaller clubs do ok, but they of course have smaller stadiums to fill. QPR is now a pretty wealthy club and they're looking to get bigger. Their stadium really isn't a Premier League standard ground so it's not suprising they're looking to a new one. And Fulham have steadily grown under Al Fayad and are looking to extend their Cravan Cottage home, so they obviously believe they have the numbers to fill an expanded stadium.
So, to anser your question, West Ham are really the only question mark here in terms of capacity overkill. If QPR go silly on the capacity of their new stadium there might be a vaccuum of seats for them as well but I don't expect them to do that.