I have a feeling apartment building owners are zero-sum - if one leaves, someone else will buy it and supply isn't affected (except that you're somewhat discouraging building new apartments). But on the "single" family home side it's another story.
I looked at moving a few years back, and one home had a cheap rental in the basement. A quick search told me that previous tenants had complained to the city multiple times about small things - getting them free rent and keeping them from being evicted. Good for them, but if I'd bought it I'd probably just convert it back to a play room and not deal with the hassle.
A neighbor recently moved in across the street from me and converted the grandfathered-in duplex to a SF home. I think this was for similar reasons, and now that unit is gone forever (it is not legal to re-establish a duplex in SF zoning).
This kind of thing surely happens all over the city. Yes, there are heartbreaking stories of bad landlords making life hard for their tenants. But with too strong of tenant protection laws the effects are much more diffuse - shutting down a unit drives everyone's rent up, and physically evicts some family from the city (you just don't always know which one).
