I think it has more to do with the fact trees require more maintenance, like raking up leaves and fruit, and having to saw off branches.

Also those roots can break pavement and pipes.