I don't think it harms the plants at all. I wouldn't worry about it. If it really bugs you water with chlorinated tap water now and then. That kills some kinds of moss. And don't fertilise more than necessary.
__________________ May the bridges I've burned light my way.