You are correct that it is simply an aesthetic issue unless the buildup is so great that it plugs the holes. Don't forget that the algae only grows where there is moisture, nutrition and light, so will not intrude into the center of the pot. If you started a preventive treatment now, you'll have no issues.