Even if they look old and withered they are still doing something for the plant. Once they are not they will turn brown or even bright orange. I always leave them until they come away on their own with a gentle tug. Then I know the plant has completely finished with them and is willing to give them up. I never cut leaves unless they are sick (as with your previous one).