Here's a title of a Nature paper that explores this very issue. Didn't read it (am on my phone), but it seems pretty targeted at your question. "Seed dispersal: Directed deterrence by capsaicin in chillies." Abstract says capsaicin deters some animals (the new world monkey seed predators like Saki monkeys spring to mind) without deterring good seed dispersers.