Hello Zinda!
Aphids are famous for going from "not noticed" to "swarms" all of a sudden, especially this time of year.
Insecticidal soap is probably your least-toxic option. Peroxide, alcohol, sulfur and such are more likely to damage the plant and less likely to kill the aphids. These products are all "chemicals" too.
Don't let the word "insecticidal" spook you. This is just soap that has been especially formulated to NOT damage the plant. Look for a product that has "potassium salts of fatty acids" as the active ingredient. Again, this is just soap, by its chemical name. Dish soap has been specially formulated to clean dishes, and is more likely to damage the plant. "Insecticidal Soap" is made for plants. It kills bugs on contact, there is no residual or systemic effect. It is just soap and water, but made to use on plants. It kills aphids.
Follow the label directions. You'll need to thoroughly spray the whole plant. After about half an hour you can rinse it off with plain water, this also helps lessen the chance of phytotoxicity. You'll need to repeat this 2-3 times a week for several weeks, then give it a rest but keep a close eye out for more bugs. You may need to continue to spot treat for awhile. Also keep an eye on your other plants. It's easier when you catch it early on, and now you know what to look for.
Here's a link to an article with a good explanation as to why you should use "insecticidal soap" instead of "Grandma's home remedy dish soap formula"
Control Houseplant Insect Pests Safely With Insecticidal Soap | News