"fall back on"
In the following sentence would it be correct to translate "acaban cayendo en ellas" as "falling back on those"?:
Es curioso ver como muchas personas que afirman no ser supersticiosas acaban cayendo en ellas "por si acaso".
3 Answers
The infinitive is "recurrir a"
Es curioso ver como muchas personas que afirman no ser supersticiosas acaban (or better, "acaban por recurrir") recurriendo a ellas (las supersticiones) "por si acaso".

Follow up question/comment:
I wonder whether "acaban cayendo en" has the sense of "recurrir a," as in "turning to," "resorting to," and/or "falling back on" OR "falling for."
In other words, which translation works best?: (1) It's odd to see how many people who claim that they aren't superstitious end up turning to or resorting to them "just in case." (2) It's odd to see how many people who claim that they aren't superstitious end up falling for them "just in case." (Or is neither translation right?) There is a difference. The first translation doesn't make the decision not to reject superstitions sound completely illogical. In other words, maybe a person turns to a superstition without necessarily falling for it. The second translation tends to imply that a person doesn't reject superstition because he or she actually believes in it, i.e., he or she has fallen for it.
I think your thoughts are okay, but our dictionary has another word that I like as well:
http://www.spanishdict.com/translate/caer
11 (to commit)
a. to succumb
Me siento culpable porque estoy a dieta, pero caí en la tentación de un pastel de chocolate.I feel guilty because I'm on a diet, but I succumbed to the temptation of a chocolate cake.
I also would pick on them over on those as the more literal of en ellas.
...end up succumbing to them just in case.
Or you could use the more casual giving in to them.