The Bachelorette has featured many men vying for the heart of one lucky lady that is looking for love, but some of them realized that pursuing that relationship just wasn’t in the cards for them at that moment. All in all, it seems like falling in love on a TV show takes a bit more of a struggle than these men thought it would take. It isn’t easy, after all, to see the woman that you’re falling for also pursuing a handful of other guys right in front of you. Moreover, lingering feelings for someone else or even the realization that they don’t have as many feelings for The Bachelorette lead could contribute to some of these men deciding to leave the show.

The first man to ever quit the competition was Frank Neuschaefer, who competed for Ali Fedotowsky during The Bachelorette season 6. Frank made it all the way to Ali’s Final 3 but decided to quit the competition after reconnecting with an ex-girlfriend and realizing that there were still feelings left on his part. This left the leading Bachelorette sobbing in tears once he broke off the relationship. At the time, he was a frontrunner in the competition. Ever since this occurrence, many other men have also quit the ABC series.

Related: Bachelorette: Chris Harrison’s Age, Girlfriend, Instagram, & Net Worth

There are also other men who quit The Bachelorette and became known as the villain of the season. Take Bentley Williams for example. He competed for Ashley Herbert’s heart on season 7. Bentley decided to quit the competition during week three, saying that Ashley was an “ugly duckling” and that he was hoping Emily Maynard was chosen as the lead for the season. Despite saying this, Bentley returned in week six, where he asked for a second chance. But Ashley declined to give him, prompting him to finish in 15th place.

Other men who have decided to quit The Bachelorette include Bryden Vukasin and Brooks Forester (season 9), both after realizing that they were not in love with Desiree. There was also Ron Worrell (season 10), who had to deal with the very unexpected death of a close friend. Then, Brady Toops (season 11) chose to leave after the season started out with two Bachelorette leads and the men ended up choosing Kaitlyn to stay. Brady wanted to pursue Britt instead, so he left to be with her. A few years later, Clay Harbor (season 14) left after injuring his wrist during a group date that required surgery. Last but not least, Jason Foster (season 16), found himself not being able to commit to Tayshia after Clare left.

Even though the premise of the show is to search for (and find) true love, some contestants realize that the journey just wasn’t what they expected it to be. This show is certainly not an easy experience, and many come to terms that it was not for them or that the leading lady was just not right for them.

Next: Bachelorette: Everything To Know About Zac Clark’s Sober Living Facility

The Bachelorette airs Tuesdays at 8PM EST on ABC.