Men cannot babysit small children?

I have just made a crucial decision- I have to quit my part-time job because my husband is giving up the task of babysitting his 2 sons while I'm not around for that twice weekly, 4 hour per session job. It's really sad and quite unbelievable that a man could not babysit his own flesh & blood! He just couldn't take the whining & crying anymore. And what about me? "Oh.... you're their mother. OF COURSE you can handle it.", he says.