Would Christ tell the church to just "shut up and do what I say?"
Is your husband loving you as Christ loves the church?
Is there somewhere you and the kids can go? A safe house where you could stay for sometime? I think you need to make some preparations---find a place to stay---or better yet, find him a place to stay as that will be less disruptive to the kids..........and then confront him, preferably with a pastor at your side.
I think you hit on two key fears that are keeping you from acting, the fear of seeing yourself as having failed and the fear of being alone. You have not failed to keep your vow of fidellity to your husband,; he has failed to keep it to you. While children do not offer the same type of relationship that a spouse does, hopefully they would help keep some of the loneliness at bay. Loneliness would likely be difficult, though were that to happen, I pray that God would fill that void with special presence of His Holy Spirit.
For the purposes of marriage, love is not a feeling, it is a choice. You are choosing to behave lovingly toward your husband, 1 cor. 13:4-7, but he is not behaving lovingly toward you. Only God can show you where the line is crossed from choosing to love versus enabling.