That's really sad, but sometimes we crave company so much we are unable to tell good company from indifferent and toxic. I'd suggest you go through therapy and dig a little into yourself. Focus on yourself, not other people.
Honestly, I wouldn’t take weaker erections as a sign that you’re less attractive. If it keeps happening, it would be a good idea for your husband to talk to a doctor rather than putting the blame on either of you.