Your answer was insufficient. If you didn’t want to work that night, a simple response like “Sorry, I’m not available tonight” would suffice.
As mentioned in the other answers, you need to think about your attitudes towards work. It is important to set reasonable limits on the extra time you will spend at work. His attitude towards work in general, however, is quite poor. “She has to solve all the problems…other stupid employees…that’s not my role”, are all blatant red flags.