Yes elitism exists among everyone, including among women, among black people, among poor people, even disabled people.
That doesn't mean it's not fed by and feeding into racism, classism, sexism, ableism etc.
Patriarchy is elitism, it devalues women, poor men, queer men, black men, disabled men.