Not a problem to date anyone on your standard of immutable characteristics. Human beings do this all the time.
I’m sure many men would love to be able to ask ‘what do you weigh?’ At the beginning of an interaction. If they did, they’d be eviscerated. That, men are told, is fat shaming and misogynistic. By the same logic, what you are doing is ‘height shaming’ - whether it saves you time or not (and it does) you cannot argue for equality and both of those positions to be true.
I suspect that men are rallying against the double standard rather than you personally. Either we have a world where preferences can be stated and asked (that’d be my choice and yours I suspect) - or we have a world where the feelings of others matter and a society prepares to police it. We can’t have a society that polices it one way only - that’d be an unequal and discriminatory on the grounds of sex, something which goes against the arguments put forward by feminists.