Gaydar study calibration

The other day the New York Times had a Gray Matter science piece by the authors of a study in PLoS One that showed some people could identify gays and lesbians based only on quick flashes of their unadorned faces. They wrote:

We conducted experiments in which participants viewed facial photographs of men and women and then categorized each face as gay or straight. The photographs were seen very briefly, for 50 milliseconds, which was long enough for participants to know they’d seen a face, but probably not long enough to feel they knew much more. In addition, the photos were mostly devoid of cultural cues: hairstyles were digitally removed, and no faces had makeup, piercings, eyeglasses or tattoos.

Even when viewing such bare faces so briefly, participants demonstrated an ability to identify sexual orientation: overall, gaydar judgments were about 60 percent accurate.

Since chance guessing would yield 50 percent accuracy, 60 percent might not seem impressive. But the effect is statistically significant — several times above the margin of error. Furthermore, the effect has been highly replicable: we ourselves have consistently discovered such effects in more than a dozen experiments.

This may be seen as confirmation of the inborn nature of sexual orientation, if it can be detected by a quick glance at facial features.

Sample images flashed during the “gaydar” experiment.

There is a statistical issue here that I leave to others to consider: the sample of Facebook pictures the researchers used was 48% gay/lesbian (111/233 men, 87/180 women). So if, as they say, it is 64% accurate at detecting lesbians, and 57% accurate at detecting gay men, how useful is gaydar in real life (when about 3.5% of people are gay or lesbian, when people aren’t reduced to just their naked, hairless facial features, and you know a lot of people’s sexual orientations from other sources)? I don’t know, but I’m guessing not much.

Anyway, I have a serious basic reservation about studies like this — like those that look for finger-length, hair-whorl, twin patterns, and other biological signs of sexual orientation. To do it, the researchers have to decide who has what sexual orientation in the first place — and that’s half the puzzle. This is unremarked on in the gaydar study or the op-ed, and appears to cause no angst among the researchers. They got their pictures from Facebook profiles of people who self-identified as gay/lesbian or straight (I don’t know if that was from the “interested in” Facebook option, or something else on their profiles).

Sexual orientation is multidimensional and determined by many different things — some combination of (presumably many) genes, hormonal exposures, lived experiences. And, for some people at least, it changes over the course of their lives. That’s why it’s hard to measure.

Consider, for example, a scenario in which someone who felt gay at a young age married heterogamously anyway — not too uncommon. Would such a person self-identify as gay on Facebook? Probably not. But if someone in that same situation got divorced and then came out of the closet they probably would self-identify as gay then.

Consider another new study, in the Archives of Sexual Behavior, which used a large sample of people interviewed 10 years apart. They found changes in sexual orientation were not that rare. Here is my table based on their results:

Overall, 2% of people changed their response to the sexual orientation identity question. That’s not that many — but then only 2.5% reported homosexual or bisexual identities in the first place.

In short, self identification may be the best standard we have for sexual orientation identity (which isn’t the same as sexual behavior), but it’s not a good fit for studies trying to get at deep-down gay/straight-ness, like the gaydar study or the biological studies.

And we need to keep in mind that this is all complicated by social stigma around sexual orientation. So who identifies as what, and to whom, is never free from political or power issues.

12 Comments

Filed under In the news, Research reports

12 responses to “Gaydar study calibration

  1. But what’s the rate of random measurement error in that survey? It could easily be on the order of 2%

    • I should have mentioned: the sample of Facebook pictures they used was 48.6% gay/lesbian. What this means about the practical effectiveness of gaydar in real life, I don’t know.

  2. Can someone check my math, because I think the 60% number vastly overstates the usefulness of this type of “gaydar.” The study reports that on average respondents correctly identified 37% of gay or lesbian faces as gay or lesbian, and 74% of straight faces as straight. For simplicity sake, let us assume that 95% of the population identifies as straight and 5% as lesbian or gay. The average respondent in this survey would guess that 70 (.74*95) of the straight people were straight, and about 3, ((1-.37)*5), of the lesbian and gay population was straight. This means that 96% of the people who were tagged as straight do identify that way. That leaves 27 people who are tagged as lesbian or gay–this is the relevant set for checking the accuracy of gaydar. Of those, 25, or 92.5%, actually identify as straight. So, even assuming lesbian or gayness is fixed identity, while it might correctly classify 60% of lesbian and gay people, I figure that this face-based measure of gaydar is wrong more than nine out of ten times.

    Also, fun fact: “only photographs of White-appearing individuals who self-identified ages of 18–29 were included.”

  3. Ron

    Plz help clarify a couple of questions about the “10 years later” table:

    1) Does it say that 8 women changed identification from heterosexual to homosexual?

    2) Were there really 96.64% (1324/1370) hetero women?

  4. If they wanted to demonstrate that gaydar is effective, they need only have used gay people to view the participants. Gays have much better tuned gaydar than straights, particularly when viewing members of their own sex.

  5. Pingback: Interrace Today | Interrace Today

  6. So maybe you already did this with your “ten years apart” table. I added up all the people who EVER said (either at T1 or T2) that they were bisexual or homosexual, that’s 46 women and 47 men. Then I calculated a) the percent of people who ever reported themselves as homosexual or bisexual who changed categories, which is 78% or women and 40% for men and b) the percent of the people who ever reported themselves as bisexual or homosexual who reported themselves as heterosexual at the other time and got 61% for women and 30% for men. That’s huge. It’s not only not rare, it is more common than not, suggesting that that table is more in the zone of respondent error than actual change although, obviously, you can’t know for each person. This is more relevant I guess to the scatterplot methodology debate http://scatter.wordpress.com/2012/06/23/bad-science-not-about-same-sex-parenting/ than this one.

    But as your Soc Images cross-post is going to bring traffic, I thought I’d comment here.

  7. Thank you. I was thinking the same thing looking back at this after the scatterplot discussion. Given that the numbers are so small, it would make sense to simply follow up to verify that they are changing their response – rather than try to identify response errors after the fact.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s