Tag Archives: same-sex

On artificially intelligent gaydar

A paper by Yilun Wang and Michal Kosinski reports being able to identify gay and lesbian people from photographs using “deep neural networks,” which means computer software.

I’m not going to describe it in detail here, but the gist of it is they picked a large sample of people from a dating website who said they were looking for same-sex partners, and an equal number that were looking for different-sex partners, and trained their computers to learn the facial features that could distinguish the two groups (including facial structure measurements as well as grooming things like hairline and facial hair). For a deep dive on the context of this kind of research and its implications, and more on the researchers and the controversy, please read this post by Greggor Mattson first. These notes will be most useful after you’ve read that.

I also reviewed a gaydar paper five years ago, and some of the same critiques apply.

This figure from the paper gives you an idea:

gd4

These notes are how I would start my peer review, if I was peer reviewing this paper (which is already accepted and forthcoming in the Journal of Personality and Social Psychology — so much for peer review [just kidding it’s just a very flawed system]).

The gay samples here are “very” gay, in the sense of being out and looking for same-sex partners. This does not mean that they are “very” gay in any biological, or born-this-way sense. If you could quantitatively score people on the amount of their gayness (say on some kind of scale…), outness and same-sex attraction might be correlated, but they are different things. The correlation here is assumed, and assumed to be strong, but this is not demonstrated. (It’s funny that they think they address the problem of the sample by comparing the results with a sample from Facebook of people who like pages such as “I love being gay” and “Manhunt.”)

Another way of saying this is that the dependent variable is poor defined, and then conclusions from studying it are generalized beyond the bounds of the research. So I don’t agree that the results:

provide strong support provide strong support for the PHT [prenatal hormone theory], which argues that same-gender sexual orientation stems from the underexposure of male fetuses and overexposure of female fetuses to prenatal androgens responsible for the sexual differentiation of faces, preferences, and behavior.

If it were my study I might say the results are “consistent” with PHT theory, but it would be better to say, “not inconsistent” with the theory. (There is no data about hormones in the paper, obviously.)

The authors give too much weight to things their results can’t say anything about. For example, gay men in the sample are less likely to have beards. They write:

nature and nurture are likely to be as intertwined as in many other contexts. For example, it is unclear whether gay men were less likely to wear a beard because of nature (sparser facial hair) or nurture (fashion). If it is, in fact, fashion (nurture), to what extent is such a norm driven by the tendency of gay men to have sparser facial hair (nature)? Alternatively, could sparser facial hair (nature) stem from potential differences in diet, lifestyle, or environment (nurture)?

The statement is based on the faulty premise that they are “nature and nurture are likely to be as intertwined.” They have no evidence of this intertwining. They could just as well have said “it’s possible nature and nurture are intertwined,” or, with as much evidence, “in the unlikely event nature and nurture are intertwined.” So they loaded the discussion with the presumption of balance between nature and nurture, and then go on to speculate about sparse facial hair, for which they also have no evidence. (This happens to be the same way Charles Murray talks about race and IQ: there must be some intertwining between genetics and social forces, but we can’t say how much; now let’s talk about genetics because it’s definitely in there.)

Aside from the flaws in the study, the accuracy rate reported is easily misunderstood, or misrepresented. To choose one example, the Independent wrote:

According to its authors, who say they were “really disturbed” by their findings, the accuracy of an AI system can reach 91 per cent for homosexual men and 83 per cent for homosexual women.

The authors say this, which is important but of course overlooked in much of the news reporting:

The AUC = .91 does not imply that 91% of gay men in a given population can be identified, or that the classification results are correct 91% of the time. The performance of the classifier depends on the desired trade-off between precision (e.g., the fraction of gay people among those classified as gay) and recall (e.g., the fraction of gay people in the population correctly identified as gay). Aiming for high precision reduces recall, and vice versa.

They go on to give a technical, and I believe misleading example. People should understand that the computer was always picking between two people, one of whom was identified as gay and the other not. It had a high percentage chance of getting that choice right. That’s not saying, “this person is gay”; it’s saying, “if I had to choose which one of these two people is gay, knowing that one is, I’d choose this one.” What they don’t answer is this: Given 100 random people, 7 of whom are gay, how many would the model correctly identify yes or no? That is the real life question most people probably think the study is answering.

As technology writer Hal Hodson pointed out on Twitter, if someone wanted to scan a crowd and identify a small number individuals who were likely to be gay (and ignoring many other people in the crowd who are also gay), this might work (with some false positives, of course).

gd1

Probably someone who wanted to do that would be up to no good, like an oppressive government or Amazon, and they would have better ways of finding gay people (like at pride parades, or looking on Facebook, or dating sites, or Amazon shopping history directly — which they already do of course). Such a bad actor could also train people to identify gay people based on many more social cues; the researchers here compare their computer algorithm to the accuracy of untrained people, and find their method better, but again that’s not a useful real-world comparison.

Aside: They make the weird but rarely-necessary-to-justify decision to limit the sample to White participants (and also offer no justification for using the pseudoscientific term “Caucasian,” which you should never ever use because it doesn’t mean anything). Why couldn’t respondents (or software) look at a Black person and a White person and ask, “Which one is gay?” Any artificial increase in the homogeneity of the sample will increase the likelihood of finding patterns associated with sexual orientation, and misleadingly increase the reported accuracy of the method used. And of course statements like this should not be permitted: “We believe, however, that our results will likely generalize beyond the population studied here.”

Some readers may be disappointed to learn I don’t think the following is an unethical research question: Given a sample of people on a dating site, some of whom are looking for same-sex partners and some of whom are looking for different-sex partners, can we use computers to predict which is which? To the extent they did that, I think it’s OK. That’s not what they said they were doing, though, and that’s a problem.

I don’t know the individuals involved, their motivations, or their business ties. But if I were a company or government in the business of doing unethical things with data and tools like this, I would probably like to hire these researchers, and this paper would be good advertising for their services. It would be nice if they pledged not to contribute personally to such work, especially any efforts to identify people’s sexual orientation without their consent.

11 Comments

Filed under Research reports

A party like it’s 2014 (marriage equality edition)

My question for Marco Rubio is, what are you going to do about this gay marriage you are still so against?

In his closing statement at last night’s debate, Marco Rubio said,

Our culture’s in trouble. Wrong is now considered right, and right is considered wrong. All the things that once held our families together are under constant assault. … If you elect me president we are going to re-embrace free enterprise, so that everyone can go as far as their talent and their work will take them. We are going to be a country that says that life begins at conception, and life is worthy of the protection of our laws. We’re gonna be a country that says that marriage is between one man and one woman.

Here it is:

This wrong-right thing is not exactly specified, but in context it clearly refers to abortion and gay marriage — so wrong, but not “considered right.”

What does it mean to say, “We’re gonna be a country that says that marriage is between one man and woman”? What does a country say? Does anyone really listen to what these people say?

Yes, they do. Because as of the morning of yesterday’s debate Rubio has a Marriage & Family Advisory Board to make sure that his words have meaning, and that right returns to right, while wrong is again returned to its proper place: hidden, shamed, and reviled.

Here’s the charge of the board:

This morning, the Marco Rubio for President campaign is excited to announce the formation of Marco Rubio’s Marriage & Family Advisory Board. Marco believes the family is the most important institution in society. He understands that in a vibrant culture of marriage and family everyone benefits, but in a culture where the importance of families is neglected all sorts of problems result. You cannot have a strong nation without strong people, and you cannot have strong people without strong values. Right and wrong. Good and bad. That is learned from your values instilled in you in the family. It is irreplaceable.

Strong statements for strong times. (In fact, you cannot have strong times without strong statements.) These are the board’s members:

  • Ryan T. Anderson, Ph.D., Senior Research Fellow, The Heritage Foundation
  • Joseph Backholm, Executive Director, Family Policy Institute of Washington
  • Ambassador Ken Blackwell, Senior Fellow, Family Research Council
  • David S. Dockery, President, Trinity Evangelical Divinity School
  • Sherif Girgis, J.D./Ph.D. candidate, Yale Law & Princeton
  • Alan Hawkins, Ph.D., Professor, Brigham Young University
  • Kay Hymowitz, William E. Simon Fellow, Manhattan Institute
  • Jonathan Keller, CEO, California Family Council
  • Caitlin La Ruffa, Executive Director, Love and Fidelity Network
  • Robert Lerman, Emeritus Professor of Economics, American University
  • Everett Piper, Ph.D., President, Oklahoma Wesleyan University\
  • Bill Wichterman, former special assistant to President George W. Bush
  • Bradford Wilcox, Senior Fellow, Institute for Family Studies & Visiting Scholar, American Enterprise Institute

I wish the Republicans would debate this a little more seriously. Ted Cruz has proposed a Constitutional amendment, Jeb Bush and John Kasich have complained about marriage equality but not argued for overturning it, Trump says he opposes marriage equality but doesn’t really care. So what’s Rubio’s plan. Either you think it can be reversed, which is dumb, or you’re just attacking gays and lesbians as “wrong,” which is mean.

On Rubio’s board, Wilcox, Lerman, Hawkins, and Hymowitz are Family Inequality regulars. Of course he doesn’t really need policy advice at this point in the campaign, so this is just about signaling — it’s Rubio showing donors the direction he’s taking, and it’s these people deciding to put their names on his campaign. (Somehow, though, I’m sure they will also still be able to describe themselves as “non-partisan,” because wrong is now right.) It’s also the first time I know of that Wilcox has publicly opposed marriage equality, which is a promising turn in his maturation as a partisan hack.

2 Comments

Filed under In the news

That time when your research is used to justify ripping a baby from the arms of its loving adoptive parents

UPDATE: Judge Johansen has rescinded his order

Brad Wilcox and Mark Regnerus lost in their attempt to turn the federal courts against marriage equality. The work they did culminated in a paper published under Regnerus’s name, and Regnerus is the name most associated with its bogusness, but it was Wilcox who led the effort to raise the money (some of which he kept), helped direct the study, and weaseled it into the journal by serving as a peer reviewer for its publication. (Two subsequent studies reanalyzed the Wilcox/Regnerus data, and thoroughly debunked its results — here and here; you can get the full story by following the links in this post.)

Although they failed in their quest to affect the Supreme Court, their work lives on in the very small, evil minds of anti-gay fanatics around the world, who continuously cite the original paper. One of those men is Judge Scott Johansen, a juvenile court judge in Carbon County, Utah (the state’s seventh district), who has cited unspecified “research” to justify his decision to take a one-year-old baby from the home of Beckie Peirce and April Hoagland, a married lesbian couple who are the child’s foster parents. With the approval of the baby’s biological mother and child welfare authorities — who did the routine thorough investigation and vetting that all adoptive parents (including me) have endured — the two were moving ahead with plans to legally adopt the baby when Johansen, a law graduate of the Mormon Brigham Young University, handed down his decision. The decision is set to take effect next Tuesday (November 17). His decision is not public, but he told the couple his own research showed it was better for children to be raised by a heterosexual couple. We don’t need to ask what research he has in mind.

Legal efforts continue, and officials — including the governor of Utah — have asked the judge to reconsider.

If your research was used like this, what would you do?

So, this is the point of all the work Wilcox and Regnerus did. We must assume they wanted exactly this decision, but on a much larger scale; they wanted same-sex couples to be denied the right to adopt children, and children to be denied the right to have married gay and lesbian parents. They would apparently rather see a one-year-old child who has spent three months with a loving family ripped from that family rather than face the fate of having lesbian parents.

If I’m wrong, and I would be especially happy to be wrong in this case, then Wilcox and Regnerus should be the first experts lining up to convince Judge Johansen that he’s making a mistake, that the actual well-being of the child, and the civil rights of its parents, should come before slavish devotion to religious dogma. In fact, speaking up right now might actually do some good.

Wilcox has gone out of his way to sing the praises of the “deep normative and religious commitments to marriage and to raising children within marriage” in Utah specifically. But he doesn’t comment on this aspect of Utah’s holiness — the deep commitment that has led the Mormon church to announce a wretched, hateful policy under which it will not bless or baptize the children of gay and lesbian couples unless they denounce their parents.

Now might be a good time for Wilcox’s sham Institute for Family Studies — which has yet to ever use the words “lesbian,” “gay,” or “homosexual” on its web pages — to break its silence and take a stand for children and family well-being.

I’ll be holding my breath.

7 Comments

Filed under In the news

Marriage equality is official now that it’s in The Family textbook

Well, actually, it’s in a special addendum to the textbook that W. W. Norton is just releasing.

The book I wrote, The Family: Diversity, Inequality, and Social Change, hit the streets a year ago today. Marriage equality plays a significant part in the story, much larger than the proportion of the population that is directly affected by the changing law. That’s because of the high-stakes nature of the debate for so many people, and because of its symbolic acceptance of rising family diversity — the main theme of the book.

So when the law suddenly, and fundamentally, changed this summer, we decided we needed an update for instructors teaching this fall. The three-page supplement reviews the political and legal events leading up to the June 26 Obergefell decision, and the logic of the legal questions addressed — along with a little context on the place of marriage equality in the story of family change. I hope it’s helpful for you.

The update is now available on the Norton website, here, and on my teaching page. While you’re at it, you should visit the book’s homepage, and see what we have in store for you if you teach family sociology (and request an exam copy), here.

Also:

  • A symposium with 12 writers and researchers addressing the concept, “After marriage equality,” which Syed Ali and I edited for Contexts.
  • My whole series of blog posts on marriage equality is archived under the homogamy tag.

whitehouserainbow

1 Comment

Filed under Me @ work

To tell the truth (right-wing front edition)

lyingcross

The mission of the Institute for Family Studies is “strengthening marriage and family life, and advancing the well-being of children, through research and public education.” As of this morning, this includes not a single use of the words “gay,” “lesbian,” or “same-sex” anywhere on their website, according to Google. They routinely post links to articles and research “of note,” that might interest readers who believe in their mission. So, why never mention the gay?

Or — dramatic pause — is that really their whole mission? The IFS website lists seven “senior fellows.” Don’t tell the others, but W. Bradford Wilcox is the only one getting paid $50,000 per year (in 2013). Their 2013 fundraising included $50,000 from the Bradley Foundation, which also supported Wilcox’s effort to fund the Regnerus study; and $20,000 from the Vine and Branches foundation, which lists the purpose of the donation as “religious” (the foundation’s eligibility criteria include, “Christian organizations that overtly express their faith through programming”).

So, do you really believe this?

As a nonpartisan, nonsectarian, and not-for-profit institute committed to the study of family life, IFS works with scholars, writers, and supporters without regard to academic discipline, party, or ideology.

The only thing that bothers me about this, besides the values, is the blatant, routine dishonesty. Why do respectable people just tolerate that?

Not to get into minutiae, but also, would it kill him to have any women among the nine officers of his shadowy, bogus non-profit foundation?

Note: I first wrote about IFS here, but only some of that info is still accurate.

10 Comments

Filed under In the news

Things are getting better/worse, and we’re not going to take it anymore

There is a whole social science to the optimal balance of victory and defeat in social movements and social change. Trying to sort that out recently reminds me of the time in 1980 when the Williams pinball machine company introduced Black Knight, which featured four flippers, 2- and 3-ball multiball™ play, and magna-save (don’t ask). And it talked. It was hard to get to sleep that week, with the ringing in my ears, the flashing lights burned into my eyes, and the endless strategic possibilities bouncing around in my head (though, looking at it now, I find this all hard to believe).

So, too, in the last week. Consider two political cartoons by Mike Luckovich. This from June 21:

Luckovich-slider

And this one from June 25:

luckovich

Did he really just demand the removal of the Confederate flag and then mock people who would celebrate its removal? Is that how much things change in a week? But in periods of social change, moving the goal posts is what it’s all about. And there’s nothing wrong with that.

The Charleston massacre was a horrific reminder of how it seems some things never change. But they do change. Dylann Roof was caught and may be put to death, legally. And it turned out that, not only had the Confederate flag only been flying at the South Carolina capitol for a few decades, but it actually could be taken down in response to public outrage. And yet, that’s not the end of racism. (Four flippers, three balls, magna-save.)

Anthea Butler, a religion and Africana studies professor at Penn, who wrote an op-ed in the Washington Post, was on the On Point radio show last week. She was talking to host Tom Ashbrook, when she got this:

Tom Ashbrook: If you ask me, I understand that feeling and that vivid response. At the same time, I, and maybe you, Anthea Butler, Dr. Butler, don’t want to lose, or not recognize, or lose the progress that has been made. And this is nowhere near paradise…

Anthea Butler: But what kind of progress? What kind of progress? This is what we keep talking about. And I don’t understand, when you say, “We’ve made progress.” How have we made progress when the president of the United States has been constantly questioned because he is partially a Black man? And so you talk progress — and this is the kind of talk we’re going to hear all week long after this.

TA: But he’s president, madam.

AB: He is president.

TA: Well, that’s a pretty big deal…

AB: That is a big deal, but to some people in this country, like Dylann Roof, that is the end of this country. That’s why you had the kind of phrase that he said, that all your politicians, the right Republican politicians have been saying, “Take our country back.” And so, I want to talk about the rhetoric that’s happened…

Ashbrook has a point about progress, of course, but it’s just the wrong time to say that, days after a racist massacre that seems as timeless as a Black-church burning. At that moment there could be no progress.

For whatever reason, Ashbrook turned to progress on the interpersonal level:

TA: We did see White people in South Carolina, in Charleston, pour into the churches alongside African Americans over this weekend.

AB: Yes we did. But you need to understand the distinction here. I don’t doubt that there are well-meaning, good White people, good White Christians, who are appalled at this. I understand that. But when you have a structural system that continues to do this kind of racial profiling, the kinds of things that are going on with the police in this country, the kinds of issues that we’ve had. The problem becomes this: you can talk about progress all you want, but reality is another thing altogether.

Again, it’s progress, but focusing on it at that moment is basically #AllLivesMatter. President Obama also tried to keep his eyes on the prize, in his appearance on the WTF podcast:

Racism, we are not cured of it. And it’s not just a matter of it not being polite to say “nigger” in public. That’s not the measure of whether racism still exists or not. It’s not just a matter of overt discrimination. Societies don’t, overnight, completely erase everything that happened 200 to 300 years prior.

Outrage ensued about his use of “nigger,” but White House Press Secretary Josh “earnest non-racist white guy” Earnest doubled down:

The President’s use of the word and the reason that he used the word could not be more apparent from the context of his discussion on the podcast.  The President made clear that it’s not possible to judge the nation’s progress on race issues based solely on an evaluation of our country’s manners.  The fact is that we’ve made undeniable progress in this country over the last several decades, and as the President himself has often said, anyone who lived in this country through the ‘50s and the ‘60s and the ‘70s and the ‘80s notes the tremendous progress that we’ve made.  That progress is undeniable. But what’s also undeniable is that there is more work that needs to be done, and there’s more that we can do.  And the fact is everyone in this country should take some inspiration from the progress that was made in the previous generation and use that as a motivation and an inspiration to try to make further progress toward a more perfect union.

Now is no time to talk about progress, some say. With Black church members being gunned down and churches burning, and one appalling, outrageous video after another showing the abuse of Black citizens by police, having a Black president is not a victory. So much so that maybe he’s not really Black at all. Frank Roberts writes of Obama’s “Amazing Grace” moment:

With Obama … blackness has been reduced to a theatrical prop; a shuck-and-jive entertainment device that keeps (black) audiences believing that the President “feels their pain” — at precisely the same time that he fails to provide a substantive policy response to black unemployment, over-incarceration, and/or racialized state violence.

The social scientist in me objects, because the rate of progress is not determined by the victory or tragedy of the moment, or by the blackness of a man. And Obama probably has done more than any other president (at least recently) to address Black unemployment, incarceration, and racialized state violence. That’s not a moral or political statement — and it doesn’t imply “enough” — it’s an empirical one.

Movements use good news for legitimacy, and bad news for urgency.  When something goes well, they need to claim credit and also make sure their supporters know there is more work to be done. When something awful happens they place the troubles in the context of a narrative of struggle, but they don’t want to appear powerless because that saps support as well, and undermines morale.

Case in point, marriage equality

In that old psychology study of lottery winners and paraplegic accident victims, the researchers concluded that we put too much weight on the fleeting reactions of others to good or bad events, falsely assuming that these events will define them permanently. Since gay marriage will not actually make their lives worse, I have to assume that the doom-and-gloom gang on pathetic display in a mordantly morose, delightfully depressive, symposium on the Supreme Court decision at the religious conservative First Things site will soon again return to being their sunny selves.*

And these people will go back to being miserable soon enough. (Photo by James Buck)

And these San Francisco Pride marchers will go back to being miserable soon enough. (Photo by James Buck)

In the meantime, the family right will use SCOTUS to stoke their movement — after an oh-so-dramatic display of what Jeffrey Toobin called a “religiously themed retreat into victimology.”

But the anti-equality right has to be careful, or their nattering negativity will undermine their appeal, especially among young people who haven’t yet given up all hope of being the change they want to see. For Ted Cruz to call this — that is, people getting married — “some of the darkest 24 hours in our nation’s history,” to declare a “day of mourning,” or to see a vision of Jesus weeping, all may be a little much for the youth vote. (Not that kids these days know how to spell anymore, but I think they’ll get the difference between “morning” and “mourning” in America.) It’s one thing for religious conservatives to entice others to join them on the holier-than-them side of the fence with a little martyrdom (after all, whole religions have been built on it). But who wants to join a movement — much less have Thanksgiving dinner — with a guy who wallows in his own defeat like this?

While many have pointed to the 1973 decision of Roe v. Wade as an obvious historical analogue for the Obergefell decision, to my mind, the insistence that all must conform to the new, official definition of marriage that no civilization has ever endorsed until yesterday seems to be more aptly compared to life under Communism. … The “monopoly of violence” possessed by the State is now a main weapon in perpetuating this lie, and will be used mercilessly and without cessation against those who persist on pointing out that it seeks to perpetuate a lie. But violence will serve as a last resort, merely backstopping the education system, the economic players, and even family members who will work to correct wayward thinkers.

That poor persecuted soul, by the way, is a tenured professor.

Four flippers, three balls, magna-save.

* The First Things symposium was linked without explicit endorsement by Ross Douthat, who on marriage equality day did not pause to congratulate a gay couple (whose wedding he would rather not attend anyway) before mean-spiritedly besmirching the movement by speculating on the coming legalization of polygamy.

7 Comments

Filed under Politics

How random error and dirty data made Regnerus even wronger than we thought

The news is nothing I have to say, but the new article, available in prepublication form, by Simon Cheng and Brian Powell, which methodically flays the infamous Regnerus paper, leaving nothing but a wisp of foul-smelling ill-will trailing from its remains. (The paper is here, where it is paywalled; feel free to email me. Follow the whole story at the Regnerus tag.)

Cheng and Powell reanalyzed the Regnerus data, the New Family Structures Survey (NFSS), and see what would happen if Regnerus had done the data processing and analysis right. This goes beyond the logical flaws and biases that were inherent in the study design (discussed here), to find the coding and analysis errors. A few examples:

  • So much for “raised by…” 24 of the 236 people coded as having a “lesbian mother” or “gay father” — because they reported one of their parents ever had a same-sex romantic relationship (I’ll use LM and GF here to refer to Regnerus’s codes, not reality) — never lived with the parent in question! We had known previously that a large number (138) had never lived with the partner in the romantic relationship, but this is a whole nother level of wrong. A total of 58 of the LM/GF sample were reported to have lived with the supposedly gay or lesbian parent for a single year or less.
  • Bad cases. The most ridiculous is the “25 year-old man who reports that his father had a romantic relationship with another man, but also reports that he (the respondent) was 7-feet 8-inches tall, weighed 88 pounds, was married 8 times and had 8 children.” Another reported being arrested for the first time at age 1. Real data collectors scrutinize cases like that and throw them out or find a way to fix them. (Really good data collectors stop the person — or the data entry — right when they say something outrageous, to see if they’re sure.)
  • Illogical cases. There are a lot of these, including the person who reported “having always lived alone but also claims to have always lived with mother, father, and two grandparents.”

Then there are a series of bad analysis and modeling decisions Regnerus made, such as coding people who refused to answer a question as 0 instead of missing, or using the wrong kind of statistical model for the particular outcome.

When they get done with it, there really is no reliable, significant negative outcome associated with having lived any appreciable amount of time with a parent who might have been gay or lesbian. There’s more to it, but I don’t want to discourage you from reading the paper.

Random error, correlated outcome

Some of the “misclassified or uncertain” cases also report serious problems in adulthood, exhibiting higher-than-average rates of suicidality, depression, drinking to get drunk, and having a poor relationship with their mothers. So those could be people whose difficult lives rendered them unable to complete the life history calendar correctly. But there is also a chance that, like the 7’8″ guy, there are people just answering some of the question at random. These were people taking the survey alone on a computer, with no supervision, and getting paid to be part of the sample. Clicking at random is not out of the question (one person only took 10 minutes to complete the lengthy survey).

Contrary to what you might assume, clicking at random does not always produce random results. I’ll illustrate this with an example. First, here’s another tidbit from Regnerus, which might fit this point. Speaking to some Franciscans in 2014, Regnerus (just after 9:00 of this video) was going on about sexual fluidity as a condition of modernity, when he dropped in this fact from the NFSS:

Despite comprising a mere 1.3 percent of the population, respondents in the NFSS [New Family Structures Survey] who said that their mothers have had a same-sex sexual relationship made up 15 [50?] percent of all the asexual identifiers in the NFSS. So, 15 [50?] percent of them come from 1.3 percent of the population. [I originally transcribed those as 50%, but on second listening I think he said 15%, but I can’t be sure.]

His raised eyebrow here is to indicate the deeply depraved nature of lesbian mothers — maybe it’s genetic, or maybe it’s child abuse — but… he lets the numbers speak for themselves. Lesbian mothers, asexual children.

Here’s how this works. If you are trying to find people in two rare conditions — for example, those with lesbian mothers and those who are asexual — and a small portion of your sample answers questions at random, not only will you have a relatively large number of false positives on your conditions, your rare conditions will also falsely appear to be correlated.

I’m sure I didn’t discover this, and I don’t have a mathematical proof for it, but it’s logical. And I confirmed it with an experiment, as follows.

Say you have a sample of 1000 people, and you’re studying two conditions that occur on average in one out of every 500 cases. I’ll call them “climbing Mt. Everest” and “going to the moon.” In your thousand cases, you will on average have 2 people who did each thing. The chances that the same person did both are probably really low (you do the maths). But, if just 1% of your sample — 10 people — answer those two yes/no questions at random, look out!

I created this scenario using Excel’s random-number function. With 990 people answering truthfully — that is, given a 1/500 chance of saying yes to each question — and 10 answering them both randomly, this is what I got: 6 people who had climbed Mt. Everest, and 8 people who had gone to the moon. But shockingly, there were 4 people who had done both — that is 67% of the mountain climbers and 50% of the moonshotters. You can’t know, from looking at the data, but I can, that all of the people who went on both adventures were in the tiny group of random answerers.

Here are the 1000 cases in random order, with green showing Everest-only cases, blue showing moon-only cases, and red showing positive answers to both questions. And here’s the statistic: in the total sample — 990 serious survey takers and 10 jokers — the correlation between climbing Mt. Everest and going to the moon is .53! Click to enlarge:

rare event errors.xlsx

Maybe Regnerus is just an incredibly, irresponsibly bad researcher, who didn’t conduct the simplest data checks before rushing to publish his paper. Or maybe he is a diabolical genius, and he realized that high random error rates in both his rare independent variable and his rare dependent variables would produce results showing poor outcomes for children of gays and lesbians.

In the Cheng and Powell paper, their various procedures and corrections wipe out many of Regenerus’s negative outcomes for GF/LM respondents before they tackle the “misclassified or uncertain” cases. But when they do that, some of the last coefficients to fall to non-significance are indeed relatively rare: having suicidal thoughts (7%), not being “entirely heterosexual” (15%), having had an STI (11%), and having had forced sex (13%). Each of these becomes non-significant when the bad cases are controlled in the Cheng and Powell models. I haven’t worked out a proof (ever), but I reckon that the rarer they are, the more likely they are to be correlated with the rare independent variable (LM/GF) if some people are answering at random — which they apparently were.

Anyway, the Cheng and Powell paper speaks for itself. But I find it interesting that unchecked data error produces false positive (that is, negative) outcomes for marginal groups. Look out!

15 Comments

Filed under Research reports