Policy, politics, and promoting education versus marriage

Here are three ideas I disagree with:

1. Most people aren’t smart enough to make going to college worth it.

Maybe the best-known purveyor of this idea is Charles Murray, who argued in his 2008 book Real Education (offshore bootlegged copy here) that the “consensus intellectual benchmark” for understanding real college-level material is an IQ of 115, which by definition is only 16% of the population — but probably only 10% are really, truly smart enough (and efforts to improve education at lower levels to prepare more people for college are futile, so don’t even think about spending more on education, because so many people are “born lazy“).

2. We’ve done so much for poor people, it’s time for them to do something for themselves.

This is clearly related to idea #1, insofar as the government spends billions of dollars educating people for college — and subsidizing the colleges they attend — who could instead just work hard and enjoy life in a job requiring less education. But it extends to all kinds of social welfare and anti-poverty programs, as illustrated by the exasperated people in the policy establishment from Brookings to Heritage.

3. Poor women should get married before they have children.

This idea is pervasive, as I’ve discussed many times under the single mothers tag, in response to people blaming single mothers for rising inequality, poverty, low upward mobility, and crime.

One response

Here I offer one response to these three ideas combined. It is possible to increase access to college education, which would increase stability and opportunity for poor people and their children.

In demography, there is a long-running debate over whether there is a biological limit to human longevity, and whether and how fast we may be approaching it. Regardless of the ultimate answer, so far it’s clear that projections based on an inevitable tapering off of increases in life expectancy have repeatedly proved wrong (here’s a review and a recent paper). The same might be said of college education. Here is the trend in 25-34 year-old U.S. civilians with at least a BA degree, from Census numbers:

There was more talk about hitting the limits of college access 10 years ago, but even then it was increasing rapidly among women. Yes, we can and should improve college education. But I see nothing here to suggest a ceiling approaching. Still, people keep assuming that expanding education isn’t feasible.

For example, while Murray holds forth on the intelligence limitations among the poor, his colleague Brad Wilcox argues for a cultural press on those with less than a college degree:

They can go down the road of not having marriage as the keystone to their family formation, family life, or we can hold the line, if you will, and try to figure out creative strategies for strengthening marriage in this particular middle demographic in the United States.

In addition to upscaling their deficient values, however, couldn’t we also move them out of the less-than-college category altogether? Not so fast, says Wilcox in a recent interview:

On the education front, the U.S. spends a ton of money and devotes unparalleled attention to college. But the reality is that only one-third of adults, even today, will get a college degree, a B.A. or B.S. We can do a lot better in both funding and focusing on vocational education and apprenticeship training.

Really, America, be reasonable: Our “ton of money” is “unparalleled.” Don’t set your sights too high. Who do you think you are, anyway, Poland (college graduation rate: 53%), Ireland (46%), or Portugal (41%)? From OECD numbers:

I know expanding college access (the real kind, not the for-profit kind) suggests expanding a broken financial aid system, and the economic returns aren’t guaranteed, but for my purposes it’s not just about getting a better job. People who go to college — and those who know they are going to go to college before they do — usually delay having children, not because some moralizing think tank tells them it’s wrong, but because they’re trying to rationally sequence their lives. Of course, married couples have relatively low poverty rates, but even for parents who aren’t married, higher education sure helps. From the American Community Survey via IPUMS.org:

Trying to get more poor people to get married is both offensive and useless. But increasing access to higher education is both uplifting and useful. The choice between early birth with low education and later birth with higher education is not hard to make, but unless it’s feasible — with a readily apparent, practical, path toward completion — there is no choice to make.

The increase in college education has already helped keep child poverty levels from rising as marriage rates have fallen. Among women old enough to have finished college (ages 22-44) the percentage of babies born to mothers with college degrees (married or not) has increased from 23% in 1990 to 35% in 2010. From the Current Population Survey via IPUMS.org:

Promoting marriage among the poor is a moralizing salve for the self-esteem — and anti-tax self-interest — of pious elites, with zero proven success in helping anybody poor. Promoting access to higher education is good policy and good politics.

Repeated misinterpretation is not causation

The other day I criticized Brad Wilcox and Bob Lerman for claiming that increasing marriage would reduce inequality. In that post I passed up the chance to reinforce a lesson about misleading claims regarding selection and unobserved factors by members of the right-wing family social science community.

Which comes first, social advantages or marriage?
Photo by Jonathan Tellier from Flickr Creative Commons.

Passages like the following have become standard for Wilcox when he makes overblown claims regarding the benefits of marriage. Here is the latest:

Notwithstanding this report’s extensive data analysis, we do not claim that the associations we find among family structure while growing up, marriage as an adult, and economic outcomes are definitively causal. … Even after netting out the effects of many observed differences among individuals, both marriage and economic well-being may be the result of some third factor, such as unobserved differences in personality or character … Moreover, most of the evidence in this report is descriptive and does not derive from a causal model. For all these reasons, this report cannot definitively assert that adolescent family structure and adult marital status have a causal impact on individual and family economic well-being. …  Nevertheless, the evidence is widespread and consistent enough to suggest strong, causal positive roles for being raised in an intact family and for current marriage on a range of important economic outcomes for the average American.

So this is the criteria for evaluating whether selection and omitted variables are a problem — whether the “evidence is widespread and consistent enough”? No. The volume of evidence is irrelevant; what matters is what it means. If people with various kinds of advantages and privileges are more likely to get and stay married, then research that fails to take that into account will always show married people doing better than people who aren’t married. The evidence will be “widespread and consistent,” and that does not mean it means marriage is the cause of their advantages.

I think that repeating this over and over, having been corrected on it many times, qualifies as demagoguery, or the practice of a demagogue, as the OED defines it:

…a political agitator who appeals to the passions and prejudices of the mob in order to obtain power or further his own interests; an unprincipled or factious popular orator.

It’s appealing to passions and prejudices, and taking advantage of the credulity of the friendly media — and abusing their status as professional researchers, in Wilcox’s case with academic tenure — in the service of their own ideological and material interests.

I made this argument in a previous post, which demonstrated widespread and consistent evidence for an assertion that is probably not true because of obvious selection bias: Cars improve child health. There is no end to the ways you can demonstrate this pattern (and I controlled for income, to show how far that gets you), but the ubiquity of the evidence does not correspond with the veracity of the claim that the relationship is causal.

Real research addendum

If, unlike Wilcox and Lerman, you want to consider this set of issues seriously, I must say I don’t mean to imply that there is no causal effect of marriage on anything. But real research that rigorously takes selection into account usually finds the remaining (probable) effects of marriage are small, if still theoretically important. With earnings, for example, a substantial part of men’s marriage effect is due to selection — that is, men who are either already earning more or who are headed for higher earnings are more likely to get married. For example, this recent study by Christopher Dougherty finds that men’s earnings start rising on average more than five years before marriage. You could attribute this to the cultural power of marriage if you think it shows men getting their act together and earning more because they want or plan to get married, but there’s no evidence for that over the interpretation that marriage is a windfall that follows from other advantages — such as physical or mental traits or health, or social advantages such as rich networks of job and relationship connections (none of which is measured directly in the kinds of data we have for these purposes). Alexandra Killewald and Margaret Gough report a similar pattern, although it’s not the focus of their paper. Killewald, in another good piece, does a lot of selection checking for the positive effect of married fatherhood on men’s earnings, before coming down on the side of  a causal story. But her effect, although important, is not anywhere near large enough to lift poor people out of poverty or substantially reduce income inequality in the unlikely event that marriage increased among low-income parents. That’s a different discussion. As I argued the other day, even if marriage is good for married people, those who aren’t married are very unlikely to get the same benefit from marrying that we observe among the married population. They’re different people with different (mostly fewer) assets to capitalize on in marriage.

Turns out marriage and income inequality go pretty well together

Diatribe first, then critique.

Brad Wilcox and Bob Lerman have a new report arguing, among other things:

Had marriage rates not declined substantially among parents, many more families would have attained middle-class incomes, and the inequality across families would have increased at a slower rate.

It’s well established that falling marriage rates are contributing to family income inequality. However, increasing inequality is not an inevitable result of low marriage rates. In general, among rich countries, higher marriage rates are associated with higher levels of income inequality. The USA is a clear outlier here:

It’s possible marriage increases income inequality in general. It’s also possible that people don’t get married as much when they’re not worried about inequality. Regardless, this shows high marriage rates are quite compatible with high inequality.

Falling marriage does contribute to rising inequality in the USA, because of how it’s manifesting: increasing selectivity in marriage, so that richer people are getting and staying married more; and increasing social class endogamy, so that there are more two-high-income families lording over more one-low-income families. And all of that is exacerbated by widening underlying inequality, with high-end incomes pulling away from low-end incomes, relatively unchecked by income redistribution.

One obvious solution is to take money away from married high-income people and give it to single low-income people. With all the benefits that married people get — many of them through no special effort of their own, but rather as a result of their social status at birth, race, health, good looks, legal perks, or lucky breaks — it seems reasonable to tax marriage, like a windfall profits tax, or an inheritance tax, or a progressive income tax. But, if you’re squeamish about taxing something “good” like marriage, then just taxing wealth a little more would accomplish much the same thing. This elegant solution would decrease inequality, increase well-being for poor people, and equalize life chances for children (who are the future, I believe). In other words, it’s out of the question.

A second, less-obvious (but more-often mentioned) solution is more marriage. Low-income single people could become high-income married people. Or, failing that (which they would) they could settle for becoming low-income married people. Besides the fact that efforts to promote marriage have been a complete failure, would this even make poor single people and their children better off?

The family science right-wing establishment says Yes. To the poor singles, they say: “See how well married people are doing? Get married and you’ll be like them (also: you won’t get raped so much, you sluts.)” To their rich donors and political allies, they say, “Make them earn their benefits by demonstrating their moral fiber and manning up.” The welfare reform attempted this, and successfully forced many single mothers into the labor force in the cause of character development  — but it failed in its goal of marrying them off.

So more marriage is the new agenda — and the family right has a plan that leads inexorably to success (for them): either by successfully raising marriage rates among the poor (extremely unlikely), or by justifying the continued denial of basic welfare to the poor and shoring up the political case against economic redistribution (extremely likely).

A few notes on the first part of their report

Question: Why should we think the unmarried people would get the same benefits from marriage that currently married people do? If marriage is becoming increasingly selective, then you can’t assume the benefits observed among actually married people would be reaped by those who have been left out (or opted out) of the increasingly stringent marriage selection process. They may not have the assets that lead to marriage benefits — skills of many kinds, wealth, social networks, and so on.

Wilcox and Lerman say family income would have risen more — and there would be less inequality — if more people were married, because married couple incomes rose faster than average. They show this:

Setting aside the completely misleading use of an area chart, and the gruesome y-axis truncation, this shows that married-parent families have had faster than average income growth. One obvious reason for this is women’s rising labor force participation, at least into the 1990s. That has a big effect on income at the median, which is the line this is showing for each group (though the area form makes it look like it’s some kind of distribution). Rising income at the median would reduce income inequality. The fact that single-parent families are dragging down the average contributes to growing inequality and a stagnant overall median.

But the top is where most inequality is being generated. Looking at the top will help us see not just growing inequality, but also why getting poor people to get married won’t help them as much as Wilcox and Lerman think it would. Let’s add the 90th and 10th percentiles to the married parent income trends. My figure shows that the married parent family’s 90th percentile’s income has risen 39% since 1979, while the median has risen 14%. But the 10th percentile’s income has fallen 12%.

So, if poor single people finally get with it and start getting married, which married parents are they going to look like?

The chart shows dramatically increasing inequality among married-couple families. Pouring more married couples into the bottom of the distribution doesn’t seem likely to fix that. And, as Jordan Weissman pointed out, the family structure story has nothing to do with the huge rise in incomes in the top 1% and .1%, which are central to the inequality story.

Till now I’ve skirted some thorny technical issues to make a comparison comparable to Wilcox/Lerman’s data. But assessments of family income inequality are tricky. Marrying two low earners creates one family household with twice the income. That shows up as a rise in incomes per family, but what is the real gain? They get economies of scale, but most descriptions (like Wilcox/Lerman’s) don’t take that into account. And the children might increase their consumption from greater access to the second income, but that’s hidden within the family black box.

To see how changes in family income distributions affect children, it’s useful to use a family size adjustment. I like one in here that counts kids as seven-tenths of an adult, and scales the family income by .65. (So you just divide family income by this: [(adults+(.70*kids)).^65].) Now you can track children’s cash on hand much better. I also prefer to use household rather than family income and composition, because the Census definition of families is narrow. In the charts so far, for example, parents’ cohabiting partners’ income is not included.

So here is the inequality trend for children — using the Gini index for needs-adjusted household income (code here) — by parents’ marital status:

This shows that the increase in family inequality has been much more dramatic for married-couple families than single-parent families. That’s those high-income couples pulling away from the middle and the bottom. On the other hand, inequality has been and remains higher for single-parent families. Note that the inequality for all children is not just the average of the two other lines, because it also includes the inequality between married-couple and single-parent families.

So moving people from single to married would have reduce inequality more in 1980 than now, but just on composition it might still help if it boosted cash per kid through access and efficiency. Whether that benefit would outweigh the costs is not clear. If people not married yet aren’t just like the people who are — they may have lower skills and resources of various kinds, for example — marriage might not facilitate those transfers. Plus, it’s only good if the people want to be married.

Anyway, point is, married-couple families are doing pretty well at increased income inequality all by themselves.

My rejection of the National Marriage Project’s “Before ‘I Do'”

All day today, “The Decisive Marriage” has topped the New York Times most-emailed list. The piece is a Well Blog post, written by Tara Parker-Pope, which reports on a report published by the National Marriage Project and written by Galena Rhoades and Scott Stanley, “Before ‘I Do': What Do Premarital Experiences Have to Do with Marital Quality Among Today’s Young Adults?”

I have frequently criticized the National Marriage Project, run by Bradford Wilcox (posts listed under this tag), and I ignore their work when I can. But this report is getting a lot of attention now and several people have asked my opinion. Since the research in the report has not been subject to peer review, and the Pope piece does not include any expert commentary from non-authors, I figured I’d structure this post like the peer review report I would dash off if I had been asked to review the piece (it’s a little different because I have access to the author and funding information, and I wouldn’t include links or graphics, but this is more or less how it would go if I were asked to review it).

Before “I Do”

This paper reports results from an original data collection which sampled 1,294 people in 2007/08, and then followed an unknown number of them for five years. The present paper reports on the marriage quality of 418 of the individuals who reported marrying over the period (ages 18-40). The authors provide no information on sample attrition or how this was handled in the analysis, or the determinants of marriage within the sample. Although they claim (without evidence) that the sample was “reasonably representative of unmarried adults,” they note it is 65% female, so it’s obviously not representative. More importantly, the analysis sample is only those who married, which is highly select. Neither sexual orientation of the respondents, nor gender composition of the couples described is reported.

The outcome variable in the study is a reasonable measure of “marital quality” based on a four-item reduced-form version of the Dyadic Adjustment Scale (originally developed by Graham Spanier), which includes these items:

• How often do you discuss or have you considered divorce, separation, or terminating your relationship?
• In general, how often do you think that things between you and your partner are going well?
• Do you confide in your mate?
• Please circle the dot which best describes the degree of happiness, all things considered, of your relationship.

The authors provide no details on the coding of these items, but say the scale ranges from 0 to 21, and their sample included people who scored from 0 to 21. However, the mean was 16.5 and the standard deviation was 3.7, indicating a strong skew toward high scores. Inexplicably, for the presentation of results the authors dichotomize the dependent variable into those they classify as “higher quality,” the 40% of respondents who scored (19-21), versus everyone else (0-18). To defend this decision, the authors offer this non-explanation, which means exactly nothing:

This cut point was selected by inspection of the distribution. While it is somewhat arbitrary, we reasoned that these people are not just doing “above average” in their marriages, but are doing quite well.

The average marriage duration is not reported, but the maximum possible is 5 years, so we are talking about marriage quality very early in these marriages.

The main presentation of findings consists of bar graphs misleadingly labeled “Percent in Higher-Quality Marriages, by…” various independent variables. These are misleading because, according to the notes to these figures, “These percentages are adjusted for race/ethnicity, years of education, personal income, religiousness, and frequency of attendance at religious services.” Here is one:

The method for arriving at these “adjusted” percentages is not given. This apparently confused Parker-Pope, who reported them as unadjusted percentages, like this:

People who lived with another person before marrying also reported a lower-quality relationship. In that group, 35 percent had higher-quality marriages. Among those who had not lived with another romantic partner before marriage, 42 percent had higher-quality marriages.

The statistical significance of this difference is not reported. However, if this were a simple difference of proportions, the difference would not be statistically significant at conventional levels (with a sample of 418, 39% of whom lived with someone else before, the test for difference of proportions for .42 and .35 yields a z-score of 1.43, p=.15). The full report includes an appendix which says they used multilevel modeling, but the form of the regression is not specified. The regression table provided includes no fit statistics or variance components so the efficacy of the model cannot be evaluated.

Regression says: Adding 100 people to the wedding party 5 times would not equal the effect on marital quality of not being Black.

Much is made here (and in the Pope article about these findings) about the wedding-size effect. That is, among married couples, those who reported bigger weddings had higher average marriage quality. The mean wedding size was 117. In the regression model, each additional wedding guest was associated with an increase in marriage quality (on the 0-21 scale) of .005. That is, if this were a real effect, adding 100 wedding guests would increase marital quality by half a point, or less than 1/7 of a standard deviation. For comparison, in the model, the negative effect of being Black (-2.69) is more than 5-times greater than the effect of a 100-guest swing in wedding attendance. (The issue of effect size did not enter into Pope’s description of the results.)

The possibility of nonlinear effects of wedding size or other variables is not discussed.

Are the results plausible?

It is definitely possible that, for example, less complicated relationship histories, or larger weddings, do contribute to marital happiness early in the marriage. The authors speculate, based on psychological research from the 1970s, that the “desire for consistency” means “having more witnesses at a wedding may actually strengthen marital quality.”

Sure. The much bigger issue, however, is two kinds of selection. The first, which they address — very poorly — concerns spurious effects. Thus, the simplest explanation is that (holding income constant) people with larger weddings simply had better relationships to begin with. Or, because personal income (not couple income — and note only one person from each couple was interviewed) is at best a very noisy indicator of resources available to couples, big weddings may simply proxy for wealthier families.

Or, about the finding that living with someone else prior to the current relationship is associated with poorer marriage quality, it may simply be that people who have trouble in relationships are more likely to have both lived with someone else and have poor quality marriages later. Cherlin et al. have reported, for example, that women with a history of sexual abuse are more likely to be in transitory relationships, including serial cohabiting relationships, so a history of abuse could account for some of these results. And so on.

The authors address this philosophically, which is all they can do given their data:

One obvious objection to this study is that it may be capturing what social scientists call “selection effects” rather than a causal relationship between our independent variables and the outcome at hand. That is, this report’s results may reflect the fact that certain types of people are more likely to engage in certain behaviors—such as having a child prior to marriage—that are correlated with experiencing lower odds of marital quality. It could be that these underlying traits or experiences, rather than the behaviors we analyzed, explain the associations reported here. This objection applies to most research that is not based on randomized experiments. We cannot prove causal associations between the personal and couple factors we explore and marital quality.

However, because they have rudimentary demographic controls, and the independent variables chronologically precede the outcome variable, they think they’re on pretty firm ground:

With the help of our research, we hope current and future couples will better understand the factors that appear to contribute to building a healthy, loving marriage in contemporary America.

This is Wilcox’s standard way of nodding to selection before plowing ahead with unjustified conclusions. This is not a reasonable approach, for reasons apparent in today’s New York Times. Tara Parker-Pope does not mention this issue, and her piece will obviously reach many more people than the original report or this post.

They hope people will take their results as relationship advice. In Pope’s piece, Stanley offers exactly the same advice he always gives. If that is to be the case, the best advice by far — based on their models — is to avoid being Black, and to finish high school. Living with both one’s biological parents at age 14 helps, too. In relationship terms, unfortunately, most of the results could just as easily reflect wealth or initial relationship quality rather than relationship decisions, and thus tell us that people who have healthy (and less complicated) relationships before marriage have healthy relationships in the first few years after marriage.

Perhaps more serious, however, for this study design, is the second kind of selection: selection into the sample (by marriage). Anything that affects both the odds of marrying and the quality of marriage is potentially corrupting these results. This is a big, complicated issue, with a whole school of statistical methods attached to it. Unless they attend to that issue this analysis should not be published.

On the funding

The authors state the project was “initially funded” by the National Institute of Child Health and Human Development, but the report also acknowledges support from the William E. Simon Foundation, a very conservative foundation that in 2012 gave hundreds of thousands of dollars to the Witherspoon Institute (which funded the notorious Wilcox/Regnerus research on children of same-sex couples), the Heritage Foundation, the Hoover Institute, the Manhattan Institute, and other conservative and Christian activist organizations. Details on funding are not provided.

The National Marriage Project is well-known for publishing only work that supports their agenda of marriage promotion. Some of what they publish may be true, but based on their track record they cannot be trusted as honest brokers of new research.

James Wright’s recounting of the Regnerus review process wasn’t true

(And Brad Wilcox lied continuously, too.)

It may seem like a footnote to the Regnerus scandal (last summary here), but I think it is worth reporting that we now know Social Science Research editor James Wright apparently lied in his published description of the process by which the Regnerus paper was published.

In the “Introductory Remarks” that Wright published in the November 2012 issue of SSR, he described the sequence of events leading up to the paper’s publication, writing in part (with false portion highlighted):

The [Loren] Marks paper was submitted to SSR on October 3, 2011, and had already been accepted for publication (subject to some pretty significant revisions) when the Regnerus paper was submitted on February 1, 2012. Like most journals, SSR often tries to co-publish topically linked papers … and given the obvious topical similarity of these two papers, publishing them at the same time seemed sensible (assuming, as goes without saying, that both fared well in peer review). The email sent to prospective reviewers of the Regnerus paper therefore stated, “I would greatly appreciate the quickest possible turnaround on your review” but was otherwise identical to the form letter sent to all prospective reviewers when requesting reviews.

In this telling, Wright’s motivation for encouraging a quick turnaround was that he wanted to publish the two papers together, and that’s why (“therefore”) he asked the reviewers to expedite their reviews.

But, Straight Grandmother has published the email that Wright sent to reviewer Brad Wilcox, and it does not match Wright’s published description. In that email, Wright wrote:

We have received a manuscript that we think may interest you. We would very much appreciate your reading it and rendering a critique.

We have also learned that a report on this study will be released sometime this coming summer and if the paper is destined to appear in SSR, it would be nice to have the paper accepted (and available online) before the report is released. So I would greatly appreciate the quickest possible turnaround on your review.

Here is the grainy public-records version, for authenticity:

Clearly, the highlighted passage in the first quote was not the only passage that made the Regnerus request different. In his “Introductory Remarks,” Wright omitted mention of the summer report deadline. And the email to Wilcox does not mention the goal of publishing the Regnerus and Marks papers together.

Why would Wright change the story, from one about trying to publish the Regnerus paper in time for the summer report (told to Wilcox) to one about trying to publish two topically-related papers together (told to the public)? The answer, I conclude, is that in his published accounting Wright was attempting to distance himself from the appearance (fact) of coordination with Regnerus and his backers (including Wilcox).

Wright’s story of the dog wagging the tail is reversed. Regnerus and Wilcox needed to have the peer-reviewed paper accepted and online before they could release the “report” publicly, because they wanted that legitimacy (this is apparent in the first document dump). Wright’s actions made that strategy successful. (When it appeared, the “report” was just an animated website rehashing the contents of the paper.)

Wilcox lies, too

Brad Wilcox will say this was not a lie, because he thinks he carefully did not lie, but it was a lie, because lying is about deception, not just about uttering words that are literally untrue. Take it from the Catechism of the Catholic Church, in which Pope John Paul II wrote:

[Quoting St. Augustine] “A lie consists in speaking a falsehood with the intention of deceiving.” … To lie is to speak or act against the truth in order to lead someone into error.

I don’t know this first-hand, but I’m told God does not give partial credit for lies of omission. In his response to Wright’s reviewer request, Wilcox’s entire reply was this:

Dear Jim:

I’m happy to do this. Just want to let you know that I serve on the advisory board for this project — as does Kelly Raley and others on the SSR board. Ok?

Brad

You could call this a head-fake disclaimer. What is the relevance of the advisory board? It is certainly not the most important fact about Brad’s involvement with the study. We probably don’t know all that he did, but we do know that Brad coordinated the fundraising for the study, recruited Regnerus to be the lead researcher, advised Regnerus on how to handle co-authorship with Cynthia Osborne, suggested to Regnerus that they send the paper to Wright at SSR, and referred to the research project as “our dataset.”

So, sure, the email exchange contains a disclosure — one that puts Wilcox on the same level of involvement as other fleeting consultants — but it is far from the most important thing to disclose. That’s lying.

Did Wright lie some more?

After Brad’s response to the reviewer request, they exchange two more emails, which read, in their entirety:

Wright: Understood.

Wilcox: thanks.

So why, in his email to Inside Higher Ed, did Wright say this?

Amato and Wilcox mentioned their prior involvement with the Regnerus study in response to my initial reviewing request.  I asked, as I always do, whether this involvement precluded their writing an objective review. Both said no and so both were asked to proceed.

Perhaps there was a followup exchange in which Wright wrote to Brad, “Oops, forgot to ask, as I always do: Will this involvement preclude you writing an objective review?” But if there wasn’t, then Wright lied again. One can’t help suspecting that Wright did not expect his actual email exchange to be published.

In Darren Sherkat’s report on the journal’s review process, incidentally, he wrote:

Two of the reviewers indicated that they had a potential conflict of interest related to consulting on the Regnerus paper but both averred that this consulting relationship would not preclude an objective, critical assessment.

If this is supposed to be a description of the Wright-Wilcox exchange Straight Grandmother has published, then it also appears not to be true — Wilcox didn’t tell that particular lie. I don’t know the source of Sherkat’s information on that point, but it might well just be Wright’s say-so.

The shifting boilerplate

I don’t know the content of all of Wright’s requests to reviewers, or what he “always” asks, but I have some circumstantial evidence. A review request that Wright sent to someone I know the same month as the Regnerus paper is identical to the one Straight Grandmother published to Wilcox, except for the part about the summer report and the quick turnaround. So that appears to have been a form letter (the typos match as well). In that letter, Wright says SSR has single-blind reviews because:

…we feel it is important to give our reviewers an opportunity to be forthcoming about potential bias prior to rendering a critique or decline to review for fear of compromising professional ties with the authors.

It doesn’t ask them whether anything “precluded their writing an objective review.” However, the boilerplate seems to have changed. The last review request I received, in early 2013, included a passage that is not in the email he sent to Wilcox or my informant:

Agreeing to review a paper for this or any journal is simultaneously an affirmation  that you harbor no conflicts of interest or past or current relationships with the author(s) that would preclude you from writing an honest, objective critique.  If this is not the case, our assumption is that you will decline to do the review.

So I guess Wright might say that he “always” asks this now, but it does not appear that he asked it of Wilcox (at least in the documents we have). Maybe he’s improving his practice. Maybe he’s covering his bases.

So, some of you may still be reviewing for James Wright at Social Science Research, or sending your papers to him. My question is, Why?

Deciphering a well-told data story, cars are good for kids edition

Brad Wilcox has written up his best case for how marriage protects women and girls from violence. I discussed his initial post earlier, but the blowup has prompted me to provide more general advice for the critical data citizen — reader, writer, and editor — who has to decide what to believe when someone comes at them with a data story.

I have some tips about that at the end, but first this elaborate setup.

The information in this section is true

Consider three stories:

• When Melanie Thernstrom’s toddler, Kieran, first ate cheese, he immediately had a massive allergic attack. His face swelled, his skin turned red and scaly, and he started gasping for breath. They jumped in their car and rushed to the hospital, where doctors were able to save him.
• Chicago mother Tynisha Hilliard had six children in the car when someone opened fire. “Mommy, I’m shot,” said her nine-year-old boy from the back seat. Hilliard immediately sped to the nearest hospital. “My reaction was to save my son. That’s all I can do, save my son,” she said. After emergency surgery for a gunshot wound to the chest, the boy was expected to survive.
• When Dodgers catcher A. J. Ellis’s wife, Cindy, went into labor, they hopped in the car and headed for NYU hospital, normally a 35-minute drive. Despite racing through traffic with a police escort, they didn’t make it in time – the baby was born in the back seat – but they arrived at the hospital moments later, met by an emergency crew that whisked mother and child to care and safety in the hospital.

What do these stories have in common? Children’s lives saved by cars.

Is this part of a wider phenomenon? I know what you’re thinking: The pollution from cars hurts children, the vast resources devoted to infrastructure for cars could be spent instead in ways that help children, the need for gas causes wars all the time, and the individualism promoted by car culture contributes to social isolation instead of community efficacy.

Maybe. But let’s theorize a little. Here are three ways cars might be good for children’s health:

• Kids whose families have cars can get them to doctors in an emergency. Considering that in modern societies a lot of what kills children is various kinds of accidents and medical emergencies, this could be a major advantage.
• Say what you want about individualism, but it’s emerged as a modern character trait in tandem with the cultural shift that brought us the view of children as priceless individuals. Car culture is a major prop of individualism, so it’s reasonable to hypothesize that people who drive individual cars are more totally devoted to their priceless individual children’s well-being (rather than, say, the well-being of children in general).
• Being able to transport oneself at will — any time, any place — may create a sense of self-efficacy, of mastery over one’s environment, which makes people refuse to accept failure (or illness or death), and thus devote themselves more confidently to their survival and the survival of their children.

Don’t take a theoretical word for it, though — let’s go to the data. Here are three small studies.

Cars and children’s health across countries

First we examine the relationship between the number of passenger cars per capita and the rate of child malnutrition in 110 countries (all the countries in the World Bank’s database that have measures of both variables in the last 10 years — mostly poor countries). The largest — India, China, Brazil, and the USA — are highlighted (click to enlarge).

This is a very strong relationship. This single variable, cars per capita, statistically explains no less than 67% of the variation in child malnutrition rates.

But, you liberals object, cars are surely more common in wealthier countries, so this relationship may be spurious. Sure, income and cars are positively correlated (r=.86, in fact). But when I fit a regression model with both per capita income and per capita cars, cars still have a highly significant statistical association with malnutrition (p<.001). (All the regression models are in the appendix at the end.)

Cars and child death rates across US states

Second, we take a closer look within the United States.  Here there is a lot less variation in both the number of cars and the condition of children. Still, there is a clear relationship between private cars per person and the death rate of children and teenagers: Children are substantially less likely to die in states with more privately owned passenger cars (click to enlarge).

Again, there is less variation in income between U.S. states than there is between countries of the world. But to make sure this is not just a function of state income, I fit a regression model with cars and a control for median household income. The statistical effect of private cars remains significant at the p<.05 level, confirming it is unlikely to be due to chance.

Car commuting and children’s disabilities within the US

Third, let’s go still further, not just comparing US states but comparing children according to the car-driving habits of their parents within the US. For this I got data on children’s disabilities (four kinds of disability) and the means of transportation to work for their parents using the 2010-2012 American Community Survey, with a sample of more than 700,000 children ages 5-11.

Sure enough, children who live with parents who drive to work are substantially less likely to have disabilities than those who don’t live with a parent who drives to work:

Again, could this be because richer families are more likely to include car-driving parents? The regressions (below) show that, although it is true that children in richer households are less likely to have disabilities, the statistical effect of parents’ commuting method remains highly significant in the model that includes household income.

In summary: Children are less likely to be malnourished if they live in a country with more cars per person; they are less likely to die if they live in a state with more cars per person, and they are less likely to have disabilities if they live with parents who commute to work by car. All of these relationships are statistically significant with controls for income (of the country, state, or family). These are facts.

One interpretation

Compare this analysis to the question of marriage and violence. In their piece for the Washington Post (discussed here), Brad Wilcox and Robin Fretwell Wilson wrote about #YesAllWomen:

This social media outpouring makes it clear that some men pose a real threat to the physical and psychic welfare of women and girls. But obscured in the public conversation about the violence against women is the fact that some other men are more likely to protect women, directly and indirectly, from the threat of male violence: married biological fathers. The bottom line is this: Married women are notably safer than their unmarried peers, and girls raised in a home with their married father are markedly less likely to be abused or assaulted than children living without their own father.

With the facts above I can accurately offer this parallel construction:

Some cars pose a real threat to the health and safety of children. But obscured in the public conversation about auto safety, pollution, and environmental degradation is the fact that some other cars are more likely to protect children, directly and indirectly, from threats to their health and safety: cars driven by their own, responsible, caring parents. The bottom line is this: Children in places with more cars — and in families where parents commute by car — are notably healthier than peers without cars.

At the end of his followup post, Brad concludes:

Of course, none of these studies definitively prove that marriage plays a causal role in protecting women and children. But they are certainly suggestive. What we do know is this: Intact families with married parents are typically safer for women and children. … That’s why the conversation about violence against women and girls … should incorporate the family factor into efforts to reduce the violence facing women and girls.

I am equally confident in my conclusion:

Of course, my brief studies don’t definitively prove that cars plays a causal role in protecting children’s health and safety. But they are certainly suggestive. What we do know is this: Societies and families with cars are typically safer and healthier for children. That’s why the conversation about children’s well-being should incorporate the car factor into efforts to reduce the harms too many children continue to experience.

Another interpretation

Both the marriage story and the car story are misleading data manipulations that substitute data volume for analytical power and present results in a way intended to pitch a conclusion rather than tell the truth.

When is a non-causal story “certainly suggestive”? When the person giving you the pitch wants you to believe the conclusion.

Please do not conclude from this that all data stories are equally corrupt, and everyone just picks the version that agrees with their preconception. Not all academics lie or distort their findings to fit their personal, political, or scientific conclusions. I may be more motivated to criticize Brad Wilcox because I disagree with his conclusions (and there may be people I agree with who use bad methods that I haven’t debunked), but that doesn’t mean I’m dishonest in my interpretation and presentation of evidence. Like a real climate scientist debunking climate-change deniers, I am happy that discrediting him is both morally good and scientifically correct (and I think that’s not a coincidence).

There are two main problems with both the cars story and the marriage story. First is selection into the independent variable condition (marriage and car ownership). People end up in these conditions partly because of their values on the dependent variable. For example, women in marriages are less likely to be raped on average because women don’t want to marry men who have raped them, or likely will rape them — the absence of rape causes marriage. In the case of children with disabilities, there is evidence that children’s disabilities increase the odds their parents will divorce (which means at least one of the parents isn’t in the household and so can’t be a car-commuting parent in the ACS data).

The other main problem is omitted variables. Other things cause both family violence and children’s health, and these are not adequately controlled even if researchers tell you they control for them. Controlling for household income (and other easily-measured demographics) does not capture all the benefits and privileges that married (or car-owning) people have and transfer to their children. For tricky questions of selection and omitted variables, we need to get closer to experimental conditions in order to provide causal explanations.

Tips for critical reading

So, based on Wilcox’s car story and my car story, here are practical tips to help you avoid getting hoodwinked by a propagandist with a PhD — or a data journalist looking at a mountain of data and a tight deadline. These are some things to watch out for:

Scatter plot proof

Impressive bivariate relationships; they may be presented with mention of control variables but no mention of adjusted effect size. That’s what I did with my scatter plots above. If you have adjusted results but don’t show them, it’s selling a small net effect with a big unadjusted label. (Wilcox examples here; Mark Regnerus does this, too.)

Axis truncation

A classic example is the Obama food stamp meme, but Wilcox had a great example a few years ago when he wanted to show the drop in divorce that resulted from hard times pulling families together during the recession. If you assume divorce is always going up (it fell for decades), this looks like a dramatic change (he called it “the first annual dip since 2005″):

No head-to-head comparison of alternative explanations

This is a lot to ask, but real social scientists take seriously the alternative explanations for what they observe, and try to devise ways to test them against each other. Editors often see this as a low-hanging fruit for removal, because cutting it both shortens the piece and strengthens the argument. In the rape versus marriage story, Wilcox nodded to the alternative explanation that “women in healthy, safe relationships are more likely to select into marriage” — which he called “part of the story” — but he offered nothing to help a reader or editor adjudicate the relative size of that “part” of the story. This connects to the next red flag.

Greater than zero proof

Sometimes just showing that something exists at all is offered as evidence of its importance. That’s why I included three anecdotes about children being saved by private passenger cars — it happened, it’s real. The trick is to identify whether something matters in addition to existing. Here’s a Wilcox example where he showed that a tiny number of people said they didn’t divorce because of the recession; here’s an example in which Nate Cohn at the NYTimes Upshot said that 2% of Hispanics changing their race to White was “evidence consistent with the theory that Hispanics may assimilate as white Americans.” Neither of these provide any comparison to show how important these discoveries were relative to anything else — other reasons people delay divorce? other reasons for race-code changes? — they just exist. This is reasonable if you’re discovering a new subatomic particle, but with social behavior it’s less impressive.

Piles of studies

The reason I presented the car results as the three separate “studies” was to make the point that you can have a lot of studies, but if none of them prove your point it doesn’t matter. For example, in his post Wilcox linked to a series of publications about how children whose parents weren’t married were more likely to be sexually abused, but none of them handle the problem of selection into marriage I described above. Similarly, a generation of research showed that women who have babies as teenagers suffer negative economic consequences, but those effects were all exaggerated because people didn’t take selection into account (women with poor economic prospects are more likely to have babies as teenagers).

Describing one side of inequality as a social good

Let’s say that, in street fights, the person with a gun beats the person with a knife more than 50% of the time. Do we conclude people should have more guns? Some benefits are absolute and have no zero-sum quality to them. (I can’t think of any, but I assume there are some.) Normally, however, we’re talking about relative benefits. The benefits of marriage, or the economic benefits of education, are measured relative to people who aren’t married or schooled.

The typical description of such a pattern is, “This causes a good outcome, we should have more of it.” But we should always consider whether the best thing, socially, might be to reduce the benefit — that is, solve the problems of the people who don’t have the asset in question — rather than try to increase the number of people with the asset.

The benefit of cars that comes from being able to get to the hospital quicker may only be relative to the poor suckers stuck in an ambulance while your personal cars are blocking up Manhattan.

Ambulance stuck in Manhattan, by Philip Cohen

Appendix: Regression results

Final proof there is no human tragedy Brad Wilcox will not exploit in order to promote marriage

I’m not going to dignify this with a thorough debunking, but here’s a quick note to highlight the evil that walks among us in academic robes.

Brad Wilcox and Robin Fretwell Wilson wrote a piece for “Post Everything” at the Washington Post that was originally titled like this:

The post didn’t specifically say what’s in the headline, but in this case I have to give credit to the overreaching headline writer for accurately capturing the basic message of the piece. What Brad wants to do is make people think that without exactly saying it. Erin Gloria Ryan on Jezebel wrote a good alternate headline for it, too: “Violence Against Women Will End When You Sluts Get Married, Says WaPo.”

Their audience is married people who feel superior to women who aren’t married, who want to coerce women into marriage — or cast them out. The friendly side of this is paternalistic shaming, the unfriendly side is violent shaming; both are expressions of patriarchal outlook. Their conclusion:

And, most fundamentally, for the girls and women in their lives, married fathers provide direct protection by watching out for the physical welfare of their wives and daughters, and indirect protection by increasing the odds they live in safe homes and are not exposed to men likely to pose a threat. So, women: if you’re the product of a good marriage, and feel safer as a consequence, lift a glass to dear old dad this Sunday.

I can’t help reading this without hearing a voice that says, “We can do this the easy way, or the hard way.”

After the interwebs’ head exploded over the headline, Brad tweeted, “Working to match title w text,” and then a new headline appeared:

The new headline is supposed to be less offensive, I suppose, but it amounts to the same thing. And it’s based on the same correlations in the post. There is still nothing in the post to show that adding marriage to a random relationship would reduce the odds or level of intimate partner violence. So the implication is the same: shame on you.

On Twitter, Marina Adshade pointed out that marriage rates and violence rates have both been falling for several decades. Brad’s response was, “Fair enough. But the question is this: Would they have fallen even more if marriage was stronger?” That’s a question he should probably have asked before writing the piece.

Can you imagine what he would do if he had the opposite result to work with — an increase in violence during a period of decreasing marriage?

We don’t have to imagine, actually, because he and his marriage-promoting compatriots at the National Marriage Project were all over that in the 1990s. To choose one example I have handy, William Galston, who sits on Brad’s board of advisors at NMP, wrote in 1991 in the New Republic (12/2/91) that, “The American family has changed dramatically in the past generation, and it is children who have paid the price.” We needed, he said, to “relegitimate the discussion of the links between family structure and a range of social ills.” Indeed, “theft, violence, and the use of illicit drugs are far more prevalent among teenagers than they were thirty years ago.” Now, as “revolution in the American family” has reached unprecedented levels, crime has fallen for two decades. <Crickets>

As a spoof — but with real data — I illustrated Adshade’s point. Here is the relationship between marriage prevalence and intimate partner violence rates:

That curvilinear statistical relationship explains 84% of the variance in intimate partner violence rates. If you add the linear time trend, the variance explained jumps to 92% and the effects of marriage remain highly significant.

Wow.

If I were like Brad on the other side of this debate, the news story would read like this:

“We had reason to believe marriage was harmful, on average,” said Prof. Cohen. “But I was surprised by the strength of the relationship, especially the fact that the effect seems to accelerate at higher levels of marriage, as if marriage feeds off itself in a violence loop.” Although further research will be needed to confirm the findings, he added, the statistical association is very strong. “The bottom line is that intimate partner violence is much less common in years when marriage is more rare.”

However, I am not seriously suggesting that the decline in marriage has caused the decline in violence (although reduced exposure of women to men in general may be one factor). In fact, if you add the curvilinear effect of time, the variance explained rises to 95% — and marriage effects disappear. But the fact that violence has dropped so much while marriage has plummeted means Brad has a steeper hill to climb to make his case. It’s not enough to say, maybe violence would have declined even more. This is not one of those random spurious correlations, these are two large social trends affecting whole swaths of the population, and the correlation directly contradicts his theory. When there is a plausible connection, or the trends at least affect the same people, the burden is on the one going beyond the existing evidence to reconcile the hypothesis with the available circumstantial evidence.

But none of this matters to Brad*, or, apparently, Robin Fretwell Wilson. Their conclusion is predetermined. There is nothing that would lead them to conclude that society would not be improved by more marriage. It’s just a case of picking a subject in the news, picking some facts, and repeating their conclusions. And I think it’s appalling.

* If you’re wondering why I seem to be picking on Brad individually, please rest assured it’s nothing personal. If there was any other sociologist who behaved as poorly as he consistently does I would pick on them, too. For endless details, follow the National Marriage Project tag.