A tale of two studies: poor research leads to poor findings on minimum wage

June 26, 2017 | Marilyn Watkins

Photo: elmundoenbici via Flickr Creative Commons

Photo: elmundoenbici
via Flickr Creative Commons

Seattle’s economy is booming: construction everywhere, crowded streets and transit, housing costs soaring, bustling neighborhood restaurants, and a 2.6% unemployment rate. Much of this growth is driven by high wage-tech jobs and the spillover effect of all those workers eating out, shopping, and paying premium prices.

It’s in this context that Seattle instituted its higher minimum wage ordinance in 2015. In the past week, two studies have come out with very different conclusions on the impacts of those wage increases on low wage work – one says it’s positive, and the other negative. But the two studies are not created equal.

The first study, led by Michael Reich and Sylvia Allegretto based at the University of California, Berkeley, concludes that the 2015 and 2016 increases to $11 and $13 an hour had the intended effects of raising incomes for low-wage workers without having discernible impact on the number of jobs. These findings are consistent with the bulk of economic studies of minimum wage increases over the past couple of decades.

In the second, a University of Washington team concluded that the 2016 wage increase reduced the number of low-wage jobs by 9% and actually lowered the incomes of low-wage workers. This diverges from the majority of economic research. Across the U.S., city, state, and federal governments have changed minimum wages dozens of times over the past two decades. Multiple economists from across the ideological spectrum have studied these changes, and even opponents of minimum wage increases have not found impacts anywhere near the scale of the UW team.

The UW’s counter-intuitive findings underscore several methodological flaws:

  • They limit their study only to single-site establishments, because their data could not distinguish whether employees of multi-site chains – think Molly Moon’s, Mud Bay, Mod Pizza, Starbucks – actually worked inside or outside the city limits. That leaves 40% of workers excluded from their study. It also means that leaving a job at small business for a job at a larger company counts incorrectly as a job loss.
  • The UW team created a control by comparing Seattle’s employment statistics with other parts of the state. But there is no place in Washington that has a similar economy to Seattle. Seattle has an economy more like San Francisco or New York than Everett or Spokane. The Berkeley team used the more accepted methodology of generating a control from similar areas across the country, rather than just the state. Moreover, the Berkeley team compared numbers for the previous 5 years, while the UW only looked at the previous 9 months.
  • The UW study focused on jobs paying $19 an hour or less, making the assumption that fewer jobs in this bracket meant lost opportunity for workers who used to be in this pay range. But what we’re seeing in Seattle is that jobs that used to pay $18 an hour now pay $20 due to competition for employees. In the UW study, this was unaccounted for and incorrectly counted as job loss.

The quality of a study hinges on the quality of its methods. But the UW study was too myopic in its lens. It eschewed all of the hallmarks of good science – including all the data, equivalent control group, breadth of time. There’s a reason its findings go against what the vast majority of previous studies found: the UW study isn’t as academically rigorous.

If something seems too bad to be true, it probably is.

Tagged with: , , ,
Posted in A Fair Deal at Work, Minimum Wage


  1. jakefreebergravatar says:

    “housing costs souring” ? I think someone may want to proofread this once more.

  2. Ryan says:

    I’m not sure if you read the paper but they addressed all three of your concerns…

    1. They surveyed multi-site firms and found them more likely to reduce workers.
    2. Comparing against the rest of WA is more appropriate than the control counties in the Berkeley paper. Not sure if you looked at the control counties they used but most were in FL, OH, and MO. Counties hardly experiencing an economic boom like WA.
    3. Did you see the total decline in jobs for workers making below $13 and $19? Both buckets lost a significant number of jobs. Sure a handful of workers that were making $18 are now making $20, but it’s doubtful to have a statistical impact. You would need to find 12-15k workers (15% of earners making less than $19) to have made this jump.

    Both Jared Bernstein and David Autor couldn’t find holes in the UW study, that says something.

    • Economic Opportunity Institute says:

      A detailed response from the Economic Policy Institute provides a review of the most important shortcomings in the UW analysis:

      “The authors’ analysis…suffers from a number of data and methodological problems that bias the study in the direction of finding job loss, even where there may have been no job loss at all. One initial indicator of these problems is that the estimated employment losses in the Seattle study lie far outside even those generally suggested by mainstream critics of the minimum wage (see, for example, Neumark and Wascher [2008]) — as the authors themselves acknowledge.

      In this report, we describe the most important shortcomings in the new analysis and make suggestions for how the researchers can attempt to correct for these problems in future iterations of their long-term study of the Seattle minimum wage. These shortcomings include:

      • The employment responses estimated by the authors are well outside the bounds of most published research, and indeed all of the research cited by the authors implies much smaller and even no employment changes in response to wage increases similar to those experienced so far in Seattle. After accounting for Seattle’s much higher wage structure, the increase of the minimum wage to $13.00 in the city is within the range of increases that other research has found to have had little to no effect on employment.
      • The study implausibly finds employment changes due to the minimum wage in parts of the labor market where there should have be none. The study’s own estimates inaccurately imply the minimum wage caused large gains in the number of jobs paying above $19.00 per hour and in the number of hours worked in those jobs—even though those jobs are well above the wage range where the $13.00 minimum wage should be having measurable effects. These spurious results strongly suggest that the study’s methodology fails to account properly for the booming Seattle labor market during the period being studied—a labor market that has been shifting employment from lower-paid to higher-paid jobs.
      • The study excludes an important group of workers, representing roughly 40 percent of the workforce: those working for employers with businesses in multiple locations. By omitting all multi-location businesses, such as chains, in Seattle, the authors bias their results toward showing job loss if there has been a shift in employment from small, single-location establishments toward larger firms with multiple locations.

      The study’s findings of employment losses well outside even those generated by a large body of earlier research raises immediate concerns about the study’s methodology. The exclusion of roughly 40 percent of employment from the analysis—especially when multi-site employers are large employers of low-wage workers and may be expected to increase their share of low-wage employment after a minimum wage increase—is a major limitation of the study and one that will be difficult to circumvent. Our best explanation for the study’s outsized findings is that the statistical techniques employed were not capable of isolating the effects of the minimum wage from a range of other simultaneous changes in the Seattle labor market. In particular, the strong performance of the Seattle labor market in recent years appears to have overwhelmed the ability of the authors’ methodology to measure accurately the specific wage and employment impacts of the wage ordinance.”

      You can read EPI’s full report here.

    • sandypacific101com says:

      I don’t think it even makes logical sense to conclude only low wage workers are losing jobs in an economy with a 2.6% unemployment rate. Besides, there are other cities that have seen the same phenomenon. Raise wages and businesses with low wage workers see their business increase due to those same workers spending money.

    • Fred Jack says:

      1st report UW released July 2016 the mayor, city council, & EOI claim success.

      2nd report UW released June 2017 the mayor, city council, – EOI claim it’s a fraud.

      Both reports utilize the same methodology!
      Can you see the problem, hypocrite?

  3. anonymous says:

    If the two studies are so different in their methodology, why do they arrive at the exact same results? When looking at the restaurant industry, both find no employment effect. UW finds large job losses among low-wage workers as a whole, and Berkley does not even attempt to determine whether this is correct or not.

    I also note that the author of this post complains that wages are going from $18 to $20 and this is not being counted, while the comment above mine by EOI/EPI claims the EXACT opposite.

    You also complain about the exclusion of chains, however you decided not to mention that the study conducted a survey of all business owners, including these chains, and the chains were more likely to report that they cut hours and jobs

  4. Billy Walla says:

    This reminds me of the conservative meme that said “92 million Americans are out of work in the USA”. That counted retirees, babies, elementary school children, people who don’t want jobs, or basically everyone who didn’t have a job, regardless of the reasoning.

  5. Steven Vandor says:

    I believe UW is one of the largest, if not THE largest, recipients of research funding from the U.S. federal government. Given changes over the past six months in Washington, D.C., particularly at funding agencies heretofore focused on science-based research, one should expect less rigor in that research and findings that hew to “alternative facts,” not science or reality as scientists have come to understand it. A sorry state of affairs for a once-great research university.

  6. NW Citizen says:

    I am concerned about the effect of the minimum wage increase on small businesses. Does the EOI have a comment on that?

  7. Shane S says:

    > 1. They surveyed multi-site firms and found them more likely to reduce workers.

    How does this address ignoring 40% of the workforce? You can’t wave your hands and make 4 out of 10 low wage workers just disappear from the eqution based on assumption that multi-site employers are “more likely” to cut hours or layoff workers. You have no real idea whether they hired, fired, cut or increased hours – and that calls every conclusion drawn into question.

    “Likely” is not data, and never will be. Ignoring more than 1/3 of your target population *is* a major methodological flaw, and there is simply no way around that.

    > 2. Washington Counties

    Are you asserting that using “non-boom” counties whose economic base more closely resembles Seattle’s is an inferior choice to other Washington counties whose economic base are profoundly different that Seattle’s, and who are also not experiencing the unprecedented “boom”? That is counter intuitive to say the least. How do you justify that claim

    > 3. Lost Jobs – I’ll point you back to #1. Ignoring 40% of a workforce has an effect… and you have absolutely no idea what that effect was, nor any way to even make a reasonable attempt to within the study parameters.

    I might be less skeptical if they didn’t deviate so widely from norms in other studies –
    but they do, and I just don’t believe the data and methodology are good enough to justify their truly unexpected and unprecedented conclusions.

  8. Marie says:

    I would like to know if each study was peer reviewed and the results. I didn’t read the whole article yet and comments…am very interested in the peer review status before spending time digesting the info.

Leave a Reply

Search the blog

Subscribe to the blog

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Like what you’re reading?
Reader support helps preserve our independent voice for the middle class - please chip in to help out!