These 32 hours have 20 falsifiable forecasts

Phillip Metaxas and Andrew Leigh sat through 32 hours of Australian talking-head TV.* In all that talking, they were able to find only 20 falsifiable forecasts from Australian pundits. The rest were so qualified that they were unfalsifiable.

They suggest that talk show hosts demand falsifiable predictions:

A nice start would be for Insiders host Barrie Cassidy to insist on ‘falsifiable predictions’ in his final segment, rather than the current ‘observations or predictions’ from the pundits. While such an innovation would be helpful, assessing whether a prediction has proven true or false is likely to provoke objections from the pundits involved. As Gardner (2010) notes, when predictions go awry, pundits often invoke an array of excuses to exonerate themselves. Such excuses include the ‘I was almost right’ defence, and the ‘wait and see’ defence. In our analysis, had David Marr and Lenore Taylor been incorrect about public opinion changing on the mining tax, they could easily have invoked the ‘wait and see’ option to defend themselves. 

For this reason, we advocate a Tetlock-style experiment involving Australia’s leading political pundits. Such an experiment substantially removes the ability of pundits to invoke the common defence mechanisms when their predictions go astray. An appropriate study would require our pundits to make falsifiable predictions about the political world in the future. Predictions could cover a range of topics such as the party likely to win the next federal election, the next leaders of each party, and the passage of key legislation. There would be a sufficient number of questions to ensure that lucky hits do not skew the results. We need more evidence than our analysis could provide to properly separate flukes from prescience. Furthermore, an anonymous survey like Tetlock’s would help determine whether our pundits as a group are worth the airtime they are afforded. An open survey could also give us the ability to rank our political pundits based on their predictive powers.

I'd love to see a talk show partner up with one of the betting markets to refine the opposing pundits' claims into a single falsifiable test prior to the show. Launch the contract at the start of the show, unless it's a contract that already exists, like "Who will win the next election", then have them argue about the appropriate price for the contract while the audience submits bids and asks. Let the ticker move over the course of the show, then insist that both of them put, say, $1000 on it at the conclusion of the debate if their argued price differs from the market price.

This would be really easy for things like "Who will win the next election?" or "Will Julia Gillard lead Labor into the next election?". And it isn't that much harder for combinatorials like "Labor would win more (less) seats under Gillard than under Rudd".

There would be nothing banning such things in Australia or New Zealand. There are political betting markets in both places, even if Senator Xenophon keeps trying to ban them in Oz. The constraint seems more likely to be that viewers aren't really all that interested in getting falsifiable predictions and accurate odds. Theatre and that cherished fantasies never ever be shown to be wrong - that's where it's at.


* Ok, they read transcripts.

The value of outreach

I enjoyed the CBC's radio show, The Invisible Hand. Rather than take a Freakonomics-style "wow, isn't this counterintuitive" take, they instead simply presented standard economic theory as it is understood by professional academic economists.

• Episode 1 covered price gouging, with a nice contribution from Mike Munger. (audio)
• Episode 2 takes on value theory. In the event of a zombie apocalypse, chickens rise in value relative to gold (as the Lannisters may yet discover when winter comes). (audio)
• Episode 3 explains why profit isn't a bad word and shows that rather a few capitalists - people who make their living from investments - look rather more like our parents and grandparents than like Mr Scrooge. (audio)
• Episode 4 covers love, marriage and economics. (audio)
• Episode 5 explains moral hazard and insurance. (audio)
• Episode 6, on sweatshops, includes Ben Powell along with a former sweatshop worker. (audio)
• Episode 7 demonstrates that David Suzuki knows absolutely nothing about economics: we have ways of dealing with externalities. (audio)
• Episode 8 explains a bit of macroeconomics: the old Paradox of Thrift. (audio)
• Episode 9 walks through one of my favourite topics: perverse incentives (see the masthead). Frances Woolley helps out. (audio)
• And Episode 10 explores some of the behavioural economics literature. (audio)
• [Update: the audio at the CBC link isn't working, but the (audio) link should take you to the podcast version. Thanks to Wobbly in the comments!]

Worthwhile Canadian Initiative's Stephen Gordon provided academic assistance for the project.

The CBC more typically airs standard economic fallacies as fact, or at least it did back when I was in the country, so this was really rather nice.

What reaction did they get? Here's the show's producer Matthew Lazin-Ryder. Start listening at the 14 minute mark. At 15:55 he talks about his "honest to goodness depression" about the show's being criticized for being "brazen right-wing propaganda". He says [transcription errors mine]:

We wanted to make a show that had a completely different perspective from the things most people hear. And our probably naive anticipation was that people would take it in that way. We didn't honestly expect the angry backlash that we got. ... Our agenda was to present how mainstream economists think about things. 

He also tweeted: 

"Lightning Rod For Hatemail" was the working title for the series. RT @danieloong: A show like that'd be a lightning rod for hate mail.
— The Invisible Hand (@CBCTheHand) April 12, 2013

There is a body of things that economists know about the economy. Sure there's stuff we argue about, but especially in microeconomics, we kinda know what's going on. And the basic set of things about which economists agree diverges wildly from how the public thinks the economy works. The profession attaches perhaps too high of reward for deriving the results of some model when you change a plus to a comma in a utility function when the first order welfare gains are in just getting the voting public to appreciate principles-level economics.

I get irritated when bog-standard economics is cast as having a "right wing" agenda. Mainstream economics helps you figure out what works and what doesn't work for achieving any particular end and the trade-offs that are involved. If you want a fair bit of redistribution, that's entirely consistent with mainstream economics so long as you set up the transfers appropriately; heck, it drops out of most models where you assume diminishing marginal utility of income.* But bog-standard mainstream economics in Canada says a lot of unpalatable things: ditch supply management to reduce milk prices; get rid of barriers to both interprovincial trade and labour mobility; get rid of all the zany exemptions in the GST and adopt New Zealand's version instead.

Imagine a genie gave you a button. If you push the button, every voter in the country thoroughly and intuitively understands principles-level economics. At the same time, the most recent n issues of every academic journal in economics disappear along with all knowledge of their results: we would need to re-invent or rediscover every one of them, with some chance of never finding them at all. Up to what value of n do you leap to push the button? 5 years' worth? More?

Imagine a world where the physicists and engineers spent most of their time figuring out how to get internal combustion engines from 20 to 22 percent efficiency but where, outside of the lab, everyone else is riding horses because they think engines are evil and witchcraft and tools of capitalist oppression. Maybe it's not quite that bad in economics, but it isn't far from it.**

Update: Brennan McDonald suggests, or at least this is what I draw from his comment, that there may be little potential trade-off between high-powered theorem building and public conversion efforts since the public broadly isn't truth-seeking. In that he echoes Patri Friedman's complaint about folk activism. But we are all part of the equilibrium, and I do think that we could use to move a bit at the margin.

* But be careful! Cowen points out that utilitarian theories may be less egalitarian than you'd like. I asked a couple years ago about appropriate egalitarian policy when we start opening up the margins:

Pity the borderline Asperger's investment banker who, despite his financial success, seems at a bit of a disadvantage in dating. Reddit posted the 1600 word email that the would-be suitor sent to the woman who dumped him after the first date; it's since shown up all kinds of places. But folks snickering at it seem an awful lot like a rich one-percenter laughing at a pleading email from a starving man.

If I can play armchair psychiatrist, the same Asperger tendencies that helped this poor guy in investment banking have killed him in dating.

If you're an egalitarian, what is appropriate policy? Is this guy better or worse off than the poor musician who dates easily? With whom would you rather trade places, taking both their positions and their characteristics? If we redistribute income because the investment banker's last dollar is worth less to him than it would be to the poor musician, think too about the marginal utility of the musician's last date relative to the banker's.

 And should we compensate the beauty-challenged?

** See, for example:

• Caplan on systematically biased beliefs
• Me on economic thinking in NZ

Shame and voting

I'll be talking this coming weekend with the fine Australian libertarian community about public choice problems with voting; I'll also be in a panel discussion on nanny state issues.

Voting's been something of a longstanding problem in public choice. On purely instrumental accounts, where people vote to make it slightly more likely that their preferred policies are enacted, it's hard to explain voting at existing levels of turnout. The infinitessimal probability of casting the decisive vote wipes out any potential utility difference that could attach to the different candidates' likely policies. And so we look to intrinsic accounts of voting where people derive utility from the act of voting itself.

But even those intrinsic accounts have problems. If people vote out of civic duty, and if there are myriad things people could do instead to demonstrate their dutifulness, why vote instead of spending the same amount of time donating blood or helping at a homeless shelter? You'd get the intrinsic benefits of doing something dutiful while also potentially doing something useful. If people vote because team or group affiliation make them feel good, or if whatever motivates people to get to the polling booth doesn't also motivate them to put time and effort into their choice, then there's no reason to believe that there's any connection between the median voter's choice and desirable outcomes. We have rather a fair bit of evidence that voters do not seem to know very much.

So why do people vote? Stefano DellaVigna, John List and Ulrike Malmendier say it's to avoid the pain of having either to lie or to admit to others that you didn't bother voting. In a rather nice field experiment, they varied payment to survey respondents, varied the up-front estimate of the time cost of the survey and varied how much longer they said it would take voters to complete the survey relative to non-voters. As they had voting records that told them which respondents had in fact voted, they could use the design in combination with differential response and lying rates by voters and non-voters to elicit the disutility of having to talk about having voted. In a year without a Presidential election, that value was estimated at $10-$15: being asked about whether you voted, when you hadn't voted, cost the non-voter $10-$15; the figure would presumably rise in a Presidential election as more people would ask.

The Wall Street Journal quotes Malmendier:

The study has implications for campaigns’ get-out-the-vote efforts, which often involve enlisting people to ask individuals whether they voted, Ms. Malmendier said. “They ask friends and neighbors to ask if you plan to vote. They will ask you and then call you on Election Day… Our paper is saying we should not forget that people get a ‘disutility’ from that. They really don’t like it. Even if they’re voters, they really don’t like being asked.”

Ah, but that cost is what makes it work if the disutility comes either from saying you won't vote or from fearing that they'll know you didn't. Campaigns are acting rationally and probably reasonably expect that their supporters won't vote for the other guy out of spite after a GOTV call.

Political parties aren't subject to the US national "do not call" registry; it's perhaps worth re-thinking that exemption.

HT: WSJ via Politix

Morning roundup

Item the first: New York Supreme Court Judge Milton Tingling blocked Bloomberg's ban on Big Gulps. I agree with the Judge here:

Judge Tingling found the Board of Health's mission is to protect New Yorkers by providing regulations that prevent and protect against diseases. Those powers, he argued, don't include the authority to "limit or ban a legal item under the guise of 'controlling chronic disease.' "

The board may supervise and regulate the city's food supply when it affects public health, but the City Charter clearly outlines when such steps may be taken: According to Judge Tingling, the city must face imminent danger due to disease.

"That has not been demonstrated," he wrote.

Judge Tingling also suggested that Mr. Bloomberg overstepped his powers by bringing the sugary drink rules before the Board of Health, which is solely appointed by him. The City Council, he wrote, is the legislative body "and it alone has the authority to legislate as the board seeks to do here."

But a malicious part of me really wishes this had gone ahead. The ban looked to be an intractable nightmare that New York, having elected Bloomberg, really deserved to endure. They could have served as example unto others. Instead, the lesson is that judges will bat back things that are entirely too crazy, so there's no harm in electing Bloombergs. Sometimes, the electorate does deserve to get what it wants, and that right hard.

Item the Second: Rasmussen polls show 36% of Americans think it's at least somewhat likely that Fukushima radiation "did significant harm to the United States." I would bet that a survey of scientists specialising in radiation would find no more than 1-2% thinking it did significant harm to the US, and those would be saying that the negative effects were through reduced public acceptance of nuclear power. I really really wish that polls of this sort, where there is a right scientific answer, or at least a likely strong consensus among scientists, would mention it. The survey doesn't mean that there's a 36% chance that it did harm, it means that 36% of Americans are scientifically illiterate.

Why we cannot have nice things

Economists disagree on some policies; you can usually find an economist willing to support some policy proposition that his colleagues won't like. But, surely, were there some set of policies agreed upon by economists from across the spectrum, it would probably be a good one, right?

Over the last decade, Bryan Caplan produced a series of papers and a book showing that economists' views on matters of positive economics diverged widely from the views of the public, even after correcting for income, ideology, education, and a host of other covariates. Broadly speaking, relative to economists, the public were shown to be pessimistic, xenophobic, anti-business, and to love make-work projects.

Paola Sapienza and Luigi Zingales have added* to this literature with a new Chicago Booth working paper: Economic Experts vs Average Americans. They compare public and economist support for a series of policy propositions and conclude that the set of policies that would command substantial agreement among economists would have little chance of electoral success.

I've a few quibbles with their method: they collapse "agree" and "strongly agree" into single categories; there can be useful information in strength of agreement. When we're thinking about giving up a bit on a less-important policy to get concessions on something that's more important, preference-intensity matters.

I'll also quibble a bit with the Booth Panel as being that representative a sample. First off, the survey shows that Booth Panel economists trust government more than does the general public. I'm not sure that that's a finding that would be found in a broader survey of economists, but it could be. But I do know that the Booth panel's views on the minimum wage differ vastly from every prior survey of economists I've seen on it: the Booth Panel had 47% support for the $9 US minimum wage (albeit with many supporters saying in the comments that hiking the EITC would be even better); Whaples 2006 (see here or here) had 87% of economists agreeing that minimum wages hurt the job prospects of the young and unskilled, and 47% wanting to abolish the minimum wage entirely. Whaples 1996 had 87% of economists agreeing that minimum wages hurt the job prospects of the young and unskilled while Whaples 2006 had 47% wanting to abolish the minimum wage entirely. [Thanks to David Friedman, in comments, catching that I'd meant to put "1996 and 2006, respectively," rather than 2006. I'd linked both sources but screwed up the attribution in text. It matters because consensus could have deteriorated over the decade.]

But, the differences remain striking. Even among a sample of high-trust-in-markets Democrat members of the public, average differences in agreement between economists and the public across questions was thirty percentage points.

Well, maybe this is all because people can always find some economist who'd agree with their position. But in that case, we'd expect to find smaller differences between economists and the public where economists agree with each other more strongly; they find the opposite. They put the difference up to specific training in economics and expert knowledge among economists - about what Caplan had concluded a decade ago. But they add a very nice twist: one wave of the public survey included information in the relevant questions, "Nearly all economic experts agree that ...". Telling the public that economists agree on something usually very slightly reduced disagreement, but sometimes made things worse.

They probe further into disagreements and find some evidence that the public have different ceteris paribus assumptions. Economists pretty much all agree that carbon taxes are much better than CAFE regs. But economists also tend to assume that carbon taxes will be handled in some revenue-neutral way with offsetting transfers. And the public doesn't trust government to actually make things revenue-neutral.** They then put most of the differences down to differential auxiliary assumptions around ceteris paribus. But Caplan's prior work was on purely positive questions like whether foreign aid spending is a major reason that the economy is performing poorly. I'm not sure what auxiliary assumptions might have driven the vast differences he was finding.

I don't have particular reason to doubt their results around trust in government. But even if you didn't trust the government to make sure a carbon tax were revenue-neutral, CAFE standards are a really wasteful way of achieving any particular goal: you force the public to pay $1 for enhanced fuel economy that they maybe value at $0.80.*** Wouldn't you then only prefer CAFE over carbon taxes if you valued the marginal dollar of government spending somewhere south of $0.80? That's plausible for me, but is it plausible in a general sample? Well, maybe if they have systematic misperceptions about the composition of the federal budget and where the marginal dollar goes....

HT: Andres Marroquin

* And, in the next draft of their paper, I hope that they make some note of Caplan's pioneering work here. I've found similar results in a New Zealand sample.

** I'm not sure the public's wrong here either. Yeah yeah, we put up some nice revenue-neutral policy with tax cuts. But once a new domain's been opened to taxation, it's a lot easier to ratchet back the income tax cuts while keeping the new tax. So is the problem then voters' lack of trust, or governments' inability to make binding commitments?

*** Numbers here purely for illustration; I've not looked around to see if anybody's tried to estimate this. I know that people value fuel economy and that the value of fuel economy varies with petrol prices, but I've not ready numbers on the costs of increasing fuel economy via CAFE regs.

A rather neat field experiment

My first reaction: what an awesome idea.

My second reaction: they lied to subjects about the purpose of the experiment and didn't fully inform respondents about what the surveys were about; their human ethics committee is far more awesome than others.

My third reaction: I hope this gets published despite that many of the main effects are statistically insignificant.

I've read a bit of the literature on preferences for homogeneity. It depresses me, but I'd hoped that longer exposure to more diverse groups would overcome some of those effects. And as it's hard to tease out causality in population based measures, you could argue the toss on it.

Ryan Enos at Harvard's Department of Government approached things a bit more scientifically. Here's his experimental design. Get a bunch of train stations in ethnically homogenous white areas of Boston. Half of them are controls. The other half get a pair of obviously ethnic Hispanic confederates who are going to ride the morning commute for a week while chatting with each other in Spanish - other commuters could assume that a Hispanic family had moved into the neighbourhood and were commuting into town. Pre-survey people at all stations and get them to give an email address for follow-up surveys. Among the questions, ask about attitudes towards immigration. Half of respondents were invited to answer the survey again three days after treatment began; the other half were invited to do so ten days later.

He finds:

• Ideological conservatives' views did not much respond to treatment, but remained hostile towards immigration. Liberals and moderates became more hostile to immigration as consequence of treatment. But results were not always significant.
• Negative responses were sharper in the three day than in the ten day treatment. But the 95% CI around the effect only failed to include zero for one survey question and only in the three-day treatment: "Should the number of immigrants be increased?"
• Income didn't moderate the effects: the same negative shift in attitudes was pretty uniform across respondent income.
• Treated respondents were more likely to say their neighbourhood had recently become poorer, more diverse, and more unsafe. But the confidence interval is wide.
• Effects were largest among those with low numbers of Hispanic friends; for those with few Hispanic friends, effects were significantly negative on all three questions.
• There was no change in policy attitudes other than those associated with immigration.

Figure 2, replicated below, has the main results.

My take-away is that I should expect increased immigration to make people more hostile towards immigration but that the effect is likely to wash out over time. 

No transitional gains traps?

Don Wittman is right: the transitional gains trap is a bit of a puzzle.

Recall first how the transitional gains trap works. One a rent-seeker has a rent conferred upon him, the value of that rent is capitalized into whatever draws the rent: the quota permit for Canadian dairy farmers; the taxicab medallion for New York taxi firms; the liquor licence for permit holders in places where licences are in restricted supply, for example. After that capitalization happens, the owner of the permit earns only a normal return on the total value of his capital, including the capitalized value of his regulatory rents. Permits change hands such that whoever earned the windfall initial gain takes his rent and leaves; eventually, nobody who currently owns the permits has earned any kind of excess return by having owned them. But try to get rid of the regulatory inefficiency that draws the rent and each and every one of these permit holders will scream blue murder as you're wiping out a good chunk of their capital: some permit holders could easily go bankrupt over it if they took out loans to buy the business and both they and the bank were counting on a continuing flow of regulatory rents.

Now, Wittman would rightly point out that if this is really so inefficient, there has to be a move that buys out the losers out of the gains to the winners. If it's Kaldor-Hicks efficient, this has to be the case. If you run the compensation, then the policy switch is Pareto-efficient.

The usual answer is that the transactions costs are too high to prevent the move towards more efficient policy. But in the case of taxis, or the Canadian dairy cartel, that really doesn't seem to be the case. For dairy, as I've suggested many times, all you need to do is put a tax on dairy products in Canada at the same time as you abolish all of the tariffs on imports and abolish supply management. The tax keeps the price to consumers a bit below where it was prior to the shift and is sufficient to pay off the bonds you issue to buy out the quota holders.

But, there's a reason that opaque transfers are preferred. That reason? Voters. Don't believe me? Read the comments section on Stephen Gordon's Globe and Mail piece where he suggests my "tax dairy and buy out the cartel" solution. For example:

professor_x

2:56 PM on November 16, 2011

I read the word screaming clear TAX.

We want to TAX dairy to make it even more expensive to 34 million Canadians who have to pay off $30 billion dollars in outstanding quota values.

Opaque transfers are opaque. Nobody understands tax incidence, never mind this kind of thing.

Add in the generalized worries about trade, insecurity issues about the Americans, and just general weirdness about food, and you wind up with voter support for a policy that makes them worse off. I'd batted back some of these fallacies. Even if Canada gets rid of supply management, Canada will still have a dairy sector; if Canadians want to ban GE milk, or milk where hormones are used in production, they can do it by direct regulation; and, if Canadian dairy farmers want to form a voluntary cooperative to get some efficiencies of scale while avoiding being contract operators for others, Canada has a strong tradition of agricultural cooperatives.

The best counter-argument I've heard is that the government can't constrain itself against bailing out farmers, so the one-off payment is likely to be followed by some additional support down the line. But isn't it better for government to try to come up with some mechanisms for self-discipline? It's a general purpose technology worth developing. And it's hard to believe that the costs of any potential future support package would trump the cost the supply management system imposes every year with certainty.

Martha Hall Findlay, Canadian Liberal Party leadership contender, makes the case for abolishing supply management in combination with a temporary tax on dairy used to fund a transitional support package for dairy farmers. She suggests the main problem is overcoming dairy farmer resistance and points out that dairy farmers are a trivially small proportion of the voting population; there's no reason that the Conservatives, or anyone else, couldn't just abolish the system, lose every single dairy farmer vote, and not expect much difference in the allocation of seats in Parliament.

I love Findlay's proposal. But I worry that the problem isn't the angry dairy farmers voting against incumbents. Rather, it's angry dairy farmers putting up ads on TV scaring voters about imported milk combined with voters really not understanding that a temporary tax on milk, under this system, reduces the cost of milk rather than increasing it.

I expect that the Canadian Dairy Cartel will use the threat of this kind of public campaign to negotiate for a bigger payout. So it's good to see the folks at EconomyLab helping to inoculate voters against the "make voters dumber" campaign that's likely to come. But if Stephen Gordon or Mike Moffatt were to put something up slowly explaining why free trade in agriculture won't mean that Canadian consumers are suddenly forced to drink poisoned milk, that would probably also be pretty useful. I know it's obvious to us, but it isn't obvious to the folks who can veto the play.