The New York Times and others have rightly made a big deal out of the scandal that an actuary paid by unions was relied on by the State Legislature in its estimate that a bill that would offer early retirement to city workers would not cost a cent. But a not too dissimilar reliance on a partisan source regarding Atlantic Yards raised nary an eyebrow.The May 16 Times article, headlined Unions Bankrolled Analyst Vetting Pension Bill, made the front page. Later that day, faced with a storm of press criticism, Assembly Speaker Sheldon Silver decided that any bills with fiscal notes prepared by that actuary would be placed on hold, with new, more objective analyses sought. The headline on the Times's City Room blog: Speaker Tosses Work of ‘Voodoo’ Actuary.
Critical editorial
On May 24, the Times's editorial page was in high dudgeon, in an editorial headlined Have We Got a Deal for You:
Just when it appears that the New York State Legislature has hit bottom, we find a false floor. The Times’s Danny Hakim recently uncovered another appalling example of how a politically powerful group can order up laws in Albany, even skewing the estimates of how much it will cost taxpayers.
... When asked about his estimating methods, Mr. Schwartz told The Times that his job is “a step above voodoo.”
New York City officials would certainly agree since they estimate the bill’s cost at $200 million a year.
AY "voodoo"
Curiously, Atlantic Yards backers openly relied on a partisan expert, but were never called to account for it. For example, the ESDC and governor's office both on 3/4/05 issued press releases relying on revenue projections made by the developer’s paid consultant, Andrew Zimbalist, rather than commission their own analysis. The press release stated:The project is expected to create 15,000 construction jobs and over 10,000 permanent jobs. According to an economic analysis completed earlier this year for FCRC by the economist Andrew Zimbalist, the net fiscal benefit to the City and State from the Atlantic Yards project is estimated to be at least $2.819 billion over thirty years, or a present value of at least $812.6 million.
Unlike with the pension bill, there was no "other side" sought out to argue that the figures in the governmental projection were off. The city's press release, interestingly enough, cited a study by the NYC Economic Development Corporation that had not been released. I wonder whether city officials were more wary of appearing to endorse a partisan study.
This episode does not directly parallal that of the "voodoo" actuary, given that it involved a press release rather than a bill to be passed by the legislature. However, there was an "other side," just disregarded. Some seven months earlier, the Times reported briefly on a study critiquing Zimbalist's report. But the newspaper gave Zimbalist the last word and never sought independent review of the costs of the project.
Zimbalist's study was, arguably, "voodoo," ignoring costs that, for example, the Independent Budget Office later found. The Zimbalist study was enormously flawed, and should not have been relied on. But it was and nobody blew the whistle on it at the time.
Just as the Times pointed out in recent coverage that city officials reached very different conclusions than that of the "voodoo" actuary, so coverage of Atlantic Yards should have acknowledged that there were alternate views.
For example, see 3/29/07 testimony of Brad R. Humphreys of the University of Illinois at Urbana-Champaign, testifying on public financing for sports facilities, during a hearing of the Subcommittee on Domestic Policy of House of Representatives' Committee on Oversight and Government Reform.
Humpreys stated:
Since “promotional” economic impact studies are forecasts, they have the same inherent weaknesses as any other economic forecast, like a forecast of the growth rate of GDP over the next five years. But “promotional” economic impact studies always project a high degree of precision. Rather than being stated in terms of a predicted value plus or minus some margin of error, the forecasts in these studies are always a single number, implying a higher degree of precision than other economic forecasts, even though there is no evidence that they are more precise.
(Note that Zimbalist's report assumes a timetable for construction and an office space allotment that no longer exist, thus rendering his forecast significantly imprecise. Add to that his reliance on counting taxes paid by new residents, which economist James Parrott commented, "I don't know of any serious cost-benefit analyses of mixed-used economic development projects that count the taxes of residents.")
The importance of peer review
Zimbalist's study got no backup from anyone but Forest City Ratner spokesman Barry Baum, whom the Times quoted.) Humphreys pointed out that "promotional" studies don't get peer review:
The most important difference between evidence from academic research and evidence from “promotional” economic impact studies is the degree of scrutiny they undergo. “Promotional” studies are typically carried out by consultants. They are released with great fanfare in the local media, and typically get widespread coverage for a brief time. The press releases and sound bites associated with these studies are typically short on details and long on large round numbers. Very few people ever read the entire reports. The vast majority of these “promotional” studies disappear within a few days of their release. The methodology used in “promotional” studies, and the results, are not reviewed or evaluated in any way. I do not know of a single instance where the predicted outcomes from a “promotional” economic impact study have been systematically evaluated for accuracy after a sports facility was completed.
In stark contrast, academic research on the economic impact of professional sport published in scholarly journals goes through a rigorous peer review process. In this process, the papers are
distributed to other experts in the field, often stripped of identifying information about authorship, who are asked to anonymously evaluate the quality of the research.
Zimbalist vs. Zimbalist
Zimbalist himself has criticized "promotional" studies. For Forest City Ratner, however, he wrote one. Humphreys, in his testimony, laid out the difference:
In contrast, journals that publish academic research on the economic impact of professional sports charge researchers submission fees to consider their papers for publication. They do not pay royalties to research who write the papers they publish. A researcher in this area has no personal financial stake in the outcome of the research.
Humphreys concluded:
It is imperative that those who make decisions on sports subsidies understand this important difference in the evidence about the economic impact of professional sports. Results that have been through the peer-review process should be given much more credence by decision makers than “promotional” economic impact studies. We do not make health policy decisions based solely on the claims of pharmaceutical companies, and we should not make decisions on subsidies for professional sports based solely on the claims made by professional sports team owners and others proponents of these subsidies.
Comments
Post a Comment