The Legacy of “Doing Business” and recent data manipulation revelations
In September 2021, the World Bank Group admitted the immense scope of “data irregularities” in the Doing Business (DB) reports of 2018 and 2020, and decided to discontinue the publication and move to alternatives. Presumably, this decision was triggered by the backlash the Bank received not only from those countries that received lower rankings than in previous years, but also from others who criticized the rankings’ methodologies, including lumping together countries, some of which had not even been included in the data collection process.
The DB project, which has been met with both praise and criticism, produces annual reports calculating aggregate scores of each jurisdiction based on the “ease of doing business” criteria. Essentially, “every year, DB produces eleven numerical indicators, offering a ‘snapshot’ of business regulations in 190 countries, on themes ranging from access to electricity to conditions of employment.” A notable illustration of “the quiet power of indicators,” DB became a much-used measurement instrument which, through its deceptively simple and eye-catching rankings, offered investors and policymakers what was touted as a ‘clear picture# of a country’s business regulatory infrastructure. Since its launch in 2003, the DB project has expanded from ranking 133 economies to 189 in 2020. This ranking is created based on an array of indicators, and has become one of the most widely watched indices issued by the World Bank, along with the Global Findex Database, the Global Reporting Initiative (GRI), and the World Justice Project’s Rule of Law index.
For decades, the DB index has remained heavily scrutinized with regard to its profound implications for different stakeholders within the ranked states. Given the wide interest in and broad impact of the DB project, the allegations of its ranking manipulation have periodically made the news. Recently, both the World Bank internal inquiry and the external review carried out by the law firm WilmerHale concluded that data “irregularities” occurred in the 2018 and 2020 reports. According to the WilmerHale report, China and Saudi Arabia were able to successfully pressure DB staff into manipulating the rankings in their favour.
Since this report by WilmerHale was commissioned by the current World Bank management, some readers interpret the investigation as a sign of power struggles within the World Bank leadership. Others like Joseph Stiglitz, the Nobel-winning American economist, display their support for the DB project and claim the WilmerHale review as an attempted coup against Ms. Kristalina Georgieva, the previous Chief Executive of the World Bank and current Managing Director of the IMF.
What, exactly, does DB measure? The Answer is “289”
By producing “quantitative indicators” based on “hard data,” the DB index once pioneered analyzing institutional structure with a synthetic score. The DB team first surveys law firms with around 10,000 questionnaires covering eleven specific “data points” (initially it covered five but has since grown). The central part of this data collection process, according to Timothy Besley, is asking law firms and businesses to count the number of proceedings, the cost, time, or pieces of laws and regulations involved in the life of a business. Based on a series of quantifiable goals and empirical evidence, DB experts create a score for each set of indicators, then convert those into a simple unweighted average of the ranks, and finally, provide an aggregate score of any given jurisdiction, leading to a global ranking.
However, there are defects in those seemingly objective DB indicators. First, for those areas that are difficult to quantify—courts, labour, or other aspects in business regulation—the DB methodology simplifies them into routinized, one-size-fits-all indicators. This standardization is deceptive, however. There is a cost when measuring a complicated court system based on simple statistics. In extreme cases, this use of indicators tempts some countries to game the index with self-reported data, making it susceptible to manipulation, as the 2021 External Panel Review revealed.
Moreover, the inherent problem with the DB measurement may stem from their choice of those eleven specific indicator sets, which, as Adam Tooze suggests, reflect the decades-old “Washington consensus.” At the end of the cold war, Washington, D.C.-based think tanks prescribed a set of neoliberal economic principles for the development of emerging economies, enabling a free and efficient transition to the market economy. To calculate efficiency, the DB founders developed a number of, purportedly, easy-to-use toolkits. When Hernando de Soto set out to open a small garment company in Peru, he recorded how long it took to obtain the government permits: 289 days. Inspired by such a simple method to measure efficiency, the DB experts designed a handful of questionnaire spreadsheets listing variables such as days, costs, or procedures in opening and managing a small- or medium-sized business. This approach revealed is limitations when used to assess the role of courts.
How Toronto’s courts fell out of favor with the DB Index
The DB indicators are further problematic when it comes to the ranking of local courts with regard to their contribution to creating an attractive economic environment. The “Enforcing Contracts” index comes down to the following: “Doing Business measures the time and cost for resolving a commercial dispute through a local first-instance court and the quality of judicial processes index, evaluating whether each economy has adopted a series of good practices that promote quality and efficiency in the court system.” (Djankov et al, “Courts”, 2003). Through questionnaires, the DB working groups collect raw data such as items of reform, numbers of cases, quality and number of proceedings, cost estimates (% of claim) and time estimates (days), which are then transformed into a quantitative value between 0 and 1.
Once exceptional weights are assigned to the time variable, fortune always favours the nimble ones. With speedy proceedings, China could easily beat many counterparts, including Canada. In the 2017 DB rankings, Canada witnessed significant downward shifts, with the Enforcing Contracts index falling from 49th to 112th. The days required to enforce contracts through courts increased from an average of 570 days to 910 days in Toronto, Canada’s benchmark area. In China, with a party-run, concentrated court system, and a business-friendly legal system, contract disputes can allegedly be resolved within a comparatively speedy 452.6 days.
Out of Sync? Numerical Rankings and the Struggle for Law and Justice
Yet, there is a much deeper problem when it comes to tying the duration of court proceedings to a country’s ability to create a welcoming business environment for local and foreign investors. The DB’s counting game treats the judiciary along with the norms that govern its operation and the claims that judges preside over as a single-purpose, input-output mechanism. But, by assessing a country’s judicial system through an exclusive business climate lens, the judiciary is reduced to not much more than an obstacle course to get through. What the indicators don’t grasp is the actual place of the judiciary in a country’s evolving legal, political, cultural and socio-economic fabric. The indicators take numerical probes of what is, fundamentally, a continuing struggle not just for business-friendly legal norms, but justice. And it is this productive tension in a country’s legal culture between law and justice that animates the ongoing efforts to create as well as to contest, to reform and to improve the law so that it may get closer to the ideals of justice. At the heart of this effort lies a democratic practice that must be protected and supported also by the courts. How courts can fulfil that function surely cannot be reduced to the label ‘business enabling.’
The same bias has governed the DB’s ranking of other regulatory-related measurements such as registering property, resolving insolvency, or, prior to 2020, employing workers, all of which share common factors such as time, cost, outcome, and the formality of regulation or institutional structure. While those numbers are meant to convey some degree of cost-efficiency, they don’t account for the actual function of a judiciary to be a forum in which to engage not only the validity but also the meaning of a country’s law.
It is no surprise, then, that the DB’s judiciary rankings have been criticized since day one. The Indicators’ “embedded policy preferences” tacitly endorse a legal system bending to the will of businesses, and the indicators for measuring judicial proceedings investigate businesses’ perspectives only. The DB portfolio embraces a court system prioritizing business interests and undermining the courts’ true goals: justice and fairness.
Against this background, the DB indicators get it all wrong. One last example, where the Enforcing Contracts benchmark calculates how many pieces of law and regulation are published in any given year, they pay little attention to evaluating the substantive effects of those documents. This is exacerbated when there is a flooding of the index with “favourable” data relating to allegedly pro-business laws and regulations. As identifying a greater number of “laws and regulations” is rewarded with a higher place on the index, this strategy has led a Chinese working group to instruct law firms to count every single official document issued by regulatory agencies.
What’s Next for “Doing Business”, not just for China?
Interestingly, the WilmerHale report revealed favouritism towards China in the 2018 rankings. With the help of “data irregularities,” China’s overall ranking rose from 85 to 78. However, even where there was no suspicion of “manipulation”, China still showed a great leap forward, to 46 in 2019, and 31 in 2020. Therefore, with or without “data irregularities,” countries may find other “effective” ways to game the rankings.
Back in 2017, when the Chinese officials complained to officials of the World Bank, then-World Bank President Kim urged them to “focus on enacting economic reforms that would boost the country’s ranking under the DB report’s current methodology,” and asserted that “this was the same response he gave to every country”. Indeed, placing a heavy emphasis on institutional changes, DB has always applauded “robust” reform agendas in developing countries, awarding them with an exciting raise in rankings, persuading more to join the club of globalization.
The core aim of reforms is to increase “efficiency in governance” and to regulate informal business activities. To facilitate the transformation of informal practices into formal institutions, de Soto and other pro-market protagonists suggest governments should write, pass and revise “good laws” efficiently. Under the baton of reform, the top awards go to the countries that performed poorly previously and are exhibiting more reform measures in writing. That may explain why, although with similar data of time and costs in the Enforcing Contracts index, China’s ranking jumped from 35 to 5 during 2015-2020. But, by writing and publishing many laws and regulations, official documents or perfect paperwork, the government of China could also take the other path of reform—one which generates piles of legislation but doesn’t address much in substantive justice—at a much lower cost. After all, if the Chinese characteristics fit with the DB methodology so well, is that perhaps inevitable?
The long-term goal – ‘reform’?
All this only underscores the contested and competing meanings of “reform.” At this point in time, the discussion around a reform of the DB rankings risks leaving the law in action untouched. Worse, the many challenges of doing business in some jurisdictions may be eclipsed by a better score of de jure reform. But, sooner or later, the ongoing data scandal could cast a shadow over what the World Bank has always encouraged developing countries to engage in, and what the DB has proudly taken the credit for: reform. This reform would, ideally, be closer to the afore-mentioned idea of a country’s citizens, through democratic practice, law making and its judiciary, striving for law and justice.
Meanwhile, the measuring of real life through numbers and standardized indicators doesn’t seem to go away any time soon. For decades, the World Bank has been a key driver in quietly building norms of governance by indicators relying on their own knowledge products. The amount of resources dedicated to the DB rankings, including the political and judicial resources spent at the national level, indicates how powerful international institutions and global governance are in shaping domestic judiciaries and influencing state actors’ behaviour. Further, they show how realpolitik, equipped with bad science, could distort soft governance, or in more extreme cases, facilitate a corrupt court system.
The World Bank Group is currently formulating an updated product to assess the business climate, and even considering abandoning the notorious ranking system altogether. As long as the DB project continues its mission as a transnational legal indicator, the prevailing approach to measuring judicial processes needs to be revisited. That methodology reflects biased perspectives in the role of law, narrow-sighted indicators of the court system, and an oversimplified ranking that allows for falsification of data that is misleading and undermines its credibility. Regardless, the current rumours of reform may suggest that a search may be under way for a clearer vision beyond the obsession with efficiency, and a deeper insight into a regulatory framework to advance justice.