Kajitani Kai, Professor, Kobe University Graduate School

 

Recently, there has been a growing awareness in Japan of the importance of “evidence” in reporting and policymaking. Especially when making decisions about highly opaque topics like China, which is my specialty, it is extremely important to support and verify evidence such as statistical data.

On the other hand, some voices are sounding the alarm that politicians and large corporations are waiving evidence and objectivity while ignoring the small voices of those affected, causing divisions in society. While there are some valid aspects to this criticism, I cannot agree with an attitude that places too much emphasis on individual narratives. Rather, we should pay attention to how much of the “evidence” that the government and others present is “falsifiable” by third parties.

For example, since Russia’s invasion of Ukraine in February 2022, more attention has been paid to Taiwan as the region where the next major conflict could occur. At the time of the invasion, the Japanese media frequently referred to the statement made by Admiral Davidson, then commander of the US Pacific Command, in the US Senate in March 2021 that a crisis over the Taiwan Strait would emerge by 2027. However, it is difficult to say that these reports were falsifiable. The number “2027” has no clear basis other than to indicate the end of Xi Jinping’s third term, and there was no evidence to disprove it in the first place.

In an article for Toyo Keizai Online, Toyo Keizai columnist Nishimura Gota described the overheated coverage of the “Taiwan Emergency 2027 Theory” as a kind of “bubble.” He added that security experts have realized that the theory is unreasonable. Furthermore, Nishimura pointed out that in order to achieve the goal of expanding [the US military’s] defense capabilities, “[the unreasonable parts of this theory] were covered up as inconvenient truths.”^[1]

On the other hand, the “NHK Special” broadcast on November 5, 2023, Chosa-hodo, Sinseiki File 1 Chugoku “Keizai shissoku” no Shinjitsu (Investigative reporting, new century File 1: The truth about China’s “economic stall”)^[2] was a meaningful attempt. This program collected and analyzed data from statistical materials in each region to reveal the hidden economic activities of China, especially in each region, and used this information to predict the future. The program makers created an annual chart of the types of data published in databases commonly used by researchers, and showed that the number has decreased significantly in recent years. The simple but labor-intensive verification process was introduced, including behind the scenes.

However, there were some parts of the show that seemed a bit misleading. The program makers presented research by Assistant Professor Luis Martinez of the University of Chicago^[3] on the verification of GDP (Gross Domestic Product) statistics using night time lights (NTL) data from satellite imagery. They introduced his argument that “China’s GDP growth rate is overestimated by more than 30% compared to estimates based on NTL, which cannot be ‘cheated.’” This suggested that China’s GDP statistics were inflated because China has become reluctant to release data.

In fact, research using NTL intensity as an alternative indicator of economic activity has been active in recent years. However, it should be treated with caution. A country’s economic activities are not only reflected in NTL, and the relationship with GDP varies depending on factors such as daylight hours and population density, so NTL cannot be used as a substitute variable for GDP. Therefore, when estimating GDP using NTL, it is first necessary to determine what kind of “stable relationship” exists between the two. Based on this premise, it is necessary to find a “less fraudulent” value of GDP from the former data. The question is, what kind of data should be used to estimate a “stable relationship” between the two?

If the official GDP figures are consistently inflated, it is because there is no “stable relationship” between the two in the first place. Therefore, Martinez uses the following method. First, he analyzed the relationship between each country’s NTL increase/decrease rate and GDP growth rate, and compared authoritarian and democratic countries. The results showed that the former has a larger “value indicating the percentage increase in the real GDP growth rate (elasticity value)” when NTL increases by 1%. Using this result, Martinez estimated GDP on the assumption that if an authoritarian country such as China were in the same situation as a democratic country, the above elasticity value would be lower.

But this raises some questions. Authoritarian countries have less developed statistical systems and less GDP coverage than democratic countries. Therefore, the relationship with NTL may also change with economic growth. In other words, Martinez’s research shows that the official GDP figures of the former have a common bias, but it does not necessarily clarify what that “bias” is.

However, I felt that the program was trying to give the impression that the Chinese government’s attitude of trying to hide its economic slump was the true nature of its “bias” by presenting Martinez’s research after reporting on such issues as the over-indebtedness of China’s local government financing vehicle (LGFV).

The above criticisms may seem to come from a rather technical perspective. However, for evidence-based reporting to take root in Japan, it is essential that it be verified by experts. NHK’s website provides links to primary sources and papers obtained through independent research. Publishing data in this way to ensure “reproducibility” is the foundation of evidence-based analysis, which is why I was able to make this criticism. The author appreciates this attitude of NHK.

Conversely, we should always maintain a skeptical attitude toward the information that lacks “reproducibility” and “falsifiability,” such as the reports based on the above-mentioned “2027 crisis theory” of the Taiwan emergency. In this era of increasing uncertainty, we must protect ourselves from information overload by using evidence as a weapon.

Translated from an original article in Japanese written for Discuss Japan. [March 2024]

 

Keywords

• Kajitani Kai
• Graduate School of Economics
• Kobe University
• evidence-based reporting
• reproducibility
• falsifiability
• policymaking
• China
• Toyo Keizai Online
• Nishimura Gota
• NHK
• China’s economic stall
• night-time lights
• GDP
• Taiwan Emergency 2027 Theory

---