A major global vendor of political risk analytics gave a talk at the Fletcher School today. They rate political risks according to a 5-point scale, with copious documentation of how countries’ political life is compressed down into this one metric. As a colleague noted, nobody uses the political risk metric for anything, because it’s such a vague number. I disagree. At the end of the day, people would much prefer to use a simple, bad heuristic to complex information that doesn’t permit quantitative comparison. Many people will categorize countries on the basis of a 5-point scale largely because of the analyst’s good reputation.

Reputable academics use simple indices in quantitative studies all the time explicitly because someone has already standardized the observations. Social science variables are cost-prohibitive to measure otherwise. This phenomenon is especially true for studies that compare countries’ performance on democracy, corruption, and many other socially defined, abstract categories. The big danger is that, due to competitive business dynamics, the leading provider will define political risk analysis, and that the whole industry will coordinate its expectations and analytic frames around that algorithm.

Definitions of risk abound. Mathematically, risks most often refer to the probability that something will happen. (Frequently, risk can also refer to the expected value of a loss from the occurrence of an event. More on that in another post, another day.) That’s the reason that financiers borrowed the term risk to describe sovereign default risk. They wanted to compare the probabilities that countries would default, and then be able to transact those risks one way or another.

The political analysis masquerading as risk analysis has serious underlying problems. First, there are no units. Probabilities and values at risk are still units of analysis, even if they don’t have standards like the kilogram and the meter. These numbers are just numbers on a scale. If the scale measured one thing, it would be more useful. The fact that it measures many things means that offsetting variation in two of the components doesn’t necessarily change the index value.

Second, the cardinal system (a scale of 1 to 5) doesn’t imply anything that a cardinal number system ordinarily implies. There isn’t twice as much of anything at 2 on the scale as there is at 1. There isn’t twice as much difference between 3 and 1 as between 2 and 1. There aren’t logaritms, complex numbers or any other succinct mathematical ways of describing the differences between levels on the scale.

Third, a much weaker mathematical standard falls down too. The scale does not provide sound ordinal information about countries. It’s not true that a country rated 3 is unambiguously riskier than a country rated a 2. Because of the huge number of categories of political analysis lumped together in political risk, reasonable people could disagree with the formula used to come up with a single number for political risk. The fact country A comes out slightly higher than similar country B is as much an artifact of the algorithm as an observation of the country.

The equation used to calculate the index also doesn’t necessarily relate to what any single actor will find indicative of risk. The political bias toward democracies and market investments surely doesn’t provide as good a description of risk for Chinese state-owned firms, as it does for US firms doing business overseas. The index describes risk from a particular perspective, and that perspective is evident in the description of the components of the algorithm.

The true wealth of observations on the company’s database more than compensates for this. In fact, I think the tool is useful particularly because it contains specific observations about political systems, investment laws, business practices, corruption measures, etc. The data points themselves are the interesting feature of the database, rather than the measures of risk used to aggregate them.

Read all the fine print if you plan to refer to a political risk index in your papers, and if you use them in business be prepared for questions about the methods.

(Thank you Patrick.)

Excellent, thanks for posting, Ben.