The desire of explaining things and trends around us has been a decisive component of wisdom. The complexity of nature challenges human thought and experience to answer the question of “why.” The answers have been wide-ranging, from religion to experimental science. The desire to explain and tackle the “challenge of complexity” is invaluable. For most, it is the differentiator between human and animal, as the former has the ability to ask “why” and “how” before reacting to events while the latter acts on natural instincts. Being able to ask these questions gives humanity opportunities to behave against their natural instincts and make unexpected but useful discoveries. It was the questions like, “Why did this apple fall?” that led Newton to the law of gravity, which then was used to develop many useful mechanical devices for human beings.

Every human being asks the question “why,” though at different levels, to explain the unexplained. It follows a pattern of questions, like “Why did the financial crisis in the U.S. happen in August 2008?” “Why did the space shuttle Challenger explode?” “Why did the terrorists commit the September 11 attacks?” In statistical terms, such unexpected events are named “outliers,” however, they are part of the system and among the components constituting the overall system’s complex behavior. Thus, they need to be part of the explanation in order for the explanation to be complete. We are naturally tempted to come up with universal explanations of the complexity behind these major events so that we can be ready when a similar thing happens again. Though simple mathematical equations or relationships relate to us better and provide a universal explanation, they are typically practical only when the outliers are excluded from the system behavior. Statistics help us greatly in quantifying and characterizing the outliers, especially in the form of probabilistic expressions, such as “there is a 30% chance of a hurricane next week.”

Understanding the complexity around us involves the development of a model that is simple enough for us to comprehend but yet universal enough to capture most of the dynamics of the complexity. The simpler and the more universal the model, the more powerful it is. The universality of a model, however, is hindered by the potential inability to capture something unexpected. The tradeoff between simplicity and universality exists in all modeling efforts; and the models finding the delicate balance in this tradeoff are the most effective ones. A simple mathematical relationship known as “the power law” has been used extensively to characterize and model various natural and social phenomena.

What is the Power Law?

The “power law” does not refer to a misconception that “whoever has power will rule,” but rather it refers to a particular way of characterizing dependency between two quantities. When the number or frequency of an object or event varies as a power of some attribute of that object (e.g., its size), the number or frequency is said to follow a power law. In more general terms, there exists a power law relationship between x and y if y is growing or reducing polynomially when x is growing linearly (y  _͌ x^–α). Mathematically speaking, this means that the relationship between y and x is mainly characterized by the exponent -a. An exponent is simply shorthand for multiplying that number of identical factors. So, 4³ is the same as 4x4x4; that is three identical factors of 4. As shown in Figure 1, a quantity with an exponent has three components: the base, the exponent, and the coefficient. So, for 4³, the base is 4, the exponent is 3, and the coefficient is an implicit 1.

Figure 1: Description of an exponent in a power law relationship.

Figure 2: Sample power law relationships between x and y, where y = x^–α.

The power law relationships are traditionally expressed with a negative exponent, which simply means the inverse of the quantity. That is, y  _͌ x^–α is equivalent to y  _͌ 1/x^α. For example, when a is 2, y will reduce from 1/4 (i.e. 0.25) to 1/9 (i.e. ~0.11) if x grows from 2 to 3. Likewise, when a is 0.5, y will reduce from 1/2 (i.e. 0.5) to 1/3 (i.e. 0.33) if x grows from 4 to 9. For those who enjoy graphs, Figure 2 illustrates these mathematical relationships in linear and logarithmic scales.

Power law on different scales: Atoms to planets

To start with, gravitation, acoustics, electrostatics, and light and electromagnetic radiation, all exhibit a form of power law in that physical quantity or strength that is inversely proportional to the square of the distance, which corresponds to a power exponent of 2. [1] Gravitational force between two particles, the electrostatic force of attraction between two electrically charged particles, the intensity of sound signals coming from a source, and finally the intensity of light or electromagnetic field coming from a source all follow a power law with respect to the distance.

What makes the power law relationships more interesting is their independence from scale or size of the measures being related to each other. This is why we sometimes call power law relationships as “scale-free” relationships or “scale-invariance.” For example, the gravitational force between two spherical particles decays with a power exponent of 2 regardless of the sizes of the particles though the actual force is certainly dependent on the particle sizes. So, the particles can be at nano scales (e.g. a group of atoms) or macro scales (e.g. a planet), but the relationship stays the same!

Power law in frequency: Wealth, terror, and earthquakes

A common usage of power law relationships has been to model and understand frequency of a varying measure. A power law typically very well represents the distribution of wealth in a society. [2] According to a recent study, the distribution of wealth in China during the years 2003–2005 follows a power law with an exponent ranging from 1.758 to 2.285. If we consider an average exponent of 2 for Chinese wealth distribution, this means that if there are 1 million Chinese people who owned $1000 there were 1000 that owned $1M. Thus, the power law essentially expresses how skewed the distribution of a frequency is (see Figure 2). The larger the power exponent, the more skewed the distribution. In this case, a larger power exponent means a more imbalanced wealth distribution while a power exponent of 1 refers to an evenly distributed wealth.

Many other social patterns exhibit power law. A recent study showed that it exists even in terror events! The number of casualties per insurgent event and the number of insurgent events per day follow a power law. [3] Historical data for the last two centuries show further that the number of casualties per war or a terror attack follows a power law distribution. What is even more interesting is that the number of casualties and the number of attacks within an insurgent conflict both follow power law. That is, when only a particular conflict between two countries or ethnic groups is considered, the number of casualties per insurgent event and the number of insurgent events per day follow the power law. This suggests a “self-similar” pattern. Likewise, traffic measurements for many systems show power law distributions of size. For instance, if one observes the data traffic on an Internet connection and counts the number of bytes being transmitted per hour over that connection, a power law distribution of the count of bytes will emerge. Further, if this counting is done per minute instead of per hour, a similar distribution will still emerge – again showing a self-similar pattern. [4]

The power law has been observed in several natural phenomena as well. The frequency of earthquake magnitudes follows a power law. [5] This refers to the intuitive notion that the number of earthquakes with small magnitudes (which humans do not even feel) is much larger than the number of earthquakes with large magnitudes, (which can kill many humans). Small earthquakes are the norm while large ones the outliers. However, without the outliers, there is no power law distribution! Thus, the power law distribution of a quantity comes with an interesting observation: If a quantity is indeed following a power law distribution, then the likelihood of an outlier event increases as the time goes by without an outlier event. This is why geoscientists would make comments like “The region X is due for a major earthquake!” indicating that the region X has not been receiving a major earthquake (i.e. an outlier) for several years. The issue, though, is determining the threshold for an outlier is typically ambiguous and may require many years of measurements and data, which may be impractical.

Power law in growth: Rich get richer

Growth of systems also exhibit power law in various ways. Social growth follows power law due to the well-known “rich get richer” rule, which refers to the intuition that “important” people in the society attract more of the attention of newcomers. This dynamic situation is observed, for example, in the growth of the Internet. Several studies [6] showed that the connections between Internet Service Providers (ISPs) (e.g., AOL, Yahoo!, AT&T, Sprint) follow a power law distribution in that the number of connections per ISP (which shows how well an ISP is connected to the rest of the world) is represented by power law. In other words, there are few ISPs with many connections to other ISPs while most ISPs have a few connections to the others. This is believed to be due to the “rich get richer” rule since an existing ISP with many connections is more likely to gain the business of a new ISP who is joining to the Internet. So, it is somewhat an economic pattern too.

If economics (or the money) is taken out of the picture, social growth still exhibits power law. Online social networks such as Facebook, LinkedIn, and Flickr are clearly following a power law distribution. It is found that the power exponents are in the range of 2.5 to 3.7, indicating a highly imbalanced social growth pattern where few people are at the “center” of the social network with hundreds or thousands of friends, and many people have only one or two friends. [7] Again, the typical explanation for this growth pattern has been the “rich get richer” rule, but “richness” refers to the number of existing friends in this context rather than money.

Physical growth shows power law too in many ways. For instance, roughness of a growing surface as time goes by follows a power law distribution with an exponent ranging between 0 and 1 where an exponent of 0 refers to a smooth growth and 1 refers to a stiff growth. The surface roughness is measured by the variance of heights of surface locations. [8]

Does it really exist? Why does it exist?

Verifying existence of a power law distribution is not easy and requires enough number of samples to show the “tail” of the distribution. The tail of the distribution refers to the samples with large (or rare) values. For example, for the power law distributions in Figure 2, the portion of the distribution when x is greater than 10 (i.e. x>10) roughly corresponds to the “tail.” The tail corresponds to the rare samples. Though statistical theory calls those rare samples “outliers,” the distribution will not be a power law distribution without them. They are strictly parts of pieces that constitute a power law relationship, and observing them typically requires long periods or large numbers of measurements. Due to this difficulty, the existence of the power law is questioned for many real systems. Most of the time, claims of the existence of the power law typically come with an error factor indicating the confidence of the claim. The bottom-line is to observe trends in the samples and thus establish sufficient confidence (e.g., more than 95%) that the power law distribution does exist in the samples.

For those systems with clear exhibition of power law, the root causes of it have been of high interest. The “rich get richer” rule is intuitively one of the root causes, and it is intuitively a natural dynamic to get attracted by a rich member rather than a poor one. Growth certainly naturally follows the “rich get richer” rule, but we have system components slowing their growth, flattening, and then deteriorating. So, not everything is growing, and actually, we have as many things deteriorating as growing. For instance, participants join or leave the Internet or the social networks, and likewise, people join (i.e. birth) or leave (i.e. death) society. How does the power law stay in such systems then?

Due to the “rich get richer” intuition, the power law is considered to be the signature of “self-organization.” The fact that so many natural or synthetic systems are exhibiting this signature deserves the question: “Is it really self-organization?” Maintaining a global power law distribution for a system requires either (i) every member joining or leaving the system according to the “rich get richer” rule and having global knowledge of the whole system or (ii) somebody who knows everything about the system and gives explicit direct orders to each member when they are joining or leaving. Which one is more likely?

Murat Yuksel is an Assistant Professor at the CSE Department of The University of Nevada - Reno (UNR), Reno, NV.

References

[1] Wikipedia, “Inverse-square law,” http://en.wikipedia.org/wiki/Inverse-square_law

[2] M. A. Santos, R. Coelho, G. Hegyi, Z. Néda, and J. Ramasco. 2007. “Wealth distribution in modern and medieval societies,” The European Physical Journal, Volume 143, Number 1, pages 81-85.

[3] J. C. Bohorquez, S. Gourley, A. R. Dixon, M. Spagat, and N. F. Johnson. 2009. “Common ecology quantifies human insurgency,” Nature, Volume 462, December, pages 911-914.

[4] T. Karagiannis, M. Molle, and M. Faloutsos. 2004. “Long-Range Dependence: Ten Years of Internet Traffic Modeling,” IEEE Internet Computing, September/October, pages 57-64.

[5] T. Lay and T. Wallace. 1995. Modern Global Seismology, Academic Press, San Diego, CA.

[6] M. Faloutsos, P. Faloutsos, and C. Faloutsos. 1999. “On power-law relationships of the Internet topology,” ACM Computer Communication Review, Volume 29, Issue 4.

[7] R. Kumar, J. Novak, and A. Tomkins. 2006. “Structure and evolution of online social networks,” Proceedings of ACM SIGKDD, pages 611-617.

[8] A. L. Barabasi and H. E. Stanley. 1995. Fractal Concepts in Surface Growth, Cambridge University Press, Cambridge, England.