Abstract
A challenging problem in the study of complex systems is that of resolving, without prior information, the emergent, mesoscopic organization determined by groups of units whose dynamical activity is more strongly correlated internally than with the rest of the system. The existing techniques to filter correlations are not explicitly oriented towards identifying such modules and can suffer from an unavoidable information loss. A promising alternative is that of employing community detection techniques developed in network theory. Unfortunately, this approach has focused predominantly on replacing network data with correlation matrices, a procedure that we show to be intrinsically biased because of its inconsistency with the null hypotheses underlying the existing algorithms. Here, we introduce, via a consistent redefinition of null models based on random matrix theory, the appropriate correlation-based counterparts of the most popular community detection techniques. Our methods can filter out both unit-specific noise and system-wide dependencies, and the resulting communities are internally correlated and mutually anticorrelated. We also implement multiresolution and multifrequency approaches revealing hierarchically nested subcommunities with “hard” cores and “soft” peripheries. We apply our techniques to several financial time series and identify mesoscopic groups of stocks which are irreducible to a standard, sectorial taxonomy; detect “soft stocks” that alternate between communities; and discuss implications for portfolio optimization and risk management.
- Received 16 November 2013
DOI:https://doi.org/10.1103/PhysRevX.5.021006
This article is available under the terms of the Creative Commons Attribution 3.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.
Published by the American Physical Society
Popular Summary
A challenging problem in the analysis of complex systems, from stock markets to the brain, is that of unveiling mesoscopic structure, otherwise known as the community structure of a system. This structure is composed of groups of objects that bear a certain resemblance to each other, more so than with other objects in the system. Community detection is an area of research that has so far focused on this task for networks. However, the problem is relevant to many systems other than networks and, in particular, systems whose internal units have correlated temporal activity. An example of such a complex system is that of publicly traded companies on a stock exchange, where the performance of each company is correlated with the performances of other companies to a certain degree.
Identifying communities of highly correlated stocks provides critical risk information that can be used to balance and optimize one’s portfolio. Current applications of community detection to correlation matrices have merely replaced the network structure with the raw correlations, leading to less-than-optimal community structures. We instead employ a method that appropriately adapts some of the most popular community detection algorithms traditionally applied to networks. Using random-matrix theory, we redefine the null models upon which these algorithms are based to ensure consistency with the properties of a correlation matrix. We then benchmark our algorithms against synthetic data with a well-controlled community structure to assess their accuracy, and we finally apply these techniques to a variety of international stock markets, including the S&P 500 and the Nikkei 225. We find that our model, which we apply to different sectors such as telecommunications and energy, is able to filter out noise and reveal smaller-scale correlations (i.e., hierarchically nested “subcommunities”).
We expect that our findings will be useful for optimizing financial portfolios and tracking the substructure of complex systems. We also expect promising applications to functional brain network research.
