Nowcasting is a modern approach to monitoring economic conditions in real-time. It makes financial market trading more efficient because economic dynamics drive corporate profits, financial flows and policy decisions, and account for a large part of asset price fluctuations. The main technology behind nowcasting is the dynamic factor model, which condenses the information of numerous correlated ‘hard’ and ‘soft’ data series into a small number of ‘latent’ factors. A growth nowcast can be interpreted as the factor that is most correlated with a diverse representative set of growth-related data series. The state-space representation of the dynamic factor model formalizes how markets read economic data in real-time. The related estimation technique (‘Kalman filter’) generates projections for all data series and estimates for each data release a model-based surprise, called ‘news’. In recent years machine learning models, such as support vector machines, LASSO, elastic net and feed-forward artificial neural networks, have been deployed to improve the predictive power of nowcasts.

Sources are listed below.

Cursive text and text in brackets have been added for clarity.
The post ties up with this site’s summary on quantitative methods for macro information efficiency.

What is nowcasting?

Now-casting is defined as the prediction of the present, the very near future and the very recent past. The term is a contraction for now and forecasting and has been used for a long-time in meteorology…Now-casting is relevant in economics because key statistics on the present state of the economy are available with a significant delay. This is particularly true for those collected on a quarterly basis, [such as] Gross Domestic Product. [Bańbura, Giannone, Modugno, Reichlin]

“Nowcasting literally means to forecast the present…It characterizes the modern approach to monitoring current economic conditions in real time…The main idea of nowcasting is to analyze and interpret the macroeconomic news flow by continuously updating the predictions of key variables, like real GDP growth, for each data release.” [Time Series Analysis Team]

“Nowcasting is the dynamic process of making short-term estimates of lagging target variables — that is, estimates of economic variables that are announced relatively infrequently and with long delays…As there is often a significant delay in the information flow, by the time a provisional estimate is made (and often revised), we learn more about the recent past than about the present or future. Paradoxically, we have to  ‘forecast’ the recent past and present, as well as the future, to seek accurate results.” [Kozlov, Karaivanov, Tsonev, Valkov]

Why is nowcasting important for investment management?

Macroeconomic surprises explain a large part of asset price fluctuations, up to one-third of the quarter-to-quarter fluctuations in government bond yields…No one indicator can be a silver bullet that solves the problem of accurately tracking the evolution of the economy in real time. A more promising approach is, instead, combining the information contained in many available releases…Unlike professional forecasters who combine a variety of unrelated models and apply some form of judgment, using a single formal model that allows for a transparent…analysis of the real-time data flow. The model, in essence, codifies within an econometric framework the best practice and expert knowledge in business cycle analysis.” [Time Series Analysis Team]

“The current state of the economy is an important driver of asset returns. The cycle drives business plans, company profits, employment levels and consumer purchasing power. It directs monetary policy decisions and impacts economic policy…GDP…is an official measure covering the whole economy but, from the perspective of an investor, it is not very useful. First, it is only available quarterly and published with a significant delay (the first estimate is usually released four weeks after the end of the reference quarter, far too late to make decisions in real time). Second, it is noisy and, despite being an official measure, it is heavily revised – sometimes even years after the first release…Nowcasting can fill the gap for investors and provide a complete signal about underlying economic activity and in real-time.” [Little, Sonntag]

“Nowcasting…has developed into an essential tool for real-time analysis based on facts, not stories. It digests high-velocity asynchronous data at arbitrary arrival rates to estimate a low-velocity target variable using statistical techniques. At present, nowcasting is unmatched by any other technique in its ability to handle complexity and make use of vast amounts of data.” [Kozlov, Karaivanov, Tsonev, Valkov]

Dynamic factor model, state space, and Kalman filter

“The nowcasting model uses a large and heterogeneous set of predictors, including both ‘hard’ and ‘soft’ data (e.g., everything from unemployment statistics to consumer surveys)…The estimation procedure exploits the fact that these data series, although numerous, co-move quite strongly so that their behaviour can be captured by few factors. All output series are generated by a dynamic factor model…This type of model was chosen in order to cope with the so-called ‘curse of dimensionality’ (large numbers of correlated series) as it requires us to estimate only a limited number of parameters for a large dataset. The model assigns weights to the series, optimally exploiting the dynamic relationships among them. The nowcast can be interpreted as that component of growth which is highly correlated with all of the input data series: it disregards idiosyncratic information but it captures common signals given by all macroeconomic data releases including surveys.” [Now-Casting.com]

“The premise of a dynamic factor model is that a few latent dynamic factors drive the co-movements of a high-dimensional vector of time-series variables [such as economic time series], which is also affected by a vector of mean-zero idiosyncratic disturbances…The latent factors follow a time series process, which is commonly taken to be a vector autoregression…An important motivation for considering dynamic factor models is that, if one knew the [latent] factors…one can make efficient forecasts for an individual variable using…regression of that variable on the lagged factors and lags of that variable…Dynamic factor models have the twin appeals of being grounded in dynamic macroeconomic theory and providing a good first-order description of empirical macroeconomic data, in the sense that a small number of factors explain a large fraction of the variance of many macroeconomic series.” [Stock, Watson]

“The engine of [nowcasting] is [often] the dynamic factor model, equipped with advanced filtering techniques, of the kind used in robotics…These techniques are very common in big data analytics since they effectively summarize the information contained in large data sets through a small number of common factors…The development of nowcasting has been made possible thanks to recent advances in the econometrics of high-dimensional data. That data set covers essentially everything from manufacturing and inventories to the sentiment of purchasing managers, from labor market indicators to transportation services and international trade. The approach is based entirely on automated procedures…The news flow is naturally processed in the same way as by any informed person. First, the surprise component is extracted from the data. Second, these surprises are translated into a common unit, which is their impact on key macroeconomic indicators, say, the GDP nowcast or corporate profits.” [Time Series Analysis Team]

“A significant breakthrough in nowcasting came with the introduction to the models of what’s known in statistics as state-space representation, a technique for analyzing systems with multiple inputs and outputs. State-space representation expresses variables as vectors — that is, as quantities that have direction and magnitude. In this representation, a vector of observed variables is assumed to be explained by a number of unobserved factors. These ‘latent’ or hidden factors follow an autoregressive relationship, which means that their current values depend on their past values.” [Kozlov, Karaivanov, Tsonev, Valkov]

State space…frameworks allow formalising…how market participants…read macroeconomic data releases in real time, which involves: monitoring many data, forming expectations about them and revising the assessment on the state of the economy whenever realizations diverge sizeably from those expectations. This is possible because for a model in a state space representation the Kalman filter generates projections for all the variables considered and therefore allows to compute, for each data release, a model-based surprise, the news. Further, now-cast revision can be expressed as a weighted average of these news.” [Bańbura, Giannone, Modugno, Reichlin]

“Nowcasting relies on state-space representation…to represent the evolution of a variable through time in a way that depends on its past values and the evolution of other variables…The general idea is that the observed economic factors depend upon an unobserved state (latent factor)…It is a natural representation for handling mixed frequencies (monthly/quarterly/yearly) and non-synchronicity of data releases. The Kalman Filter is then used to make the nowcast. It extracts co-movements in the timeseries data as a latent [unobservable] factor, use it to estimate past and present values of the observed data, make corrections when new data comes in, and nowcast the current state and values of the variables.” [Yip]

“Economic data releases occur at different points of the month and with different frequencies. It is therefore important to have a technique to deal with the idiosyncrasies of the data calendar. When a value is missing, the Kalman Filter replaces the missing value with an estimate. This feature enables the model to easily handle missing data points and gradually improve the measure of economic activity in a particular month, as more data is released.” [Little, Sonntag]

“The model is also designed to capture the essential characteristics of the now-casting problem in real time: that is, updating the estimates in relation to the flow of data releases throughout the day, the week and the month.  This implies that the estimates are computed with missing observations in some of the series at period ends due to varying publication lags (“jagged edged” data sets)…Specifically, we cast the model in state space, estimate the parameters by ‘maximum likelihood’, and apply the Kalman filter with a smoother to cope with jagged edged and mixed frequency data series. The model allows us to compute a joint forecast of predictors and target series and, at each release, to calculate the surprise component of the published data release (what we call the ‘news’). The revision of the now-cast of quarterly GDP growth can then be described as the product of the weight of each series (estimated on historical data) and the news for each release.” [Now-Casting.com]

The role of machine learning

“We train a range of popular machine learning algorithms over an expanding window and replicate an actual nowcasting situation… Our results show that the majority of the machine learning models produce point nowcasts that are superior to the simple autoregression benchmark. The top-performing models such as the support vector machines, Lasso (Least Absolute Shrinkage and Selection Operator) and neural networks are able to reduce the average nowcast errors by approximately 16-18%…Moreover, combining the nowcasts of the ML models using various weighting schemes leads to further improvements in performance.” [Richardson, van Florenstein Mulder, Vehbi]

“We rely on combinations of nowcasts obtained from a range of statistical models and machine learning techniques which are able to handle high-dimensional information sets. The results of our pseudo-real-time analysis indicate that a simple nowcast combination based on these models provides faster estimates of GDP, without increasing substantially the revision error.” [Fornaro, Luomaranta]

“We have built a nowcasting model using a machine learning technique known as Elastic Net Regularisation and Variable Selection…The Elastic Net Regularisation and Variable Selection…is a machine learning algorithm that augments standard statistical techniques (such as Ordinary Least Squares regression) by introducing a ‘penalty factor’…that constricts the size of the impact [of] different variables…[preventing] the model from ‘over-fitting’ past data…The program ‘learns’ the right penalty size to optimise the model’s out of sample performance using a well-known method called ‘K-fold cross-validation’…[This] allows the use of highly similar variables…It creates an invisible ‘net’ around such groups of variables which can stretch to accommodate additional variables.” [Hinds et al]

“[We] demonstrate that feed-forward artificial neural networks can be applied to nowcasting, as well as to forecasting…The implementation of artificial neural networks can be automatized in the process of now- and forecasting…We compare the feed-forward artificial neural network forecasts of GDP growth to…state of the art dynamic factor models and the Survey of Professional Forecasters…The results indicate that the neural network outperforms the dynamic factor model in terms of now- and forecasting, while it generates at least as good now- and forecasts as the Survey of Professional Forecasters.” [Loermann, Maas]

Key empirical findings

“Nowcasting models can provide a similar degree of accuracy to [official] preliminary estimates of GDP growth, despite the nowcasting estimates being more timely. Academic research also shows that the nowcasts represent a significant improvement on standard statistical forecasting models that only make use of past data.” [Hinds et al]

“Results in the literature on nowcasting [of quarterly GDP] have provided support for several general conclusions.

  • First, gains of institutional and statistical forecasts of GDP relative to the naıve constant growth model are substantial only at very short horizons and in particular for the current quarter…
  • Second, the automatic statistical procedure performs as well as institutional forecasts which are the result of a process involving models and judgement…
  • Third, the nowcasts become progressively more accurate as the quarter comes to a close and the relevant information accumulates, hence it is important to incorporate new data as soon as they are released.
  • Fourth, the exploitation of timely data leads to improvement in the nowcast accuracy. In particular, the relevance of various data types is not only determined by the strength of their relationship with the target variable, as it is the case in traditional forecasting exercises, but also by their timeliness. Soft information has been found to be extremely important especially early in the quarter when hard information is not available.” [Bańbura, Giannone, Modugno, Reichlin]

Kalman-filtering techniques and a dynamic factor model approach…[habe] been tested for accuracy in many countries, including large developed economies (the euro area, Italy, France, Germany, Spain, the United Kingdom, Japan, and Canada), small open economies (Australia, Ireland, Belgium, New Zealand, the Czech Republic, and Scotland), fast-growing economies (Brazil, Russia, India, China, and South Africa), and developing economies (Mexico, Indonesia, and Argentina)… Kalman-filtering techniques and a dynamic factor model. The approach has a number of desirable features.” [New York Fed Nowcasting Report]

“Small data approaches came first, and they typically involve maximum likelihood estimation. Subsequent Big Data approaches, in contrast, typically involve two-step estimation based on a first-step extraction of principal components…Big Data nowcasting approaches are not necessarily better. First, they are more tedious to manage, and less transparent. Second, they may not deliver much improvement in factor extraction accuracy, which increases and stabilizes quickly as the number of indicators increases. Third…a poorly-balanced set of indicators can create distortions in the extracted factor, whereas small data approaches promote and facilitate hard thinking about a well-balanced set of indicators.” [Diebold]

Applications in financial research

“There are a range of different approaches that can be taken to produce nowcasts. The institutions that have developed nowcasting models most intensively are central banks…[The table below] sets out how five notable monetary authorities use nowcasting models.” [Hinds et al]

“The [New York Fed nowcasting] report tracks the evolution of the New York Fed Staff Nowcast of GDP growth and the impact of new data releases on the forecast.” [New York Fed Nowcasting Report]

“The growth rate of real gross domestic product (GDP) measured by the U.S. Bureau of Economic Analysis (BEA) is a key metric of the pace of economic activity. It is one of the four variables included in the economic projections of Federal Reserve Board members and Bank presidents for every other Federal Open Market Committee (FOMC) meeting…The Atlanta Fed GDPNow model also mimics the methods used by the BEA to estimate real GDP growth. The GDPNow forecast is constructed by aggregating statistical model forecasts of 13 subcomponents that comprise GDP.” [GDPNow]

Packages for nowcasting

“[R] package ‘nowcasting’ contains the tools to implement dynamic factor models to forecast economic variables. The user will be able to construct pseudo real time vintages, use information criteria for determining the number of factors and shocks, estimate the model, and visualize results among other things.” [nowcasting]

“The python ‘statsmodels.tsa.statespace.dynamic_factor’ module…contains classes and functions that are useful for time series analysis using state space methods.” [statsmodel.org]

References

Bańbura, Marta, Domenico Giannone, Michele Modugno and Lucrezia Reichlin (2013), “Now-casting and the real-time data flow”, ECB Working Paper 1564, July 2013
https://www.ecb.europa.eu/pub/pdf/scpwps/ecbwp1564.pdf

Castle , Jennifer, David Hendry and Oleg Kitov (2013), “Forecasting and Nowcasting Macroeconomic Variables: A Methodological Overview.”, University of Oxford, Department of Economics, Discussion Paper, Number 674 September 2013.
https://www.economics.ox.ac.uk/department-of-economics-discussion-paper-series/forecasting-and-nowcasting-macroeconomic-variables-a-methodological-overview

Diebold, Francis (2020), “Real-Time Real Economic Activity: Exiting the Great Recession and Entering the Pandemic Recession.”
https://arxiv.org/pdf/2006.15183.pdf

Fornaro, Paolo and Henri Luomaranta (2018), “Nowcasting Finnish Real Economic Activity: a Machine Learning Approach.” May 2018
https://www.stat.fi/static/media/uploads/tup/nowcasting_empecon.pdf

GDPNow, Federal Reserve Bank of Atlanta
https://www.frbatlanta.org/cqer/research/gdpnow.aspx?panel=1

Giannone Domenico, Argia Sbordone, and Andrea Tambalotti, Hey, Economist! How Do You Forecast the Present?, Liberty Street Economics, JUNE 16, 2017
https://libertystreeteconomics.newyorkfed.org/2017/06/hey-economist-how-do-you-forecast-the-present.html

Hinds, Sam, Lucy Rimmington, Hugh Dance, Jonathan Gillham, Andrew Sentance and John Hawksworth (2017), PwC UK Economic Outlook July 2017
https://www.pwc.co.uk/economic-services/ukeo/pwcukeo-section4-nowcasting-july-2017.pdf

Kozlov, Michael, Svetoslav Karaivanov, Dobroslav Tsonev and Radoslav Valkov (2018), “The News on Nowcasting”, Worldquant Perspectives.
https://www.weareworldquant.com/en/thought-leadership/the-news-on-nowcasting/

Little, Joseph and Marcus Sonntag (2017), “Finding a signal in noisy economic data – Nowcasting the economy to develop a better investment strategy”, HSBC Global Asset Management, White Paper, January 2017
http://www.frinfo.assetmanagement.hsbc.com/inews-issue-64/nowcast.pdf

Loermann, Julius and Benedikt Maas (2019), “Nowcasting US GDP with artificial neural networks”, MPRA Paper No. 95459, 08 Aug 2019
https://mpra.ub.uni-muenchen.de/95459/1/MPRA_paper_95459.pdf

New York Fed Nowcasting Report, updated each Friday
https://www.newyorkfed.org/research/policy/nowcast/overview.html

Nowcasting R-package,
https://github.com/nmecsys/nowcasting

Now-Casting.com, “Methodology”.
https://www.now-casting.com/resources/methodology

Richardson, Adam. Thomas van Florenstein Mulder, and Tuğrul Vehbi (2018), “Nowcasting New Zealand GDP using machine learning algorithms”, Bank Indonesia International Workshop and Seminar on “Big Data for Central Bank Policies / Building Pathways for Policy Making with Big Data” Bali, Indonesia, 23-26 July 2018
https://www.bis.org/ifc/publ/ifcb50_15.pdf

statsmodel.org, “Time Series Analysis by State Space Methods”
https://www.statsmodels.org/stable/statespace.html

Time Series Analysis Team, “Nowcasting and Forecasting with Big Data”, Federal Reserve Bank of New York October 8, 2019
https://www.newyorkfed.org/medialibrary/media/research/policy/nowcast/2019-central-banking-seminar-presentation.pdf?la=en

Yip, Jason (2020), “Macroeconomic Nowcasting with Kalman Filtering”, Towards Data Science post
https://towardsdatascience.com/macroeconomic-nowcasting-with-kalman-filtering-557926dbc737