My Finance

Updated annual volatility for 2024

My undergraduate student Angel Piotrowski updated annual realized volatility for 2024. Previously she computed it for 1928-2023, each year. She took log change in daily closing prices of the Standard & Poor 500, or its predecessor, Standard & Poor 90, and computed empirical standard deviation. Given this annual volatility data $V(t)$ she analyzed this. First, she computed the autocorrelation function for $V(t).$

This strongly suggests using the autoregression model, which is called the Heston model in quantitative finance:

$V(t) = \alpha + \beta V(t-1) + W(t).$

Results after fitting this simple linear regression using ordinary least squares method are:

$\alpha = 0.003920,\, \beta = 0.626196$

Now let us analyze residuals (innovations) $W(t)$ which are supposed to be independent identically distributed mean-zero Gaussian: $W(t) \sim \mathcal N(0, \sigma^2).$ Angel did this by making the autocorrelation function plots for $W(t)$ and for $|W(t)|$ , as well as the quantile-quantile plot of $W(t)$ versus the normal distribution:

We see that the ACF plot for $W(t)$ corresponds to white noise but the ACF plot for $|W(t)|$ does not. A few first lags have significant autocorrelation. Less importantly but also unfortunately, the quantile-quantile plot shows the innovations $W(t)$ are not Gaussian.

Yet another problem with this Heston model: Volatility can go negative according to this model, but this is impossible in real life. As a standard deviation of market fluctuations, volatility is always supposed to stay positive.

Next, Angel modeled the resulting series $V(t)$ as an autoregression of order 1 on the logarithmic scale:

$\ln V(t) = \alpha + \beta \ln V(t-1) + W(t).$

For updated data, $\alpha = -1.776087$ and $\beta = 0.620147$ so there is mean-reversion. She did not test for unit root but I am very sure this hypothesis (that $\beta = 1$ ) would be rejected.

See the autocorrelation function plots for innovations $W(t)$ and for their absolute values $|W(t)|$ which show these $W(t)$ can be modeled as independent identically distributed. And the quantile-quantile plot versus the normal distribution shows these are Gaussian.

The resulting stationary distribution $\ln V(\infty)$ is Gaussian with mean -4.68 and variance 0.218. Using the moment generating function for the normal distribution, we can compute $\mathbb E[V(\infty)] = 0.0104$ and variance $\mathrm{Var}(V(\infty)) = 1.34\cdot 10^{-4}.$

Updated data for 1928-2024 volatility, nominal and real returns, and index prices, can be found on my web site

The code and data for the current post can be found on https://github.com/asarantsev/Annual-Volatility

February 19, 2025
Annual dividend growth terms does not become white noise after dividing by annual volatility

This is the work by my undergraduate student Ian Anderson, continued from the previous post. He showed that annual earnings growth (nominal or real) are not Gaussian white noise. But after dividing these growth terms by annual volatility (computed by my other undergraduate student Angel Piotrowski) they indeed become Gaussian white noise.

Ian continued his work for dividend growth instead of earnings growth. The data is taken from Robert Shiller’s data library, as for earnings. But the results are negative in this case. The autocorrelation function plots for nominal dividend growth $G(t)$ is shown on the left. After dividing by annual volatility $V(t)$ the autocorrelation plot for $Q(t)/V(t)$ is shown on the right. It is clear there is significant autocorrelation with lag 1. Both plots seem to be for white noise, no autocorrelation.

I think that this is because dividends are persistent: Companies are reluctant to cut dividends even in poor times. This is why there are significant autocorrelations.

February 19, 2025

Make S&P Returns IID Gaussian

My undergraduate student Angel Piotrowski continued her work, started with annual volatility 1928-2023. First, she updated the annual realized volatility for 2024. The resulting series 1928-2024 is still well modeled by log Heston model, see another post. The research in this post is done in GitHub/asarantsev repository.

Then she computed annual returns of S&P 500 (and its predecessor, S&P 90) 1928-2024 in four versions:

nominal (not adjusted for inflation) or real (adjusted for inflation);
price (due only to price changes) or total (including dividends paid).

We take nominal annual dividend: $D(t)$ and December Consumer Price Index $C(t)$ We take the price $S(t)$ at the close of the last trading day of the year $t$ . Price returns are computed as $\ln\frac{S(t)}{S(t-1)}$ and total returns are $\ln\frac{S(t) + D(t)}{S(t-1)}$ for nominal versions. But for real versions, we need to subtract $\ln\frac{C(t)}{C(t-1)}$ from each of these. You see that all returns are logarithmic (geometric), so there is no problem of compound interest. If wealth at end of year $t$ is $\mathcal W(t)$ then $\mathcal W(t) = \exp(Q(1) + \ldots + Q(t))\mathcal W(0)$ where $Q(t)$ is returns during year $t$ .

In each of these four cases, returns are IID but not normal. However, dividing them by volatility keeps them IID but makes them normal. Just to illustrate, let us take real price returns $Q(t)$

The autocorrelation function for $Q(t)$ (left panel) and for $|Q(t)|$ (right panel) show these are close to zero. So it is reasonable to model these as independent identically distributed random variables. However, the below quantile-quantile plot versus the normal distribution shows these are not normal.

Next, repeat this analysis for normalized

And see that $Q(t)/V(t)$ are also well modeled as independent identically distributed. But unlike the previous example, the quantile-quantile plot shows that $Q(t)/V(t)$ is Gaussian:

This is confirmed by results of two statistical tests for normality: Shapiro-Wilk and Jarque-Bera. See their $p$ -values below. One can clearly see we reject normality hypothesis for original but not normalized returns, for all four versions of returns, and for each of two tests.

Returns	Original Total Real	Normd Total Real	Original Total Nominal	Normd Total Nominal	Original Price Real	Normd Price Real	Original Price Nominal	Normd Price Nominal
Shapiro-Wilk p	0.00164	20%	0.00027	11%	0.00087	28%	0.00009	11%
Jarque-Bera p	0.00627	63%	0.00005	60%	0.00217	86%	0.00000	60%

February 19, 2025

Dividing annual earning growth by volatility makes them Gaussian white noise

This work was done by my undergraduate student Ian Anderson, using the volatility data computed by my other undergraduate student Angel Piotrowski, see the previous post.

Ian took 1927-2023 net earnings of Standard & Poor 500 (since 1957; or its predecessor Standard & Poor 90) $E(t)$ and compute annual growth $G(t) = \ln(E(t)/E(t-1)).$ We do this first for nominal earnings, without adjustment for inflation. We analyze $G$ whether it is Gaussian independent identically distributed. We make the quantile-quantile (QQ) plot versus the normal distribution.

And we plot the autocorrelation function for $G$ and another plot for the autocorrelation function for $|G|.$

We see from the QQ plot that, unfortunately, earnings growth terms are not Gaussian. The autocorrelation function for earnings growth corresponds to white noise: It shows that $G(t)$ and $G(t-k)$ are uncorrelated. But with absolute values, this is not true. There is a significant autocorrelation of lag 1: $\mathrm{corr}(|G(t)|, |G(t-1)|).$

Then divide the earnings growth by annual volatility and get $G(t)/V(t).$ Does this division improve these terms to make them closer to Gaussian independent identically distributed? In fact, yes!

We see that now both autocorrelation plots show lack of significant autocorrelations. And the quantile-quantile plot is much closer to linear. Thus it makes sense to model $G(t)/V(t)$ as independent identically distributed Gaussian.

The same happens if we consider real earnings (inflation-adjusted) instead of nominal earnings, using December data for the Consumer Price Index. Thus we have joint model for earnings and volatility, annual 1927-2023:
$\ln V(t) = \alpha + \beta \ln V(t-1) + W(t)$ $\ln\frac{E(t)}{E(t-1)} = V(t)(Z(t) + g)$
$(W(t), Z(t)) \sim \mathcal N_2([0, 0], \Sigma)$ are independent identically distributed bivariate normal, with mean zero and covariance 2×2 matrix $\Sigma$ This works for both nominal and real annual earnings, but not for dividends.

January 24, 2025

autoregression, volatility
Does dividing by volatility improve fit for bond spread?

This work is of my undergraduate student Ian Anderson using the annual January data for American bond rate spreads. It uses annual volatility data complied by my other undergraduate student Angel Piotrowski. See the previous post.

Consider the spread between 10-year Treasury rate and 1-year Treasury rate. Usually, long-term rates are higher than short-term rates, since investors want extra compensation for committing their money for a long time. Another way to express this is that long-term bonds are more exposed to interest rate risk: If bond rates rise then bond prices fall. The coefficient is called the duration, and it is higher for long-term bonds.

But sometimes, long-term rates are lower than short-term rates. This is usually not a good sign, and a harbinger of a recession. Expecting a recession soon, investors anticipate short-term interest rate cuts by the Federal Reserve. This influences current long-term rates, which incorporate current and expected future short-term rates.

Denote this spread by $S(t).$ Apply an autoregression of order 1: $S(t) = a + bS(t-1) + Z(t).$ We expect mean reversion for this spread, which is true for $a \in (0, 1).$ This is different from $a = 1$ where this process is a random walk, when future movements are independent of the past.

Let us now analyze the innovations, otherwise called regression residuals: $Z(t).$ Apply the quantile-quantile plot versus the normal distribution. Next, divide these innovations by annual volatility and make the quantile-quantile plot again.

We do not see much difference… Does not seem to be normal. But let us apply the autocorrelation function to these innovations $Z(t).$ Next, divide by the volatility and plot the autocorrelation function again, now for $Z(t)/V(t).$ Both plots seem to be for white noise, no autocorrelation.

Finally, apply the autocorrelation function to the absolute value $|Z(t)|$ and see whether it is truly independent? It is not. Next, divide by volatility and apply the autocorrelation function to $|Z(t)/V(t)|$ to see whether it improves the result. It does not, in fact!

But now let us fit the same for the bond spread between AAA Moody’s rate and 10-year Treasury rate. The AAA rating is the highest reserved for corporate and municipal bonds, which have the lowest default risk.

QQ plot of innovations before normalization, close to normal but not quite. QQ plot of innovations after normalizations: Much closer to normal!

Autocorrelation function for innovations before and after normalizing: Good!

Autocorrelation function for absolute values of innovations before and after normalizing: Not so good, but acceptable. We need some further white noise testing.

The AAA-1YTR spread results are the same as for 10YTR-1YTR. Here TR stands for Treasury Rate.

Thus, the answer to the question in the title: Using volatility improves autoregression of order 1 for credit risk spread AAA-10YTR but not for long-short term spread 10YTR-1YTR and not for combined spread AAA-1YTR.

January 23, 2025

autoregression, bond-spread, volatility
Annual Volatility for Standard & Poor 500

My undergraduate student Angel Piotrowski computed annual volatility for Standard & Poor 500 (and its predecessor, Standard & Poor 90). For each year 1928 — 2023, she took daily index values $S(t)$ with day $t$ in this year, and computed log returns $\ln(S(t)/S(t-1)).$ Then she computed standard deviation of these log returns for day $t$ in any given year. Let $V(s)$ be this standard deviation, usually called volatility, for year $s.$ The data is available on my web page.

Next, Angel created a time series model for this volatility: Autoregression of order 1 on the log scale. The motivation comes from plotting the autocorrelation function for $X(t) = \ln V(t).$ It is defined as $k \mapsto \rho(X(t), X(t-k)).$ This looks like an autocorrelation function for an autoregression of order 1.

Here is the equation for this autoregression: $\ln V(s) = \alpha + \beta \ln V(s-1) + W(s)$ with $\alpha = 0.62$ and $\beta = -1.776.$ Let us test whether the innovations $W(s)$ are Gaussian. Apply the quantile-quantile plot versus the normal distribution. This looks like pretty close to a Gaussian law!

Next, plot the autocorrelation function for innovations $W(s)$ and another plot of an autocorrelation function for their absolute values $|W(s)|$ to see that they correspond to white noise.

Thus the model fits well: $\ln V(s) = \alpha + \beta \ln V(s-1) + W(s).$ We have numerical estimates $\beta = 0.620, \alpha = -1.775.$ Therefore, this autoregression has a stationary distribution, or an invariant probability measure, $\Pi$ such that $\ln V(t) \sim \Pi \Rightarrow \ln V(t+1) \sim \Pi.$ Angel has computed that mean and variance of this stationary distribution.

See updates here.

January 22, 2025
S&P 500 vs 400 vs 600

There are three S&P indices: Large-Cap 500, Mid-Cap 400, Small-Cap 600. These have exchange-traded funds (ETFs) by iShares. We use their monthly data to study equity premia: Total returns (including dividends) minus risk-free returns (measured by 3-month Treasury bill). As suggested previously, we divide these by monthly average volatility (measured by VIX) and get $L(t), M(t), S(t)$ . Regress the last two upon the large to do Capital Asset Pricing Model: $M(t) = \alpha_M + \beta_ML(t) + \delta_M(t)$ and $S(t) = \alpha_S + \beta_SL(t) + \delta_S(t)$ . We get: $\alpha_M,\, \alpha_S$ are not significantly different from zero, so let us ignore them. But $\beta_M = 1.05, \beta_S = 1.09$ .

Thus investing in mid-cap and especially small-cap stocks creates a slightly larger market exposure than the benchmark S&P 500 but does not have extra return on top of the one predicted by this extra risk assumed.

The residuals are independent and identically distributed, judging by the autocorrelation function plot, and Gaussian for mid-cap but not for small-cap stocks. For the latter, they seem to be well-described by the Laplace distribution.

February 4, 2024
Zero Dividend Stocks Are Bad

Let us do alpha-beta analysis (simple linear regression) of zero-dividend stocks upon dividend-paying stocks.

Consider monthly equity premia (total returns, including dividends, minus risk-free returns measured by 3-month Treasury rate). We subtract risk-free returns because this is the benchmark upon which we measure stock returns. Indeed, what good is it to get $6\%$ this year from risky stocks if you could get $8\%$ from risk-free Treasury bonds?

I used Kenneth French data library for stocks and Federal Reserve Economic Data for 3-month Treasury rates. All stock portfolios are equal-weighted.

Let $P(t)$ be the equity premium for zero-dividend stocks and let $Q(t)$ be the equity premium for dividend-paying stocks. Then $P(t) = -0.005 + 1.24Q(t) + e(t)$ , where $e(t)$ are residuals. This implies that excess return $\alpha = -0.005$ for month and so $-0.06 = -6\%$ for a year. And market exposure $\beta = 1.24$ (for a month or a year). The $R^2 = 67.5\%$ . This makes the following financial sense:

When you invest in zero-dividend stocks, you expose yourself to more risk that dividend-paying stocks, since $1.24 > 1$ , but you lose $6\%$ per year. In fact, annualized equity premium for dividend stocks is $8.8\%$ and for zero-dividend stocks is $5\%$ . The annualized standard deviation for dividend stocks is $16\%$ and for zero-dividend stocks is $24\%$ . More risk, less reward!

Many of these zero-dividend stocks are startups. But historically, investing in startups is a recipe for trouble. Although individual IPOs outperform the market (Microsoft, Walmart, Starbucks, Amazon), on average they lag far behind. This is another argument for good old value investing.

February 4, 2024
Stay Away From Negative Earnings

is it better to invest in stocks with zero or negative recent net income? The answer is no. Consider the monthly data from 1954 ( $N = 838$ months). Compute total returns of negative earnings stocks and subtract risk-free returns (measured by 3-month Treasury rates), this is the equity premium $P(t)$ . Then do the same for the benchmark: top large $30\%$ stocks, get $P_0(t)$ .

Simple linear regression with Capital Asset Pricing Model gives us $P(t) = -0.0022 + 1.36P_0(t) + W(t)$ , where $W(t)$ are residuals. Statistical tests show that W are close to independent identically distributed, but not quite Gaussian. The $R^2 = 63.7\%$ . Here, $\alpha = -0.22\%$ monthly (so $-0.22\%\cdot12 = -2.64\%$ annually), thus excess return is negative. But $\beta = 1.36 > 1$ , so market exposure is much greater than 1! More risk, less reward!

A corroboration of value investing. This portfolio is capitalization-weighted: Each stock is weighted according to its market weight. This means we invest in a slice of the total market. Most index funds are capitalization-weighted. Rate data is from Federal Reserve Economic Data (FRED) web site, and stock data is from Kenneth French’s Data Library at Dartmouth College.

February 4, 2024
Market Volatility

A financial econometrician often hopes against hope that returns exhibit the following properties: (A) normal, following the bell curve; (B) IID, meaning independent identically distributed. Unfortunately, this is almost never true.

Here, let me describe a nifty trick: Dividing each data point by the standard deviation normalizes the data. The standard deviation of stock market returns (in financial econometrics this is called volatility) is not constant over time. Periods of economic calm and steady growth when volatility is small alternate with economic crises and financial crashes when volatility is large. It is not clear that dividing by overall volatility makes sense.

Another way to say is that volatility is stochastic: It is a random process by itself. How to extract this stochastic volatility, and how to model it? Here autoregressive conditional heteroscedastic (ARCH/GARCH) models are very useful. They model dependence of current volatility upon the past volatility and returns, with fixed number of lags. These are widely used in Quantitative Finance, often combined with classic linear autoregressive and moving average (ARMA) models. Every time series of returns would have

By chance, we discovered another trick, which obviates the need for GARCH-type models and allows to simply reduce all it to a couple of linear regressions. The miracle is that we can observe volatility independently and separately from the main time series!

Enter VIX, the index created and maintained by the Chicago Board of Options Exchange. Options are contracts which give the holder the right (but not the obligation) to buy or sell a stock or a stock index in the future (at a maturity) at a predetermined price (the strike). The celebrated Black-Scholes formula computes the option price based on current stock price, the strike price, the maturity, and the volatility.

Lost of options are traded in Chicago based on S&P 500. Comparing the market price of each option at a given moment, we solve the Black-Scholes formula backward and find the unknown volatility. This is called the implied volatility. Taking a weighted average of all options, we get the overall implied volatility of the index, which is called VIX. It was computed daily for S&P 500 from the start of 1990, and for S&P 100 (a subset of S&P 500) from 1986.

Below is the ACF plot. Take monthly returns of Wilshire Total Stock Market Index, including dividends, from 1990. Below is the plot of their autocorrelation (correlation between month $T$ and month $T + K$ , for each $K = 1, 2, \ldots$ ) All correlations are very small and lie within a narrow blue band. This blue band is the fail-to-reject region, where you do not reject the null hypothesis that all returns are IID. It looks like this is, in fact, IID.

However, if we take autocorrelations for the absolute values, this is not IID anymore!

Finally, if we divide total returns by monthly average VIX, we get the autocorrelation functions for returns to be well-behaved, that is, corresponding to IID.

And for their absolute values as well.

We can have $X(t)$ without autocorrelation: $\mathrm{corr}(X(t), X(t+k)) = 0$ , but $|X(t)|$ with autocorrelation. In this case, plotting autocorrelation for $X$ will give us a false impression that $X$ is IID. This would be true if $X$ were Gaussian, because for these uncorrelated means independent. Unfortunately, many times data in finance is NOT Gaussian, they have heavy tails!

We discussed in the last post the model $X(t) = V(t)Z(t)$ , where $Z \sim \mathcal N(0, 1)$ IID and $V$ is auto regression or another mean-reverting process, independent of $Z$ . Then X have zero autocorrelation, but $|X|$ have positive autocorrelation. Indeed, for crisis times V(t) is large and then $V(t+1)$ is large, and thus $|X(t)|$ and $|X(t+1)|$ are both large.

When analyzing financial and economic data, one should always be mindful of this possibility. If we checked autocorrelation for both $X$ and $|X|$ , and both are zero, then it makes sense to assume X are IID.

We can also apply white noise tests to $X$ and to $|X|$ separately.

Consider correlations of $X(t), X(t+1)$ ; correlation of $X(t), X(t+2)$ ; etc. correlation between $X(t), X(t+k)$ . Are they all close to zero? We can test them separately, or combine them (sum of absolute values, sum of squares, or another function = statistics). This gives us Box-Pierce, Ljung-Box, and any other white noise test.

Doing the same for $|X|$ instead of $X$ , we can reject or fail to reject white noise for $|X|$ .

I created combined test (omnibus or portmanteau) to test $X$ and $|X|$ simultaneously. See the article IID Time Series Testing (2023) in Theory of Stochastic Processes, published by the Ukrainian National Academy of Sciences.

Moreover, it makes them normal, or much closer to normal than originally! This is true by statistical tests of normality (Shapiro-Wilk, Jarque-Bera, or others) and by the quantile-quantile plot (QQ) vs the normal law.

Finally, these properties work for any well-diversified factor market portfolio: Small, Value, Large Growth, and so on. VIX was created from S&P 500 options, but it works for indices, portfolios, and funds other than S&P 500.

Of course, the mean, variance, and other statistics of normalized returns will be different for two different portfolios. But normality and independence stay.

Portfolios are many, but volatility is universal!

February 4, 2024