My Finance

New Bubble Measure Work Replicated

This is the work with my undergraduate student Angel Piotrowski. She used her annual volatility data 1928-2023 to replicate my previous work published on arXiv and discussed in my previous blog entry about the new valuation measure for the Standard & Poor 500 and its predecessors. She used the end-of-year S&P 500 close trading price instead of the January S&P 500 daily close averaged price used by me. She used December Consumer Price Index (CPI) data instead of me using January CPI. And she used data only 1928-2023 instead of my work 1871-2023. But this almost a century of data is still enough to make conclusions.

First, adjust earnings for inflation and consider the trailing average $E(t)$ of earnings for the last 5 years. This is similar to the classic Campbell-Shiller approach, when the cyclically adjusted price-earnings ratio (CAPE) features last 10 years of earnings. But we chose 5 years to make room for more data. Next, take inflation-adjusted wealth $S(t)$ at end of year $t.$ Consider the linear regression

$\ln S(t) - \ln E(t) - ct = a + b(\ln S(t-1) - \ln E(t-1) - c(t-1)) + \delta(t).$

This has the meaning of subtracting the trend from relative growth of wealth over earnings. Historical earnings averaged 1-2% per year and wealth growth is around 6-7% per year so the value for the trend $c$ must be 4-5%. This equation shows that after detrending, this is the classic autoregression of order 1. We can rewrite this in the more standard ordinary least squares regression form:

$Q(t) = a + bc + t(c - bc) + (b-1)(\ln S(t-1) - \ln E(t-1)) + \delta(t),$

where we define for short notation the quantity which we called implied dividend yield:

$Q(t) := \ln S(t) - \ln E(t) - \ln S(t-1) - \ln E(t-1)$

This gives us $a + bc = 0.1351$ and $c - bc = 0.0101$ and $b - 1 = -0.1307.$ All three coefficients are significantly different from zero: Student T-test gives $p$ values $0.1\%, 2.9\%, 1.9\%.$ But the Jarque-Bera test shows that innovations are not Gaussian: $p = 0.1\%.$ This is confirmed by the following quantile-quantile plot below. But the autocorrelation function plots for $\delta(t)$ and for $|\delta(t)|$ below show that these are independent identically distributed.

This stays in contrast with our original research, when residuals (innovations) are independent identically distributed and Gaussian. Further research will include extending this to years 2024 and 1924-1928 when we have total returns and volatility data for shifted years.

The GitHub repository New Valuation Measure Replication contains code, data, and these graphs.

February 20, 2025
My bubble measure work

My undergraduate student Angel Piotrowski replicated my previous unpublished work A New Stock Market Valuation Measure with Applications to Retirement Planning (available at arXiv:1905.04603) with annual volatility which she computed previously. Here I try to explain my research. In a further post, I will explain Angel’s contributions.

In this manuscript, I considered total Standard & Poor 500 returns (and their predecessors before 1957) taken from Robert Shiller’s data library (published on my web site) and compared them with annual earnings growth. Often, price-earnings or price-dividend ratio is used to analyze the stock market: If such ratios are high the market is overvalued, like at the top of the dotcom bubble. The average price-earnings historical ratio is around 15-20. So if this ratio is much more than 15-20, the market is overvalued. This is classic research by John Campbell and Robert Shiller, which won Nobel Prize in Economics. They used 10-year trailing averaged earnings to reduce noise and adjust for recessions and expansions (which take on average no more than 10 years), and their ratio is called cyclically adjusted

But comparing only prices with earnings might not be enough: In the XXI century, earnings recently are used for buybacks which raise prices, not dividend payouts. This artificially increases prices but does not mean overvaluation. Use of price-earnings or price-dividend ratios will incorrectly show the market is overvalued. The true comparison must be between stock returns (total, including dividends) and earnings growth. Historically, earnings growth $G(t)$ is ~2% per year, and total market returns $Q(t)$ are ~6-7%. Here we consider only real (inflation-adjusted) returns and growth terms.

Thus if we take the difference $\Delta(t) = Q(t) - G(t)$ and it is much higher than 4-5% per year for a few years, then the stock market starts to be overvalued. We formally can write this as an autoregression of order 1 for the cumulative sum $H(t) = \Delta(1) + \ldots + \Delta(t)$ after subtracting the trend $ct$ where $c \approx 4-5\%$ . Our bubble measure is $H(t) - ct$ and it is high when the market is overvalued. Let us write linear regression:

$H(t) - ct - h = b(H(t-1) - c(t-1) - h) + \delta(t)$

After fitting this as multiple linear regression, we get: $b = 0.86, c = 4.5\%, h = 0.36.$ We can see that the estimate for $c$ is within this 4-5% range. The Student test for $b = 1$ gives $p = 0.1\%$ so we can reject the random walk hypothesis: The model is stationary after detrending. Moreover, we can apply the Student test, since the residuals (innovations) $\delta(t)$ are well modeled by independent identically distributed Gaussian $\mathcal N(0, \sigma^2)$ with $\sigma = 0.18.$ This is shown by the autocorrelation function plot for $\delta(t)$ and the autocorrelation plot for $|\delta(t)|$ as well as the quantile-quantile plot for $\delta(t)$ versus the Gaussian distribution. See Figure 5 from the article.

Here we used 5-year averaged earnings, similarly to Campbell-Shiller’s 10-year averaged earnings. But we also do a separate analysis for simple annual earnings, simple annual dividends, and 5-year averaged dividends. The results fit the best for 5-year averaged earnings. The best means the residuals are closest to independent identically distributed normal.

Previously, Campbell and Shiller did not do analysis for residuals: whether they are truly distributed as independent identically distributed normal. Unfortunately, this is common in economics research. But I did this here.

According to the analysis, the classic cyclically adjusted price-earnings ratio in 2024 is very high compared to the historical average. Thus the classic analysis shows that the Standard \& Poor 500 is overvalued, almost like the dotcom bubble. But our new stock market valuation measure $H(t) - ct$ is not historically high. So our own measure does not show the market is overvalued. The current situation is not like the top of the dotcom bubble. See Figure 7 in the article.

February 20, 2025
Updated annual volatility for 2024

My undergraduate student Angel Piotrowski updated annual realized volatility for 2024. Previously she computed it for 1928-2023, each year. She took log change in daily closing prices of the Standard & Poor 500, or its predecessor, Standard & Poor 90, and computed empirical standard deviation. Given this annual volatility data $V(t)$ she analyzed this. First, she computed the autocorrelation function for $V(t).$

This strongly suggests using the autoregression model, which is called the Heston model in quantitative finance:

$V(t) = \alpha + \beta V(t-1) + W(t).$

Results after fitting this simple linear regression using ordinary least squares method are:

$\alpha = 0.003920,\, \beta = 0.626196$

Now let us analyze residuals (innovations) $W(t)$ which are supposed to be independent identically distributed mean-zero Gaussian: $W(t) \sim \mathcal N(0, \sigma^2).$ Angel did this by making the autocorrelation function plots for $W(t)$ and for $|W(t)|$ , as well as the quantile-quantile plot of $W(t)$ versus the normal distribution:

We see that the ACF plot for $W(t)$ corresponds to white noise but the ACF plot for $|W(t)|$ does not. A few first lags have significant autocorrelation. Less importantly but also unfortunately, the quantile-quantile plot shows the innovations $W(t)$ are not Gaussian.

Yet another problem with this Heston model: Volatility can go negative according to this model, but this is impossible in real life. As a standard deviation of market fluctuations, volatility is always supposed to stay positive.

Next, Angel modeled the resulting series $V(t)$ as an autoregression of order 1 on the logarithmic scale:

$\ln V(t) = \alpha + \beta \ln V(t-1) + W(t).$

For updated data, $\alpha = -1.776087$ and $\beta = 0.620147$ so there is mean-reversion. She did not test for unit root but I am very sure this hypothesis (that $\beta = 1$ ) would be rejected.

See the autocorrelation function plots for innovations $W(t)$ and for their absolute values $|W(t)|$ which show these $W(t)$ can be modeled as independent identically distributed. And the quantile-quantile plot versus the normal distribution shows these are Gaussian.

The resulting stationary distribution $\ln V(\infty)$ is Gaussian with mean -4.68 and variance 0.218. Using the moment generating function for the normal distribution, we can compute $\mathbb E[V(\infty)] = 0.0104$ and variance $\mathrm{Var}(V(\infty)) = 1.34\cdot 10^{-4}.$

Updated data for 1928-2024 volatility, nominal and real returns, and index prices, can be found on my web site

The code and data for the current post can be found on https://github.com/asarantsev/Annual-Volatility

February 19, 2025
Annual dividend growth terms does not become white noise after dividing by annual volatility

This is the work by my undergraduate student Ian Anderson, continued from the previous post. He showed that annual earnings growth (nominal or real) are not Gaussian white noise. But after dividing these growth terms by annual volatility (computed by my other undergraduate student Angel Piotrowski) they indeed become Gaussian white noise.

Ian continued his work for dividend growth instead of earnings growth. The data is taken from Robert Shiller’s data library, as for earnings. But the results are negative in this case. The autocorrelation function plots for nominal dividend growth $G(t)$ is shown on the left. After dividing by annual volatility $V(t)$ the autocorrelation plot for $Q(t)/V(t)$ is shown on the right. It is clear there is significant autocorrelation with lag 1. Both plots seem to be for white noise, no autocorrelation.

I think that this is because dividends are persistent: Companies are reluctant to cut dividends even in poor times. This is why there are significant autocorrelations.

February 19, 2025

Make S&P Returns IID Gaussian

My undergraduate student Angel Piotrowski continued her work, started with annual volatility 1928-2023. First, she updated the annual realized volatility for 2024. The resulting series 1928-2024 is still well modeled by log Heston model, see another post. The research in this post is done in GitHub/asarantsev repository.

Then she computed annual returns of S&P 500 (and its predecessor, S&P 90) 1928-2024 in four versions:

nominal (not adjusted for inflation) or real (adjusted for inflation);
price (due only to price changes) or total (including dividends paid).

We take nominal annual dividend: $D(t)$ and December Consumer Price Index $C(t)$ We take the price $S(t)$ at the close of the last trading day of the year $t$ . Price returns are computed as $\ln\frac{S(t)}{S(t-1)}$ and total returns are $\ln\frac{S(t) + D(t)}{S(t-1)}$ for nominal versions. But for real versions, we need to subtract $\ln\frac{C(t)}{C(t-1)}$ from each of these. You see that all returns are logarithmic (geometric), so there is no problem of compound interest. If wealth at end of year $t$ is $\mathcal W(t)$ then $\mathcal W(t) = \exp(Q(1) + \ldots + Q(t))\mathcal W(0)$ where $Q(t)$ is returns during year $t$ .

In each of these four cases, returns are IID but not normal. However, dividing them by volatility keeps them IID but makes them normal. Just to illustrate, let us take real price returns $Q(t)$

The autocorrelation function for $Q(t)$ (left panel) and for $|Q(t)|$ (right panel) show these are close to zero. So it is reasonable to model these as independent identically distributed random variables. However, the below quantile-quantile plot versus the normal distribution shows these are not normal.

Next, repeat this analysis for normalized

And see that $Q(t)/V(t)$ are also well modeled as independent identically distributed. But unlike the previous example, the quantile-quantile plot shows that $Q(t)/V(t)$ is Gaussian:

This is confirmed by results of two statistical tests for normality: Shapiro-Wilk and Jarque-Bera. See their $p$ -values below. One can clearly see we reject normality hypothesis for original but not normalized returns, for all four versions of returns, and for each of two tests.

Returns	Original Total Real	Normd Total Real	Original Total Nominal	Normd Total Nominal	Original Price Real	Normd Price Real	Original Price Nominal	Normd Price Nominal
Shapiro-Wilk p	0.00164	20%	0.00027	11%	0.00087	28%	0.00009	11%
Jarque-Bera p	0.00627	63%	0.00005	60%	0.00217	86%	0.00000	60%

February 19, 2025

Dividing annual earning growth by volatility makes them Gaussian white noise

This work was done by my undergraduate student Ian Anderson, using the volatility data computed by my other undergraduate student Angel Piotrowski, see the previous post.

Ian took 1927-2023 net earnings of Standard & Poor 500 (since 1957; or its predecessor Standard & Poor 90) $E(t)$ and compute annual growth $G(t) = \ln(E(t)/E(t-1)).$ We do this first for nominal earnings, without adjustment for inflation. We analyze $G$ whether it is Gaussian independent identically distributed. We make the quantile-quantile (QQ) plot versus the normal distribution.

And we plot the autocorrelation function for $G$ and another plot for the autocorrelation function for $|G|.$

We see from the QQ plot that, unfortunately, earnings growth terms are not Gaussian. The autocorrelation function for earnings growth corresponds to white noise: It shows that $G(t)$ and $G(t-k)$ are uncorrelated. But with absolute values, this is not true. There is a significant autocorrelation of lag 1: $\mathrm{corr}(|G(t)|, |G(t-1)|).$

Then divide the earnings growth by annual volatility and get $G(t)/V(t).$ Does this division improve these terms to make them closer to Gaussian independent identically distributed? In fact, yes!

We see that now both autocorrelation plots show lack of significant autocorrelations. And the quantile-quantile plot is much closer to linear. Thus it makes sense to model $G(t)/V(t)$ as independent identically distributed Gaussian.

The same happens if we consider real earnings (inflation-adjusted) instead of nominal earnings, using December data for the Consumer Price Index. Thus we have joint model for earnings and volatility, annual 1927-2023:
$\ln V(t) = \alpha + \beta \ln V(t-1) + W(t)$ $\ln\frac{E(t)}{E(t-1)} = V(t)(Z(t) + g)$
$(W(t), Z(t)) \sim \mathcal N_2([0, 0], \Sigma)$ are independent identically distributed bivariate normal, with mean zero and covariance 2×2 matrix $\Sigma$ This works for both nominal and real annual earnings, but not for dividends.

January 24, 2025

autoregression, volatility
Does dividing by volatility improve fit for bond spread?

This work is of my undergraduate student Ian Anderson using the annual January data for American bond rate spreads. It uses annual volatility data complied by my other undergraduate student Angel Piotrowski. See the previous post.

Consider the spread between 10-year Treasury rate and 1-year Treasury rate. Usually, long-term rates are higher than short-term rates, since investors want extra compensation for committing their money for a long time. Another way to express this is that long-term bonds are more exposed to interest rate risk: If bond rates rise then bond prices fall. The coefficient is called the duration, and it is higher for long-term bonds.

But sometimes, long-term rates are lower than short-term rates. This is usually not a good sign, and a harbinger of a recession. Expecting a recession soon, investors anticipate short-term interest rate cuts by the Federal Reserve. This influences current long-term rates, which incorporate current and expected future short-term rates.

Denote this spread by $S(t).$ Apply an autoregression of order 1: $S(t) = a + bS(t-1) + Z(t).$ We expect mean reversion for this spread, which is true for $a \in (0, 1).$ This is different from $a = 1$ where this process is a random walk, when future movements are independent of the past.

Let us now analyze the innovations, otherwise called regression residuals: $Z(t).$ Apply the quantile-quantile plot versus the normal distribution. Next, divide these innovations by annual volatility and make the quantile-quantile plot again.

We do not see much difference… Does not seem to be normal. But let us apply the autocorrelation function to these innovations $Z(t).$ Next, divide by the volatility and plot the autocorrelation function again, now for $Z(t)/V(t).$ Both plots seem to be for white noise, no autocorrelation.

Finally, apply the autocorrelation function to the absolute value $|Z(t)|$ and see whether it is truly independent? It is not. Next, divide by volatility and apply the autocorrelation function to $|Z(t)/V(t)|$ to see whether it improves the result. It does not, in fact!

But now let us fit the same for the bond spread between AAA Moody’s rate and 10-year Treasury rate. The AAA rating is the highest reserved for corporate and municipal bonds, which have the lowest default risk.

QQ plot of innovations before normalization, close to normal but not quite. QQ plot of innovations after normalizations: Much closer to normal!

Autocorrelation function for innovations before and after normalizing: Good!

Autocorrelation function for absolute values of innovations before and after normalizing: Not so good, but acceptable. We need some further white noise testing.

The AAA-1YTR spread results are the same as for 10YTR-1YTR. Here TR stands for Treasury Rate.

Thus, the answer to the question in the title: Using volatility improves autoregression of order 1 for credit risk spread AAA-10YTR but not for long-short term spread 10YTR-1YTR and not for combined spread AAA-1YTR.

January 23, 2025

autoregression, bond-spread, volatility
Annual Volatility for Standard & Poor 500

My undergraduate student Angel Piotrowski computed annual volatility for Standard & Poor 500 (and its predecessor, Standard & Poor 90). For each year 1928 — 2023, she took daily index values $S(t)$ with day $t$ in this year, and computed log returns $\ln(S(t)/S(t-1)).$ Then she computed standard deviation of these log returns for day $t$ in any given year. Let $V(s)$ be this standard deviation, usually called volatility, for year $s.$ The data is available on my web page.

Next, Angel created a time series model for this volatility: Autoregression of order 1 on the log scale. The motivation comes from plotting the autocorrelation function for $X(t) = \ln V(t).$ It is defined as $k \mapsto \rho(X(t), X(t-k)).$ This looks like an autocorrelation function for an autoregression of order 1.

Here is the equation for this autoregression: $\ln V(s) = \alpha + \beta \ln V(s-1) + W(s)$ with $\alpha = 0.62$ and $\beta = -1.776.$ Let us test whether the innovations $W(s)$ are Gaussian. Apply the quantile-quantile plot versus the normal distribution. This looks like pretty close to a Gaussian law!

Next, plot the autocorrelation function for innovations $W(s)$ and another plot of an autocorrelation function for their absolute values $|W(s)|$ to see that they correspond to white noise.

Thus the model fits well: $\ln V(s) = \alpha + \beta \ln V(s-1) + W(s).$ We have numerical estimates $\beta = 0.620, \alpha = -1.775.$ Therefore, this autoregression has a stationary distribution, or an invariant probability measure, $\Pi$ such that $\ln V(t) \sim \Pi \Rightarrow \ln V(t+1) \sim \Pi.$ Angel has computed that mean and variance of this stationary distribution.

See updates here.

January 22, 2025
S&P 500 vs 400 vs 600

There are three S&P indices: Large-Cap 500, Mid-Cap 400, Small-Cap 600. These have exchange-traded funds (ETFs) by iShares. We use their monthly data to study equity premia: Total returns (including dividends) minus risk-free returns (measured by 3-month Treasury bill). As suggested previously, we divide these by monthly average volatility (measured by VIX) and get $L(t), M(t), S(t)$ . Regress the last two upon the large to do Capital Asset Pricing Model: $M(t) = \alpha_M + \beta_ML(t) + \delta_M(t)$ and $S(t) = \alpha_S + \beta_SL(t) + \delta_S(t)$ . We get: $\alpha_M,\, \alpha_S$ are not significantly different from zero, so let us ignore them. But $\beta_M = 1.05, \beta_S = 1.09$ .

Thus investing in mid-cap and especially small-cap stocks creates a slightly larger market exposure than the benchmark S&P 500 but does not have extra return on top of the one predicted by this extra risk assumed.

The residuals are independent and identically distributed, judging by the autocorrelation function plot, and Gaussian for mid-cap but not for small-cap stocks. For the latter, they seem to be well-described by the Laplace distribution.

February 4, 2024
Zero Dividend Stocks Are Bad

Let us do alpha-beta analysis (simple linear regression) of zero-dividend stocks upon dividend-paying stocks.

Consider monthly equity premia (total returns, including dividends, minus risk-free returns measured by 3-month Treasury rate). We subtract risk-free returns because this is the benchmark upon which we measure stock returns. Indeed, what good is it to get $6\%$ this year from risky stocks if you could get $8\%$ from risk-free Treasury bonds?

I used Kenneth French data library for stocks and Federal Reserve Economic Data for 3-month Treasury rates. All stock portfolios are equal-weighted.

Let $P(t)$ be the equity premium for zero-dividend stocks and let $Q(t)$ be the equity premium for dividend-paying stocks. Then $P(t) = -0.005 + 1.24Q(t) + e(t)$ , where $e(t)$ are residuals. This implies that excess return $\alpha = -0.005$ for month and so $-0.06 = -6\%$ for a year. And market exposure $\beta = 1.24$ (for a month or a year). The $R^2 = 67.5\%$ . This makes the following financial sense:

When you invest in zero-dividend stocks, you expose yourself to more risk that dividend-paying stocks, since $1.24 > 1$ , but you lose $6\%$ per year. In fact, annualized equity premium for dividend stocks is $8.8\%$ and for zero-dividend stocks is $5\%$ . The annualized standard deviation for dividend stocks is $16\%$ and for zero-dividend stocks is $24\%$ . More risk, less reward!

Many of these zero-dividend stocks are startups. But historically, investing in startups is a recipe for trouble. Although individual IPOs outperform the market (Microsoft, Walmart, Starbucks, Amazon), on average they lag far behind. This is another argument for good old value investing.

February 4, 2024