Testing my DSR estimate

Last time, I put my "Estimated Debt Service Ratio (back to 1947)" data into an HTML table so it's accessible. Now I want to access it: copy it, paste it into a worksheet, and try to evaluate it. Are my results obviously ridiculous? Could be. I mean, suppose I subtract "household interest paid" from the debt service numbers. That leaves me with principal repayment. Maybe I can look at the principal repayment numbers in comparison to household income, or in comparison to household debt, and see if the numbers look reasonable or not.

If they look reasonable, that doesn't mean they're right. But if they don't look reasonable, it could mean they are wrong. So I want to do that kind of looking.

//

Graph #1

The red line in Graph #1 is the FRED data in the form FRED offers it: as a percent of Disposable Personal Income. The blue line is my approximation of the FRED data, based only on the ratio of household debt to DPI, and the effective interest rate on that debt, in an Excel regression. The main advantage of the blue data is that it goes back to 1947. The main disadvantage is we don't know if it is even close to right.

Graph #2

Graph #2: Same data as the first graph, but shown as billions of dollars. Here it appears that the differences between red and blue are small at first, for 15 or 20 years after 1980 (where the red line starts), and bigger in later years. So it appears. But by the end of the graph, Debt Service has climbed to 1600 billion; by the year 2000 it is only half as high. So a comparable difference would look only half as big before the year 2000.

And again, by the mid '80s is only around 400 billion, so again the same difference would look only half as big in 1985 as in 2000. And by 1980, only 200 billion, so the difference would look half as big in 1980 as in 1985 on this graph.

To get a better view of the differences between the red and blue, I looked at the difference between red and blue as a percent of the red data:

Graph #3

I was hoping this graph would look familiar. If I recognized something in it, like the growth rate of debt, I would want to add that data to the regression to see if it would make the discrepancies smaller.

No such luck. But what I do see in this graph is that before the year 2000, the discrepancies get bigger and bigger as you go back in time. None of the differences shown on the graph are more than six or eight percent, and I'd say that's not so much as to be objectionable. But if there's a discrepancy in every decade, and as you go back in time each earlier one is two or three percentage points bigger than the one before you're up around a 20% error in the 1940s. That's a lot.

The error seems to get worse as you go back in time. That makes sense to me. And even if a 6% difference is not a problem, it does seem to be a problem that the error grows with every decade.

But I'm looking at the big recent discrepancy on the graph, too, in the years after 2000. Maybe that discrepancy tilted the playing field, and the trend of discrepancies in the years before 2000 is not really a trend so much as the tilt introduced by the large late discrepancy. I'm thinking I might run the regression again some time, but stop the Y-values, FRED's Debt Service data, at the year 2000. Maybe the playing field will tilt back to normal and the discrepancies will all go away.

One can always hope.

Graph #4

Graph #4: I separate household debt service (in billions) into its two components: principal and interest. For the interest portion I use "Monetary Interest Paid: Households" from FRED. Nobody ever told me that this measure of interest is the interest that's included in household debt service. But what else could it be?

Famous last words, right?

Graph #5

Graph #5: Having split debt service into its principal and interest portions, here I convert them to "percent of DPI", same as FRED's Debt Service data.

You know what? Here again my red line looks almost like the smoothed trend for the green (FRED) data, at least in the 1980s and '90s. Except my data goes up when FRED's goes down, and down when theirs goes up. Hm.

Interesting in the early years where the principal (red) stays in the neighborhood of 4% while the interest (blue) drops to near 1% by the late 1940s. Makes sense, because interest rates were lower in the early years, and accumulated debt was smaller.

Graph #6

Graph #6: The components of Debt Service shown as "percent of Household Debt".

The blue line (interest) shows the same rise-to-early-80s-peak (and decline thereafter) that generally appear in US interest rates. My calculation of the "effective" interest rate ("monetary interest paid: households" as a percent of "household debt") seems to hold up okay.

Interesting that the blue line starts with much less of an increase here than on Graph #5, and that the red line starts with much more of a decrease here than on #5. Makes sense: When interest is the small component of Debt Service, principal must be the large component.

Nonetheless I am fascinated by the red line here, principal as a percent of debt. It starts extremely high, then falls very rapidly. But the fall tapers off by 1965 and the line runs flat for the first ten years of the Great Inflation. Red drops in the latter '70s -- an adjustment, I suppose -- then runs flat again to the end of the Great Inflation, at which time the original decline apparently resumes.

First impressions.

Estimating Household Debt Service back to 1947

The other day I showed a graph of the components of household debt service, the interest and the principal:

Graph #1: Components of Household Debt Service: Principal (blue) and Interest (red) as % of DPI

The data on interest paid goes back to the 1940s. (Same for disposable income and household debt.) But the debt service data only goes back to 1980, so that's as far back as the blue line can go. I sure would like to see the behavior of that blue line in the earlier years.

I wonder if you could use interest cost and debt-to-income in a regression, to find an equation that simulates the debt service data since 1980, and then use that equation to estimate debt service back to the 1940s. I know: It's not as good as actually having the data would be. But it might be better than not having the data.

I must have been out the day they taught regression in school. The only thing I know about it is, it's way more complex and sophisticated than I am. So if you want to grab this idea and run with it, that would be great. Meanwhile, I do know how to find the "Data Analysis" window in Excel, click on "Regression", and plug numbers in. Let's see where that gets us.

At FRED I picked out some relevant data and saved it as my Early Years Debt Service Dataset #1.

When I think on TDSP, household debt service, I think it has to be related to the rate of interest and the level of debt. For the interest rate I'll figure the "effective" rate: household interest paid, as a percent of household debt. For the level of debt I'll go with household debt relative to disposable personal income, the same context used for TDSP.

From Jim Frost at Statistics by Jim:

Use regression analysis to describe the relationships between a set of independent variables and the dependent variable. Regression analysis produces a regression equation where the coefficients represent the relationship between each independent variable and the dependent variable. You can also use the equation to make predictions.

Sounds good. I'm thinkin I want to "predict" values for debt service:

• first, to see if my "independent variables" give me good estimates since 1980;
• then, to create estimates for the years before 1980.

Debt-to-income and the interest rate are my "independent" variables. They go in Excel's "X value" range as inputs for the regression. The FRED Debt Service data (which begins with 1980) goes in the "Y value" range. My "prediction" -- my calculated values based on the regression results -- should be a pretty good match to the Debt Service data. That's my plan so far.

//

Okay, I got results from Excel. The regression gave me coefficients I can multiply by the input values, giving me numbers I can compare to the original TDSP data. And since both the interest rate values and the debt-to-income values are available back to the 1940s, my estimate of debt service can be calculated back that far as well:

Graph #2: Regressing Household Debt and Interest to Approximate Debt Service

Not a bad match between red and blue. The biggest gap between red and blue is about a one percentage point difference near the value 12, so my numbers are off by less than one in 12, less than 10%. Excel's "Summary Output" gives me an R Squared of 0.844, if that's important for a regression. And my P-Values are suspiciously low (but I don't know what that means).

Let me be the first to say it: Past results are no guarantee of more remote past performance. But let me also say I'll use this estimate when I have a use for it, as a way to test whether it seems to make sense.

I did find an old article, "Recent Financial Behavior of Households" by Charles Luckett, from the Federal Reserve Bulletin of June 1980 (vol.66). Here's Luckett's Figure 6:

Graph #3: Luckett's 6th

His numbers are far higher than FRED's. The highest point in the FRED data is 13.22% of DPI in 2007. The low edge of Luckett's plot window is the 14% level!

I copied some of his gray background down (to duplicate his line spacing), down to the 6% level. Here (blue) is my result for 1973-1981, overlaid on Luckett' graph:

Graph #4: Comparison 1973-1980

The blue line going back to 1973  is from my calculation based on the regression. The short green line, down around the 10% level, shows FRED's Debt Service data for 1980 and '81. The black lines show Luckett's numbers. Mine are low. But so are FRED's.

Eh. I added 12 to all my numbers to bring them up near Luckett's. Adding 12 to the blue line gives me the double red line, which looks very much like a smoothed path for Luckett's numbers. Mine go a little low where his go low in 1974 and '75. Mine go a little high where his go high in 1978 and '79. The general trend of mine looks about right, assuming Luckett's numbers are right. That's at least a little bit interesting.

But why are my numbers so much lower than his?

//

Another old article: "Household debt burden: how heavy is it?" by Carl J Palash, from 1979. Here is Chart 1:

Graph #5: Carl Palash's First

I'll take the lower chart there, enlarge it, and overlay my numbers on it as before. But I can see already that these numbers also are far higher than mine. The overlay:

Graph #6: Comparison 1960-1979

Again, the blue line is my calculation. The red this time is twice the blue. Multiplication gives my red line more variation than the blue has, relative to Palash's, making the path of my data somewhat more similar in shape to the path of his black line: We both show flatness for a decade beginning in the mid-1960s; we both show increase before the flatness and after; and after 1975 our increases run parallel. But all of this is based on my calculated numbers being doubled.

Again I have to wonder why my numbers are so low. But so far, the only answer I have is that the numbers I was trying to duplicate are low, FRED's household debt service numbers.

Doesn't really answer the question.

//

Based on the two comparisons, I can say that my numbers show less variation than Luckett's or Palash's, and that mine and FRED's are a lot lower. I don't know why the old numbers are so high and the recent ones are so low. But in another article, "Recent Changes to a Measure of U.S. Household Debt Service" by Dynan, Johnson, and Pence, from 2003, I found this graph, showing the revised data lower than the older vintage:

Graph #7

The "Financial Obligations Ratio" they show here does go up above 18% of DPI, in the neighborhood of the older data. But the Financial Obligations Ratio includes other things in addition to Debt Service. And anyway in the FRED data today, the corresponding peak comes in at less than 18%. The numbers are still going down.

Factors presented in the article which would tend to lower the Debt Service Ratio include

• the "time to maturity" of loans, because "longer-maturity loans have lower payments"
• the failure to make payments: "According to the SCF, at any given time, payments are not being made on one-quarter to one-half of student loans. To account for the deferral of student loans, we adjusted the stock of loans to reflect only those loans on which payments are currently being made."
• mis-estimating interest rates: "we replaced the previously used proxies with [the average interest rate offered by banks on 48-month new car loans], which is 3 to 4 percentage points lower than the proxies we had been using."

One factor that would seem to raise the Debt service Ratio is

• broadening the measurement of debt: "Sallie Mae's student loans since 1977 were added to the Federal Reserve's G.19 consumer credit statistics beginning with the October 2003 release. Their inclusion did not materially change the growth rate of consumer credit, but it has raised the level an average of 2 1/2 percent since 1977."

"On net," they write, "changes to the source data led to a downward revision to the DSR of about 1½ percentage points from 1980 through 2002..." And "the lion's share of this revision [was due to] the lengthening of our assumptions about remaining maturity on these loans."

Even so, the Debt Service Ratio today is half or less than half what it was back when that data was still called the Debt Service Burden. That's a very large decrease, especially given the very large increase in outstanding debt which occurred during that same period.

Whatever. I got the result I got, and that's what I got. Do with it what you will.

by ArtS	EDSR47 dataset	Regression Result
Year	TDSP (% of DPI)	My Estimate
1947		5.696
1948		5.887
1949		6.118
1950		6.276
1951		6.337
1952		6.502
1953		6.739
1954		6.899
1955		7.071
1956		7.289
1957		7.429
1958		7.543
1959		7.688
1960		7.938
1961		8.023
1962		8.134
1963		8.333
1964		8.423
1965		8.519
1966		8.519
1967		8.449
1968		8.467
1969		8.582
1970		8.508
1971		8.521
1972		8.678
1973		8.796
1974		8.887
1975		8.750
1976		8.875
1977		9.152
1978		9.506
1979		9.921
1980	10.478	10.312
1981	10.348	10.559
1982	10.435	10.934
1983	10.383	11.021
1984	10.671	11.185
1985	11.463	11.483
1986	11.883	11.620
1987	11.936	11.522
1988	11.687	11.355
1989	11.709	11.510
1990	11.609	11.509
1991	11.362	11.400
1992	10.633	10.948
1993	10.387	10.668
1994	10.532	10.623
1995	11.095	10.963
1996	11.300	11.076
1997	11.321	11.179
1998	11.170	11.124
1999	11.455	11.255
2000	11.766	11.506
2001	12.373	11.612
2002	12.369	11.390
2003	12.246	11.436
2004	12.206	11.638
2005	12.583	12.424
2006	12.737	13.027
2007	13.033	13.495
2008	12.907	13.157
2009	12.331	12.663
2010	11.289	11.878
2011	10.622	11.076
2012	10.063	10.453
2013	10.113	10.362
2014	9.905	10.044
2015	9.927	9.896
2016	9.998	9.864
2017	9.934	9.827
2018	9.717	9.758

Debt Service = Interest + Principal

Red: the interest portion of Household Debt Service
Blue: the repayment-of-principal portion of Household Debt Service (since 1980)

Graph #1: Components of Household Debt Service: Principal (blue) and Interest (red) as % of DPI

Wow.

Recalculatin debt service

In the test-and-development version of Wednesday's post there remain some notes. This one,

ALSO
TOTAL DEBT WE BORROW IS GREATER THAN THE REPORTED NUMBERS BECAUSE WE ARE ALWAYS PAYING DEBT DOWN

and a handful of links to other debt service posts I've done.

In Debt service (3 July 2016), Graph #5 shows "Principal Repayment as a Percent of Household Debt" running near 4.5% all thru the 1990s, and averaging perhaps near 4.5% even after the 1990s. Independently, 4.5% was also my best guess number for principal repayment as a percent of household debt in mine of 22 January. So I thought I was onto something.

I was thinkin maybe I could use that graph for principal repayment numbers all the way back to the 1950s. Unfortunately, when I checked Graph #5, it turned out to be based on FRED's TDSP debt service, which means it only goes back to 1980. So the graph gains me nothing.

So then I was wonderin: How could principal repayment vary so much from a fixed percentage of the debt we owe?

It would vary with economic conditions, probably. If the economy was good, people could better afford to pay down debt. If the economy was bad, people could less afford to pay down debt. That could be part of it. Or the other way around: Maybe when the economy is good, people feel less urgency about paying down debt. But when things suddenly go bad, as in 2008, people feel more urgency, and make paying down debt a higher priority, so the repayment ratio goes up.

I don't know what to do with that.

Maybe it varies with "short term loans", loans that last less than a year. On average, maybe half of those are paid off before the end-of-year roundup when stats are tallied.
// But they do it Quarterly, Art.
Still, some of that short-term debt must slip through the cracks and miss being counted as debt.

If short term debt varies as a percent of debt, then the "slip through the cracks" number would vary as well.

I don't know how much sense this all makes, but it flashed thru my mind quicker than I could write it down, and I made a graph comparing "principal repayment as a percent of debt" to "short term loans as a percent of debt". I see some similarity:

Graph #1: Principal Repayment (blue) and Short Term Loans as Percent of Debt (red)

Some pattern similarity, yeah. Notice, though, that the red line references the right-hand scale, and the right-scale numbers are substantially bigger than the left-scale numbers. So, maybe substantially less than all of the short term debt has influence. As I imagined above.

I divided the red numbers by 6 and added 0.25 "to make the two series comparable", as Milton Friedman would say. I changed the frequency to annual, to get rid of the jiggies. And voila:

Graph #2: The same, but better

It's not a perfect match, red to blue, but it is interesting. I see the red runs higher than the blue coming out of the recessions of the early 1980s and thru the good period that followed, and again in the "goldilocks" of the 1990s. Some relation of red-above-blue to a good economy? Maybe. (BTW it looks like the red is about to go above the blue again just now.)

In the early years, the red line reaches 6.5%, over one third more than the 4.5% figure I noted above. At that rate, using the short term loan number instead of the constant value 4.5% should increase my estimate of principal repayment by as much as 30% or more. So I want to use this data, the red line here, as a guess at principle repayment as a percent of debt owed.

Yeah, you know what? The red line is higher on the left (when debt was low) and lower on the right (when debt is high). Makes sense, we can pay off a greater percentage of our debt when we don't have so much of it. I'm goin with this.


This is what I did on the 22nd:

"I can set up a calculation where I take interest paid, add some percentage of existing debt, and divide the total by disposable income."

This is what I'm doing today:

"I can tweak the percentage number..."

And again:

"The percentage number I get will be a rough estimate of how much debt we repay each year."

and it'll be based on "Short Term Loans as Percent of Debt".

I'll take my graph from the 22nd, duplicate the red line in green, then eliminate the 0.045 (4.5%) as percent of debt repaid and use the other thing.

Here's what I get:

Graph #3: Debt Service using a Guestimated Variable Measure of Principal Repayment (green)

Damn! It sure doesn't look like a 30% increase, going from red to green. Almost not worth the trouble of figuring.

I had the same trouble yesterday. Oh, first I had trouble with the short-term-loan calculation. (I was trying to do it without coffee.) Then when I finally got it right and plugged it in to the repayment-of-principal calculation, I got the same tiny increase you see in that last graph above.

I fiddled with it yesterday and couldn't get it to come out as I expected, so I set it aside. That delayed this post till today, and I put up the "tiresome warnings" post instead. This morning I created the graph again, along with my 3AM coffee.

I got the same result today, so maybe it's not wrong. But it sure doesn't look like a 30% increase. I have to check some numbers. I want to start with the red high points on Graph #2: 1950 (6.55837) and 1973 (6.49469), both well above the 4.5% of my prior calculation.

Table #1: Checking the High Points on the Red Line of Graph #2:

Year	1950	1973
Household Debt	74.845	613.133
Est. % of Debt Repaid	6.558	6.495
Est Debt Repaid (billions)	4.909	39.821
Household Interest Paid	4.155	48.779
Principal & Interest (billions)	9.064	88.600
DPI (billions)	214.823	1008.383
Principal & Interest as Percent of DPI	4.2	8.8

Checking next the percentages that are not 30: Reading off Graph #3 at FRED, for 1950 the red line's value is 3.50196. The green's value is 4.21911, or 20.5% more than the red. For 1973 the red is 7.57358. The green, at 8.78644, is 16.0% higher than the red value for 1973.

These values are figured for the dates with the highest estimates of "principal repayment as a percent of household debt" on Graph #2. As the values fall, the principle repayment percentage also falls from 20.5% (or 16.0%) to lower values. On average, then, on Graph #3 the green values are nowhere near 30% higher than the red values.

And Graph #3 is right. But why is my 30% guess so far off?

1. We add principal repayment and interest payment together to get the debt service payment. If we reduce the principal repayment but do not change the interest payment, the debt service payment changes by a smaller percentage than the principal repayment.
2. Where the values are expressed as "percent of DPI" rather than percent of household debt, those values are affected by the changes in the Debt-to-DPI ratio. When the ratio is less than 1.00, as it was for all years before 2002, the payment is a smaller percentage of DPI than of debt.

These factors, and perhaps others, push the calculated percentages down below my 30% estimate.

Looking at this picture a different way, my "short-term-debt"-based estimate of the principle repayment percentage (red line, Graph #2) is high early and low late. I think this is realistic. When the debt accumulation is small relative to income, it is fairly easy to repay a higher percentage of that debt. But when the debt accumulation is large, repaying even a small percentage of that debt consumes a large portion of income.

I wouldn't be surprised if the actual principle repayment percentage value was substantially higher in the early years than what my graphs show.

Meanwhile, having these numbers to look at is better than nothing.

Calculatin' debt service

Yesterday I thought of a way to calculate the debt service ratio with my only guess being the percentage of our debt that we pay off each year, on average. Granted, it's a guess. But I don't have to guess all the numbers; just the one.

I came up with this graph:

Graph #1: Household Debt Service since 1980 (blue) and my calculation (red)

While going thru the motions, making the graph and writing the thing up, I remembered doing something similar before. In Reverse Engineering the Household Debt Service Ratio (14 August 2016), using the change in household debt along with net private saving, both of them smoothed, weighted, and "relative to DPI", I generated a graph that to my mind was pretty darn good match to the Household Debt Service ratio.

Using that same data from 2016, this graph shows the debt service ratio (which goes back only to 1980) and my calculation (which goes back to 1952). As in the 2016 post, my calculated data is shown with a two-year lag:

Graph #2

The graph shows household debt service beginning in 1980, and my calculation beginning in the 1950s.


As a glance at the two graphs will tell you, the calculations are far apart in the early years. I put both calcs together on a graph, along with FRED's debt service data:

Graph #3: Combining the Data

Egads! It's worse than I thought.

Funny how red and green can run so close to blue for most of the years after 1980, but so far from each other in the years before.

Oh, well. At least I got a blog post out of it.

Why didn't I think of this before?

Debt service includes the payment of interest and the payment of some portion of existing debt.

FRED has household debt service (as a percent of disposable personal income). They have household debt, and household interest paid, and disposable personal income.

I can set up a calculation where I take interest paid, add some percentage of existing debt, and divide the total by disposable income. Then I can tweak the percentage number to get my calculated number close to FRED's debt service number on a graph. The percentage number I get will be a rough estimate of how much debt we repay each year.

This calculation assumes that we pay the same percentage of our income our debt every year. Is that a good assumption? Well, the graph will show us. If the lines run near parallel from start to finish, it is a good assumption. We could then say that, on average, we repay so-and-so percent of our debt every year. But if the two lines veer apart, it's not a good assumption.

My method will be to try different percentage numbers till I find one that brings the lines close together. I know there is a better way, regression or "least squares" or something, but I don't know the one and don't remember the other. So I'll just guess and see how it turns out.

Graph #1: Household Debt Service since 1980 (blue) and my calculation (red)

Okay, if I narrow the graph at FRED, part of the upper-border text goes missing. That's unacceptable. So I left the thing the default width, made it taller, and saved the image, then shrunk it down more than I normally do to fit the blog space. As a result, the text on this graph is smaller that usual and more difficult to read. Thanks a lot, FRED. (To see the graph bigger, of course, you can just click it.)

The blue runs a little above the red at the start and end, and a couple spots between; the two run tight together in a couple places; and blue runs a little below red for the rest. I'm saying they run nearly parallel and pretty close together.

The graph says that on average, we repay about 4.5% of our debt each year.

Couple thoughts:

• I can look at the discrepancy, the difference between red and blue, and try to find how the differences arise. Recession-related? Recovery-related? ExtremeDebtBurden-related?
• I can work backwards and calculate the percentage repayment numbers instead of guessing them. Might be interesting to see that.

Something else: If I'm satisfied that my red-line calculation is good, rough but good, then I should be able to say that the red line is an estimate of household debt service all the way back to the late 1940s. That would be useful.