A Sensitivity Analysis of the Robert Lichello Automatic Investment Management (AIM) System
If you take the time to look a little closer at the Automatic Investment Management (AIM) algorithm that Robert Lichello developed in the late 1970’s, some obvious questions pop up. For example, is it better to look at the portfolio value more frequently than monthly? What would happen if your initial equity investment was more (or less) than 50% of your total investment? Would rate of return increase or decrease if you selected a stock/fund/ETF that exhibits high (or low) price volatility?
This article will take a very methodical approach to answering those specific questions. Another article I wrote explains the AIM algorithm with 10+ years of back-test results, and another explains how to use the AIM system in a multi-ETF portfolio.
Sensitivity Analysis and Back-testing
For the back-test exercise, we studied the performance of the AIM algorithm using a single ETF (ticker SPY) over a specified time period in the past with the input variables set and not allowed to vary.
A sensitivity analysis utilizes the concept of back-testing to understand how the output results from the AIM algorithm will change when specific input variables are systematically changed. In other words, how “sensitive” is the output of the AIM algorithm when the input variables are allowed to change.
To perform the sensitivity analysis of the AIM algorithm we need to first select the input variables and what range they will be allowed to change. Next we need to select the output variables, then determine timeframe for back-testing. At this point we will be ready to run back-tests for each combination of input variable settings while collecting output results from each of the back-tests. At the end, we will summarize the results and make our conclusions.
Selecting AIM Input Variables
For this analysis we will select three input variables of the AIM algorithm: Frequency of assessment, % of initial equity investment, and different types of equity investments.
Frequency of Assessment
Mr. Lichello suggested looking at the stock price on a monthly frequency. We will keep this notion in our sensitivity analysis and also look at making decisions on a weekly basis. For the truly active trader we will also see how the algorithm reacts to making decisions on a daily basis.
% Initial Equity Investment
Mr. Lichello first suggested an even 50%-50% split between equity and cash. However, in later editions of his book he suggested ratios as high as 80%-20% equity to cash. We will keep both of these notions for our sensitivity analysis and also explore the space below 50%-50%. Our settings will start at 30% equity, and increase by 10% intervals until reaching 80% equity.
Type of Equity Investment
State Street Global Advisors sell ETFs that divide the S&P 500 into 9 sectors (Consumer Discretionary, Consumer Staples, Energy, Financial, Health Care, Industrial, Materials, Technology, and Utilities) they are called Select Sector SPDRs. In this analysis we will look for two sector ETFs in addition to the S & P Depository receipt ETF, ticker SPY. We will use an ETF that has higher price volatility than SPY and one with lower volatility then SPY. To measure volatility we will use a stock’s beta. Using Morningstar’s estimate of 3-year beta we find that the ETF with the most volatility (beta of 1.24) is the Energy stock, ticker XLE. The sector stock with the lowest beta of 0.18 is the Utility ETF, ticker XLU. So, we will use the SPY with a beta of 1.00, XLU with a beta of 0.18 and XLE with a beta of 1.24.
All of these input variables and settings are summarized in the table titled Input Variables and Settings.
% Initial Investment
Selecting Output Variables and Timeframe
For output variables we need the ability to accurately measure investment performance for each back-test. The measurement we will use is the annualized rate of return, also called the Internal Rate of Return. Fortunately, Microsoft Excel™ has a built in function (XIRR) that we will use to standardize the calculation. Additionally, we will capture the final portfolio value, any cash shortfall that might occur, and the total number of trades.
The time frame for the historical price data is from 12/22/1998 to 7/31/2013, slightly more than 14-1/2 years. Historical price and dividend data are from the Yahoo! finance website.
To summarize, let's lay out all of the back-test cases we will run for this analysis. There are 54 distinct combinations of variables and settings which we will change simultaneously. All fifty-four test cases are displayed in a graphical format, see the figure titled Test Cases.
Each test case represents a single back-test, for example, one test case is to set the AIM algorithm to 30% initial equity investment, set assessment frequency to daily, and use historical price data for the XLU - Utility ETF. Run the data through the AIM algorithm, calculate the internal rate of return, capture the final portfolio value, any cash shortfall, and total number of trades.
Assumptions for Testing AIM
It is always necessary to document the assumptions when doing an empirical analysis, here is the list for this analysis:
- Total Initial investment amount is $10,000.
- Initial purchase is the open price on 12/22/1998.
- AIM decisions are based on the closing price of the stock on the last trading day of the month for monthly assessment frequency, last trading day of the week for weekly assessment frequency or closing price for that day for daily assessment frequency.
- Buy or sell price is the open price of the stock on the next trading day following an AIM decision.
- Buy or sell orders are triggered only if AIM market order is +/- 5% of the current equity value of the portfolio.
- Cash shortfalls will be funded and the cash account will be set to zero until a sell order is executed.
- Stock trading commission is not taken into consideration, however we can estimate overall commission cost by using the total number of trades.
- Rate of return on Cash reserve is 0.5% APR.
- Dividends are reinvested in additional shares.
Which input variable do you think will have the greatest effect on Rate of Return?
The table titled Back-Test Results presents the outcomes of all 54 back-tests. We used regression analysis to determine which of the three input variable have the most significant effect on rate of return and the results are:
Type of ETF - Most Significant
% initial equity investment - Significant
Frequency of Assessment - insignificant
In fact, the two significant variables, type of ETF and % initial equity investment account for 94% of the variation that we see in the rate of return (for the statistically minded the adjusted r-square value is 0.937)
Note that a significant cash shortfall was observed when investing in SPY and XLU which occurred at every level of assessment frequencies and with initial equity investments as low as 50%. However, there was no cash shortfalls when investing in XLE regardless of assessment frequency or % initial equity investment.
To understand why there was no cash shortfall when investing in the XLE we need to deconstruct the bull market from mid 2002 to the peak of that bull run at the end of 2007. From 7/23/2002 to 12/26/2007 XLE price ranged from $19.80 to $80.55 a 306.8% increase. AIM would issue multiple sell signals during that ascent, building cash reserves for buying opportunities during the inevitable market decline that followed. The SPY and XLU experienced a similar bull run from late 2002 to late 2007, but the increase was not as dramatic. XLU grew 191.4% and SPY grew 100.4%. So, because XLE is a higher beta stock, it resulted in a higher rate of price increase, allowing AIM to capture more profits. This resulted in sufficient cash in the coffers to take advantage of multiple buy signals during the steep market decline from late 2008 to mid 2009.
We also see that the number of trades increase as the assessment frequency increases, and as ETF beta increases. Intuitively that makes sense as we would expect more trading opportunities if we are checking our portfolio value more frequently or if the price of the ETF swings up/down more violently.
Looking at the graph titled Effects of Investment Type we see that the energy ETF, ticker XLE, had the most significant effect on rate of return with an average of 11% and a range from 7.1% to 14.5%.
Now let's look at the graph titled Effects of Initial Equity Investment. We see that the average rate of return increases linearly from 5.3% with a 30% initial equity investment all the way up to 11% with an 80% initial equity investment. Note that the lowest rate of return that we observed was 3.8% and the highest was 14.5%.
Finally, looking at the graph titled Effects of Assessment Frequency, we see that the average rate of return does not change very much from daily to monthly assessments. In fact, there was only a slight difference of 0.6% average rate of return between daily and monthly assessments.
Since assessment frequency is measured in time we can look at it from a different point of view. We can calculate a payback, in dollars per hour, for time spent assessing the next buy/sell/hold decision. To do this, we need to estimate the average increase in final portfolio value for more frequent assessments and the total number of hours spent for assessments.
For instance, if we spend 5 minutes each time we update the AIM algorithm then over the 14.7 years of this study we would have spent 14.7 total hours for monthly assessments, 63.7 hours for weekly, and 318.5 hours for daily. Looking at the graph titled Effects of Assessment Frequency on Final Portfolio Value, we see that the average final portfolio value was $21,445 for monthly assessments, $23,772 for weekly, and $25,044 for daily.
Based on this information the payback for increasing assessment from monthly to weekly is calculated as follows:
(increase in final portfolio value)/(additional time for assessment) =
(23,772 - 21,445)/(63.7 - 14.7) = $2,370/49 = $47.49 per hour
So, we increased our average portfolio by $2,370 by taking 49 additional hours to update the AIM algorithm for a payback of $47.49 per hour, not a shabby salary.
The payback for increasing assessment from monthly to daily is $11.85 per hour and $4.99 per hour for increasing assessment from weekly to daily.
From our first AIM article we saw that you can improve on Buy/Hold investing by using AIM with the highly diversified ETF - SPY. From this article we see that more improvement can be gained by disassembling SPY and using AIM on individual business sectors. This is due to the individual industry ETFs having a different degree of volatility (measured by Beta) than the aggregated SPY. That difference allows AIM to capture more of the inherent volatility not available to SPY.
This is further verified by the regression analysis of our back-test data. We can conclude that the most important factor to consider if you are going to use AIM to control a portfolio of equity investments is the type of stock/mutual fund/ETF that you choose. To be more specific, it appears that the AIM algorithm is more efficient with higher beta/more volatile investments. A word of caution though, this analysis is limited to ETFs with beta's that range from 0.18 to 1.24, we did not explore those ultra volatile ETFs that are two and three-times more volatile than the standard ETFs. So, it is probably not safe to extrapolate our results to those type of investment vehicles.
There is a detailed article on stock selection in the archives of the A.I.M. users website. Although it is focused on selection of stock in individual companies, the concept should be easy to apply to ETF selection.
The next factor that shows a significant effect on rate of return is the % initial equity investment. Because the rate of return increases linearly as the % initial equity invested increases then we should use this factor as a risk/return lever. For example, if you are a conservative investor and willing to accept a lower rate of return for that safety then only invest 30-50% initially in the ETF. Conversely, if you are willing to take on the full force of risky investments then go for the gusto of a 60-80% initial equity investment.
Finally, the last factor, frequency of assessment appears to be insignificant relative to rate of return. However, when looking at the payoff for extra time spent assessing the AIM algorithm we see that our increase in portfolio value is the best when increasing assessment frequency from monthly to weekly (average of $47.49 per additional hour spent assessing the AIM algorithm).
Of course, you could treat assessment frequency as a convenience factor. If you have the time or predisposition to check your portfolio daily by all means have at it. If you don’t have that much time but have a short period on the weekends then do your AIMing weekly. If your days and weeks are filled with other activities then maybe monthly portfolio checks are for you. In any scenario, you would expect to see similar rates of return, however, be aware that your total trading commission costs will go up as the frequency of assessment increases.
AIM Based Software
- Automatic Investor: Mechanical, Automated Stock Investment Software for Long Term Investing
Automatic Investor: A Powerful, Automated, Mechanical Stock Investment Software Package Designed to Increase Your Returns, Minimize Your Risk and Save You Time.