"Trading is statistics and time series analysis." This blog details my progress in developing a systematic trading system for use on the futures and forex markets, with discussion of the various indicators and other inputs used in the creation of the system. Also discussed are some of the issues/problems encountered during this development process. Within the blog posts there are links to other web pages that are/have been useful to me.

## Thursday 16 December 2010

### Bayesian Analysis Update

## Wednesday 13 October 2010

### Bayesian Analysis

## Friday 8 October 2010

### Posting to another forum

## Tuesday 21 September 2010

### Recent coding work

## Sunday 19 September 2010

### Update on First Trades

## Monday 13 September 2010

### First Trades

## Monday 6 September 2010

### Highpass filter added

## Thursday 2 September 2010

### Addition of new Bandpass Indicator

## Monday 9 August 2010

### Completion of Speed Optimised Code

## Saturday 17 July 2010

### The Dedicated Dekalog Website

## Wednesday 14 July 2010

### Addition to title header bar

## Monday 5 July 2010

### Discussion of work so far

In this discussion I will be talking about the indicators etc. that have been developed so far and the reader is referred to all previous posts and especially the candlestick chart in the post dated 3rd July and the version thereof above. The topmost plot above is the same as the topmost in the previous post, whilst the lower plot is a plot of the default coloured bars only (see post dated 3rd July).

Looking at the extreme left of this candlestick chart we can see that a cybercycle zero line cross has just occurred (@ x-axis 3720) and at x-3724 the candles change colour to cyan, meaning that the turn indicator shows a possible turn in 2 days time. A few days later, candles turn yellow, meaning that prices are still in the "turn window" but no longer satisfy the conditions for being a sideways market according the formula for momentum/abs(ucl-lcl) of cybercycle. At this time, the momentum indicators are following the pattern that can be expected for a cyclic top to be in place. All in all, I would characterise this as a high probability short set up and subsequent price action confirms this to be a good call as prices decrease to the price lcl levels (@ x-3738).

At these new lower levels (@ x-3738) we now see a new set up to go long. Again prices are indicated to be in a "turn window" and the momentum indicators are following the pattern that can be expected for a cyclic low to be in place. Again this turns out to have been a good call as prices subsequently increase to the price ucl levels (@ x-3747).

As prices reach these new high levels (@ x-3747) there is again a change to cyan bars indicating another cyclic turn is imminent, this being confirmed by the momentum indicators patterns. However, subsequent price action obviously fails the assumed cyclic model as prices move into a narrow sideways channel with both the cybercycle amplitude decreasing and the cybercycle flip-flopping around the zero line. Also the expected cyclic pattern of the momentum indicators breaks down with all three period lines converging to, or flip-flopping around the zero line. There is an interesting series of posts on Brett Steenbarger's blog about approaching trading in a scientific manner, with a trading hypothesis being either confirmed (profitable trade) or being disconfirmed (unprofitable trade) and thus providing new market insight. Taking this approach, the failure of the turn indicated at x-3747 (and possible trading loss) provides new market insight i.e. trying to trade cyclic market turns isn't going to be profitable, even though the prices are indicated by the predominantly green and cyan bars to be moving sideways. If the market is trading sideways but it isn't profitable to pick highs and lows to take advantage of mean reversion, may be it will be more profitable to trade breakouts of the sideways channel?

So, the question now is how to trade the breakout, and in which direction? The momentum indicators perhaps give a clue. All the default colour bars, apart from 4, are blue, indicating upwards momentum. Also, the full period momentum never goes negative throughout this sideways channel. I would take this as a strong indication to look for an upwards breakout, which in fact occurs at (@ x-3774). The market then runs upwards for approx. 2 weeks. During this time period trailing a stop would seem to be the most appropriate approach as there are no cyclic or sideways market indications i.e. no green or cyan coloured bars.

This 2 week upward run ends at x-3792 with a sharp red down bar. If not already stopped out, and in the absence of either green or cyan bars, and with all three period momentum lines indicating either falling or negative momentum, this would appear to be a straight forward, go with the flow, short signal. If taken, short profits accrue quite quickly until the third cyan bar (@ x-3798) when there is a bullish engulfing bar followed by a short retracement of the down move. The indicated "turn window" immediately prior to this short retracement might indicate a new long position is in order, or perhaps only take short profits. However, militating against this action is the fact that both prices and the cybercycle have penetrated below their respective lcl levels. It is difficult to say whether, in reality, either action would have been taken if a short position had been established. However, it is easy to say that the hypothesis of a cycling market is quickly disconfirmed by the sharp downward bar at x-3802, which closes below the low of a possibly perceived, cyan bar, cyclic low, and which is immediately followed by a "hammer" candlestick pattern and then 4 red bars. Whether the 4 red bars would indicate a new short would be debatable as 2 of the momentum indicators at this point show bullish divergence.

After these 4 red bars we get 3 blue bars, which following bullish divergence and a hammer signal would indicate a new long is in order. Additionally the full period momentum indicator is flattening out, which would indicate possible high momentum to the upside if the bars were actually green and hence possibly cycling. If taken, this long moves into profit until we get green and cyan bars at the price ucl level at (@ x-3815), which of course indicates that a possible cyclic top is imminent. However, this hypothesis of cycling behaviour again is quickly disconfirmed by prices moving higher, the presence of another hammer, the momentum indicators not following the "expected" pattern for a cyclic top and finally the bars turning blue for a nice set up for the long up trend to the red bar (@ x-3858).

Of course, the period starting with the red bar (@ x-3858) is very interesting as it incorporates the record breaking, huge, intra day move and subsequent large oscillatory swings and increased volatility. Given these unique conditions I think it would be too much to expect any systematic approach to behave "normally," and equally I think it would be unproductive to try to adjust the system parameters to cope with such conditions as this would be gross over fitting of system parameters. However, having said that, how might the current implementation the "Dekalog" system have coped?

Well, the fact that there is a red bar 8 days before the meltdown means that any open trades would probably have been short trades, hence benefiting from the big move. In fact the day of the big move is the first green bar for approx. 40 trading days, and hence no cyclic action inspired trades would have been open on the day of this move. Given that no "cyclic" trades would have been open, the default trade direction of short (red bars) was in place for 4 consecutive days before the big move to the downside. After the big move the bars are green for 9 more days before turning red again, indicating possible sideways/cyclic market conditions. Would cyclic trades have been placed during this period? Well, who knows? The "Dekalog" system rules have not been tested enough and are not yet refined enough to say for sure what cyclic positions if any, would have been taken. Certainly, after such a big move in one's favour it might have been psychologically difficult to take a contrary position. Also, during the first part of this 9 day period both prices and the cybercycle were below their respective lcl levels, suggesting that it might not be time to try to trade any cycles.

After this 9 day, possibly cycle period, there is again a period of default bar colours, but with obviously higher volatility than normal. If trading had occurred during this period a couple of whipsaw losses might have been realised. However, at bar (@ x-3890) the bars turn green again for a probable small, long side profit, quickly followed by an almost perfectly indicated cyclic high which results in a nice profitable short trade down to and through the price lcl level. The last, yellow bar is the daily price bar for 2nd July.

All of the above should be read in the spirit of "me thinking out aloud" about how the Dekalog approach should be applied, and does not represent hard and fast, tested system rules. The testing and systematic application of these concepts to different price series will be the focus of immediate future work, and in this regard some ideas that come to mind are listed below.

- The default bars look back period is available for optimsation as a stand alone momentum/trend following system to act as a benchmark for all other incarnations of the Dekalog system. This "Default Bars" system itself is to be benchmarked against other common "systems."
- The above "me thinking out aloud" is, of course, very subjective and could be forward tested in real time using my Collective2 account. This will also act as a period of observation of the behaviour of the "Dekalog" approach and enable me to get a sense of how it performs and to develop trading rules. This will also require speed optimisation of current Octave implementation of ideas for practicality.
- Some methodology to distinguish between sideways markets suitable for cyclic top and bottom picking and those best suited to a breakout only methodology needs to be developed.

## Saturday 3 July 2010

### Update on recent efforts #5

An additional benefit to the repeated median slope*period length change is that I now have a form of normalised momentum indicator and the first plot above shows three applications of this to a sine wave, the three look back periods being a full cycle period (green), a half period (red) and a third period (light blue). The point of plotting this is to see how this indicator behaves in ideal cyclic circumstances to help formulate rules for use on real prices.

This indicator has the interesting properties listed below

- the third period line turns up/down almost exactly at the peak and crosses the half period line prior to this as an early warning signal of an impending turn.
- the half period line flattens out over the peaks and crosses the zero line at the same time the sine wave crosses the zero line.
- the full period line crosses the zero line as the sine wave peaks and flattens out over the section of the sine wave that actually has greatest directional movement, which is understandable given the nature of the calculation of the indicator.

Firstly, the lowest sub plot is the normalised momentum indicator described above. The full period momentum line is coloured blue or red depending on whether the momentum is positive or negative, the half period momentum is coloured light blue and the third period momentum is the yellow line.

Above this there is the short term, market turn, timing chart as described in my earlier post dated 30th May. A slight addition to this is the red half period line and the fact that if the count line is above this, it is also coloured red. The interpretation of this particular chart is that red means a cyclic zero cross is "overdue" and is an indication that cyclic behaviour, if present, may be breaking down.

The third, uppermost sub plot is the yellow cybercycle and its MC optimised ucl and lcl lines. The cybercycle line is coloured red when the timing chart indicates that a market turn is due (plus and minus 2 days around the quarter period count from the most recent zero line cross).

Finally, the topmost plot is the price candlestick chart, each bar separately colour coded according to the indicators so far mentioned. Bars are green if the full period, repeated median price momentum, normalised by the absolute difference of the cybercycle ucl and lcl is within its MC optimised limits. The bars that are yellow or cyan correspond to the timing turning points, similar to the red on the cybercycle plot, the difference being that the cyan bars also meet the condition for being green bars; this is to be interpreted as a strong indication that a market turn is imminent. By default if none of the above conditions are met then bars are blue if the third period repeated median momentum is positive or red if negative. The "bands" around the candlesticks are the full period ucl and lcl for the typical price (H+L+C)/3 and the same separately for the highs and lows, which are to be interpreted as support and resistance levels when prices are determined to be cycling or moving sideways.

I think the value of the approach taken so far, and the understanding of market action that can be inferred, is evident from this plot, but a more detailed discussion of this will be the subject of my next post.

## Sunday 20 June 2010

### Update on recent efforts #4

By using this "new" measure of cyclic tendency the intent/hope is that prices can be said the be trending or going sideways dependent upon whether the normalised slope is outside of, or between, the levels calculated above. This idea, along with the others previously detailed, will be tested on real world data very soon. The details of this testing and a discussion of the rationale of the intended system input will be the subject of my next posting.

## Saturday 19 June 2010

### Update on recent efforts #3

In my last post I spoke about conducting a Monte Carlo test to determine a maximum and minimum slope value for the repeated median straight line fit, the purpose being to establish levels outside of which it could be said that price is unlikely to be moving sideways according to the modelling of prices as a sine wave. A plot of the first step in this procedure is shown above.

The plot shows the MC test output of the slope of the repeated median straight line fit at various multiples of the period of the sine wave model. At 6 on the x-axis, representing a particular multiple, it can be seen that the standard deviation of the test run output is at a minimum, meaning this is the value that sees the least variation between MC runs, and the mean and median of the output at this point is the expected value of zero. This exact pattern repeats for all periods of interest (10 to 50 inclusive). This result is gratifying in that the actual value of the look back period multiplier seems to make sense theoretically, and the fact that it is the same across all periods means that it is a robust parameter. However, because the standard deviation is not zero, the next MC test will be to determine the normalised upper and lower levels already mentioned, using the optimised look back period multiplier represented above.

## Sunday 13 June 2010

### Update on recent efforts #2

## Sunday 6 June 2010

### Update on recent efforts

What I'm trying to code (as a dynamically loaded C++ Oct file function) is an adaptive, moving window, repeated median, straight line fit, the inspiration for which I got from Meyers Analytics and the system on this site which is called the Robust Repeated Median Velocity System. I don't intend to use this system as such, but I have an intended use for the idea of a statistically robust, straight line fit. More on this in due course.

## Sunday 30 May 2010

### Timing short term market turns

The basic theory behind the projected use of this indicator is simple: extract the cyclic component of price using the Cybercycle, measure the period of the cycle, and when the Cybercycle crosses its zeroline it can be anticipated that in one quarter period's time the turn will occur. This anticipated turning point is indicated when the white sawtooth line (the time elapsed since the last Cybercycle zeroline cross) reaches the three horizontal green lines (2 days before, 1 day before and the actual "turn" day counted out from the last Cybercycle zeroline cross). The white sawtooth is reset to zero each time a Cybercycle zeroline cross occurs (shown by the red lines). This indicator has been optimised using MC techniques to account for the lead/lag at different periods of the Cybercycle compared to the original input price series.

This new indicator, combined with the previously described Tukey Chart, will provide decision making input to the final system. Projected uses might be something like:

- if a Cybercycle zeroline cross occurs whilst prices are within the Tukey upper and lower contol limits, can anticipate a short term market turn in 1/4 period's time
- if prices reach either Tukey control limit at the same time the sawtooth reaches the green lines, a signal is given to take profits, take a new position etc.

## Wednesday 26 May 2010

### Monte Carlo Optimisation of Control Limits Complete

The MC optimisation of the various ucl and lcl multipliers is now complete and the "finished product" is displayed above on the same data as in the plot of post dated 16th May 2010. Simple visual comparison of the plots shows that recent adjustments to the MC methodology of selecting the ucl and lcl multipliers makes very little difference, except in the case of the lower plot of the Fisher Transformation of the Cybercycle indicator. This tells me that further work on more MC optimisation of this would probably be a waste of time, so I now consider this optimisation work to be complete.

The idea of applying control chart theory directly to market prices and derived indicators now needs to be tested on historical market data.

## Monday 24 May 2010

### Monte Carlo Optimisation of Control Limits #9

## Sunday 16 May 2010

### Application of Tukey Chart

Above is a screen shot of current work in progress regarding Tukey Charts.

This is a shot of the continuous back-adjusted es mini contract from early January 2010 to mid May 2010, including the 1 day, 1000 point drop! The top chart shows price bars with the optimised Tukey Chart applied directly to prices (the cyan lines) and also the Tukey Chart applied to the Cybercycle* and scaled and super-imposed on price bars (the yellow lines). The price bars are blue if the price is above both outer ucls and red if below both outer lcls, white otherwise. The middle chart is the Cybercycle* with its Tukey Chart, and the lower chart is the Fisher Transformed Cybercycle* with its Tukey Chart. The ucl and lcl multipliers used for the Cybercycle* and Fisher Transformed Cybercycle* are the same as those used from testing detailed below. These are obviously available for MC optimisation in their own right and are on my "to do" list.

*see the technical papers section at http://www.mesasoftware.com for more information.

## Monday 10 May 2010

### Monte Carlo Optimisation of Control Limits #8

I think that to continue with MC testing on theoretical sine waves would result in diminishing value, so this third MC test will be the last on such theoretical wave forms. These last few tests have resulted in different optimised ucl and lcl multipliers for 3 "ranges" of periods: periods 14 and less, 15 to 24, and 25 and greater.

The next step will be to rewrite the indicator with these new optimised multipliers and test it on actual market prices.

## Sunday 9 May 2010

### Monte Carlo Optimisation of Control Limits #7

## Saturday 8 May 2010

### Monte Carlo Optimisation of Control Limits #6

Looking at the plot from post #4 it is obvious why all the failures are concentrated at the low periods, as this is the area where the plot shows most variation in the range of the multipliers. I have therefore decided that the optimised ucl and lcl multipliers that were tested in this test shall apply only to periods less than 15 in the final indicator. I shall repeat the MC test with new ucl and lcl optimised boundary multipliers for periods 15 to 50 inclusive.

## Wednesday 5 May 2010

### Monte Carlo Optimisation of Control Limits #5

To reiterate so far, the indicator:

- is designed to show when prices are moving sideways or moving in a sideways channel by containing prices between a support and a resistance zone
- prices have been modelled as a sine wave to represent this sideways price action
- the upper and lower boundaries of each zone are based upon the application of a control chart directly to prices
- Monte Carlo techniques and statistical tests have been employed to optimise the boundaries of each zone such that approximately 95% of the time the peaks and troughs of the sine wave should occur within their respective zone boundaries

### Monte Carlo Optimisation of Control Limits #4

This post is about the selection of the ucl and lcl multipliers. From the tests performed so far it has been established that all the data for any single sine wave period can be aggregated into one overall distribution for that period. The nature of each distribution, as seen from looking at the density plots, suggests to me that taking the extreme values of each distribution would be acceptable as the sharp drop offs in the tails mean that these values would not be far removed from those that would chop off the tails but still encompass the vast majority of the data in said distribution (we are talking here about differences of only a few hundredths of a decimal point in the multiplier values). So that is exactly what I have done: aggregated all the distributions and taken the maximum and minimum values and plotted them against the sine wave period on the x-axis (plot shown above).

The thing that strikes me when looking at this plot is that there seems to be a natural upper and lower boundary that is consistent across all periods, which is quite fortuitous as this means it will not be necessary to write a complicated function with numerous if statements to check the period and apply a unique multiplier value for that period; it will simply suffice to apply the upper and lower boundary values regardless of period. For the time being at least, I have decided to set these boundaries values at the 0.025 and 0.975 quantile levels because

- "Quantiles are useful measures because they are less susceptible to long-tailed distributions and outliers. Empirically, if the data you are analyzing are not actually distributed according to your assumed distribution, or if you have other potential sources for outliers that are far removed from the mean, then quantiles may be more useful descriptive statistics than means and other moment-related statistics."

The next thing to do is write this function and test it.

## Monday 3 May 2010

### Monte Carlo Optimisation of Control Limits #3

Following on from the previous blog post, further investigation of the Kolmogorov-Smirnov test failures for periods 16, 24, 32, 40 and 48 density plots, for both amplitude 1 and variable amplitude, show that in each case the distributions "collapse to a unique value" and essentially there is no variation as the max and min values in each distribution are the same, 0.2071068. This is of no concern regarding the result of the Kolmogorov-Smirnov test, it just means for these periods there will not be two separate multipliers for the ucl and lcl: in effect there will just be a line of support/resistance instead of a zone, although I have an idea to address this which will be outlined later.

For Kolmogorov-Smirnov test failures for periods 27, 38, 45 and 49 refer to the density plot above, which is similar for all four period failures. It can be seen that the tails are almost identical and that the "plateau" area shows some variation: I surmise that it is this variation in the plateau that leads to the rejection of the null hypothesis. However, for the purposes of setting the ucl and lcl multipliers it is the tails that are of interest, and since the tails so similar to each other these test failures are also of no consequence.

The conclusion to be drawn from these tests is that

- in regard to choosing the optimum values for the ucl and lcl multipliers for our theoretical sine wave for any single, constant period sine wave the phase and amplitude of the sine wave has no noticeable effect on the optimum choice, as the distributions for any single period multiplier are, at least in the tails of interest, statistically the same.

- as a result of this the separate distributions for each single period sine wave can be combined in to one larger distribution for that period for the actual determination of the multipliers.

## Tuesday 27 April 2010

### Monte Carlo Optimisation of Control Limits #2

Having run the routine again on sine waves of various amplitudes I have plotted boxplots and kernal density plots of the ucl and lcl optimised multipliers for sine waves of periods n=10 to 50 inclusive, using R. Visual inspection of the plots suggest that, in general* (see note below), there are no real differences between the distributions and a typical plot is shown above (again this is for a sine wave of period n=20). The black plots are

- optimised multiplier for ucl of sine wave with amplitude of 1
- optimised multiplier for lcl of sine wave with amplitude of 1
- optimised multiplier for ucl of sine wave with variable amplitude
- optimised multiplier for lcl of sine wave with variable amplitude

Obviously just looking at plots is not statistically robust, so the next thing I did was to run Kolmogorov-Smirnov tests in R to see if the null hypothesis that the four separate distributions come from the same distribution can be rejected. For this purpose I used a p-value of 0.05 for the significance level. The comparative test runs were

- ucl sine wave amplitude 1 and lcl sine wave amplitude 1
- ucl sine wave variable amplitude and lcl sine wave variable amplitude
- ucl sine wave amplitude 1 and ucl sine wave variable amplitude
- lcl sine wave amplitude 1 and lcl sine wave variable amplitude

if A=B and C=D and A=C and B=D then A=B=C=D.

The actual test in R that was used was the Bootstrap Kolmogorov-Smirnov package ks.boot(), the R documentation description for which is quoted below:-

"This function executes a bootstrap version of the univariate Kolmogorov-Smirnov test which provides correct coverage even when the distributions being compared are not entirely continuous. Ties are allowed with this test unlike the traditional Kolmogorov-Smirnov test"

It is obvious from the discrete nature of the raw data being tested that this test is more appropriate than the traditional Kolmogorov-Smirnov test, but more importantly the symmetrical nature of a sine wave almost guarantees that there are likely to be ties in the data, hence the choice of this test. The number of bootstraps performed for each test was n=1000. The p-value was sufficiently low to reject the null hypothesis for the following periods (numbered bullet points correspond to tests identified above)

- Periods 16 ,24, 32, 40 and 48
- Periods 16, 24, 27, 32, 40, 45, 48, 49
- Periods 38 and 49
- Periods 38 and 49

*The exceptions to the general pattern mentioned above will be the subject of a later blog entry.

## Sunday 25 April 2010

### Monte Carlo Optimisation of Control Limits #1

The overnight run went without a hitch and I now have a text file of 3000 rows (the number of iterations in the MC routine) and columns for the ucl and lcl multipliers for sine waves of periods 10 to 50 inclusive.

As a first simple analysis of these I have used the simple R commands boxplot(ucl) and plot(density(ucl)) to plot graphs for quick visual examination of the distribution of these multipliers, each graph being either ucl or lcl for one sine wave period only. A generic example of boxplot(ucl) is the first graph above (this happens to be ucl for sine wave period 20). As can be seen, there are no outliers beyond the whiskers and it would appear that the distribution is evenly spread about the thick median line.

The second graph is plot(density(ucl)), a plot of the kernal density. Note the steep drop offs at about 2.00 and 2.15. This is a pattern that repeats for almost all periods for both the ucl and lcl, although of course with different values at the drop off points. This second graph suggests that in fact two multipliers could be used to create a "zone" ucl and lcl. It can be conjectured that

- if prices are outside this zone, either to the upside of the ucl zone or downside of the lcl zone, there would be a high probability that in fact prices are trending,
- whilst if prices are between the zones they could be considered to be in a sideways channel
- whilst if prices are actually between the lines that create either zone they could be considered to be in a high probability area for a price reversal in a sideways market, or alternatively in a zone where a breakout from a sideways market is imminent.

## Saturday 24 April 2010

### First step in optimising Tukey Chart

## Wednesday 21 April 2010

### Applying Control Charts to price series

For my first substantive post I thought I would post a screenshot (http://www.gnuplot.info) of what I'm currently working on. The red line is a sine wave and the lines of interest are the yellow upper control line (ucl) and the blue lower control line (lcl). For a description of what the ucl and lcl are see http://gunston.gmu.edu/708/frTukey.asp

The basic idea is to apply the concept of a Tukey Control Chart to prices to determine whether prices are moving sideways or not. To that end, the above sine wave models a sideways moving market and represents the pre-intervention period for constructing the ucl and lcl. The idea is that if prices are outside either of the control limits then prices are NOT moving sideways in a cyclic fashion and by default must be either trending up or down. This in turn will determine what trading approach to take given current market conditions and what indicators to use.

The above ucl and lcl do not use Tukey's recommended 1.5 multiplier for the Fourth Spread but instead use a different multiplier depending on the period of the sine wave. This has been determined simply by looking at numerous charts and adjusting the multiplier so that the control lines are as close as possible, visually, to the peaks and troughs of the sine wave. Future work to be done is to write a program/routine - I use Octave (http://www.gnu.org/software/octave) and C++ functions written for Octave, and occasionally the statistical software R (http://www.r-project.org) in my development work - to optimise this multiplier for each period length. For help in coding this I have posted on the Octave Nabble forum and I am waiting for any responses. The hope/expectation is that on a real price chart the ucl and lcl will indicate support and resistance areas when prices are in a sideways moving channel as well as indicating the fact that current price is actually moving in a fashion that, statistically speaking, is indistinguishable from a sideways moving channel.

## Saturday 17 April 2010

### Reasons for starting this blog

Another reason for starting the blog is the idea that it will become my online trading journal.