Bayesian Analysis Update

Having spent the last 6-8 weeks researching and trying out various coding attempts, I've come to the conclusion that, for the moment at least, my time will be spent more usefully doing some of the other things on my to do list. Bayesian analysis is on the back burner for the nearest future.

Bayesian Analysis

The thing I'm working on at the moment is applying Bayesian analysis techniques to the trading system, and I'm currently in the research phase consisting of lots of reading and occasional queries to on-line statistical forums. The idea behind this is to develop a methodology to switch between different sets of trading rules, each rule set being applicable to defined market types. More on this as and when I have something relevant to report.

Posting to another forum

Readers of this blog might be interested in a recent forum posting of mine here, under babelproofreader.

Recent coding work

Have been working on the various .oct compiled functions and am happy to say that the work on MC re-optimisation of .oct functions for ucl & lcl for price, cybercycle, fisher transformed cybercycle, cybercylce relative to price bars and fisher transformed highpass filter has now been completed and the "to do page" updated accordingly.

Update on First Trades

The Feeder Cattle trade has been stopped out for a loss in both the Collective2 account and the live trading account. However, the EURCAD forex trade remains open and is showing an open profit of $1602 (on Sunday evening before the markets open) in the Collective2 account. The EURCAD trade was not taken in the live account because the risk was too great for the account size.

First Trades

Today, following signals given by the Bandpass filter, the first two trades of the revised Dekalog system have been opened in the Collective2 account; long Feeder Cattle and long the EURCAD forex pair. The feeder cattle trade has also been taken in a live trading account too. At the time of posting, both are still open. Details will be posted in a follow up blog post.

Highpass filter added

From the same source as indicated in the previous post I have now also added a compiled .oct function for a highpass filter and its adaptive Fisher transformation. Again this indicator is to be used when prices are determined to be cyclic in nature to help identify turning points.

Addition of new Bandpass Indicator

I have recently written a .oct function to calculate the Bandpass Indicator, the white paper for which can be found here. This indicator will be used with a leading signal to help pin-point price series turns when the said price series is determined to be cycling according to the Dekalog approach.

Completion of Speed Optimised Code

It has been a few weeks since my last post and during this time I have been busy working on writing compiled C++ Octave functions (in the interests of speed optimisation) for the previously mention indicators. This has been more difficult and time consuming that I imagined it would be, but I am happy to say that the bulk of this work is now complete. I have also written a C++ Octave function based on these indicators to create synthetic price series for optimisation purposes. At the moment the look back period for this synthetic price series construction is fixed at 21, but now I am in a position to make this look back period adaptive to the measured period of the synthetic price series, just as the indicators are intended to be applied to "live" data. My "things to do" page has been updated to reflect this completed work, and new ideas stemming from this have been added.

The Dedicated Dekalog Website

I have just created a website dedicated to the Dekalog trading approach HERE. It is my intention in future to upload results of tests etc. of the Dekalog system to this new website. However, I will continue blogging here and refer readers to the Dekalog site whenever appropriate.

Addition to title header bar

I have just read an interesting blog post about the source of trend following returns (price-distributions-trend-following) and one of the comments on this blog post includes this - "Trading is statistics and time series analysis. I’m amazed how many 'trading books' fail to cover these relevant topics." I like this so much that I thought I would adopt this as a motto for the Dekalog approach and I have included it in this blog's header bar. I think this neatly sums up the Dekalog approach, which applies the precepts of robust statistics directly to price and derived indicators.

Discussion of work so far

In this discussion I will be talking about the indicators etc. that have been developed so far and the reader is referred to all previous posts and especially the candlestick chart in the post dated 3rd July and the version thereof above. The topmost plot above is the same as the topmost in the previous post, whilst the lower plot is a plot of the default coloured bars only (see post dated 3rd July).

Looking at the extreme left of this candlestick chart we can see that a cybercycle zero line cross has just occurred (@ x-axis 3720) and at x-3724 the candles change colour to cyan, meaning that the turn indicator shows a possible turn in 2 days time. A few days later, candles turn yellow, meaning that prices are still in the "turn window" but no longer satisfy the conditions for being a sideways market according the formula for momentum/abs(ucl-lcl) of cybercycle. At this time, the momentum indicators are following the pattern that can be expected for a cyclic top to be in place. All in all, I would characterise this as a high probability short set up and subsequent price action confirms this to be a good call as prices decrease to the price lcl levels (@ x-3738).

At these new lower levels (@ x-3738) we now see a new set up to go long. Again prices are indicated to be in a "turn window" and the momentum indicators are following the pattern that can be expected for a cyclic low to be in place. Again this turns out to have been a good call as prices subsequently increase to the price ucl levels (@ x-3747).

As prices reach these new high levels (@ x-3747) there is again a change to cyan bars indicating another cyclic turn is imminent, this being confirmed by the momentum indicators patterns. However, subsequent price action obviously fails the assumed cyclic model as prices move into a narrow sideways channel with both the cybercycle amplitude decreasing and the cybercycle flip-flopping around the zero line. Also the expected cyclic pattern of the momentum indicators breaks down with all three period lines converging to, or flip-flopping around the zero line. There is an interesting series of posts on Brett Steenbarger's blog about approaching trading in a scientific manner, with a trading hypothesis being either confirmed (profitable trade) or being disconfirmed (unprofitable trade) and thus providing new market insight. Taking this approach, the failure of the turn indicated at x-3747 (and possible trading loss) provides new market insight i.e. trying to trade cyclic market turns isn't going to be profitable, even though the prices are indicated by the predominantly green and cyan bars to be moving sideways. If the market is trading sideways but it isn't profitable to pick highs and lows to take advantage of mean reversion, may be it will be more profitable to trade breakouts of the sideways channel?

So, the question now is how to trade the breakout, and in which direction? The momentum indicators perhaps give a clue. All the default colour bars, apart from 4, are blue, indicating upwards momentum. Also, the full period momentum never goes negative throughout this sideways channel. I would take this as a strong indication to look for an upwards breakout, which in fact occurs at (@ x-3774). The market then runs upwards for approx. 2 weeks. During this time period trailing a stop would seem to be the most appropriate approach as there are no cyclic or sideways market indications i.e. no green or cyan coloured bars.

This 2 week upward run ends at x-3792 with a sharp red down bar. If not already stopped out, and in the absence of either green or cyan bars, and with all three period momentum lines indicating either falling or negative momentum, this would appear to be a straight forward, go with the flow, short signal. If taken, short profits accrue quite quickly until the third cyan bar (@ x-3798) when there is a bullish engulfing bar followed by a short retracement of the down move. The indicated "turn window" immediately prior to this short retracement might indicate a new long position is in order, or perhaps only take short profits. However, militating against this action is the fact that both prices and the cybercycle have penetrated below their respective lcl levels. It is difficult to say whether, in reality, either action would have been taken if a short position had been established. However, it is easy to say that the hypothesis of a cycling market is quickly disconfirmed by the sharp downward bar at x-3802, which closes below the low of a possibly perceived, cyan bar, cyclic low, and which is immediately followed by a "hammer" candlestick pattern and then 4 red bars. Whether the 4 red bars would indicate a new short would be debatable as 2 of the momentum indicators at this point show bullish divergence.

After these 4 red bars we get 3 blue bars, which following bullish divergence and a hammer signal would indicate a new long is in order. Additionally the full period momentum indicator is flattening out, which would indicate possible high momentum to the upside if the bars were actually green and hence possibly cycling. If taken, this long moves into profit until we get green and cyan bars at the price ucl level at (@ x-3815), which of course indicates that a possible cyclic top is imminent. However, this hypothesis of cycling behaviour again is quickly disconfirmed by prices moving higher, the presence of another hammer, the momentum indicators not following the "expected" pattern for a cyclic top and finally the bars turning blue for a nice set up for the long up trend to the red bar (@ x-3858).

Of course, the period starting with the red bar (@ x-3858) is very interesting as it incorporates the record breaking, huge, intra day move and subsequent large oscillatory swings and increased volatility. Given these unique conditions I think it would be too much to expect any systematic approach to behave "normally," and equally I think it would be unproductive to try to adjust the system parameters to cope with such conditions as this would be gross over fitting of system parameters. However, having said that, how might the current implementation the "Dekalog" system have coped?

Well, the fact that there is a red bar 8 days before the meltdown means that any open trades would probably have been short trades, hence benefiting from the big move. In fact the day of the big move is the first green bar for approx. 40 trading days, and hence no cyclic action inspired trades would have been open on the day of this move. Given that no "cyclic" trades would have been open, the default trade direction of short (red bars) was in place for 4 consecutive days before the big move to the downside. After the big move the bars are green for 9 more days before turning red again, indicating possible sideways/cyclic market conditions. Would cyclic trades have been placed during this period? Well, who knows? The "Dekalog" system rules have not been tested enough and are not yet refined enough to say for sure what cyclic positions if any, would have been taken. Certainly, after such a big move in one's favour it might have been psychologically difficult to take a contrary position. Also, during the first part of this 9 day period both prices and the cybercycle were below their respective lcl levels, suggesting that it might not be time to try to trade any cycles.

After this 9 day, possibly cycle period, there is again a period of default bar colours, but with obviously higher volatility than normal. If trading had occurred during this period a couple of whipsaw losses might have been realised. However, at bar (@ x-3890) the bars turn green again for a probable small, long side profit, quickly followed by an almost perfectly indicated cyclic high which results in a nice profitable short trade down to and through the price lcl level. The last, yellow bar is the daily price bar for 2nd July.

All of the above should be read in the spirit of "me thinking out aloud" about how the Dekalog approach should be applied, and does not represent hard and fast, tested system rules. The testing and systematic application of these concepts to different price series will be the focus of immediate future work, and in this regard some ideas that come to mind are listed below.

1. The default bars look back period is available for optimsation as a stand alone momentum/trend following system to act as a benchmark for all other incarnations of the Dekalog system. This "Default Bars" system itself is to be benchmarked against other common "systems."
2. The above "me thinking out aloud" is, of course, very subjective and could be forward tested in real time using my Collective2 account. This will also act as a period of observation of the behaviour of the "Dekalog" approach and enable me to get a sense of how it performs and to develop trading rules. This will also require speed optimisation of current Octave implementation of ideas for practicality.
3. Some methodology to distinguish between sideways markets suitable for cyclic top and bottom picking and those best suited to a breakout only methodology needs to be developed.
   

Update on recent efforts #5

I have spent the last week or so working on and refining my ideas connected with the use of the repeated median. The first thing is that the formula from the earlier post has been altered to (repeated median slope*period length)/(max-min)values of sine wave. In effect this then becomes a form of normalised momentum indicator. MC testing has been done such that I now have upper and lower limits for this at each period of interest; i.e. if the momentum across the measured period is outside of these limits, prices are highly likely to be trending and not cycling sideways. In actual application to price, the (max-min)values of sine wave in the above formula will be replaced by the absolute difference between the ucl and lcl of the cybercycle, the theoretical MC optimised limits of the extracted cyclic component of price.

An additional benefit to the repeated median slope*period length change is that I now have a form of normalised momentum indicator and the first plot above shows three applications of this to a sine wave, the three look back periods being a full cycle period (green), a half period (red) and a third period (light blue). The point of plotting this is to see how this indicator behaves in ideal cyclic circumstances to help formulate rules for use on real prices.



This indicator has the interesting properties listed below

1. the third period line turns up/down almost exactly at the peak and crosses the half period line prior to this as an early warning signal of an impending turn.
2. the half period line flattens out over the peaks and crosses the zero line at the same time the sine wave crosses the zero line.
3. the full period line crosses the zero line as the sine wave peaks and flattens out over the section of the sine wave that actually has greatest directional movement, which is understandable given the nature of the calculation of the indicator.
   

The above comments seem to hold true across all periods. Knowing this "ideal" behaviour will aid interpretation of price action when the indicator is applied to price. The second chart, below, is an application of all the ideas so far discussed to an actual price series. It is the same chart as that shown previously (S&P E-mini), but updated to include recent prices. It may seem confusing to readers, so an explanation will help.



Firstly, the lowest sub plot is the normalised momentum indicator described above. The full period momentum line is coloured blue or red depending on whether the momentum is positive or negative, the half period momentum is coloured light blue and the third period momentum is the yellow line.

Above this there is the short term, market turn, timing chart as described in my earlier post dated 30th May. A slight addition to this is the red half period line and the fact that if the count line is above this, it is also coloured red. The interpretation of this particular chart is that red means a cyclic zero cross is "overdue" and is an indication that cyclic behaviour, if present, may be breaking down.

The third, uppermost sub plot is the yellow cybercycle and its MC optimised ucl and lcl lines. The cybercycle line is coloured red when the timing chart indicates that a market turn is due (plus and minus 2 days around the quarter period count from the most recent zero line cross).

Finally, the topmost plot is the price candlestick chart, each bar separately colour coded according to the indicators so far mentioned. Bars are green if the full period, repeated median price momentum, normalised by the absolute difference of the cybercycle ucl and lcl is within its MC optimised limits. The bars that are yellow or cyan correspond to the timing turning points, similar to the red on the cybercycle plot, the difference being that the cyan bars also meet the condition for being green bars; this is to be interpreted as a strong indication that a market turn is imminent. By default if none of the above conditions are met then bars are blue if the third period repeated median momentum is positive or red if negative. The "bands" around the candlesticks are the full period ucl and lcl for the typical price (H+L+C)/3 and the same separately for the highs and lows, which are to be interpreted as support and resistance levels when prices are determined to be cycling or moving sideways.

I think the value of the approach taken so far, and the understanding of market action that can be inferred, is evident from this plot, but a more detailed discussion of this will be the subject of my next post.

Update on recent efforts #4

MC testing on the slope of the repeated median straight line fit is now complete. Normalising the optimal length look back slope of a sine wave by its amplitude ( slope/(max-min)values of sine wave ) gives a ratio. A ratio distribution has a tendency to be heavy tailed (wiki/Ratio_distribution) and as quantiles are "useful measures because they are less susceptible to long-tailed distributions and outliers" I have simply used R to calculate quantiles and determined outliers to be 1.5 * IQR above and below Q3 and Q1.

By using this "new" measure of cyclic tendency the intent/hope is that prices can be said the be trending or going sideways dependent upon whether the normalised slope is outside of, or between, the levels calculated above. This idea, along with the others previously detailed, will be tested on real world data very soon. The details of this testing and a discussion of the rationale of the intended system input will be the subject of my next posting.

Update on recent efforts #3

In my last post I spoke about conducting a Monte Carlo test to determine a maximum and minimum slope value for the repeated median straight line fit, the purpose being to establish levels outside of which it could be said that price is unlikely to be moving sideways according to the modelling of prices as a sine wave. A plot of the first step in this procedure is shown above.

The plot shows the MC test output of the slope of the repeated median straight line fit at various multiples of the period of the sine wave model. At 6 on the x-axis, representing a particular multiple, it can be seen that the standard deviation of the test run output is at a minimum, meaning this is the value that sees the least variation between MC runs, and the mean and median of the output at this point is the expected value of zero. This exact pattern repeats for all periods of interest (10 to 50 inclusive). This result is gratifying in that the actual value of the look back period multiplier seems to make sense theoretically, and the fact that it is the same across all periods means that it is a robust parameter. However, because the standard deviation is not zero, the next MC test will be to determine the normalised upper and lower levels already mentioned, using the optimised look back period multiplier represented above.

Update on recent efforts #2

I have been working on my new idea for the past week or so and I have finally managed to get a compiled .oct function which works, although it is not optimised for speed. Above is a plot of this, which shows two sine waves with their respective repeated median straight line fits. Both sine waves are one complete cycle, have the same period and amplitude, and only differ in their phase. As can be seen the straight line fits display different slopes, so the next thing I'm going to do is a Monte Carlo test to determine upper and lower values for the slopes at different periods, normalised to the amplitudes. The point of this to determine levels of normalised slope above and below which it can be said that price is not likely to be moving sideways. Conceptually, this could be thought of as a switch between trending and non-trending market environments.

Update on recent efforts

In my last post I spoke about a new idea that I want to code. Well, I have been working on this and the coding has turned out to be frustratingly difficult and I am having to rely on the generosity of forum members (Nabble-Octave-General) to help me. However, I'm sure that eventually it will be successfully coded.

What I'm trying to code (as a dynamically loaded C++ Oct file function) is an adaptive, moving window, repeated median, straight line fit, the inspiration for which I got from Meyers Analytics and the system on this site which is called the Robust Repeated Median Velocity System. I don't intend to use this system as such, but I have an intended use for the idea of a statistically robust, straight line fit. More on this in due course.

Timing short term market turns

The top plot above shows the sine wave modelled prices in white, with the Cybercycle indicator plotted in Cyan and blue. The bottom plot shows an indicator I have created to time the turns in the original, white series based upon the Cybercycle.

The basic theory behind the projected use of this indicator is simple: extract the cyclic component of price using the Cybercycle, measure the period of the cycle, and when the Cybercycle crosses its zeroline it can be anticipated that in one quarter period's time the turn will occur. This anticipated turning point is indicated when the white sawtooth line (the time elapsed since the last Cybercycle zeroline cross) reaches the three horizontal green lines (2 days before, 1 day before and the actual "turn" day counted out from the last Cybercycle zeroline cross). The white sawtooth is reset to zero each time a Cybercycle zeroline cross occurs (shown by the red lines). This indicator has been optimised using MC techniques to account for the lead/lag at different periods of the Cybercycle compared to the original input price series.

This new indicator, combined with the previously described Tukey Chart, will provide decision making input to the final system. Projected uses might be something like:

• if a Cybercycle zeroline cross occurs whilst prices are within the Tukey upper and lower contol limits, can anticipate a short term market turn in 1/4 period's time
• if prices reach either Tukey control limit at the same time the sawtooth reaches the green lines, a signal is given to take profits, take a new position etc.

These ideas will be tested on market data, along with the Tukey Charts, in due course. However for the moment I now want to work on a new idea which, if I can successfully code it, I believe will complement the above.

Monte Carlo Optimisation of Control Limits Complete

The MC optimisation of the various ucl and lcl multipliers is now complete and the "finished product" is displayed above on the same data as in the plot of post dated 16th May 2010. Simple visual comparison of the plots shows that recent adjustments to the MC methodology of selecting the ucl and lcl multipliers makes very little difference, except in the case of the lower plot of the Fisher Transformation of the Cybercycle indicator. This tells me that further work on more MC optimisation of this would probably be a waste of time, so I now consider this optimisation work to be complete.

The idea of applying control chart theory directly to market prices and derived indicators now needs to be tested on historical market data.

Monte Carlo Optimisation of Control Limits #9

The above is the optimisation of the cybercycle ucl and lcl, adjusted so that the control lines can be plotted on price bars, i.e. the yellow lines in the plot posted on 16th may below. The green and light blue boundary lines are polynomial line fits, which will be the actual multipliers used. This plot is somewhat different to that in the post of 5th May (Monte Carlo Optimisation of Control Limits #4). I put this down to the fact that at longer periods the cybercycle tends to lead prices and so the peaks/troughs in the cybercycle do not occur at the same time as those in the underlying data. The result is the upward sloping fit seen above.

Application of Tukey Chart

Above is a screen shot of current work in progress regarding Tukey Charts.

This is a shot of the continuous back-adjusted es mini contract from early January 2010 to mid May 2010, including the 1 day, 1000 point drop! The top chart shows price bars with the optimised Tukey Chart applied directly to prices (the cyan lines) and also the Tukey Chart applied to the Cybercycle* and scaled and super-imposed on price bars (the yellow lines). The price bars are blue if the price is above both outer ucls and red if below both outer lcls, white otherwise. The middle chart is the Cybercycle* with its Tukey Chart, and the lower chart is the Fisher Transformed Cybercycle* with its Tukey Chart. The ucl and lcl multipliers used for the Cybercycle* and Fisher Transformed Cybercycle* are the same as those used from testing detailed below. These are obviously available for MC optimisation in their own right and are on my "to do" list.

*see the technical papers section at http://www.mesasoftware.com for more information.

Monte Carlo Optimisation of Control Limits #8

The third, repeated MC test for periods 25 to 50 give these results: for periods 26 and 30 only the peaks and troughs occur between the ucl and lcl zones without actually entering the zones, and in each case the number of times this happened represents less than 0.01% of the total runs. At all other periods the peaks and troughs of the sine waves occur within their respective zones.

I think that to continue with MC testing on theoretical sine waves would result in diminishing value, so this third MC test will be the last on such theoretical wave forms. These last few tests have resulted in different optimised ucl and lcl multipliers for 3 "ranges" of periods: periods 14 and less, 15 to 24, and 25 and greater.

The next step will be to rewrite the indicator with these new optimised multipliers and test it on actual market prices.

Monte Carlo Optimisation of Control Limits #7

The second, repeated, MC test has been run on periods 15 to 50 inclusive giving these results: 0.2% of total runs peak/trough outside the ucl and lcl zones at period 17, 0.2% are entirely within the zones at period 18 and 0.17% are entirely within the zones at period 22. For the same reasons as outlined in post #6, the results from this test lead me to decide that the optimised multipliers tested in this test shall apply to periods 15 to 24 inclusive in the final indicator. A third repeated MC test will be run on periods 25 to 50 inclusive.

Monte Carlo Optimisation of Control Limits #6

I have now run the Monte Carlo tests of the boundary limits. The test consisted of 3000 phase randomised sine waves per period for constant periods 10 to 50 inclusive. The amplitudes of the sine waves were also randomised, up to a maximum of 20 (20*rand). This makes a total of 123,000 random sine waves. Any time the peak or trough of a sine wave occured above the resistance zone or below the support zone this was recorded. Similarly if the peak or trough of a sine wave occured between these zones without penetrating or touching them this was separately recorded. There were 683 instances (0.56 % of total runs) that peaked/troughed outside the zones, and 379 instances (0.31 %) that peaked/troughed completely within the zones without actually touching them. All of the above 683 instances occured at period 13, whilst the 379 instances were split 265/114 between periods 10/14.

Looking at the plot from post #4 it is obvious why all the failures are concentrated at the low periods, as this is the area where the plot shows most variation in the range of the multipliers. I have therefore decided that the optimised ucl and lcl multipliers that were tested in this test shall apply only to periods less than 15 in the final indicator. I shall repeat the MC test with new ucl and lcl optimised boundary multipliers for periods 15 to 50 inclusive.

Monte Carlo Optimisation of Control Limits #5

Writing the function was a relatively simple matter of reusing some of the code that had already been written to perform the tests detailed so far. A graphical plot of the appearance of the indicator is shown above.

To reiterate so far, the indicator:

• is designed to show when prices are moving sideways or moving in a sideways channel by containing prices between a support and a resistance zone
• prices have been modelled as a sine wave to represent this sideways price action
• the upper and lower boundaries of each zone are based upon the application of a control chart directly to prices
  
• Monte Carlo techniques and statistical tests have been employed to optimise the boundaries of each zone such that approximately 95% of the time the peaks and troughs of the sine wave should occur within their respective zone boundaries

The next step will be to write a Monte Carlo routine to test whether this indicator performs as expected on a new "out of sample" set of randomised sine waves, and this will be the subject of the next post.

Monte Carlo Optimisation of Control Limits #4

This post is about the selection of the ucl and lcl multipliers. From the tests performed so far it has been established that all the data for any single sine wave period can be aggregated into one overall distribution for that period. The nature of each distribution, as seen from looking at the density plots, suggests to me that taking the extreme values of each distribution would be acceptable as the sharp drop offs in the tails mean that these values would not be far removed from those that would chop off the tails but still encompass the vast majority of the data in said distribution (we are talking here about differences of only a few hundredths of a decimal point in the multiplier values). So that is exactly what I have done: aggregated all the distributions and taken the maximum and minimum values and plotted them against the sine wave period on the x-axis (plot shown above).

The thing that strikes me when looking at this plot is that there seems to be a natural upper and lower boundary that is consistent across all periods, which is quite fortuitous as this means it will not be necessary to write a complicated function with numerous if statements to check the period and apply a unique multiplier value for that period; it will simply suffice to apply the upper and lower boundary values regardless of period. For the time being at least, I have decided to set these boundaries values at the 0.025 and 0.975 quantile levels because

• "Quantiles are useful measures because they are less susceptible to long-tailed distributions and outliers. Empirically, if the data you are analyzing are not actually distributed according to your assumed distribution, or if you have other potential sources for outliers that are far removed from the mean, then quantiles may be more useful descriptive statistics than means and other moment-related statistics."

It can be expected that approximately only 5% of the peaks and troughs of any sine wave will fall outside these levels. The quantile function in R was used to calculate these levels, with R-8 as the type used (http://en.wikipedia.org/wiki/Quantile).

The next thing to do is write this function and test it.

Monte Carlo Optimisation of Control Limits #3

Following on from the previous blog post, further investigation of the Kolmogorov-Smirnov test failures for periods 16, 24, 32, 40 and 48 density plots, for both amplitude 1 and variable amplitude, show that in each case the distributions "collapse to a unique value" and essentially there is no variation as the max and min values in each distribution are the same, 0.2071068. This is of no concern regarding the result of the Kolmogorov-Smirnov test, it just means for these periods there will not be two separate multipliers for the ucl and lcl: in effect there will just be a line of support/resistance instead of a zone, although I have an idea to address this which will be outlined later.

For Kolmogorov-Smirnov test failures for periods 27, 38, 45 and 49 refer to the density plot above, which is similar for all four period failures. It can be seen that the tails are almost identical and that the "plateau" area shows some variation: I surmise that it is this variation in the plateau that leads to the rejection of the null hypothesis. However, for the purposes of setting the ucl and lcl multipliers it is the tails that are of interest, and since the tails so similar to each other these test failures are also of no consequence.

The conclusion to be drawn from these tests is that

• in regard to choosing the optimum values for the ucl and lcl multipliers for our theoretical sine wave for any single, constant period sine wave the phase and amplitude of the sine wave has no noticeable effect on the optimum choice, as the distributions for any single period multiplier are, at least in the tails of interest, statistically the same.

• as a result of this the separate distributions for each single period sine wave can be combined in to one larger distribution for that period for the actual determination of the multipliers.

The actual determination of these multipliers for each period will be the subject of the next post.

Monte Carlo Optimisation of Control Limits #2

Having run the routine again on sine waves of various amplitudes I have plotted boxplots and kernal density plots of the ucl and lcl optimised multipliers for sine waves of periods n=10 to 50 inclusive, using R. Visual inspection of the plots suggest that, in general* (see note below), there are no real differences between the distributions and a typical plot is shown above (again this is for a sine wave of period n=20). The black plots are

• optimised multiplier for ucl of sine wave with amplitude of 1
• optimised multiplier for lcl of sine wave with amplitude of 1
• optimised multiplier for ucl of sine wave with variable amplitude
  
• optimised multiplier for lcl of sine wave with variable amplitude

The red line is a kernal density plot of all four of the above combined into a single, global distribution for that period sine wave.

Obviously just looking at plots is not statistically robust, so the next thing I did was to run Kolmogorov-Smirnov tests in R to see if the null hypothesis that the four separate distributions come from the same distribution can be rejected. For this purpose I used a p-value of 0.05 for the significance level. The comparative test runs were

1. ucl sine wave amplitude 1 and lcl sine wave amplitude 1
2. ucl sine wave variable amplitude and lcl sine wave variable amplitude
3. ucl sine wave amplitude 1 and ucl sine wave variable amplitude
4. lcl sine wave amplitude 1 and lcl sine wave variable amplitude

The logic behind this choice is:-
if A=B and C=D and A=C and B=D then A=B=C=D.

The actual test in R that was used was the Bootstrap Kolmogorov-Smirnov package ks.boot(), the R documentation description for which is quoted below:-

"This function executes a bootstrap version of the univariate Kolmogorov-Smirnov test which provides correct coverage even when the distributions being compared are not entirely continuous. Ties are allowed with this test unlike the traditional Kolmogorov-Smirnov test"

It is obvious from the discrete nature of the raw data being tested that this test is more appropriate than the traditional Kolmogorov-Smirnov test, but more importantly the symmetrical nature of a sine wave almost guarantees that there are likely to be ties in the data, hence the choice of this test. The number of bootstraps performed for each test was n=1000. The p-value was sufficiently low to reject the null hypothesis for the following periods (numbered bullet points correspond to tests identified above)

1. Periods 16 ,24, 32, 40 and 48
2. Periods 16, 24, 27, 32, 40, 45, 48, 49
3. Periods 38 and 49
4. Periods 38 and 49
   

*The exceptions to the general pattern mentioned above will be the subject of a later blog entry.

Monte Carlo Optimisation of Control Limits #1

The overnight run went without a hitch and I now have a text file of 3000 rows (the number of iterations in the MC routine) and columns for the ucl and lcl multipliers for sine waves of periods 10 to 50 inclusive.

As a first simple analysis of these I have used the simple R commands boxplot(ucl) and plot(density(ucl)) to plot graphs for quick visual examination of the distribution of these multipliers, each graph being either ucl or lcl for one sine wave period only. A generic example of boxplot(ucl) is the first graph above (this happens to be ucl for sine wave period 20). As can be seen, there are no outliers beyond the whiskers and it would appear that the distribution is evenly spread about the thick median line.

The second graph is plot(density(ucl)), a plot of the kernal density. Note the steep drop offs at about 2.00 and 2.15. This is a pattern that repeats for almost all periods for both the ucl and lcl, although of course with different values at the drop off points. This second graph suggests that in fact two multipliers could be used to create a "zone" ucl and lcl. It can be conjectured that

• if prices are outside this zone, either to the upside of the ucl zone or downside of the lcl zone, there would be a high probability that in fact prices are trending,
• whilst if prices are between the zones they could be considered to be in a sideways channel
  
• whilst if prices are actually between the lines that create either zone they could be considered to be in a high probability area for a price reversal in a sideways market, or alternatively in a zone where a breakout from a sideways market is imminent.

However, before this particular idea is investigated further there is one more run of the MC routine I would like to do. The above results come from a run where the sine wave amplitude is one. Of course real prices, if they are in a sideways channel, won't be oscillating with this amplitude but will have variable amplitudes. Therefore the MC routine shall be run again with sine waves of various amplitudes, and the results compared with those obtained so far.

First step in optimising Tukey Chart

I have now written a simple Monte Carlo routine in Octave script to loop through a series of sine waves of periods 10 to 50 inclusive, with an amplitude of 1, to extract a series of optimised ucl and lcl multipliers such that each multiplier ensures that the ucl and lcl for that particular sine wave has a value of 1, i.e. perfectly encloses said sine wave between the control limits. The code is certainly "clunky" and not optimised for speed, but this is not important as it will only need to be run a few times. I calculate that it will take about 10 hours to obtain a total sample of 3000 multipliers for each sine wave period. I shall run this overnight tonight while I am sleeping and have the results ready for analysis from tomorrow.

Applying Control Charts to price series

For my first substantive post I thought I would post a screenshot (http://www.gnuplot.info) of what I'm currently working on. The red line is a sine wave and the lines of interest are the yellow upper control line (ucl) and the blue lower control line (lcl). For a description of what the ucl and lcl are see http://gunston.gmu.edu/708/frTukey.asp

The basic idea is to apply the concept of a Tukey Control Chart to prices to determine whether prices are moving sideways or not. To that end, the above sine wave models a sideways moving market and represents the pre-intervention period for constructing the ucl and lcl. The idea is that if prices are outside either of the control limits then prices are NOT moving sideways in a cyclic fashion and by default must be either trending up or down. This in turn will determine what trading approach to take given current market conditions and what indicators to use.

The above ucl and lcl do not use Tukey's recommended 1.5 multiplier for the Fourth Spread but instead use a different multiplier depending on the period of the sine wave. This has been determined simply by looking at numerous charts and adjusting the multiplier so that the control lines are as close as possible, visually, to the peaks and troughs of the sine wave. Future work to be done is to write a program/routine - I use Octave (http://www.gnu.org/software/octave) and C++ functions written for Octave, and occasionally the statistical software R (http://www.r-project.org) in my development work - to optimise this multiplier for each period length. For help in coding this I have posted on the Octave Nabble forum and I am waiting for any responses. The hope/expectation is that on a real price chart the ucl and lcl will indicate support and resistance areas when prices are in a sideways moving channel as well as indicating the fact that current price is actually moving in a fashion that, statistically speaking, is indistinguishable from a sideways moving channel.

Reasons for starting this blog

The purpose of this blog is for me to document the development of a trading system that I intend to use for my own trading and to hopefully receive constructive criticism from those who care to offer it. I have been testing different elements of this system on and off and in isolation from each other for some time now on my computer and in "demo account" mode with Collective2 (press clickable Collective2 icon at the top of the blog), with an occasional live trade, and I now want to completely overhaul and improve each element and integrate them so that together they constitute a complete trading system. At the moment I don't envisage that this system will ever be 100% mechanical or fully automated in the sense of a trading robot or EA, but I do hope that this process of development and possible feedback from readers will lead to a set of indicators/rules/guidelines that will prove to be profitable in the future.

Another reason for starting the blog is the idea that it will become my online trading journal.