Expressing an Indicator in Neural Net Form

Recently I started investigating relative rotation graphs with a view to perhaps implementing a version of this for use on forex currency pairs. The underlying idea of a relative rotation graph is to plot an asset's relative strength compared to a benchmark and the momentum of this relative strength and to plot this in a form similar to a Polar coordinate system plot, which rotates around a zero point representing zero relative strength and zero relative strength momentum. After some thought it occurred to me that rather than using relative strength against a benchmark I could use the underlying relative strengths of the individual currencies, as calculated by my currency strength indicator, against each other. Furthermore, these underlying log changes in the individual currencies can be normalised using the ideas of brownian motion and then averaged together over different look back periods to create a unique indicator.

This indicator was coded up in the "traditional" way that will be familiar to anybody who has ever tried coding a trading indicator in any coding language. However, I thought that if the calculation of this indicator could be expressed in the form of a Feedforward neural network then the optimisation opportunities available using backpropagation and regularization could be used to tweek the indicator in more useful ways than just varying a look back length and amount of averaging. After some work I was able to make this work in just these two lines of Octave code:

indicator_middle_layer = tanh( full_feature_matrix * input_weights ) ;
indicator_nn_output = tanh( indicator_middle_layer * output_weights ) ; 

Of course, prior to calling these two lines of code, there is some feature engineering to create the input full_feature_matrix, and the input weights and output_weights matrices taken together are mathematically equivalent to the original indicator calculations. Finally, because this is a neural net expression of the indicator, the non-linear tanh activation function is applied to the hidden middle and output layers of the net.

The following plot shows the original indicator in black and the neural net version of it in blue 

over the data shown in this plot of 10 minute bars of the EURUSD forex pair. 

The red indicator in the plot above is the 1 bar momentum of the neural net indicator plot.

To judge the quality of this indicator I used the entropy measure (measured over 14,000+ 10 minute bars of EURUSD), the results of which are shown next.

entropy_indicator_original = 0.9485
entropy_indicator_nn_output = 0.9933

An entropy reading of 0.9933 is probably as good as any trading indicator could hope to achieve (a perfect reading is 1.0) and so the next thing was to quickly back test the indicator performance. Based on the logic of the indicator the obvious long (short) signals are being above (below) the zeroline, or equivalently the sign of the indicator, and for good measure I also tested the sign of the momentum and some simple moving averages thereof.

The following plot shows the equity curves of this quick test where it is visually clear that the blue equity curves are "the best" when plotted in relation to the black "buy and hold" equivalent equity curve. These represent the equity curves of a 3 bar simple moving average of the 1 bar momentum of both the original formulation of the indicator and the neural net implementation. I would point out that these equity curves represent the theoretical equity resulting from trading the London session after 9:00am BST and stopping at 7:00am EST (usually about noon BST) and then starting trading again at 9:00am EST until 17:00pm EST. This schedule avoids the opening sessions (7:00 to 9:00am) in both London and New York because, from my observations of many OHLC charts such as shown above, there are frequently wild swings where price is being pushed to significant points such as volume profile clusters, accumulations of buy/sell orders etc. and in my opinion no indicator can be expected to perform well in that sort of environment. Also avoided are the hours prior to 7:00am BST, i.e. the Asian session or overnight session.    

Although these equity curves might not, at first glance, be that impressive, especially as they do not account for trading costs etc. my intent on doing these tests was to determine the configuration of a final "decision" output layer to be added to the neural net implementation of the indicator. A 3 bar simple moving average of the 1 bar momentum implies the necessity to include 4 consecutive readings of the indicator output as input to a final " decision" layer. The following simple, hand-drawn sketch shows what I mean:

A discussion of this will be the subject of my next post.

Currency Strength Revisited

Recently I responded to a Quantitative Finance forum question here, where I invited the questioner to peruse certain posts on this blog. Apparently the posts do not provide enough information to fully answer the question (my bad) and therefore this post provides what I think will suffice as a full and complete reply, although perhaps not scientifically rigorous.

The original question asked was "Is it possible to separate or decouple the two currencies in a trading pair?" and I believe what I have previously described as a "currency strength indicator" does precisely this (blog search term ---> https://dekalogblog.blogspot.com/search?q=currency+strength+indicator). This post outlines the rationale behind my approach.

Take, for example, the GBPUSD forex pair, and further give it a current (imaginary) value of 1.2500. What does this mean? Of course it means 1 GBP will currently buy you 1.25 USD, or alternatively 1 USD will buy you 1/1.25 = 0.8 GBP. Now rather than write GBPUSD let's express GBPUSD as a ratio thus:- GBP/USD, which expresses the idea of "how many USD are there in a GBP?" in the same way that 9/3 shows how many 3s there are in 9. Now let's imagine at some time period later there is a new pair value, a lower case "gbp/usd" where we can write the relationship

                    (1)     ( GBP / USD ) * ( G / U ) = gbp / usd

to show the change over the time period in question. The ( G / U ) term is a multiplicative term to show the change in value from old GBP/USD 1.2500 to say new value gbp/usd of 1.2600, 

e.g.                ( G / U ) == ( gbp / usd ) / ( GBP / USD ) == 1.26 / 1.25 == 1.008

from which it is clear that the forex pair has increased by 0.8% in value over this time period. Now, if we imagine that over this time period the underlying, real value of USD has remained unchanged this is equivalent to setting the value U in ( G / U ) to exactly 1, thereby implying that the 0.8% increase in the forex pair value is entirely attributable to a 0.8% increase in the underlying, real value of GBP, i.e. G == 1.008. Alternatively, we can assume that the value of GBP remains unchanged,

 e.g.                G == 1, which means that U == 1 / 1.008 == 0.9921

which implies that a ( 1 - 0.9921 ) == 0.79% decrease in USD value is responsible for the 0.8% increase in the pair quote.

Of course, given only equation (1) it is impossible to solve for G and U as either can be arbitrarily set to any number greater than zero and then be compensated for by setting the other number such that the constant ( G / U ) will match the required constant to account for the change in the pair value.

However, now let's introduce two other forex pairs (2) and (3) and thus we have:-

                    (1)     ( GBP / USD ) * ( G / U ) = gbp / usd

                    (2)     ( EUR / USD ) * ( E / U ) = eur / usd

                    (3)     ( EUR / GBP ) * ( E / G ) = eur / gbp

We now have three equations and three unknowns, namely G, E and U, and so this system of equations could be laboriously, mathematically solved by substitution. 

However, in my currency strength indicator I have taken a different approach. Instead of solving mathematically I have written an error function which takes as arguments a list of G, E, U, ... etc. for all currency multipliers relevant to all the forex quotes I have access to, approximately 47 various crosses which themselves are inputs to the error function, and this function is supplied to Octave's fminunc function to simultaneously solve for all G, E, U, ... etc. given all forex market quotes. The initial starting values for all G, E, U, ... etc. are 1, implying no change in values across the market. These starting values consistently converge to the same final values for G, E, U, ... etc for each separate period's optimisation iterations.

Having got all G, E, U, ... etc. what can be done? Well, taking G for example, we can write

                    (4)     GBP * G = gbp

for the underlying, real change in the value of GBP. Dividing each side of (4) by GBP and taking logs we get

                    (5)     log( G ) = log( gbp / GBP )

i.e. the log of the fminunc returned value for the multiplicative constant G is the equivalent of the log return of GBP independent of all other currencies, or as the original forum question asked, the (change in) value of GBP separated or decoupled the from the pair in which it is quoted.

Of course, having the individual log returns of separated or decoupled currencies, there are many things that can be done with them, such as:-

• create indices for each currency
• apply technical analysis to these separate indices
• intermarket currency analysis
• input to machine learning (ML) models
• possibly create new and unique currency indicators

Examples of the creation of "alternative price charts" and indices are shown below

where the black line is the actual 10 minute closing prices of GBPUSD over the last week (13th to 18th August) with the corresponding GBP price (blue line) being the "alternative" GBPUSD chart if U is held at 1 in the ( G / U ) term and G allowed to be its derived, optimised value, and the USD price (red line) being the alternative chart if G is held at 1 and U allowed to be its derived, optimised value.

This second chart shows a more "traditional" index like chart

where the starting values are 1 and both the G and U values take their derived values. As can be seen, over the week there was upwards momentum in both the GBP and USD, with the greater momentum being in the GBP resulting in a higher GBPUSD quote at the end of the week. If, in the second chart the blue GBP line had been flat at a value of 1 all week, the upwards momentum in USD would have resulted in a lower week ending quoted value of GBPUSD, as seen in the red USD line in the first chart. Having access to these real, decoupled returns allows one to see through the given, quoted forex prices in the manner of viewing the market as though through X-ray vision. 

I hope readers find this post enlightening, and if you find some other uses for this idea, I would be interested in hearing how you use it.
 

A Forex Pair Snapshot Chart

After yesterday's Heatmap Plot of Forex Temporal Clustering post I thought I would consolidate all the chart types I have recently created into one easy, snapshot overview type of chart. Below is a typical example of such a chart, this being today's 10 minute EUR_USD forex pair chart up to a few hours after the London session close (the red vertical line).

The top left chart is a Market/Volume Profile Chart with added rolling Value Area upper and lower bounds (the cyan, red and white lines) and also rolling Volume Weighted Average Price with upper and lower standard deviation lines (magenta).

The bottom left chart is the turning point heatmap chart as described in yesterday's post.

The two rightmost charts are also Market/Volume Profile charts, but of my Currency Strength Candlestick Charts based on my Currency Strength Indicator. The upper one is the base currency, i.e. EUR, and the lower is the quote currency. 

The following charts are the same day's charts for:

GBP_USD,

USD_CHF

and finally USD_JPY

The regularity of the turning points is easily seen in the lower lefthand charts although, of course, this is to be expected as they all share the USD as a common currency. However, there are also subtle differences to be seen in the "shadows" of the lighter areas.

For the nearest future my self-assigned task will be to observe the forex pairs, in real time, through the prism of the above style of chart and do some mental paper trading, and perhaps some really small size, discretionary live trading, in additional to my normal routine of research and development.

Candlestick Pattern Scanner Functions

Since my last currency strength candlestick chart post it seemed to make sense to be able to scan said charts for signals, so below is the code for two Octave functions which act as candlestick pattern scanners. The code is fully vectorised and self-contained, and on my machine they can scan more than 300,000 OHLC bars for 27/29 separate patterns in less than 0.5 seconds. Both functions have a self-explanatory help and within the body of the code there are ample comments which describe the patterns. Enjoy!

Bullish Reversal Indicator

## Copyright (C) 2020 dekalog
## 
## This program is free software: you can redistribute it and/or modify it
## under the terms of the GNU General Public License as published by
## the Free Software Foundation, either version 3 of the License, or
## (at your option) any later version.
## 
## This program is distributed in the hope that it will be useful, but
## WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
## GNU General Public License for more details.
## 
## You should have received a copy of the GNU General Public License
## along with this program.  If not, see
## .

## -*- texinfo -*- 
## @deftypefn {} {@var{bullish_signal_matrix} =} candle_bullish_reversal (@var{high}, @var{low}, @var{close}, @var{open}, @var{downtrend}, @var{avge_hi_lo_range})
##
## The HIGH, LOW, CLOSE and OPEN should be vectors of equal length, and are required.
##
## Internally the DOWNTREND is determined as a bar's CLOSE being lower than the CLOSE
## of the bar 5 bars previously. This can be over-ruled by an optional, user supplied
## vector DOWNTREND of zeros and ones, where one(s) represent a bar in a DOWNTREND.
## If a DOWNTREND vector consists solely of ones, this effectively turns off the
## discriminative ability of the DOWNTREND condition as all bars will be considered
## to be DOWNTREND bars. If the DOWNTREND vector consists solely of zeros, no bar
## will be classified as being in a DOWNTREND and those candle patterns that require a 
## DOWNTREND as a condition will not be indicated.
##
## The AVGE_HI_LO_RANGE is calculated as a rolling 5 bar simple moving average of
## the HI-LO range of bars. A bar is considered a "long line" if its HI-LO range
## is greater than the AVGE_HI_LO_RANGE. This can also be over-ruled, as above, by
## an optional, user supplied vector AVGE_HI_LO_RANGE, which consists of zeros and
## ones, with ones representing "long line" bars. An AVGE_HI_LO_RANGE vector of all
## ones or zeros will have a similar effect as described for the DOWNTREND vector.
##
## The candlestick pattern descriptions are predominantly taken from
##
## https://www.candlescanner.com
##
## and the bullish only patterns implemented are:
##
## Bullish Reversal High Reliability
##
##     01 - Bullish Abandoned Baby
##
##     02 - Concealing Baby Swallow
##
##     03 - Kicking
##
##     04 - Morning Doji Star
##
##     05 - Morning Star
##
##     06 - Piercing Line
##
##     07 - Three Inside Up
##
##     08 - Three Outside Up
##
##     09 - Three White Soldiers
##
## Bullish Reversal Moderate Reliability
##
##     10 - Breakaway
##
##     11 - Counter Attack
##
##     12 - Doji Star
##
##     13 - Dragonfly Doji
##
##     14 - Engulfing
##
##     15 - Gravestone Doji
##
##     16 - Harami Cross
##
##     17 - Homing Pigeon
##
##     18 - Ladder Bottom
##
##     19 - Long Legged Doji
##
##     20 - Matching Low
##
##     21 - Meeting Lines
##
##     22 - Stick Sandwich
##
##     23 - Three Stars in the South
##
##     24 - Tri Star
##
##     25 - Unique Three River Bottom
##
## Bullish Reversal Low Reliability
##
##     26 - Belt Hold
##
##     27 - Hammer
##
##     28 - Harami
##
##     29 - Inverted Hammer
##
## The output BULLISH_SIGNAL_MATRIX has a row length the same as the OHLC
## input vectors and 29 columns. The matrix is a zero filled matrix with the
## value 1 when a candle pattern is indicated. The row ix of the value is the
## row ix of the last candle in the pattern. The column ix corresponds to the
## pattern index given above.
##
## @seealso{candle_bearish_reversal}
## @end deftypefn

## Author: dekalog 
## Created: 2020-08-10

function bullish_signal_matrix = candle_bullish_reversal ( varargin )
 
if ( nargin < 4 || nargin > 6 )
  print_usage () ;
elseif ( nargin == 4 )
  high = varargin{1} ; low = varargin{2} ; close = varargin{3} ; open = varargin{4} ;
  downtrend = close < shift( close , 5 ) ;
  avge_hi_lo_range = filter( [0.2 ; 0.2 ; 0.2 ; 0.2 ; 0.2 ] , 1 , high .- low ) ; ## 5 bar simple moving average
elseif ( nargin == 5 )
  high = varargin{1} ; low = varargin{2} ; close = varargin{3} ; open = varargin{4} ;
  downtrend = varargin{5} ;
  avge_hi_lo_range = filter( [0.2 ; 0.2 ; 0.2 ; 0.2 ; 0.2 ] , 1 , high .- low ) ; ## 5 bar simple moving average
elseif ( nargin == 6 )
  high = varargin{1} ; low = varargin{2} ; close = varargin{3} ; open = varargin{4} ;
  downtrend = varargin{5} ; avge_hi_lo_range = varargin{6} ;
endif
 
bullish_signal_matrix = zeros( size( close , 1 ) , 29 ) ;

## pre-compute some basic vectors for re-use below, trading memory use for
## speed of execution
downtrend_1 = shift( downtrend , 1 ) ;
downtrend_2 = shift( downtrend , 2 ) ;
downtrend_3 = shift( downtrend , 3 ) ;
long_line = ( high .- low ) > shift( avge_hi_lo_range , 1 ) ;
long_line_1 = shift( long_line , 1 ) ;
long_line_2 = shift( long_line , 2 ) ;
long_line_3 = shift( long_line , 3 ) ;
long_line_4 = shift( long_line , 4 ) ;
open_1 = shift( open , 1 ) ;
open_2 = shift( open , 2 ) ;
open_3 = shift( open , 3 ) ;
open_4 = shift( open , 4 ) ;
open_5 = shift( open , 5 ) ;
high_1 = shift( high , 1 ) ;
high_3 = shift( high , 3 ) ;
low_1 = shift( low ,1 ) ;
low_2 = shift( low ,2 ) ;
low_3 = shift( low , 3 ) ;
close_1 = shift( close , 1 ) ;
close_2 = shift( close , 2 ) ;
close_3 = shift( close , 3 ) ;
close_4 = shift( close , 4 ) ;
black_body = close < open ;
black_body_1 = shift( black_body , 1 ) ;
black_body_2 = shift( black_body , 2 ) ;
black_body_3 = shift( black_body , 3 ) ;
black_body_4 = shift( black_body , 4 ) ;
white_body = close > open ;
white_body_1 = shift( white_body , 1 ) ;
white_body_2 = shift( white_body , 2 ) ;
doji = ( close == open ) ;
doji_1 = shift( doji , 1 ) ;
doji_2 = shift( doji , 2 ) ;
body_high = max( [ open , close ] , [] , 2 ) ;
body_high_1 = shift( body_high , 1 ) ;
body_high_2 = shift( body_high , 2 ) ;
body_low = min( [ open , close ] , [] , 2 ) ;
body_low_1 = shift( body_low , 1 ) ;
body_low_2 = shift( body_low , 2 ) ;
body_range = body_high .- body_low ;
body_range_1 = shift( body_range , 1 ) ;
body_range_2 = shift( body_range , 2 ) ;
body_midpoint = ( body_high .+ body_low ) ./ 2 ;
body_midpoint_1 = shift( body_midpoint , 1 ) ;
body_midpoint_2 = shift( body_midpoint , 2 ) ;
wick_gap_up = ( low > high_1 ) ;
wick_gap_down = ( high < low_1 ) ;
wick_gap_down_1 = shift( wick_gap_down , 1 ) ;
black_marubozu = ( high == open ) .* ( low == close ) ;
black_marubozu_1 = shift( black_marubozu , 1 ) ;
black_marubozu_2 = shift( black_marubozu , 2 ) ;
black_marubozu_3 = shift( black_marubozu , 3 ) ;
white_marubozu = ( high == close ) .* ( low == open ) ;

## 01 - Bullish Abandoned Baby
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     a doji candle
##     the high price below the prior low price
## Third candle
##     white body
##     the low price above the prior high price
bullish_signal_matrix(:,1) = downtrend_2 .* black_body_2 .* ...
                              doji_1 .* wick_gap_down_1 .* ...
                              white_body .* wick_gap_up ; 
## 02 - Concealing Baby Swallow
## First candle
##     a Black Marubozu candle in a downtrend
## Second candle
##     a Black Marubozu candle
##     candle opens within the prior candle's body
##     candle closes below the prior closing price
## Third candle
##     a High Wave basic candle with no lower shadow
##     candle opens below the prior closing price
##     upper shadow enters the prior candle's body
## Fourth candle
##     black body
##     candle’s body engulfs the prior candle’s body including the shadows
bullish_signal_matrix(:,2) = downtrend_3 .* black_marubozu_3 .* ...
                              black_marubozu_2 .* (open_2 < high_3) .* (open_2 > low_3) .* (close_2 < close_3) .* ...
                              (high_1 .- body_high_1 >= 3.*body_range_1) .* (low_1 == body_low_1) .* (open_1 < close_2) .* (high_1 > body_low_2) .* ...
                              black_body .* (open > high_1) .* (close < low_1) ;
## 03 - Kicking
## First candle
##     a Black Marubozu
##     appears on as a long line
## Second candle
##     a White Marubozu
##     price gaps upward
##     appears on as a long line
bullish_signal_matrix(:,3) = black_marubozu_1 .* long_line_1 .* ...
                              white_marubozu .* (low > high_1) .* long_line ;
## 04 - Morning Doji Star
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     a doji candle
##     a doji body below the previous candle body
##     the high price above the previous candle low price
## Third candle
##     white body
##     candle body above the previous candle body
##     the closing price above the midpoint of the first candle body
bullish_signal_matrix(:,4) = downtrend_2 .* black_body_2 .* ...
                              doji_1 .* (open_1 < body_low_2) .* (high_1 > low_2) .* ...
                              white_body .* (body_low > body_high_1) .* (close > body_midpoint_2) ;
## 05 - Morning Star
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     white or black body
##     the candle body is located below the prior body
## Third candle
##     white body
##     the candle body is located above the prior body
##     the candle closes at least halfway up the body of the first line
bullish_signal_matrix(:,5) = downtrend_2 .* black_body_2 .* ...
                              (body_high_1 < body_low_2) .* ...
                              white_body .* (body_low > body_high_1) .* (close >= body_midpoint_2) ;
## 06 - Piercing Line
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     white body
##     the opening below or equal of the prior low
##     the closing above the midpoint of the prior candle's body
##     the closing below the previous opening
bullish_signal_matrix(:,6) = downtrend_1 .* black_body_1 .* ...
                              white_body .* (open <= low_1) .* (close > body_midpoint_1) .* (close < open_1) ;
## 07 - Three Inside Up
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     white body
##     the candle body is engulfed by the prior candle body
## Third candle
##     the closing price is above the previous closing price
bullish_signal_matrix(:,7) = downtrend_2 .* black_body_2 .* ...
                              white_body_1 .* (body_high_1 < body_high_2) .* (body_low_1 > body_low_2) .* ...
                              (close > close_1) ;
## 08 - Three Outside Up
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     white body
##     candle’s body engulfs the prior (black) candle’s body
## Third candle
##     closing price above the previous closing price
##     white body
bullish_signal_matrix(:,8) = downtrend_2 .* black_body_2 .* ...
                              white_body_1 .* (body_high_1 > body_high_2) .* (body_low_1 < body_low_2) .* ...
                              (close > close_1) .* white_body ;
## 09 - Three White Soldiers
## First candle
##     a candle in a downtrend
##     white body
## Second candle
##     white body
##     the opening price within the previous body
##     the closing price above the previous closing price
## Third candle
##     white body
##     the opening price within the previous body
##     the closing price above the previous closing price
bullish_signal_matrix(:,9) = downtrend_2 .* white_body_2 .* ...
                              white_body_1 .* (open_1 < body_high_2) .* (open_1 > body_low_2) .* (close_1 > close_2) .* ...
                              white_body .* (open < body_high_1) .* (open > body_low_1) .* (close > close_1) ; 
## 10 - Breakaway
## First candle
##     a tall black candle
## Second candle
##     a black candle
##     candle opens below the previous closing price (downward price gap, shadows can overlap)
## Third candle
##     a white or black candle
##     candle opens below the previous opening price
## Fourth candle
##     a black candle
##     candle closes below the previous closing price
## Fifth candle
##     a tall white candle
##     candle opens above the previous closing price
##     candle closes above the second line's opening price and below the first line's opening price
bullish_signal_matrix(:,10) = black_body_4 .* long_line_4 .*...
                               black_body_3 .* (open_3 < close_4) .* ...
                               (open_2 < open_3) .* ...
                               black_body_1 .* (close_1 < close_2) .* ...
                               white_body .* long_line .* (open > close_1) .* (close > open_3) .* (close < open_4) ; 
## 11 - Counter Attack
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     white body
##     the opening price is lower than the previous closing price
##     the closing price is at or higher than the previous closing price
bullish_signal_matrix(:,11) = downtrend_1 .* black_body_1 .* ...
                               white_body .* (open < close_1) .* (close >= close_1) ;
## 12 - Doji Star
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     a doji candle
##     a body below the first candle's body
bullish_signal_matrix(:,12) = downtrend_1 .* black_body_1 .* ...
                               doji .* (open < body_low_1) ; 
## 13 - Dragonfly Doji
##     Opening, closing and maximum prices are the same or very similar
##     Long lower shadow
##     appears on as a long line
bullish_signal_matrix(:,13) = (open == close) .* (close == high) .* long_line ;

## 14 - Engulfing
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     white body
##     candle's body engulfs the prior (black) candle's body
bullish_signal_matrix(:,14) = downtrend_1 .* black_body_1 .* ...
                               white_body .* (body_high > body_high_1) .* (body_low < body_low_1) ; 
## 15 - Gravestone Doji
##     Opening, closing and minimum prices are the same or very similar
##     Long upper shadow
##     appears on as a long line
bullish_signal_matrix(:,15) = (open == close) .* (close == low) .* long_line ; 

## 16 - Harami Cross
## First candle
##     a candle in a downtrend
##     black body
##     appears on as a long line
## Second candle
##     a doji candle with two shadows
##     the candle (including shadows) is engulfed by the previous candle's body
bullish_signal_matrix(:,16) = downtrend_1 .* black_body_1 .* long_line_1 .* ...
                               doji .* (high > close) .* (low < close) .* (high < body_high_1) .* (low > body_low_1) ;
## 17 - Homing Pigeon
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     black body
##     candle’s body engulfed by the prior candle’s body
bullish_signal_matrix(:,17) = downtrend_1 .* black_body_1 .* ...
                               black_body .* (body_high < body_high_1) .* (body_low > body_low_1) ;
## 18 - Ladder Bottom
## First candle
##     a tall black candle
## Second candle
##     a tall black candle
##     candle opens below the previous opening price
## Third candle
##     a tall black candle
##     candle opens below the previous opening price
## Fourth candle
##     a black candle
##     candle closes below the previous closing price
##     candle has a long upper shadow
## Fifth candle
##     a tall white candle
##     candle opens above the previous opening price
##     candle closes at or above the third line's opening price and below the first line's opening price
bullish_signal_matrix(:,18) = black_body_4 .* long_line_4 .* ...
                               black_body_3 .* (open_3 < open_4) .* long_line_3 .* ...
                               black_body_2 .* (open_2 < open_3) .* long_line_2 .* ...
                               black_body_1 .* (close_1 < close_2) .* ((high_1 .- body_high_1) > body_range_1) .* ...
                               white_body .* (open > open_1) .* (close >= open_2) .* (close < open_5) ; 
## 19 - Long Legged Doji
##     a doji candle
##     opening and closing prices are the same or similar
##     upper and lower shadow are very long
##     body is located in the middle of the candle or nearly mid-range
##     appears on as a long line
bullish_signal_matrix(:,19) = doji .* ((high .- close) > 0) .* ((close .- low) > 0) .* long_line ;

## 20 - Matching Low
## First candle
##     a candle in a downtrend
##     black body
##     no lower shadow
##     appears as a long line
## Second candle
##     black body
##     the opening price is below the previous opening price
##     the closing price is at the level of the previous closing price
##     no lower shadow
bullish_signal_matrix(:,20) = downtrend_1 .* black_body_1 .* (close_1 == low_1) .* long_line_1 .* ...
                               black_body .* (open < open_1) .* (close == close_1) .* (close == low) ;
## 21 - Meeting Lines
## First candle
##     a candle in a downtrend
##     black body
##     appears as a long line
## Second candle
##     white body
##     the closing price is equal to the previous closing price
bullish_signal_matrix(:,21) = downtrend_1 .* black_body_1 .* long_line_1 .* ...
                               white_body .* (close == close_1) ;
## 22 - Stick Sandwich
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     white body
##     the opening price is higher than the previous closing price
##     the closing price is higher than the previous opening price
## Third candle
##     black_body
##     the opening price is higher than the previous closing price
##     the closing price is equal to or higher than first line close
bullish_signal_matrix(:,22) = downtrend_2 .* black_body_2 .* ...
                               white_body_1 .* (open_1 > close_2) .* (close_1 > open_2) .* ...
                               black_body .* (open > close_1) .* (close >= close_2) ;
## 23 - Three Stars in the South
## First candle
##     a candle in a downtrend
##     black body
##     long lower shadow
## Second candle
##     black body
##     the opening below the prior opening
##     the closing below or at the prior closing
##     the low above the prior low
## Third candle
##     a marubozu candle with black body
##     appears as a short line
##     a candle is located within the prior candle
bullish_signal_matrix(:,23) = downtrend_2 .* black_body_2 .* ((body_low_2 .- low_2) > body_range_2) .* ...
                               black_body_1 .* (open_1 < open_2) .* (close_1 <= close_2) .* (low_1 > low_2) .* ...
                               black_marubozu .* (high < high_1) .* (low > low_1) ;
## 24 - Tri Star
## First candle
##     a doji candle in a downtrend
## Second candle
##     a doji candle
##     a body below the prior body
## Third candle
##     a doji candle
##     a body above the prior body
bullish_signal_matrix(:,24) = downtrend_2 .* doji_2 .* ...
                               doji_1 .* (open_1 < close_2) .* ...
                               doji .* (open > close_1) ;
## 25 - Unique Three River Bottom
## First candle
##     black candle
##     a candle in a downtrend
## Second candle
##     black candle
##     a body within the prior body
##     the lower shadow is at least twice longer than the body
##     the low price below the prior low price
## Third candle
##     white candle
##     a body located below the prior body
##     the low price above the prior low price
bullish_signal_matrix(:,25) = downtrend_2 .* black_body_2 .* ...
                               black_body_1 .* (body_high_1 < body_high_2) .* (body_low_1 > body_low_2) .* ((body_low_1 .- low_1) >= body_range_1) .* (low_1 < low_2) .* ...
                               white_body .* (body_high < body_low_1) .* (low > low_1) ;
## 26 - Belt Hold
##     white body
##     no lower shadow
##     short upper shadow
##     appears as a long line
bullish_signal_matrix(:,26) = downtrend .* white_body .* (open == low) .* long_line ;

## 27 - Hammer
##     downtrend
##     white or black candle with a small body
##     no upper shadow or the shadow cannot be longer than the body
##     lower shadow between two to three times longer than the body
##     if the gap is created at the opening or at the closing, it makes the signal stronger
##     appears as a long line
bullish_signal_matrix(:,27) = downtrend .* (doji == 0) .* ((high .- body_high) < body_range) .* ((body_low .- low) > 2.*body_range) .* long_line ;

## 28 - Harami
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     white body
##     candle's body engulfed by the prior (black) candle's body
bullish_signal_matrix(:,28) = downtrend_1 .* black_body_1 .* ...
                               white_body .* (body_high < body_high_1) .* (body_low > body_low_1) ;
## 29 - Inverted Hammer 
## First candle
##     a candle in a downtrend
##     black body
## Second candle
##     a white or black candle with a small body
##     no lower shadow or the shadow cannot be longer than then body
##     upper shadow at least 2.5 times longer than the body
##     the open below or at the level of the previous candle's close
bullish_signal_matrix(:,29) = downtrend_1 .* black_body_1 .* ...
                               (doji == 0 ) .* ((body_low .- low) < body_range) .* ((high .- body_high) >= 2.5.*body_range) .* (open <= close_1) ;
                               
bullish_signal_matrix( 1 : 10 , : ) = 0 ;

endfunction

and Bearish Reversal Indicator

## Copyright (C) 2020 dekalog
## 
## This program is free software: you can redistribute it and/or modify it
## under the terms of the GNU General Public License as published by
## the Free Software Foundation, either version 3 of the License, or
## (at your option) any later version.
## 
## This program is distributed in the hope that it will be useful, but
## WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
## GNU General Public License for more details.
## 
## You should have received a copy of the GNU General Public License
## along with this program.  If not, see
## .

## -*- texinfo -*- 
## @deftypefn {} {@var{bearish_signal_matrix} =} candle_bearish_reversal (@var{high}, @var{low}, @var{close}, @var{open}, @var{uptrend}, @var{avge_hi_lo_range})
##
## The HIGH, LOW, CLOSE and OPEN should be vectors of equal length, and are required.
##
## Internally the UPTREND is determined as a bar's CLOSE being higher than the CLOSE
## of the bar 5 bars previously. This can be over-ruled by an optional, user supplied
## vector UPTREND of zeros and ones, where one(s) represent a bar in an UPTREND.
## If an UPTREND vector consists solely of ones, this effectively turns off the
## discriminative ability of the UPTREND condition as all bars will be considered
## to be UPTREND bars. If the UPTREND vector consists solely of zeros, no bar
## will be classified as being in an UPTREND and those candle patterns that require an 
## UPTREND as a condition will not be indicated.
##
## The AVGE_HI_LO_RANGE is calculated as a rolling 5 bar simple moving average of
## the HI-LO range of bars. A bar is considered a "long line" if its HI-LO range
## is greater than the AVGE_HI_LO_RANGE. This can also be over-ruled, as above, by
## an optional, user supplied vector AVGE_HI_LO_RANGE, which consists of zeros and
## ones, with ones representing "long line" bars. An AVGE_HI_LO_RANGE vector of all
## ones or zeros will have a similar effect as described for the UPTREND vector.
##
## The candlestick pattern descriptions are predominantly taken from
##
## https://www.candlescanner.com
##
## and the bearish only patterns implemented are:
##
## Bearish Reversal High Reliability
##
##     01 - Bearish Abandoned Baby
##
##     02 - Dark Cloud Cover
##
##     03 - Evening Doji Star
##
##     04 - Evening Star
##
##     05 - Kicking
##
##     06 - Three Black Crows
##
##     07 - Three Inside Down
##
##     08 - Three Outside Down
##
##     09 - Upside Gap Two Crows
##
## Bearish Reversal Moderate Reliability
##
##     10 - Advance Block
##
##     11 - Breakaway
##
##     12 - Counter Attack
##
##     13 - Deliberation
##
##     14 - Doji Star
##
##     15 - Dragonfly Doji
##
##     16 - Engulfing
##
##     17 - Gravestone Doji
##
##     18 - Harami Cross
##
##     19 - Identical Three Crows
##
##     20 - Long Legged Doji
##
##     21 - Meeting Lines
##
##     22 - Tri Star
##
##     23 - Two Crows
##
## Bearish Reversal Low Reliability
##
##     24 - Belt Hold 
##
##     25 - Hanging Man
##
##     26 - Harami
##
##     27 - Shooting Star
##
## The output BEARISH_SIGNAL_MATRIX has a row length the same as the OHLC
## input vectors and 27 columns. The matrix is a zero filled matrix with the
## value 1 when a candle pattern is indicated. The row ix of the value is the
## row ix of the last candle in the pattern. The column ix corresponds to the
## pattern index given above.
##
## @seealso{candle_bullish_reversal}
## @end deftypefn

## Author: dekalog 
## Created: 2020-08-10

function bearish_signal_matrix = candle_bearish_reversal ( varargin )

if ( nargin < 4 || nargin > 6 )
  print_usage () ;
elseif ( nargin == 4 )
  high = varargin{1} ; low = varargin{2} ; close = varargin{3} ; open = varargin{4} ;
  uptrend = close > shift( close , 5 ) ;
  avge_hi_lo_range = filter( [0.2 ; 0.2 ; 0.2 ; 0.2 ; 0.2 ] , 1 , high .- low ) ; ## 5 bar simple moving average
elseif ( nargin == 5 )
  high = varargin{1} ; low = varargin{2} ; close = varargin{3} ; open = varargin{4} ;
  uptrend = varargin{5} ;
  avge_hi_lo_range = filter( [0.2 ; 0.2 ; 0.2 ; 0.2 ; 0.2 ] , 1 , high .- low ) ; ## 5 bar simple moving average
elseif ( nargin == 6 )
  high = varargin{1} ; low = varargin{2} ; close = varargin{3} ; open = varargin{4} ;
  uptrend = varargin{5} ; avge_hi_lo_range = varargin{6} ;
endif

bearish_signal_matrix = zeros( size( close , 1 ) , 27 ) ;

## pre-compute some basic vectors for re-use below, trading memory use for
## speed of execution
uptrend_1 = shift( uptrend , 1 ) ;
uptrend_2 = shift( uptrend , 2 ) ;
long_line = ( high .- low ) > shift( avge_hi_lo_range , 1 ) ;
long_line_1 = shift( long_line , 1 ) ;
long_line_4 = shift( long_line , 4 ) ;
open_1 = shift( open , 1 ) ;
open_2 = shift( open , 2 ) ;
open_3 = shift( open , 3 ) ;
high_1 = shift( high , 1 ) ;
high_2 = shift( high , 2 ) ;
high_4 = shift( high , 4 ) ;
low_1 = shift( low , 1 ) ;
close_1 = shift( close , 1 ) ;
close_2 = shift( close , 2 ) ;
close_4 = shift( close , 4 ) ;
black_body = close < open ;
black_body_1 = shift( black_body , 1 ) ;
black_body_2 = shift( black_body , 2 ) ;
white_body = close > open ;
white_body_1 = shift( white_body , 1 ) ;
white_body_2 = shift( white_body , 2 ) ;
white_body_3 = shift( white_body , 3 ) ;
white_body_4 = shift( white_body , 4 ) ;
doji = ( close == open ) ;
doji_1 = shift( doji , 1 ) ;
doji_2 = shift( doji , 2 ) ;
body_high = max( [ open , close ] , [] , 2 ) ;
body_high_1 = shift( body_high , 1 ) ;
body_high_2 = shift( body_high , 2 ) ;
body_low = min( [ open , close ] , [] , 2 ) ;
body_low_1 = shift( body_low , 1 ) ;
body_low_2 = shift( body_low , 2 ) ;
body_midpoint = ( body_high .+ body_low ) ./ 2 ;
body_midpoint_1 = shift( body_midpoint , 1 ) ;
body_midpoint_2 = shift( body_midpoint , 2 ) ;
candle_midpoint = ( high .+ low ) ./ 2 ;
candle_midpoint_1 = shift( candle_midpoint , 1 ) ; 
body_range = body_high .- body_low ;
wick_gap_up = low > shift( high , 1 ) ;
wick_gap_up_1 = shift( wick_gap_up , 1 ) ;
wick_gap_down = high < shift( low , 1 ) ;
black_marubozu = ( high == open ) .* ( low == close ) ;
white_marubozu = ( high == close ) .* ( low == open ) ;
white_marubozu_1 = shift( white_marubozu , 1 ) ;

## 01 - Bearish Abandoned Baby
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     a doji candle
##     the low price above the prior high price
## Third candle
##     black body
##     the high price below the prior low price
bearish_signal_matrix(:,1) = uptrend_2 .* white_body_2 .* ...
                              doji_1 .* wick_gap_up_1 .* ...
                              black_body .* wick_gap_down ; 
## 02 - Dark Cloud Cover
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     black body
##     the opening above or equal of the prior high
##     the closing below the midpoint of the prior candle
##     the closing above the previous opening
bearish_signal_matrix(:,2) = uptrend_1 .* white_body_1 .* ...
                              black_body .* (open >= high_1) .* (close < candle_midpoint_1) .* (close > open_1) ;  
## 03 - Evening Doji Star
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     a doji candle
##     a doji body above the previous candle body
##     the low price below the previous candle high price
## Third candle
##     black body
##     candle body below the previous candle body
##     the closing price below the midpoint of the first candle body
bearish_signal_matrix(:,3) = uptrend_2 .* white_body_2 .* ...
                              doji_1 .* (open_1 > body_high_2) .* (low_1 < high_2) .* ...
                              black_body .* (body_high < close_1) .* (close < body_midpoint_2) ;
## 04 - Evening Star
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     white or black body
##     the candle body is located above the prior body
## Third candle
##     black body
##     the candle body is located below the prior body
##     the candle closes at least halfway down the body of the first line
bearish_signal_matrix(:,4) = uptrend_2 .* white_body_2 .* ...
                              (body_low_1 > body_high_2) .* ...
                              black_body .* (body_high < body_low_1) .* (close <= body_midpoint_2) ;
## 05 - Kicking
## First candle
##     a White Marubozu
##     appears on as a long line
## Second candle
##     a Black Marubozu
##     price gaps downward
##     appears on as a long line
bearish_signal_matrix(:,5) = white_marubozu_1 .* long_line_1 .* ...
                              black_marubozu .* (high < low_1) .* long_line ; 
## 06 - Three Black Crows
## First candle
##     a candle in an uptrend
##     black body
## Second candle
##     black body
##     the opening price within the previous body
##     the closing price below the previous closing price
## Third candle
##     black body
##     the opening price within the previous body
##     the closing price below the previous closing price
bearish_signal_matrix(:,6) = uptrend_2 .* black_body_2 .* ...
                              black_body_1 .* (open_1 < body_high_2) .* (open_1 > body_low_2) .* (close_1 < close_2) .* ...
                              black_body .* (open < body_high_1) .* (open > body_low_1) .* (close < close_1) ; 
## 07 - Three Inside Down
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     black body
##     the candle body is engulfed by the prior candle body
## Third candle
##     black_body
##     the closing price is below the previous closing price
bearish_signal_matrix(:,7) = uptrend_2 .* white_body_2 .* ...
                              black_body_1 .* (body_high_1 < body_high_2) .* (body_low_1 > body_low_2) .* ...
                              black_body .* (close < close_1) ;
## 08 - Three Outside Down
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     black body
##     candle’s body engulfs the prior (white) candle’s body
## Third candle
##     closing price below the previous closing price
##     black body
bearish_signal_matrix(:,8) = uptrend_2 .* white_body_2 .* ...
                              black_body_1 .* (body_high_1 > body_high_2) .* (body_low_1 < body_low_2) .* ...
                              black_body .* (close < close_1) ;
## 09 - Upside Gap Two Crows
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     black body
##     candle's body above the previous candle's body
## Third candle
##     black body
##     candle’s body engulfs the prior candle’s body
bearish_signal_matrix(:,9) = uptrend_2 .* white_body_2 .* ...
                              black_body_1 .* (body_low_1 > body_high_2) .* ...
                              black_body .* (body_high > body_high_1) .* (body_low < body_low_1) ;
## 10 - Advance Block
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     white body
##     the opening price is within the previous body
##     the closing price is above the previous closing price
## Third candle
##     white body
##     the opening price is within the previous body
##     the closing price is above the previous closing price
bearish_signal_matrix(:,10) = uptrend_2 .* white_body_2 .* ...
                               white_body_1 .* (open_1 < body_high_2) .* (open_1 > body_low_2) .* (close_1 > close_2) .* ...
                               white_body .* (open < body_high_1) .* (open > body_low_1) .* (close > close_1) ;
## 11 - Breakaway
## First candle
##     a tall white candle
## Second candle
##     a white candle
##     candle opens above the previous closing price (upward price gap, shadows can overlap)
## Third candle
##     a white or black candle
##     candle opens above the previous opening price
## Fourth candle
##     a white candle
##     candle closes above the previous closing price
## Fifth candle
##     a tall black candle
##     candle opens below the previous closing price
##     candle closes below the second line's opening price and above the first line's closing price
##     the price gap formed between the first and the second line is not closed
bearish_signal_matrix(:,11) = white_body_4 .* long_line_4 .* ...
                               white_body_3 .* (open_3 > close_4) .* ...
                               (open_2 > open_3 ) .* ...
                               white_body_1 .* (close_1 > close_2) .* ...
                               black_body .* long_line .* (open < close_1) .* (close < open_3) .* (close > close_4) .* (low > high_4) ;
## 12 - Counter Attack
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     black_body
##     the opening price is higher than the previous closing price
##     the closing price is at or lower than the previous closing price
bearish_signal_matrix(:,12) = uptrend_1 .* white_body_1 .* ...
                               black_body .* (open > close_1) .* (close <= close_1) ;
## 13 - Deliberation
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     white body
##     the opening price is above the previous opening price
##     the closing price is above the previous closing price
## Third candle
##     white body
##     the opening price is slightly lower or higher than the previous closing price
##     the closing price is above the previous closing price
bearish_signal_matrix(:,13) = uptrend_2 .* white_body_2 .* ...
                               white_body_1 .* (open_1 > open_2) .* (close_1 > close_2) .* ...
                               white_body .* (open > body_midpoint_1) .* (close > close_1) ;
## 14 - Doji Star
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     a doji candle
##     a body above the first candle's body
bearish_signal_matrix(:,14) = uptrend_1 .* white_body_1 .* ...
                               doji .* (open > close_1) ; 
## 15 - Dragonfly Doji
##     Opening, closing and maximum prices are the same or very similar
##     Long lower shadow
##     appears on as a long line
bearish_signal_matrix(:,15) = (open == close) .* (close == high) .* long_line ;

## 16 - Engulfing
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     black body
##     candle's body engulfs the prior (white) candle's body
bearish_signal_matrix(:,16) = uptrend_1 .* white_body_1 .* ...
                               black_body .* (body_high > body_high_1) .* (body_low < body_low_1) ;
## 17 - Gravestone Doji
##     Opening, closing and minimum prices are the same or very similar
##     Long upper shadow
##     appears on as a long line
bearish_signal_matrix(:,17) = (open == close) .* (close == low) .* long_line ;

## 18 - Harami Cross
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     a doji candle with two shadows
##     candle's body engulfed by the prior (white) candle's body
bearish_signal_matrix(:,18) = uptrend_1 .* white_body_1 .* ...
                               doji .* (high > close) .* (low < close) .* (high < body_high_1) .* (low > body_low_1) ;
## 19 - Identical Three Crows
## First candle
##     a candle in an uptrend
##     black body
## Second candle
##     black body
##     the opening price at or near the prior close
## Third candle
##     black body
##     the opening price at or near the prior close
bearish_signal_matrix(:,19) = uptrend_2 .* black_body_2 .* ...
                               black_body_1 .* (open_1 == close_2) .* ...
                               black_body .* (open == close_1) ;
## 20 - Long Legged Doji
##     a doji candle
##     opening and closing prices are the same or similar
##     upper and lower shadow are very long
##     body is located in the middle of the candle or nearly mid-range
##     appears on as a long line
bearish_signal_matrix(:,20) = doji .* ((high .- close) > 0) .* ((close .- low) > 0) .* long_line ;

## 21 - Meeting Lines
## First candle
##     a candle in an uptrend
##     white body
##     appears as a long line
## Second candle
##     black body
##     the closing price is equal to the previous closing price
bearish_signal_matrix(:,21) = uptrend_1 .* white_body_1 .* long_line_1 .* ...
                               black_body .* (close == close_1) ;
## 22 - Tri Star
## First candle
##     a doji candle in an uptrend
## Second candle
##     a doji candle
##     a body above the prior body
## Third candle
##     a doji candle
##     a body below the prior body
bearish_signal_matrix(:,22) = uptrend_2 .* doji_2 .* ...
                               doji_1 .* (open_1 > close_2) .* ...
                               doji .* (open < close_1) ;
## 23 - Two Crows
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     black body
##     the closing price above the prior closing price (gap between bodies)
## Third candle
##     black body
##     the opening price within the prior body
##     the closing price within the body of the first line (gap close)
bearish_signal_matrix(:,23) = uptrend_2 .* white_body_2 .* ...
                               black_body_1 .* (close_1 > close_2) .* ...
                               black_body .* (open < body_high_1) .* (open > body_low_1) .* (close < close_2) ;
## 24 - Belt Hold
##     black body
##     no upper shadow
##     short lower shadow
##     appears as a long line
bearish_signal_matrix(:,24) = black_body .* (open == high) .* (low < close) .* long_line ;

## 25 - Hanging Man
##     uptrend
##     white or black candle with a small body
##     no upper shadow or the shadow cannot be longer than the body
##     lower shadow at least two times longer than the body
##     if the gap is created at the opening or at the closing, it makes the signal stronger
##     appears as a long line
##     the body fully located above the trendline
bearish_signal_matrix(:,25) = uptrend .* (doji == 0) .* ((high .- body_high) < body_range) .* ((body_low .- low) >= 2.*body_range) ; 

## 26 - Harami
## First candle
##     a candle in an uptrend
##     white body
## Second candle
##     black body
##     candle's body engulfed by the prior (white) candle's body
bearish_signal_matrix(:,26) = uptrend_1 .* white_body_1 .* ...
                               black_body .* (body_high < body_high_1) .* (body_low > body_low_1) ;
## 27 - Shooting Star
##     white or black candle with a small body
##     no lower shadow or the shadow cannot be longer than the body
##     upper shadow at least two times longer than the body
##     if the gap is created at the opening or the closing, it makes the signal stronger
##     appears as a long line
bearish_signal_matrix(:,27) = (doji == 0) .* ((body_low .- low) < body_range) .* ((high .- body_high) >= 2.*body_range) ;

bearish_signal_matrix( 1 : 10 , : ) = 0 ;

endfunction

Currency Strength Candlestick Chart

In my previous posts on currency strength indices I have always visualised the indicator(s) as a line chart, e.g. here. However, after some deep thought, I have now created a way to visualise this as a candlestick chart using Octave's candle function, which, by the way, was written by me. Creating the candlestick body of a currency strength index was quite straight forward - just use the previous currency strength value as the bar's open and the current currency strength value as the close. A simple plot of this, with an overlaid currency strength index line chart, is

Of course the problem with this rendering is that there are no candlestick wicks.

My solution to create the wicks is showcased by the following code snippets

retval_high_wicks( ii , 6 ) = log( str2double( S.candles{ ii }.mid.h ) / max( str2double( S.candles{ ii }.mid.o ) , str2double( S.candles{ ii }.mid.c ) ) ) ;
retval_low_wicks( ii , 6 ) = log( str2double( S.candles{ ii }.mid.l ) / min( str2double( S.candles{ ii }.mid.o ) , str2double( S.candles{ ii }.mid.c ) ) ) ;

and

[ ii , ~ , v ] = find( [ retval_high_wicks( : , [32 33 34 35 36 37].+5 ) , -1.*retval_low_wicks( : , [5 20 28].+5 ) ] ) ;
new_index_high_wicks( : , 13 ) = accumarray( ii , v , [] , @mean ) ;
[ ii , ~ , v ] = find( [ retval_low_wicks( : , [32 33 34 35 36 37].+5 ) , -1.*retval_high_wicks( : , [5 20 28].+5 ) ] ) ;
new_index_low_wicks( : , 13 ) = accumarray( ii , v , [] , @mean ) ;

The first snippet shows an additional bit of code to the code here to record the log values of highs (lows) over (under) the candlestick bodies of all relevant currencies used in creating the currency strength indices.

The second snippet shows how the wicks are created, namely by taking the mean log values of high (low) wicks indexed by e.g.

[32 33 34 35 36 37].+5 and [5 20 28].+5

columns of downloaded forex crosses.

The reasoning behind this is as follows: take, for example, the EUR_USD forex pair - the upper wicks of these bars are recorded as upper wicks for the EUR index candles and as lower wicks for the USD index candles, reflecting the fact that upper wicks in EUR_USD can be viewed as intrabar EUR strength pushing to new highs or, alternatively, USD index candle's weakness pushing to new lows which, because the USD is the quote currency of the pair, also leads to new highs in the cross. A similar, reversed logic applies to the low wicks of the cross.

Below are charts of currency strength index candles created according to this methodology

The upper pane shows GBP currency strength index candles and the lower pane the same for USD. This is basically price action for Thursday, 30th July, 2020. The green vertical lines are the London and New York opens respectively, the red vertical line is the London close and the charts end at more or less the New York close. Bars prior to the London open are obviously the overnight Asian session.

My contemporaneous volume profile chart is the upper right pane below

 

From these charts it is easy to discern that the upward movement of GBP_USD during the main London session was due to GBP strength, whilst after the London close the continued upward movement of GBP_USD was due to USD weakness.

However, the point of this blog post was not to pass commentary on FX price movements, but to illustrate a methodology of creating candlestick charts for currency strength indices.

Enjoy!

Downloading FX Pairs via Oanda API to Calculate Currency Strength Indicator

In the past I have posted a series of blog posts about a Currency Strength Indicator (here, here, here and here). This blog post gives an Octave function to use Oanda's API to download all the 10 minute OHLC data required to calculate the above strength indicators on the 10 minute time frame.

## Copyright (C) 2020 dekalog
## 
## This program is free software: you can redistribute it and/or modify it
## under the terms of the GNU General Public License as published by
## the Free Software Foundation, either version 3 of the License, or
## (at your option) any later version.
## 
## This program is distributed in the hope that it will be useful, but
## WITHOUT ANY WARRANTY; without even the implied warranty of
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
## GNU General Public License for more details.
## 
## You should have received a copy of the GNU General Public License
## along with this program.  If not, see
## .

## -*- texinfo -*- 
## @deftypefn {} {@var{retval} =} get_currency_index_10m_pairs()
##
## This function gets the date and time value of the last currency index update for
## 10 minute bars by reading the last line of the file at:
##
## "/home/path/to/file"
##
## and then downloads all the currencies required to calculate new values for
## new currency index calculations, via looped Oanda API calls. 
## 
##The RETVAL is a matrix of GMT dates in the form
## YYYY:MM:DD:HH:MM in the first 5 columns, followed by the 45 required
## currency candlestick close values.
##
## @seealso{}
## @end deftypefn

## Author: dekalog 
## Created: 2020-06-01

function retval = get_currency_index_10m_pairs()
 
## cell array of currency crosses to iterrate over to get the complete set 
## of currency crosses to create a currency index
iter_vec = {'AUD_CAD','AUD_CHF','AUD_HKD','AUD_JPY','AUD_NZD','AUD_SGD',...
'AUD_USD','CAD_CHF','CAD_HKD','CAD_JPY','CAD_SGD','CHF_HKD','CHF_JPY',...
'EUR_AUD','EUR_CAD','EUR_CHF','EUR_GBP','EUR_HKD','EUR_JPY','EUR_NZD',...
'EUR_SGD','EUR_USD','GBP_AUD','GBP_CAD','GBP_CHF','GBP_HKD','GBP_JPY',...
'GBP_NZD','GBP_SGD','GBP_USD','HKD_JPY','NZD_CAD','NZD_CHF','NZD_HKD',...
'NZD_JPY','NZD_SGD','NZD_USD','SGD_CHF','SGD_HKD','SGD_JPY','USD_CAD',...
'USD_CHF','USD_HKD','USD_JPY','USD_SGD'} ;

## read last line of current 10min_currency_indices
unix_command = [ "tail -1" , " " , "/home/path/to/file" ] ;
[ ~ , data ] = system( unix_command ) ;
data = strsplit( data , ',' ) ; ## gives a cell arrayfun of characters
## zero pad singular month representations, i.e. 1 to 01
if ( numel( data{ 2 } == 1 ) )
data{ 2 } = [ '0' , data{ 2 } ] ;
endif
## and also zero pad singular dates
if ( numel( data{ 3 } == 1 ) )
data{ 3 } = [ '0' , data{ 3 } ] ;
endif
## and also zero pad singular hours
if ( numel( data{ 4 } == 1 ) )
data{ 4 } = [ '0' , data{ 4 } ] ;
endif
## and also zero pad singular minutes
if ( numel( data{ 5 } == 1 ) )
data{ 5 } = [ '0' , data{ 5 } ] ;
endif
 
## set up the headers
Hquery = [ 'curl -s -H "Content-Type: application/json"' ] ; ## -s is silent mode for Curl for no paging to terminal
Hquery = [ Hquery , ' -H "Authorization: Bearer XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"' ] ;
query_begin = [ Hquery , ' "https://api-fxtrade.oanda.com/v3/instruments/' ] ;

## get time from last line of data
query_time = [ data{1} , '-' , data{2} , '-' , data{3} , 'T' , data{4} , '%3A' , data{5} , '%3A00.000000000Z&granularity=M10"' ] ;

## initialise with AUD_CAD
## construct the API call for particular cross
query = [ query_begin , iter_vec{ 1 } , '/candles?includeFirst=true&price=M&from=' , query_time ] ;
## call to use external Unix systems/Curl and return result
[ ~ , ret_JSON ] = system( query , RETURN_OUTPUT = 'TRUE' ) ;
## convert the returned JSON object to Octave structure
S = load_json( ret_JSON ) ;
## parse the returned structure S
if ( strcmp( fieldnames( S( 1 ) ) , 'errorMessage' ) == 0 ) ## no errorMessage in S
end_ix = numel( S.candles ) ; ## how many candles?
if ( S.candles{ end }.complete == 0 ) end_ix = end_ix - 1 ; endif ## account for incomplete candles
## create retval
retval = zeros( end_ix , 50 ) ; ## 45 currencies plus YYYY:MM:DD:HH:MM columns
for ii = 1 : end_ix
 date_time = strsplit( S.candles{ ii }.time , { '-' , 'T' , ':' } ) ;
 retval( ii , 1 ) = str2double( date_time( 1 , 1 ) ) ; ## year
 retval( ii , 2 ) = str2double( date_time( 1 , 2 ) ) ; ## month
 retval( ii , 3 ) = str2double( date_time( 1 , 3 ) ) ; ## day
 retval( ii , 4 ) = str2double( date_time( 1 , 4 ) ) ; ## hour
 retval( ii , 5 ) = str2double( date_time( 1 , 5 ) ) ; ## min
 retval( ii , 6 ) = str2double( S.candles{ ii }.mid.c ) ; ## candle close price 
endfor ## end of ii loop
else
error( 'Initialisation with AUD_CAD has failed.' ) ; 
endif ## end of strcmp if

for ii = 2 : numel( iter_vec )
## construct the API call for particular cross
query = [ query_begin , iter_vec{ ii } , '/candles?includeFirst=true&price=M&from=' , query_time ] ;
## call to use external Unix systems/Curl and return result
[ ~ , ret_JSON ] = system( query , RETURN_OUTPUT = 'TRUE' ) ;
## convert the returned JSON object to Octave structure
S = load_json( ret_JSON ) ;
## parse the returned structure S
if ( strcmp( fieldnames( S( 1 ) ) , 'errorMessage' ) == 0 ) ## no errorMessage in S
end_ix = numel( S.candles ) ; ## how many candles?
if ( S.candles{ end }.complete == 0 ) end_ix = end_ix - 1 ; endif ## account for incomplete candles
temp_retval = zeros( end_ix , 6 ) ;
for jj = 1 : end_ix
 date_time = strsplit( S.candles{ jj }.time , { '-' , 'T' , ':' } ) ;
 temp_retval( jj , 1 ) = str2double( date_time( 1 , 1 ) ) ; ## year
 temp_retval( jj , 2 ) = str2double( date_time( 1 , 2 ) ) ; ## month
 temp_retval( jj , 3 ) = str2double( date_time( 1 , 3 ) ) ; ## day
 temp_retval( jj , 4 ) = str2double( date_time( 1 , 4 ) ) ; ## hour
 temp_retval( jj , 5 ) = str2double( date_time( 1 , 5 ) ) ; ## min
 temp_retval( jj , 6 ) = str2double( S.candles{ jj }.mid.c ) ; ## candle close price  
endfor ## end of jj loop

## checks dates and times allignment before writing to retval
date_time_diffs_1 = setdiff( retval( : , 1 : 5 ) , temp_retval( : , 1 : 5 ) , 'rows' ) ; 
date_time_diffs_2 = setdiff( temp_retval( : , 1 : 5 ) , retval( : , 1 : 5 ) , 'rows' ) ;

 if ( isempty( date_time_diffs_1 ) && isempty( date_time_diffs_2 ) ) 
 ## there are no differences between retval dates and temp_retval dates 
 retval( : , ii + 5 ) = temp_retval( : , 6 ) ;
 
 elseif ( ~isempty( date_time_diffs_1 ) || ~isempty( date_time_diffs_2 ) )
 ## implies a difference between the date_times of retval and temp_retval, so merge them
 
 dn_retval = datenum( [ retval(:,1) , retval(:,2) , retval(:,3) , retval(:,4) , retval(:,5) ] ) ;
 dn_temp_retval = datenum( [ temp_retval(:,1) , temp_retval(:,2) , temp_retval(:,3) , temp_retval(:,4) , temp_retval(:,5) ] ) ;
 new_dn = unique( [ dn_retval ; dn_temp_retval ] ) ; new_date_vec = datevec( new_dn ) ; new_date_vec( : , 6 ) = [] ; 
 new_retval = [ new_date_vec , zeros( size( new_date_vec , 1 ) , 45 ) ] ;
 [ TF , S_IDX ] = ismember( new_retval( : , 1 : 5 ) , retval( : , 1 : 5 ) , 'rows' ) ;
 TF_ix = find( TF ) ; new_retval( TF_ix , 6 : end ) = retval( : , 6 : end ) ;
 [ TF , S_IDX ] = ismember( new_retval( : , 1 : 5 ) , temp_retval( : , 1 : 5 ) , 'rows' ) ;
 TF_ix = find( TF ) ; new_retval( TF_ix , ii + 5 ) = temp_retval( : , 6 ) ; 
 retval = new_retval ; 
 clear new_retval new_dn dn_temp_retval dn_retval date_time_diffs_1 date_time_diffs_2 ;
 
 else 
 error( 'Mismatch between dates and times for writing to retval.' ) ; 
 endif ## TF == S_IDX check

endif ## end of strcmp if

endfor ## ii loop

endfunction

At the moment there are almost 50 separate API calls nested within a loop, which of course is non-vectorised and inefficient, and if I find out how to make a batch API call to do this I shall rewrite the function.

This function is called in a script which uses the output matrix "retval" to then calculate the various currency strengths as outlined in the above linked posts. The total running time for this script is approximately 40 seconds from first call to appending to the index file on disk. I wrote this function to leverage my new found Oanda API knowledge to avoid having to accumulate an ever growing set of files on disk containing the raw 10 minute data.

I hope readers find this useful.

Another Method of Creating Synthetic Data

Over the years I have posted about several different methodologies for creating synthetic data and I have recently come across yet another one which readers may find useful.

One of my first posts was Creation of Synthetic Data, which essentially is a random scrambling of historic data for a single time series with an attempt to preserve some of the bar to bar dependencies based upon a bar's position in relation to upper and lower price envelopes, a la Bollinger Bands, although the code provided in this post doesn't actually use Bollinger Bands. Another post, Creating Synthetic Data Using the Fast Fourier Transform, randomly scrambles data in the frequency domain.

Rather than random scrambling of existing data, another approach is to take measurements from existing data and then use these measurements to recreate new data series with similar characteristics. My Hidden Markov Modelling of Synthetic Periodic Time Series Data post utilises this approach and can be used to superimpose known sinusoidal waveforms onto historical trends. The resultant synthetic data can be used, as I have used it, in a form of controlled experiment whereby indicators etc. can be measured against known cyclic prices.

All of the above share the fact that they are applied to univariate time series only, although I have no doubt that they could probably be extended to the multivariate case. However, the new methodology I have come across is Statistical Mechanics of Time Series and its matlab central file exchange "toolbox." My use of this is to produce ensembles of my Currency Strength Indicator, from which I am now able to produce 50/60+ separate, synthetic time series representing, for example, all the Forex pairs, and preserving their inter-relationships whilst only suffering the computational burden of applying this methodology to a dozen or so underlying "fundamental" time series.

This may very well become my "go to" methodology for generating unlimited amounts of training data.

Preliminary Test Results of Time Series Embedding

Following on from my post yesterday, this post presents some preliminary results from the test I was running while writing yesterday's post. However, before I get to these results I would like to talk a bit about the hypothesis being tested.

I had an inkling that the dominant cycle period might having some bearing on tau, the time delay for the time series embedding implied by Taken's theorem, so I set up the test (described below) and after writing the post yesterday, and while the test was still running, I did some online research to see if there is any theoretical justification available.

One of the papers I found online is A Novel Method for Topological Embedding of Time-Series Data. From the abstract

"...we propose a novel method for embedding one-dimensional, periodic time-series data into higher-dimensional topological spaces to support robust recovery of signal features via topological data analysis under noisy sampling conditions...To provide evidence for the viability of this method, we analyze the simple case of sinusoidal data..."

One of the conclusions drawn is that for sinusoidal data the ideal embedding length is a quarter of the cycle period, i.e. Π/2.

Another paper I found was Topological time-series analysis with delay-variant embedding. This paper investigates "delay-variant embedding," essentially making the embedding length tau variable. From the concluding remarks

"We have demonstrated that the topological features that are constructed using delay-variant embedding can capture the topological variation in a time series when the time-delay value changes... These results indicate that the topological features that are deduced with delay-variant embedding can be used to reveal the representative features of the original time series."

My inkling was that tau should be adaptive to the measured dominant cycle, and both the above linked papers do provide some justification. Anyway, on to the test.

Reusing the code from my earlier post here I created synthetic price series with known but variable dominant cycle periods and based on real price trends, which results in synthetic series that are highly correlated with real price series but with underlying characteristics being exactly known. I then used my version of mdembedding code to calculate tau for numerous Monte Carlo replications of time series of prices based on a range of real forex pairs prices, gold and silver prices and different indices created by my currency strength indicator methodology. I then created a ratio of tau/average_measured_period measured as a global value across the whole of each individual time series replication as the test statistic, with the period being measured by an autocorrelation periodogram function: a ratio value of 0.5, for example, meaning that the tau for that time series is half the average period over the whole series.

Below is a plot of the histogram of these ratios:

which shows the ratios being centred approximately around 0.4 but with a long right tail. The following histogram is of the means of the bootstrap with replacement of the above distribution,

which is centred around a mean value of 0.436. Using this bootstrapped mean ratio of tau/period implies, for example, that the ideal tau for an average period of 20 is 8.72, which when rounded up to 9, is almost twice the theoretical ideal of a 5 day tau for 20 day cyclic period prices.

Personally I think this is an encouraging set of preliminary test results, with some caveats. More in due course.

"Black Swan" Data Cleaning

Since my last post I have been investigating training features that can be derived from my Currency Strength indicator as input for machine learning algorithms and during this work it was obvious that there are instances in the raw data that are Black Swan outliers. This can be seen in the chart below as pronounced spikes.

The chart itself is a plot of log returns of various forex crosses and Gold and Silver log returns, concatenated into one long vector. The black is the actual return of the underlying, the blue is the return of the base currency and the red is the cross currency, both of these being calculated from indices formed from the currency strength indicator.

By looking at the dates these spikes occur and then checking online I have flagged four historical "Black swan" events that occured within the time frame the data covers, which are listed in chronological order below:

1. Precious metals price collapse in mid April 2013
2. Swiss Franc coming off its peg to the Euro in January 2015
3. Fears over the Hong Kong dollar and Renminbi currency peg in January 2016
4. Brexit black Friday

The next series of charts shows the progressive reduction in the number of spikes as the data around the above events is deleted from those crosses etc. that were affected.

It can be seen that the final chart shows much more homogeneous data within each concatenated series, which should have benefits when said data is used as machine learning input. Also, the data that has been deleted will provide a useful, extreme test set to stress test any finished model. More in due course.

Update on Improved Currency Strength Indicator

Following on from my previous post I have now slightly changed the logic and coding behind the idea, which can be seen in the code snippet below

%  aud_cad
mse_vector(1) = log( ( current_data(1,1) * ( aud_x / cad_x ) ) / current_data(2,1) )^2 ;
%  xau_aud
mse_vector(46) = log( ( current_data(1,46) * ( gold_x / aud_x ) ) / current_data(2,46) )^2 ;
%  xau_cad
mse_vector(47) = log( ( current_data(1,47) * ( gold_x / cad_x ) ) / current_data(2,47) )^2 ;

Essentially the change simultaneously optimises, using Octave's fminunc function, for both the gold_x and all currency_x geometric multipliers together rather than just optimising for gold and then analytically deriving the currency multipliers. The rationale for this change is shown in the chart below,

which shows the optimisation errors for the "old" way of doing things, in black, and the revised way in blue. Note that this is a log scale, so the errors for the revised way are orders of magnitude smaller, implying a better model fit to the data.

This next chart shows the difference between the two methods of calculating a gold index ( black is old, blue is new ),

this one shows the calculated USD index

and this one the GBP index in blue, USD in green and the forex pair cross rate in black

The idea(s) I am going to look at next is using these various calculated indices as inputs to algorithms/trading decisions.