Some Basic Code Housekeeping

Since my last post, back in late November last year, I have been doing a few disparate things such as:

• improving the coding of some functions in R to use the Oanda API to automatically download data using cronjobs
• coding some Octave functions to plot/visualise the above data
• more work on Random Vector Functional Link networks 
• trying my hand at some discretionary day trading to take advantage of the increased market volitility due to the current Covid-19 pandemic

The R functions I blogged about back in 2017 here and my recent work has been to improve them by implementing some basic price sanity checks on the OHLC bars, checking the integrity of the date handling prior to writing to file and also writing to file in a format more amenable to being read by Octave.

A new addition to this work is to also download Oanda's forex order and position books using the API, which I was inspired to do by this online guide. At the moment I am testing this data using ideas from the literature on Limit Order Books and using the R Boruta package to test if the features described in the literature are suitable for this data.

To help visualise this, I have written this Octave function

clear all ;
cd /home/dekalog/Documents/octave/oanda_data/20m ;

files_10m = glob( '*_ohlc_10m' ) ;
## files_10m =
## {
##   [1,1] = aud_jpy_ohlc_10m
##   [2,1] = aud_usd_ohlc_10m
##   [3,1] = eur_aud_ohlc_10m
##   [4,1] = eur_chf_ohlc_10m
##   [5,1] = eur_gbp_ohlc_10m
##   [6,1] = eur_jpy_ohlc_10m
##   [7,1] = eur_usd_ohlc_10m
##   [8,1] = gbp_chf_ohlc_10m
##   [9,1] = gbp_jpy_ohlc_10m
##   [10,1] = gbp_usd_ohlc_10m
##   [11,1] = nzd_usd_ohlc_10m
##   [12,1] = usd_cad_ohlc_10m
##   [13,1] = usd_chf_ohlc_10m
##   [14,1] = usd_jpy_ohlc_10m
##   [15,1] = xag_usd_ohlc_10m
##   [16,1] = xau_usd_ohlc_10m
## }
file_no = input( 'Enter file no. from list, a number 1 to 16 inclusive. ' ) ;
filename = files_10m{ file_no } ;
str_split = strsplit( filename , "_" ) ;
price = strjoin( str_split( 1 : 2 ) , "_" ) ;
orders = dlmread( [ price , '_orderbook_snapshot' ] ) ; ## get latest orderbook levels
latest_price = orders( 1 , 1 ) ;
[ ~ , ix ] = min( abs( orders( 1 , 1 ) .- orders( 2 : end , 1 ) ) ) ; ix = ix + 1 ;
order_levels = 10 ;
order_max = max( max( orders( ix - order_levels : ix + order_levels , 2 : 3 ) ) ) ;
max10 = 20 / order_max ; max20 = 10 / order_max ;

## create a unix system command to read the last 'n' rows of ohlc of filename
unix_command = [ "tail -100" , " " , filename ] ; 
[ ~ , data ] = system( unix_command ) ;
data = strsplit( data , { "," , "\n" } ) ; ## gives a cell arrayfun
if ( isempty( data{ end } ) )
 data( end ) = [] ;
endif
data = reshape( str2double( data' ) , 22 , 100 )' ;
## data cols 
## open == 18, high == 19, low == 20, close == 21, vol == 22

## plot the 10 minute bars
h = findobj( 'type' , 'figure' ) ;
if ( !isempty( h ) && any( h == 10 ) )
clf( 10 ) ;
endif
figure( 10 ) ; candle( data(:,19) , data(:,20) , data(:,21) , data(:,18) ) ;
hold on ; figure( 10 ) ; barh( orders( ix - order_levels : ix + order_levels , 1 ) , orders( ix - order_levels : ix + order_levels , 2 ) .* max10 , 0.7 , 'c' ) ; 
barh( orders( ix - order_levels : ix + order_levels , 1 ) , orders( ix - order_levels : ix + order_levels , 3 ) .* max10 , 0.4 , 'm' ) ;
hold off ;
hline( latest_price , 'r-' ) ; ## the latest snapshot price
for ii = ix - order_levels : ix + order_levels
hline( orders( ii , 1 ) , 'k:' , num2str( orders( ii , 1 ) ) ) ;
endfor
title( strjoin( str_split( 1 : 2 ) , " " ) ) ;

## get ix for forming 20 minute bars
ix0 = find( ( data(:,4) == shift( data(:,4) , -1 ) ) .* ( ( data(:,5) == 0 ) .* ( shift( data(:,5) , -1 ) == 10 ) ) ) ;
ix20 = find( ( data(:,4) == shift( data(:,4) , -1 ) ) .* ( ( data(:,5) == 20 ) .* ( shift( data(:,5) , -1 ) == 30 ) ) ) ;
ix40 = find( ( data(:,4) == shift( data(:,4) , -1 ) ) .* ( ( data(:,5) == 40 ) .* ( shift( data(:,5) , -1 ) == 50 ) ) ) ;

## bars beginning on the hour
## the highs
data( ix0 , 19 ) = max( [ data( ix0 , 19 ) , data( ix0.+1 , 19 ) ] , [] , 2 ) ;
## the lows
data( ix0 , 20 ) = min( [ data( ix0 , 20 ) , data( ix0.+1 , 20 ) ] , [] , 2 ) ;
## the close
data( ix0 , 21 ) = data( ix0.+1 , 21 ) ;
## the vol
data( ix0 , 22 ) = data( ix0 , 22 ) + data( ix0.+1 , 22 ) ;

## bars beginning 20 past the hour
## the highs
data( ix20 , 19 ) = max( [ data( ix20 , 19 ) , data( ix20.+1 , 19 ) ] , [] , 2 ) ;
## the lows
data( ix20 , 20 ) = min( [ data( ix20 , 20 ) , data( ix20.+1 , 20 ) ] , [] , 2 ) ;
## the close
data( ix20 , 21 ) = data( ix20.+1 , 21 ) ;
## the vol
data( ix20 , 22 ) = data( ix20 , 22 ) + data( ix20.+1 , 22 ) ;

## bars beginning 40 past the hour
## the highs
data( ix40 , 19 ) = max( [ data( ix40 , 19 ) , data( ix40.+1 , 19 ) ] , [] , 2 ) ;
## the lows
data( ix40 , 20 ) = min( [ data( ix40 , 20 ) , data( ix40.+1 , 20 ) ] , [] , 2 ) ;
## the close
data( ix40 , 21 ) = data( ix40.+1 , 21 ) ;
## the vol
data( ix40 , 22 ) = data( ix40 , 22 ) + data( ix40.+1 , 22 ) ;

## delete rows of 10, 30 and 50 minutes past the hour
data( [ ix0.+1 ; ix20.+1 ; ix40.+1 ] , : ) = [] ;

## plot 20 minute candles
if ( !isempty( h ) && any( h == 20 ) )
clf( 20 ) ;
endif
figure( 20 ) ; candle( data(:,19) , data(:,20) , data(:,21) , data(:,18) ) ;
hold on ; figure( 20 ) ; barh( orders( ix - order_levels : ix + order_levels , 1 ) , orders( ix - order_levels : ix + order_levels , 2 ) .* max20 , 0.7 , 'c' ) ; 
barh( orders( ix - order_levels : ix + order_levels , 1 ) , orders( ix - order_levels : ix + order_levels , 3 ) .* max20 , 0.4 , 'm' ) ;
hold off ;
hline( latest_price , 'r-' ) ; ## the latest snapshot price
for ii = ix - order_levels : ix + order_levels
hline( orders( ii , 1 ) , 'k:' , num2str( orders( ii , 1 ) ) ) ;
endfor
title( strjoin( str_split( 1 : 2 ) , " " ) ) ;

which takes in 10 minute OHLC data and creates 20 minute bars from these 10 minute bars and plots both with the 10 order book levels above and below the latest order book price level. The horizontal blue bars are percentage of buy orders whilst the red bars are sell orders,

compared with the closest Oanda screen shot.

There are differences between the two, namely

• My plot above is of 20 minute bars vs Oanda 15 minute bars
• my open order histogram is fixed at the latest downloaded open order snapshot corresponding to the rightmost bar close, whereas the Oanda platform allows you to scroll back and see the order book in historical time

The 20 minute timeframe was deliberately chosen to match the maximum "refresh rate" of the API download availability and gives a nice, slightly different perspective to the 15 and 30 minute granularity provided on the Oanda platform.