Saturday 26 October 2013

Change of Direction in Neural Net Training.

Up till now when training my neural net classifier I have been using what I call my "idealised" data, which is essentially a model for prices comprised of a sine wave along with a linear trend. In my recent work on Savitzky-Golay filters I wanted to increase the amount of available training data but have run into a problem - the size of my data files has become so large that I'm exhausting the memory available to me in my Octave software install. To overcome this problem I have decided to convert my NN training to an online regime rather than loading csv data files for mini-batch gradient descent training - in fact what I envisage doing would perhaps be best described as online mini-batch training.

To do this will require a fairly major rewrite of my data generation code and I think I will take the opportunity to make this data generation more realistic than the sine wave and trend model I am currently using. Apart from the above mentioned memory problem, this decision has been prompted by an e-mail exchange I have recently had with a reader of this blog and some online reading I have recently been doing, such as mixed regime simulation, training a simple neural network to recognise different regimes in financial time series, neural networks in trading, random subspace optimisation and profitable hidden and markovian coupling. At the moment it is my intention to revisit my earlier posts here and here and use/extend the ideas contained therein. This means that for the nearest future my work on Savitzky-Golay filters is temporarily halted.

3 comments:

Anonymous said...

If Octave is your current cup of tea, I suggest you take a sip of Python. You may discover, like the rest of us former matlab/python/r sippers, that python is a much smoother drink. You likely will not regret taking the time to make the transition. If all else fails, at least you could get a job at google. Speed, elegance, power, (free)dom... python.

Dekalog said...

Anonymous,

You're quite right that Python would be a good choice of software for me to use and in fact I do have some experience with it. At some time in the future I may yet change from Octave to Python, or maybe to R. Time will tell.

Dekalog

Unknown said...

Nice blog there are many information about training