It has taken this long to get this far for a few reasons; the necessity to substantially adapt the code from the Geoff Hinton neural net course and then conducting grid searches over the hyper-parameter space for the "optimum" learning rate, number of neurons in the hidden layer and also incorporating some changes to the features set used as input to the classifier. At this halfway stage I thought I would subject the classifier to the cross validation test of my recent post of 5th December 2012 and the results, which speak for themselves, are shown in the box below.
Random NN
Complete Accuracy percentage: 99.610000
"Acceptable" Mis-classifications percentages
Predicted = uwr & actual = unr: 0.000000
Predicted = unr & actual = uwr: 0.062000
Predicted = dwr & actual = dnr: 0.008000
Predicted = dnr & actual = dwr: 0.004000
Predicted = uwr & actual = cyc: 0.082000
Predicted = dwr & actual = cyc: 0.004000
Predicted = cyc & actual = uwr: 0.058000
Predicted = cyc & actual = dwr: 0.098000
Dubious, difficult to trade mis-classification percentages
Predicted = uwr & actual = dwr: 0.000000
Predicted = unr & actual = dwr: 0.000000
Predicted = dwr & actual = uwr: 0.000000
Predicted = dnr & actual = uwr: 0.000000
Completely wrong classifications percentages
Predicted = unr & actual = dnr: 0.000000
Predicted = dnr & actual = unr: 0.000000
End NN
Complete Accuracy percentage: 98.518000
"Acceptable" Mis-classifications percentages
Predicted = uwr & actual = unr: 0.002000
Predicted = unr & actual = uwr: 0.310000
Predicted = dwr & actual = dnr: 0.006000
Predicted = dnr & actual = dwr: 0.036000
Predicted = uwr & actual = cyc: 0.272000
Predicted = dwr & actual = cyc: 0.036000
Predicted = cyc & actual = uwr: 0.344000
Predicted = cyc & actual = dwr: 0.210000
Dubious, difficult to trade mis-classification percentages
Predicted = uwr & actual = dwr: 0.000000
Predicted = unr & actual = dwr: 0.000000
Predicted = dwr & actual = uwr: 0.000000
Predicted = dnr & actual = uwr: 0.000000
Completely wrong classifications percentages
Predicted = unr & actual = dnr: 0.000000
Predicted = dnr & actual = unr: 0.000000
This classifier has 3 sigmoid logistic neurons in its single hidden layer and during training early stopping was employed. I also tried adding L2 regularization but this didn't really seem to have any appreciable effect, so after a while I dropped this. All in all, I'm very pleased with my efforts and the classifier's performance so far. Over the next few weeks I shall continue with the training and when this is complete I shall post again.On a related note, I have recently added another blog to the blogroll because I was impressed with a series of posts over the last couple of years concerning that particular blogger's investigations into neural nets for trading, especially the last two posts here and here. The ideas covered in these last two posts ring a bell with my post here, where I first talked about using a neural net as a market classifier based on the work I did in Andrew Ng's course on recognising hand written digits from pixel values. I shall follow this new blogroll addition with interest!
No comments:
Post a Comment