Walk Forward Analysis - Holy Grail?

At first I have to worry you: Holy Grail does not exist. Every trading algorithm has its weak points, and will fail under some circumstances. A few examples:

1.      Statistical arbitrage - nowadays, I believe the pricing inefficiencies are captured very quickly by HFT arbitrage strategies used by many funds and proprietary traders. Without exceptionally quick order execution or even exchange collocated hosting I doubt it is possible to be successful in arbitrage in the long term. 

2.       Martingale – obvious.

3.       Trend followers, breakouts, volatility breakouts – look at GBPUSD from second half of 2010. Most of the systems that worked exceptionally well, failed bluntly since then.

I have seen quite a few discussions recently: people try to use walk forward analysis to implement “robustness” into the trading strategies that fall into 3^rd point above. Idea is to built in  two opposite trading strategies in one trading system (trend following / counter trade, volatility breakout / volatility retracement) and look for a robust process to choose which one to trade...

Walk Forward Analysis 

Walk Forward Analysis is described here or here. In classical approach, WFA should be used to determine how the profit from a given strategy would change if you would re-optimize the strategy parameters every 1 week, 1 month or 1 year. If you will select a given parameter space for optimization, it will tell you:

• how good the parameter space is
• how often you should reoptimize given strategy
• whether it's not just a local extremum that works for a given system

So in the classical approach you are not influencing the system core - the way in which it trades. But, if you will implement two counter strategies and optimize also the way in which a system trades (trend-following / counter trend), by using the WFA you could potentially find a process that will be flexible enough to detect when the trading mode of the strategy should be switched...  So for example the parameter selection process to use last X bars to optimize the strategy and repeating the optimization every Y days could have been a good way of detecting how the market changed in the past and avoid the system from losing when market changes.

I have spent a lot of time to analyze this for different systems. I used my own implementation of a volatility breakout and trend following system and the following tool to perform the WFA in MetaTrader. The tool is rubbish in my opinion, but for 30 bucks it does what its supposed to. 

Below you can see system results for GBPUSD,  2002-2010:

Being a volatility breakout / trend following system, it obviously failed in 2010 and 2011:

I also implemented a counter trend way of trading into the system and run the WFA to optimize the parameters and the way in which the system trades. Here are the WFA results for 2007, 2008, 2009, 2010 and 2011 (optimization window length of 300 days, and testing window length of 30 days):

The system is able to adapt to periods in which trend following / volatility breakouts works and do not work. If I would decide to re-optimize the strategy every 300 days, I would have a Holy Grail, right? Answer is no. Why? There is no guarantee that the market will change at the same pace in the future. So it might happen that the market will change it's characteristics every few months, and this re-optimization process will just choose the wrong way of trading all the time. 

So is there any value in the WFA?
I believe so and I am using WFA. I re-optimize the volatility breakout and trend following system every 1 month and I use past 10 months to do the optimization. I do it for two reasons:

• this produces better results than using just one set of parameters that worked throughout the years.
• if the market will change there is a chance, the system will adapt to these changes. 

However, at some point in time, this process will stop working...

GO! Standard

I have just set up a new expert advisor on the GO! Standard account. This account will be traded by a modified version of the  HumbleScalp system I described some time ago. Basically the improvements are:

• added a few additional complementary strategies to bring down drawdown. Their trading style is the same, entry conditions are just different for each strategy. This smooths the equity curve a lot, and I am a big supporter of this approach. Usually a few instances of the same strategy ran with different parameters will have much slightly lower profit but much lower drawdown than single instance with higher risk. 
• improved calculation of SL and TP - calculated based on recent volatility and not recent range. Some limitation to the maximum and minimum SL and TP still applies to avoid too low or too high levels. 
• slightly changed position management (it can take up 18 trades to follow one move). This change goes in pair with adding new strategies, simply each strategy can use just a few slots out of the available pool.

Below is a year by year 10 years EURUSD backtest (Alpari UK data, 0.8 pip spread). I know when you see backtest like this, you think this is some shitty EA that abuses the MetaTrader backtest module and opens and closes trades within one, 1-minute bar (which will produce false results, as Metatrder is considering only OHLC data from lowest available timeframe 1-mintue in the backtest). But actually the average trade time is around 1 hour. By the way I also resigned from fancy take profit trailing, because I believe Metatrader backtester is also unreliable when trailing is done with less pips then bar size.

I have a few ideas for improvement up my sleves, but I'll see how the strategy will perform on this level. I will run it with a rather high risk (nothing new though...). 

2002:

 2003:

 2004:

 2005:

 2006:

 2007:

 2008:

 2009:

2010:

 2011:

 2012:

Time series similarity

One of the things I have been working on in the last months are time series similarity algorithms. Why? In principle, most trading systems use past data to enter a trade. So for instance you use 5 technical indicators to enter the market. Or maybe you are using classical technical analysis: double top/bottom or head and shoulders as entry signals... Basically, you are trying to find chart patterns that occurred in the past and produced more profit then loss.

My idea is: let's take last X prices. And look in all the history for a given instrument what were the most similar X prices  in the past and what happened in the next Y prices. For example:

• Let's look at last 500 open prices on 1 minute chart
• Find the most similar 500 open prices in the historical data
• Check whether the price risen or fallen in the next 200 bars

Of course the challenge is: how to find the most similar piece of data from the past. My initial idea was rather trivial: take two data series and calculate an euclidean distance in each data point. If all the distances would be the same it would mean that the series are identical. However this approach wouldn't be very robust in presence of noise or for time shifted data... 

So I implemented an algorithm called Dynamic Time Warping. It is a rather simple approach to determining time series similarity, but it gives good results for vast variety of different classes of time series data. I will not write about the algorithm itself, as it is well described in Internet. I tried two things (not going into calculation details):

1. Finding one most similar pattern from the past. 
2. Finding many similar patterns and seeing whether there was a consistent behavior in directly subsequent data.

When I created a simple trading system leveraging above as an entry signal results were confusing in both cases. Some combinations of input parameters  were profitable, but small modification of the parameters resulted in no profit at all...  I done all the tests on EURUSD 2006-2011 1-minute and 5-minute data which is very consistent in its characteristics. Of course I was normalizing each data point by subtracting mean  and dividing by the standard deviation of a given probe.

As for now I am parking this idea, even though I believe there is a lot of value in it... But to further explore it I would need to spent much more time to implement other similarity algorithms:

• more precise on varied datasets,
• much faster in calculation. 

If you want to spend some time on this, I would suggest continue reading here: http://www.eecs.umich.edu/db/files/sigmod07timeseries.pdf

Loss averaging martingale trading systems

I see that more and more people are using loss averaging martingale systems. In principle I am against this method, especially if it is used in trading systems that don't have any other logic... Basically due to the spread those systems are doomed to fail, spread works here like 0 (green) in roulette.

It is a popular method of trading GBPUSD which is ranging since second half of 2010. Here is an example:
http://www.myfxbook.com/members/rsmereka/forex-hacked/218967 Adding also a screenshot - usually those systems are removed just after blowing up the account:

Description of three best three best martingale systems:

System 1: tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok BANG!


System 2: tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok,tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok,tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok, tik, tok  BANG!

System 3: tik, tok, tik, tok  BANG!

Calling C# from MetaTrader

Some time ago I have discovered a way of linking C# DLL libraries with MetaTrader (I am using it on MetaTrader 4 but it should also work on MetaTrader 5). And I love this, because C#:

• is much faster in running complex calculations than MT4 is,
• it allows to implement multithreading,
• provides all the flexibility of a modern Object Oriented Language. 

The method is well described here: https://sites.google.com/site/robertgiesecke/ It is just a Visual Studio Project Template that will modify the compilation options and expose managed methods for unmanaged calls. Short problem background: MetaTrader is written in C++ and allows linking external libraries with functions, variables, etc... If you create a library in C++, the code is compiled and the *.dll can be executed directly (so the *.dll is a set of instructions that can be sent to the CPU and executed). Linking C++ libraries with MetaTrader is extremely easy. However, in managed languages like C# or VB.NET, the code is compiled into an Intermediate Language (MSIL). This is then run by the run-time engine (.NET Framework) and translated into the machine code that is executed by the CPU. In general there are many methods of calling the managed code from unmanaged one (to name a few: C++/CLI wrappers or COM interop), but this one is definitely the easiest.

So how I am using C# libraries? I have a SQL Server database containing good quality 1-minute data for several instruments. And I use C# to execute algorithms against this data set. Example: In the current project I am working on I have completely moved the processing logic to C# through three functions:


C# Definitions:

MQL Imports:

I use the C# library in the following way:

1. When the Expert Advisor starts (MQL init function) I call the initDB function. What this function does, it will initialize the C# algorithm and load the data from SQL Server to memory (so the calculations will be placed fully in C# to improve calculations performance). 
2. Then I just pass some data from MetaTrader to getProbability function in C#. This will do all the maths, based on the most recent price data and settings received.  It will return calculation results back to MQL and MQL will execute the trading logic. 
3. When the expert advisor will shut down, I am just calling the deInitDB to cleanup after the algorithm. 

More information about what I am trying to achieve through this will follow...

Historical data

I have spent some time looking for good source of 1 minute intraday data. While good quality forex data seems to be pretty easy to get, even for free (for example through Alpari or Dukascopy trading platform), it's much harder with other instruments.

I was interested in getting DAX futures history. EUREX, is selling DAX data, at 200 Euro per month for the tick data set.  But I was able to find a decent quality 1 min data and I paid just 280 Euro ($400) for ca. 10 years 1-minute data. I tried a few other sources, but anything cheaper than that was just rubbish (with a lot of bad ticks or gaps which made it impossible use it for strategy testing of any kind).

There are a few vendors, I do not want to encourage or discourage you to use any particular one, but I tried TickDataMarket and TickData and was satisfied with both.

Simplicity is the ultimate sophistication

One of my favorite trading systems is a scalper of my own that I use at GO! Standard account. I love this strategy because it is so damn simple and it works much, much better than hundreds more complex things I tried...  

It uses only a few moving averages, as well as calculate price deviations. By doing this it looks at three dimensions of price movement:

1. direction 
2. deviation
3. volatility  

Idea in short: if there is some consistent direction of the price movement, and price moves sharply* enough against it (without breaking the most recent retracements deviation pattern) - the system enters trade in the direction of the main move.

Sharply* is very relative but it is the key. Obviously the system doesn't try to match the main trend volatility, but rather the most recent volatility. For example: it looks for a sharp retracement in the morning compared to nightly volatility. This volatility is in fact very small compared to day-time one, but the short term deviation will be negative and main trade direction maintained. In general, smaller volatility periods are present very often during all kind of trends and the trades are distributed almost equally throughout the day.

This system works on multiple timeframes and symbols (currencies as well as indices), however my favorite one is EURUSD 1M. The Risk to Reward ratio is dynamic, depends on pair / timeframe. 

I even thought about publishing source code of this EA... But I won't. Why? Why should I...? EURUSD 1M backtest:

China

Right now I am spending some time on a business trip in Shanghai. And while everyone thinks about the Eurozone debt crisis, in China things aren’t going well either. In the center there are lots of tall living houses, in which two bedroom flats cost ca. $700 k. One family houses nearby cost $4 mln each... Those prices are close to insane compared to what you can get for this money in other parts of the world.. But still in Shanghai there are thousands of construction sites... PMI fallen recently below 50, and GDP is not growing as fast as it used to grow. Also when I look at for instance IT services rates, those are becoming closer to those in Central European rates, so growing at 7% pace will not be as easy as it used to be in the past… Government is focusing on innovations and services, however transformation from an industrial economy will take many years.

Not only in Europe things aren't looking very promising… 

Step Money Management

Money Management is crucial... It will not make a system profitable, but it seriously affects profitability. Most of the trading systems allocate some account percentage per each trade... While this method in theory protects you before blowing up the account, it also seriously limits the profitability for some types of trading systems. Just consider the following example: trading system enters a loosing streak and decreases the lot sizes. Then when it starts winning again first winners win less amount of money that system actually actually lost (because the lot size is decreased). Especially if a system has low winning percentage but high risk:reward ratio. 

For those kind of systems I use step money management. It basically calculates the lot size not based on actual account balance but based on highest balance. This money management method will not suite all trading strategies. As said, it is best for system that have low winning percentage, but when they win - they win big (trend followers and breakout systems). So when a win comes - it will win with larger lot value. 

Code that implements discussed step money management is placed below. It requires specifying maximum  initial account balance.

   int round;
   double lotStep;
   double tickvalue = MarketInfo(Symbol(), MODE_TICKVALUE);
   if (Digits == 5) tickvalue = 10.0 * tickvalue;
   if (Digits == 3) tickvalue = 10.0 * tickvalue;
   lotStep = MarketInfo(Symbol(), MODE_LOTSTEP);
   if (lotStep == 0.01) round = 2;
   if (lotStep == 0.1) round = 1;
   if (lotStep == 1.0) round = 0;


   if(AccountBalance()>maxAccountBalance)
      maxAccountBalance=AccountBalance();
   
   return(MathAbs(NormalizeDouble(riskPercent1 * maxAccountBalance / 100.0 / mmStopLoss / maxOrders / tickvalue, round)));
  

Broker selection

What really amazes me is that most of the people think that everything depends from the Broker... And that 90% of the brokers are there to fool them... Well, brokers are there to make money - that's obvious, but don't need to steal it... There are three kind of brokers:

• MM (Market Makers, basically they are adding they commission into the spread) 
• STP NDD (Straight through processing, without dealing desk)
• ECN (well I haven't seen a true, retail ECN broker for small accounts... With an ECN broker, after placing an order, it should appear in the market depth breakdown... Does it happen? I haven't seen it so far...). 

Everyone moans around Market Makers. People say that Market Makers b-book trades (explanation: a-book trades are the trades executed on the inter-bank market, while b-book trades are trades that broker doesn't pass to inter-bank, but matches with other orders and positions he has). Of course they do, but if someone wants to sell and someone else wants to buy, why they should put this kind of order on the inter-bank? You think they do not have enough orders to do it? Well, look here: http://www.myfxbook.com/forex-broker-volume - some brokers have on average over 80 lots per minute on EURUSD and this only includes accounts monitored by MyFxBook. They must have hundreds if not thousands of lots traded per minute... You still think they are not able to much the orders on their side? Of course, how much trades is a-booked and b-booked... Nobody knows... 

Secondly, people often claim, that the charts are different at MM brokers... Yes, they are, but this what I love in those brokers. The charts have less peaks, are more smooth, there is also less ticks usually.. I especially prefer 4 digit brokers. Those are perfect condition for SOME automated trading systems. 

Thirdly, MM brokers often have fixed spread, on average the spread is worst than at an STP NDD broker. But fixed, means fixes, also during nights. Find me an STP NDD broker with 1pip spread on EURUSD through the entire night (+ commission)...!

In regard to the STP NDD brokers, the largest advantage of a STP NDD are tight stop / take profit levels (often 0 pips) as well as much faster execution times (very often close to 200ms per order, which for a retail broker is a top notch result). 

Trading style

Currently I mostly use automated strategies. I stared from Metastock software (I know most of you don't even know what it is...) and created first systems based on some simple technical indicators in 2004 and 2005. Right now I am focusing almost exclusively on MetaTrader 4. It's not a perfect platform, especially the strategy tester, however the best one available for the retail market in my opinion. The things I am missing in particular are:

• more classical approach to strategy testing & optimization: two data sets: training and probe one. Testing parameters selected during optimization on training set should be then verified on the probe one. I am not even mentioning Walk Forward Analysis...
• simple way to support tick data backtesting, without it, it's almost impossible to test any kind of HFT scalpers.. The only way is forward testing.
• concurrent backtests on several currencies / timeframes.

Biggest pros? Ability to link external dll's.