Wednesday, 25 April 2018

Importing Back-test Data in Metatrader4

Back-testing with Metatrader4 can be a bit tricky for the unwary as there are some quirks that you need to be aware of and if you're not careful you can end up back-testing with random data rather than the good quality data that you thought you were. In this blog post I go through how to ensure that your back-test is what you think it is. I also walk you through the necessary steps for importing and converting your data to different time-frames. This is particularly relevant if you want to use good quality back-test data of your own and are worried about it being "contaminated" by data from other sources. Incidentally, you may wish to take a look at the Snapdragon Historic Data Service as a good source of clean historic data for back-testing.

Part 1 - Clearing Out Old History Data
The first thing to be aware of is that the data that is used in the Mt4 back-test is stored in the history folder. You can look at this folder by going to the File | Open Data Folder menu option which will bring up the Windows Explorer open at the root folder for your installation. If you then open the history folder and select your broker folder you will be able to see all the history files which will have names like EURUSD1.hst. These are the same files that you can see when you select the  File | Open offline menu option instead, but in that instance they are all listed in a dialog box and you can't delete them. So that's where the data is stored. What you also need to know is that when Mt4 closes down, it writes out all the history data to this folder. So if you want to clear all your history data for example then you need to do it in the following way:

1. Open up the folder in Windows Explorer by going to File | Open Data Folder
2. Now close down Mt4 (but leave Windows Explorer open)
3. Now to delete all the data for EURUSD say, navigate down to the appropriate files and delete all the EURUSD*.hst files.

If you're importing your own data for a back-test then I recommend that you first clear out the old data using this method.

Part 2 - Importing Your Own Data
To import some data of your own into the history folder then this is done using the History Center dialog box, which is brought up using the Tools | History Center menu option (or just the F2 key). Once you bring up this dialog box you will see something like this 

You can see down the left-hand side are listed all the markets and by drilling down into them you can see sub-folders for each of the available time-frames. One important thing to note is that in order to select one of these sub-folders you need to double-click on it - just selecting it is not enough. If you look in the title bar of the dialog box it shows you which folder is currently active.

A really important fact to note is that when you double-click on a folder Mt4 "helpfully" goes and populates it with some data of its own. So even after you've deleted all the history data for a pair as described in Part 1 it will still go and put some more in that it's downloaded from its servers. The best way around this issue is to double-click on the M1 folder, let it populate the list with its own data and then to delete it again. To do this, highlight the top element, hold down the shift key and then scroll to the bottom and click on the last element. This should now highlight all the elements in the list. Now click on the Delete button to delete them all. This deletes all the entries in the list. 

The next thing to do is to import your own data. You do this by clicking the Import button and then navigating to the file that you want to import. Once you've imported your own data, Mt4 won't overwrite it, but it will still add stuff beyond the end of your data. There's not a lot you can do about this but as long as you know what date your data goes up to and are careful only to test up to that point then it should be OK. You should now have your own imported data in the list.

You should now see your data in the dialog box. Finally to make sure that it's saved to the history folder you should close down Mt4.

Part 3 - Populating Different Time-frames
Now, when running back-tests, the Strategy Tester sometimes pulls in data from different time-frames in order to construct the bar data that you are back-testing. The best way to ensure that you are only using your own data is to populate all the time-frames with your own data. Even if you're only back-testing one time-frame it is still recommended that you populate all the others just to be on the safe side. Now you may only have 1 minute data so you how do you create M5, M15 etc from this? The answer is to use the Period Converter script. Below are the necessary steps for doing this

1. The first thing that you need to do is to change the Max bars in history and Max bars in chart options in your Mt4 to ensure that you can load in all your data without any problems. Go to Tools | Options and then select the Charts tab from the dialog box. Set the Max bars in history and the Max bars in chart options to as big a number as you can fit in there: I just keep putting 9's in there until it won't take any more.

2. Next you need to go to the  File | Open offline menu option which will bring up the dialog box showing all the history data. You should see the file for the symbol that you've imported already, so your EURUSD,M1 file for example. Select this and click on the Open button. This will bring up a chart and because of your Max bars settings it will load all that it can into the chart.

3. Next we will need to generate the other time-frames. To do this we can use the Period Converter script that comes with your Mt4 installation. Drop the script into the chart and set the Period multiplier factor to 5 first of all to generate the 5 minute data. Note that you won't see anything happen on the chart but in the Experts tab it will say X records written.

4. Now repeat this process for all the time-frames that you are likely to need, so 15, 30, 60, 240 etc. I usually don't bother with the daily if I'm not testing a daily strategy but will do all the intraday time-frames.

5. Once you've done this then if you bring up the Offline Chart dialog (File | Open offline) then you should see all the different time-frames now listed there.

6. Finally, remember to set your two Max bars settings back to something moderate, otherwise it will slow your Mt4 right down.

7. The last step is to close down Mt4. This will write out all the history data to the folder so it's now backed up.

Part 4 - Using Your Data
That should be it! You have now created all the history data that you want populated with your own data source and you can go ahead and do your back-tests with the Strategy Tester. Just make sure that you specify the end date from the test so it stops at or before your own data does. Otherwise you'll be using some data that Mt4 has provided.

Thursday, 19 April 2018

Snapdragon Systems Historic Data Service

Clean financial market data for back-testing
Through our various trading platforms and data feeds we have accumulated a reasonable history of finanical market data over the years. What we have now done is to clean it up (where possible it's from a single reliable and tradeable source), and convert it into a format that can easily be imported either into ForexTester or into Metatrader4 for back-testing. Below are the key points of the service

  • Spot forex: all pair combinations from the 8 major markets + some key CFD markets as well
  •  Goes back to the start of 2006 in 1 minute bars
  •  Comes in Ascii format for easy importing across a variety of platforms.
  • Will load straight into the ForexTester back-testing platform and also into the Metatrader4 History Center for Strategy Tester back-tests

Data is adjusted for the daylight savings difference between the US and Europe so you can either have the New York close always at the same time or the London Open always at the same time. Both sets of data are supplied.

  •     Price is GBP 100.00 to download all the data along with weekly updates for one month.
  •    Thereafter you can get weekly updates for GBP 60.00 per quarter. NB this is optional, so if you just want a one-off data dump then simply don't renew your subscription at the end of the first month.

For more enquiries or to order please get in touch via the Snapdragon Systems contact page on our web-site

We can also source other markets and time-frames. If you have a particular requirement that isn't covered by the Historic Data Service then please ask.

Full List of Markets Convered



Thursday, 2 November 2017

Video on the Easy Order Entry Script for Forex Tester

This is a video on the "Easy Order Entry" script for ForexTester. It automates the entire position sizing calculation at the click of a button, making for much easier ForexTester back-tests! 

Monday, 4 September 2017

Riskfree Pyramid Addon Strategy for ForexTester

For users of ForexTester (and if you don't then perhaps view our review here), this is a strategy that can be used within the ForexTester software for aggressively pyramiding trades in a risk-free way.

Basically it does three add-ons and at each one the stop-loss on the combined position is moved to break-even.

By doing this multiple times very aggressive returns can be achieved with no additional risk apart from that of the initial trade.

For more details please watch our video below

Risk-free Pyramid Addon for Forex Tester

Tuesday, 29 August 2017

Forex Tester - Forex Back-testing Software

Over the years there have been any number of back-testing platforms but one which I've recently come across and which I really like is ForexTester. This has been designed very much to look like MetaTrader 4, which is my preferred FX platform anyway so I feel very much at home with it. What it enables you to do is to step through historic data on a bar by bar basis (or even tick by tick if you really want) and to place trades on the chart using a simple right-click just as you would for Mt4. It has drag-and-drop for it's stop losses and take profit levels and you can trade multiple positions, trade hedged positions in fact everything that you would expect to do on Mt4. It will keep track of all your fills, positions and account equity so it feels just like the real thing. In this way you can back-test your approach across years of historic data, thereby gaining a huge amount of market experience at a much faster pace than you would otherwise be able to if just forward testing in real time.

ForexTester in action
So, what are the pro's and con's? Well, if you're looking for a programmatic testing platform then this isn't really the ideal platform. For sure you can code up indicators, scripts and strategies on it but if you have fairly simple back-testing requirements then something like TradeStation or MultiCharts is going to be much better. The one exception to this would be if your strategy involved hedged positions (so both long and short positions simultaneously) in which case the two previous platforms can't easily accommodate it. In such situations I generally use ForexTester, coding my strategy up into a dll "EA" ForexTester Strategy in order to do all my testing.

So if this isn't a programmatic platform then what is it? Well, this is rather for more discretionary traders. If you trade using any method where there's an element of discretion or of "eyeballing the chart" then this is the perfect platform. Perhaps trend-line breaks, support or resistance or Fibonacci levels, all methods which are very hard to do programmatically, for all such approaches this would be the ideal platform. For those of you who are keen on indicators it has all the common ones at your disposal as well as various drawing tools such as Andrew's Pitchfork, wave counts, Fibonacci etc.

In fact I've become so enamoured with this platform that I've started to develop and sell some add-ons for it (see here for details) to make back-testing even easier and more hassle free. Think of it as the ForexTester equivalent of scripts and EA's. In addition I've also launched a simple historic data service whereby users can download historic data which can be imported into ForexTester.

I'll leave you with this fact: of all the successful professional traders that I know (and I know quite a few these days), the vast majority of them swear by ForexTeser and attribute their success to using it.

Stop Press!
I've now negotiated a special discount on the full price of ForexTester for Snapdragon Systems! Instead of the usual price of $299 it's now $269, so $30 off! To take advantage of this offer visit 
this page here.

Tuesday, 22 August 2017

FXStrategies Web-site

FXStrategies is a great little web-site that I've just recently come across through my association with the trader behind it, Frank Walbaum. Originally from Germany, Frank now lives and trades from his adopted home in Singapore. I got to know Frank at a recent FX conference where we were both speakers and we've kept in touch ever since. Frank is a professional trader who started off working in a family office for three years before moving to managing other people's funds in 2014. He has a refreshingly simple "naked trading" approach with no indicators or bells or whistles: instead it's about candle patterns and support and resistance lines for him. Each morning (9 a.m. UK time) Frank runs a "morning call" where he goes through his trades and discusses what he's presently looking at in the markets. It's also a chance to ask him questions and learn about his approach to trading.

For subscribers who pay his modest monthly fee, his web-site has loads of content including numerous webinar recordings (include some by myself on trading psychology), tutorials, podcasts, Metatrader code and all sorts of other stuff. You also get access to his trade call updates many times a week via Twitter. As a subscriber you can also ask Frank questions directly via Skype or e-mail. He is happy for others to learn how to trade his style and this offers a great opportunity to have direct access to a top professional trader.

So all in all it's a great little site and well worth looking into.

Just some of the tutorial content that's available on the site for subscribers

Friday, 18 August 2017

Free Metatrader 4 Trailing Stop EA

Here's a nifty little trailing stop EA that you can use for free. In fact in our Mt4 programming lessons we're going to be building it ourselves but for those who can't wait here's the finished article.

The Legal Bit
Please note that this EA is supplied "as is" and no warranty is made, either expressly or implicitly as to its behavior. Please do your own due diligence on a demo account before using it on a live account and you use it at your own risk.


The TrailTypeOptions selects which trail type to use.
1 is an N-bar hi/lo trail - so e.g. 5 bar trailing low for a long position or 10 bar trailing high for a short
2 is an SMA trail, so trailing at the price of a simple moving average
3 is an EMA trail so trailing at the price of an exponential moving average

The TrailLength is the trail length to use. For the N-bar hi/lo trail this is the number of bars back for your trail. For the other options it's the moving average length

The StopOffset is how far beyond the calculated point to put your stop. So if you're running a short 10 bar trailing high stop then you might want to place your stop a couple of pips above the 10 bar high to allow for the spread etc. NB this is specified in points - the minimum price movement so if you input 20 then this is 2 pips (for a market that is quoted in pipettes).

If StopOnClose is set to true then the position will only be closed out if the market closes beyond the stop level. NB use this carefully as you don't have a stop in the market at all in this case and if you should lose your internet connection then the EA won't be able to close your position.

For help on how to install it in Mt4 see this post here
For help on how to apply it to your chart see this post here