You are not logged in or registered. Please login or register to use the full functionality of this board...

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
What hardware is adequate for tick backtesting?
#11
I personnally have dedicated server for that Smile
You are not allowed to view links. Register or Login to view.

I have VMs running 1 MT4 instances each as I am testing many pairs, but it might not be available to anyone.
I am doing many optimizations at the same time. 

The problem in case of PA would be the licensing.

What I would recommend if you plan to do Optimization (can take up to 1 or 2 months for 10 000 BTs on 3 years), is to either have dedicated Server for this, or use Virtual Machine.

You might loose 10% of calculation throughput,  but you can make snapshots or "Pause" the opt. for example. I run sometime Optimization on my personal computer, but I want to shut it down over the night. VMWare allow to pause the VM Smile
Reply
Thanks given by:
#12
LL,

In case it’s not too late and is of any value, I have the Intel 4790(K) you were considering.  It has 4 physical cores and 4 virtual cores; (I also have 8GB of RAM and a good 512GB SSD).  MT4 is a single-threaded application, so 4 instances could probably run well if you can feed all the data fast enough to each instance; 8 instances probably not quite as well.

For some reference, backtesting one instance of MT4 consumes about 13% of my 4790K CPU.  Two instances consumes about 26% and begins to generate significantly more heat.  A stock Intel CPU cooler would not be my recommendation when pushing a CPU for extended periods.

To carry your desires forward, eight instances on a 4790 would likely consume about 100% of the CPU cycles and a special liquid CPU cooling system of some sort would probably be a good idea if running for extended periods.  Yes, for more than eight instances, you begin looking at server CPUs with 6, 8, 10, or 12 cores, or even server motherboards holding multiple CPUs.  $$$$.  

I have trouble keeping track of more than two tests running at the same time.  :-)  I do like to do a survey to pick my favorites, but then concentrate on those few.

Best of luck.
Reply
Thanks given by:
#13
(07-01-2015, 07:01 PM)dcon Wrote: You are not allowed to view links. Register or Login to view.LL,

In case it’s not too late and is of any value, I have the Intel 4790(K) you were considering.  It has 4 physical cores and 4 virtual cores; (I also have 8GB of RAM and a good 512GB SSD).  MT4 is a single-threaded application, so 4 instances could probably run well if you can feed all the data fast enough to each instance; 8 instances probably not quite as well.

For some reference, backtesting one instance of MT4 consumes about 13% of my 4790K CPU.  Two instances consumes about 26% and begins to generate significantly more heat.  A stock Intel CPU cooler would not be my recommendation when pushing a CPU for extended periods.

To carry your desires forward, eight instances on a 4790 would likely consume about 100% of the CPU cycles and a special liquid CPU cooling system of some sort would probably be a good idea if running for extended periods.  Yes, for more than eight instances, you begin looking at server CPUs with 6, 8, 10, or 12 cores, or even server motherboards holding multiple CPUs.  $$$$.  

I have trouble keeping track of more than two tests running at the same time.  :-)  I do like to do a survey to pick my favorites, but then concentrate on those few.

Best of luck.

Many thanks, dcon, for your advice.

Just as you advised, simultaneous backtesting of several pairs easily increase CPU demand and $$$ demand such that I limit myself to two backtestings on my computer which seems ok now.

Many thanks again.

llforex
Reply
Thanks given by:
#14
(05-24-2015, 05:14 PM)llforex Wrote: You are not allowed to view links. Register or Login to view.Dear d4v3,

Now that I have switched to tick data from fxcm data for backtesting of PA, I realized the backtesting now is so much slower:  In 24 hours with fxcm I could backtest 15 years (2000-2015), with tick data I could hardly test several months, not even one year.  With this snail speed, my beard would be 9 inches long before I could finish the pairs with variations of the preset!

The solution for this problem is simultaneous backtesting of many pairs with different preset variations.  The time has come to upgrade my hardware.  But I know so little about computer hardware and what they can do.  On the one hand, there is no need to waste money, on the other hand I wish to get the testing done timely.  

Would you please advise:  for my purpose of simultaneously running 20 mt4 backtesting, at least running 10 mt4 backtesting, what minimum computer configuration is necessary?  Would the following configuration get the job done?

CUSTOM BUILT INTEL I7-4790 COMPUTER SYSTEM
  • Intel Core i7 Processor i7-4790 3.6GHz 8MB QUAD CORE

  • INTEL COPPER HEAVY DUTY LGA 1150 COOLING FAN

  • ZEROtherm Advanced ZT100 Thermal Grease

  • PROFESSIONAL WIRING

  • ASUS B85M-G LGA 1150 Intel B85 HDMI SATA 6Gb/s USB 3.0 uATX Intel Motherboard

  • 16GB (2 x 8GB) 240-Pin DDR3 1600MHz (PC3 12800) Dual You are not allowed to view links. Register or Login to view. (HIGH PERFORMANCE)

  • (2X) MEMORY HEAT SPREADERS

  • WD Green 3TB 3.5" Internal Hard Drive - OEM Packaging #WD30EZRX

  • 24X DUAL LAYER DVD-RW

  • INTEL HD 1GB HDMI/DVI 1080p PCI-EXPRESS VIDEO (ONBOARD)

  • REALTEK 8-You are not allowed to view links. Register or Login to view. DIGITAL SOUND ONBOARD

  • REALTEK 10/100/1000 Gigabit Network Card (onboard)

  • Antec NEW SOLUTION SERIES VSK-4000 Black SGCC steel ATX Mid Tower Computer Case

  • (2X) 120MM CASE FANS

  • hec X ORION 585 WATT POWER SUPPLY

  • 3-Year Limited Warranty Plan with Lifetime of free USA based Support Custom Hand Wiring For Ultimate Air Flow, Assembled in Cleveland, Ohio, USA

  • FED-EX GROUND SHIPPING (1-7 BUSINESS DAYS)
Thank you very much for your advice.

llforex

thats what i have its very good configuration the only thing i changed is my hard drive is SSD which makes it way much faster & boots in 10 sec. .
Reply
Thanks given by:
#15
LL,

It occurred to me to actually test up to 8 instances to see what would happen on a 4 physical core Intel 4790K CPU that has 4 additional virtual cores, that has a large heat-pipe CPU cooler.  Good news for multi-taskers: the temperature did not continue to increase in a linear fashion.

Ambient test temperature 25°C/79°F

Results by # of MT4 terminal instances running a backtest optimization, data fed by Samsung 840 Pro SSD.
(numbers moved around a little, but these seemed typical)

I   CPU%   °C     °F
0      0       32     89
1    13       46    113
2    26       53    128
3    37       56    132
4    50       59    136
5    62       61    141
6    74       63    144
7    87       65    148
8   100      66    150
Reply
Thanks given by:
#16
Many thanks, xplorfx and dcon, for sharing your computer experiences.  Next time I replace my computer, I know what to do.

Merci beaucoup!

llforex
Reply
Thanks given by:
#17
Im running algo's but while Im doing so I need 120 days of data and my current processor takes 5 to 10 minutes to get data back to me, then another 5 to 10 minutes after reloading ninja script, ideally I need the data to get back to me as quick as possible since I would like to backtest ALL DAY long, but lag is killing me and im assuming its a processor issue since im currently running a amd older processor.....what would the best processor be to get the info to me as quick as possible, fyi my internet speed is not the problem I have a download speed of 50mbs and upload around 12 thank you

You are not allowed to view links. Register or Login to view.



Reply
Thanks given by:
#18
More than a faster processor, you need Tick Data offiline so you don't have to download it everytime.

Look for TickStory or Birt's Tick Data Suite.
The Titanic was built by professionals, but Noah's Ark was built by amateurs.
The staff does not supply assistance via private mail.
Please open a new discussion in the appropriate section and you will be answered.
Reply
Thanks given by:
#19
(10-05-2017, 09:39 AM)The Tramp Wrote: You are not allowed to view links. Register or Login to view.More than a faster processor, you need Tick Data offiline so you don't have to download it everytime.

Look for TickStory or Birt's Tick Data Suite.

I know this thread has been quiet for some time, but I'd just like to add my plus one for Tick Data Suite. It's very good, with plenty of tutorials on how to have multiple instances all using a single MQL4 directory, so that when you add indicators or EAs to the first instance, you are also adding them to the rest.

On the subject of number of instances to number of cores. I've always been led to believe that you should only use one instance per physical core, trying to use virtual cores will see a drop in performance. I stand to be corrected on that one though.

Next, RAM. Try to have at least 2Gb or RAM per instance of MT4.

SSD, you must have an SSD, spinning HDDs are just too slow to cope with the frequent paging to disk that the instances of MT4 do.

Next.... before anyone spends big money building a fancy new PC, consider looking for an old Dell Workstation such as a T5400.

I got a 2 x Quad Core Xeon setup with 16Gb of RAM for £120 on eBay, to which I needed to add a 120Gb SSD and a copy of Windows 7. 

Effectively I have an 8 Core backtesting machine for £200. (~$250).

Hope this helps someone.

GG
Reply
Thanks given by:


Possibly Related Threads...
Thread Author Replies Views Last Post
  Additional CID for backtesting? hopfi2k 8 2,332 08-18-2015, 12:37 PM
Last Post: bmigette
  How to store tick data on drive D? llforex 14 3,293 07-01-2015, 06:10 PM
Last Post: bmigette



Users browsing this thread: 1 Guest(s)
Welcome, Guest
You have to register before you can post on our site.


  

Password
  





Online Users
There are currently 49 online users.
» 0 Member(s) | 49 Guest(s)

Search Forums

(Advanced Search)

Forum Statistics
» Members: 8,962
» Latest member: najjarja
» Forum threads: 2,820
» Forum posts: 25,555

Full Statistics