Differences between optimization and backtesting results

02 Feb 2019, 18:51Differences between optimization and backtesting results#1
cysecsbin.01posts: 17since: 10 Nov 2018

hi, i was optimizing a simple cbot, and when i tried to backtest one of the results trough the 'apply' option, the backtest was completely different, here i'll post some pics:

so, here is the optimization screen, as you can see i selected the best fit result

this is the backtest, the timespan selected for the test reflects the one of the optimization, both are tick data accurate, and the parameters are all correctly applied, however the results are the opposite. in the optimization's equity graph a linear growth is seen, in the bt result, a linear decay. how is this possible? is there anything wrong in any part of the procedure?

Thanks in advance

02 Feb 2019, 18:54#2
cysecsbin.01posts: 17since: 10 Nov 2018

by the way, i tried the same procedure both on fxpro's ctrader and on the open beta frome spotware's site, same strange outcome

04 Feb 2019, 10:26#3
Panagiotis Charalampousposts: 3718since: 13 Jan 2017

Hi cysecsbin.01,

Can you share the cBot with us so that we can reproduce? Also please let us know the cBot parameters you are optimizing.

Best Regards,

Panagiotis


Head of Community Management at cTrader -Join us on Telegram - https://t.me/cTrader_Official
04 Feb 2019, 11:07#4
cysecsbin.01posts: 17since: 10 Nov 2018

Thanks for your answer, i'll send the indicators' and bot's sources


using System;
using cAlgo.API;
using cAlgo.API.Internals;
using cAlgo.API.Indicators;
using cAlgo.Indicators;

namespace cAlgo
{
    [Indicator(IsOverlay = false, TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class ShutForce : Indicator
    {
        [Parameter("RSI Periods", DefaultValue = 14)]
        public int Rsi_per { get; set; }
        [Parameter("ATR Periods", DefaultValue = 14)]
        public int ATR_per { get; set; }
        [Parameter("TimeFrame")]
        public TimeFrame tf { get; set; }

        [Output("Main", Color = Colors.Black)]
        public IndicatorDataSeries Result { get; set; }

        private AverageTrueRange atr;
        private RelativeStrengthIndex rsi;
        private MarketSeries series;


        protected override void Initialize()
        {
            series = MarketData.GetSeries(tf);
            rsi = Indicators.RelativeStrengthIndex(series.Close, Rsi_per);
            atr = Indicators.AverageTrueRange(series, ATR_per, MovingAverageType.Simple);
        }

        public override void Calculate(int index)
        {
            int i = series.OpenTime.GetIndexByTime(MarketSeries.OpenTime[index]);
            if (rsi.Result[i] > rsi.Result[i - 1] && atr.Result[i] > atr.Result[i - 1])
            {
                Result[index] = 1;
                ChartObjects.DrawLine("line" + index, index, -1, index, 1, Colors.Green, 5, LineStyle.Solid);
            }
            else if (rsi.Result[i] < rsi.Result[i - 1] && atr.Result[i] > atr.Result[i - 1])
            {
                Result[index] = -1;
                ChartObjects.DrawLine("line" + index, index, -1, index, 1, Colors.Red, 5, LineStyle.Solid);
            }
            else
            {
                Result[index] = 0;
                ChartObjects.DrawLine("line" + index, index, -1, index, 1, Colors.Blue, 5, LineStyle.Solid);
            }


        }
    }
}

indicator 1


using System;
using cAlgo.API;
using cAlgo.API.Internals;
using cAlgo.API.Indicators;
using cAlgo.Indicators;

namespace cAlgo
{
    [Indicator(IsOverlay = false, TimeZone = TimeZones.UTC, AccessRights = AccessRights.None)]
    public class TickVolume : Indicator
    {
        [Parameter(DefaultValue = false)]
        public bool Segno { get; set; }
        [Parameter(DefaultValue = 10)]
        public int Periodi { get; set; }

        [Output("Main", PlotType = PlotType.Histogram)]
        public IndicatorDataSeries Result { get; set; }
        [Output("EMA", Color = Colors.Cyan)]
        public IndicatorDataSeries EMA { get; set; }

        private ExponentialMovingAverage ema;


        protected override void Initialize()
        {
            ema = Indicators.ExponentialMovingAverage(MarketSeries.TickVolume, Periodi);
        }

        public override void Calculate(int index)
        {
            if (!Segno)
                Result[index] = MarketSeries.TickVolume[index];
            else if (Segno)
                if (Math.Max(MarketSeries.Close[index], MarketSeries.Open[index]) == MarketSeries.Close[index])
                {
                    Result[index] = MarketSeries.TickVolume[index];
                }
                else
                {
                    Result[index] = -MarketSeries.TickVolume[index];
                }
            EMA[index] = ema.Result[index];
        }
    }
}

indicator 2


using System;
using System.Linq;
using cAlgo.API;
using cAlgo.API.Indicators;
using cAlgo.API.Internals;
using cAlgo.Indicators;

namespace cAlgo.Robots
{
    [Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.FullAccess)]
    public class ShutForce_Bot : Robot
    {
        [Parameter("RSI Periods", DefaultValue = 14)]
        public int Rsi_per { get; set; }
        [Parameter("ATR Periods", DefaultValue = 14)]
        public int ATR_per { get; set; }
        [Parameter("MTF RSI Periods", DefaultValue = 14)]
        public int mtf_Rsi_per { get; set; }
        [Parameter("MTF ATR Periods", DefaultValue = 14)]
        public int mtf_ATR_per { get; set; }
        [Parameter("Volume Average Periods", DefaultValue = 14)]
        public int tv_per { get; set; }
        [Parameter("SL Multiplier", DefaultValue = 1)]
        public double sl_mul { get; set; }
        [Parameter("TP to SL Ratio", DefaultValue = 1)]
        public double tp_to_sl { get; set; }
        [Parameter("Rischio %", DefaultValue = 1)]
        public double risk { get; set; }
        /*[Parameter("Trailing Start", DefaultValue = 10)]
        public double trl_start { get; set; }
        [Parameter("Trailing Step", DefaultValue = 1)]
        public double trl_step { get; set; }*/
        [Parameter("TimeFrame Secondario")]
        public TimeFrame tf { get; set; }

        private ShutForce sf, sf_mtf;
        private MarketSeries mtf_series;
        private ExponentialMovingAverage ema;
        private TickVolume tv;

        protected override void OnStart()
        {
            mtf_series = MarketData.GetSeries(tf);
            sf = Indicators.GetIndicator<ShutForce>(Rsi_per, ATR_per, TimeFrame);
            sf_mtf = Indicators.GetIndicator<ShutForce>(mtf_Rsi_per, mtf_ATR_per, tf);
            ema = Indicators.ExponentialMovingAverage(MarketSeries.Close, 200);
            tv = Indicators.GetIndicator<TickVolume>(false, tv_per);
        }
//&& MarketSeries.Close.LastValue > ema.Result.LastValue&& MarketSeries.Close.LastValue < ema.Result.LastValue
        protected override void OnBar()
        {
            CancelOrders();
            double sl = sl_mul * (MarketSeries.High.Last(1) - MarketSeries.Low.Last(1)) / Symbol.PipSize;
            double tp = sl * tp_to_sl;
            string label = MarketSeries.OpenTime.GetIndexByTime(MarketSeries.OpenTime.Last(1)).ToString();
            int index_mtf = mtf_series.OpenTime.GetIndexByTime(MarketSeries.OpenTime.Last(0));
            if (sf.Result.Last(1) == 1 && sf_mtf.Result[index_mtf - 1] == 1 && tv.Result.Last(1) > tv.EMA.Last(1))
            {
                PlaceStopOrder(TradeType.Buy, Symbol, Volume(sl), MarketSeries.High.Last(1), label, sl, tp);
            }
            else if (sf.Result.Last(1) == -1 && sf_mtf.Result[index_mtf - 1] == -1 && tv.Result.Last(1) > tv.EMA.Last(1))
            {
                PlaceStopOrder(TradeType.Sell, Symbol, Volume(sl), MarketSeries.Low.Last(1), label, sl, tp);
            }
        }

        protected override void OnTick()
        {
            //Trailing();
        }

        private void CancelOrders()
        {
            foreach (var order in PendingOrders)
            {
                CancelPendingOrder(order);
            }
        }

        private int Volume(double sl)
        {
            double _2_percent = risk * Account.Balance / 100;
            double vol = (_2_percent * Symbol.Ask) / (Symbol.PipSize * sl);
            int vol_int = 1000 * (int)Math.Round(vol / 1000, 0);
            if (vol_int < 1000)
                return 1000;
            return vol_int;
        }

        /* private void Trailing()
        {
            foreach (var pos in Positions)
            {
                if (pos.Pips > trl_start)
                    ModifyPosition(pos, pos.EntryPrice);
            }
        }*/

        protected override void OnStop()
        {
            foreach (var pos in Positions)
            {
                ClosePosition(pos);
            }
        }
    }
}

bot


these are the optimization parameters, i used grid search, tick data and a period from 21/06/2018 to 30/01/2019.

ctraders used are both spotware's open beta and fxpro's

the optimization takes a while but the issue is displayed on any positive result when transported in the backtest section.

i kindly thank you for your help, it is much appreciated.

04 Feb 2019, 11:30#5
Panagiotis Charalampousposts: 3718since: 13 Jan 2017

Hi cysecsbin.01,

Thanks I will have a look. However in the previous images timeframe is set to h1 while in the last one it is set to h4. Can you tell me which one is the correct?

Best Regards,

Panagiotis


Head of Community Management at cTrader -Join us on Telegram - https://t.me/cTrader_Official
04 Feb 2019, 15:41#6
cysecsbin.01posts: 17since: 10 Nov 2018

the correct setting for the timeframe to apply the bot to is h1, the 'timeftrame secondario' = auxiliary timeframe from italian, should be set to h4.

however the last pic with the optimization parameters settings is correct.

05 Feb 2019, 17:20#7
Panagiotis Charalampousposts: 3718since: 13 Jan 2017

Hi cysecsbin.01,

Thanks, we managed to reproduce this. The problem is with the optimization results. The backtesting results are correct. We will fix in in an upcoming update.

Best Regards,

Panagiotis


Head of Community Management at cTrader -Join us on Telegram - https://t.me/cTrader_Official
13 Aug 2019, 17:06RE:#8
leohermosoposts: 35since: 31 Jan 2016

Panagiotis Charalampous said:

Hi cysecsbin.01,

Thanks, we managed to reproduce this. The problem is with the optimization results. The backtesting results are correct. We will fix in in an upcoming update.

Best Regards,

Panagiotis

Hi, Is this problem solved? I am getting different results between opt and backtests. I think its caused by MarketData.GetSeries that MAYBE is not pre allocating the correct amount of data to initialize the indicators. 

13 Aug 2019, 17:23#9
leohermosoposts: 35since: 31 Jan 2016

BackTest 1Backtest 2Both are backtesting results

 

OPT 1OPT2

These both are optimization results.

 

 

I use daily data inside this 1-minute bot, and the parameter attached to daily data is 35 periods and its roughly the period that optimization and backtest begin to behave accordingly, so it makes me think to backtest are not correct pre-allocating the necessary data to correctly load the indicator

13 Aug 2019, 17:28#10
Panagiotis Charalampousposts: 3718since: 13 Jan 2017

Hi leohermoso,

No it has not been fixed yet.

Best Regards,

Panagiotis


Head of Community Management at cTrader -Join us on Telegram - https://t.me/cTrader_Official