scotpip
18 Sep 2023, 07:22

RE: Bug? Performance issue with tick-data backtesting

Hi Panagiotis

I'm not finding this a welcoming experience to the community - is this really how bug reports are handled here?

I've explained that I'm experiencing this issue when running the hello world default script.

What more do you want? Instead of giving me the brush off, please explain why you are dismissing this information and insisting that my report will be ignored unless I provide a different script that has the same issue?

I'd also point out yet again that others are reporting the same problem.

To repeat - when I run 8 years of server ticks through the hello world script, the client pauses for around 6 minutes at the end of the run, thrashing my CPU, before writing the stop message to the log and ungreying the run button. With shorter runs I get shorter pauses, but still much longer than I would expect given that this is a dummy script and there is nothing to analyse.

This is 100% repeatable on a Win 10 Pro workstation with 2 x 12 core Xeon processors and 96 GB of memory.

I really don't see how I can be any clearer.

If I was a developer and multiple users reported this issue I would want to check it out rather than ignoring the report. I'm happy to provide any additional information needed, provided the request makes sense and moves the issue forward. 

scotpip
15 Sep 2023, 15:40

RE: RE: RE: RE: Bug? Performance issue with tick-data backtesting

I've shown a simple way to replicate the bug - so I'm not clear what you're objecting to?

Are you backtesting with extended runs of server ticks? Because that's where the issue occurs. 

Also, as I've said, others are reporting the same problem.

It would be comforting if someone from the CTrader team responded. They don't seem to have a formal bug reporting system, so this is all we've got - hopefully they monitor the forum regularly?

scotpip
15 Sep 2023, 15:35

RE: RE: RE: Bug? Performance issue with tick-data backtesting

scotpip
14 Sep 2023, 19:00

RE: Bug? Performance issue with tick-data backtesting

Thanks for responding!

No need for my script - I think this is a generic bug.

To reproduce:

1. I created a new hello-world script and built it without alteration.
    
2. Ran EURUSD from 2014 - 2022 with server ticks.
    
3. From the point the progress bar stopped at the last tick to the point the Stop message was printed to the Log and the Run button was re-activated took very nearly 6 minutes. This is pretty extreme!

During this extended pause, the CTrader client was thrashing my CPU pretty hard. with a darky red High Power Usage flag in the Task Manager. It wasn't leaking memory.

I experience these extended pauses at the end of every tick-resolution run, whatever the script. The longer the run, the longer the pause.

I note that at least one other user has reported a similar issue in the forum.

As I've said, I have an HP Z-series workstation with Windows 10 Pro, 24 Xeon cores and 96 GB of memory, so it's not a client-side resource issue.

Seems to me that you have a significant issue on your side. I'd very much appreciate a fix.

scotpip
12 Sep 2023, 23:53

Just spotted this thread. 

As I reported in my own thread, I'm experiencing the same thing.

My script is simply streaming the ticks for a single symbol to run some simple analysis of the spread.

Then it writes out half a dozen lines to the Log from the OnStop() handler. 

Couldn't be simpler.The bot is not trading, so nothing for CTrader to analyse there.

For short runs of a few weeks, the pause is around 100 secs. For longer runs, the pause can be truly painful - I have to leave the client running in the background and get on with something else.

I have a Win 10 Pro workstation with 24 cores and 96 GB of memory, so it's not a resource issue. Running the script barely registers on the Resource Monitor.

There surely must be something wrong here? With a workstation of this power it's hard to imagine what the client could be doing that would take so long.

scotpip
12 Sep 2023, 18:44

You will find life MUCH more pleasant and productive if you take advantage of CTrader's integration with Visual Studio, VS Code or Rider.

The built-in editor is basic, to put it kindly, and isn't meant for anything serious.

IDEs demand a bit of a learning curve, but it pays off handsomely in the longer run.

scotpip
10 Jan 2023, 23:10

RE: RE: RE:

Hi

It's still in development, but nearly done.

For a developer, generating price bars from the runtime tickstream is trivial. 

If you want to speed things up by serialising completed bars and using them in your backtests you're in for a bigger project, because you'll have to simulate the cTrader exchange. cTrader itself will only know about the bar you specified in the bot instance, and will know nothing of your own bars.

The charting was fairly easy too. I strongly recommend the AnyChart javascript library, which is free for personal use. It is powerful performant and well documented - much the best of the bunch that I evaluated.

There is a feature to backtest against bars in CSV files, but here too they seem to have hard-coded for time-based aggregations.

Indicators are more of an issue. In my use-case accessing the cTrader indicators was more trouble than it's worth - especially as an API class I'd want to extend is sealed. I only use a couple of simple indicators so have built my own system. There are examples of indicator logic in C# and other languages all over github and on TradingView as well as here in the cTrader community if you need anything more exotic.

For me the main development effort has been elsewhere - for serious trading I'm having build a lot of functionality that's not available out of the box. The price-based bars have been a relatively minor part of the project. 

scotpip
30 Nov 2022, 22:12

RE:

Update - seems this was an issue at my end.

When I click on Edit With Visual Studio on a .Net Framework 4.x file, it now opens VS and displays a message that the file is incompatible with the version of VS, which is what I'd have expected.

Haven't changed anything, so this is just one of these mysterious MS issues...

scotpip
29 Nov 2022, 18:15

RE:

I have the same requirement.

I've given up on waiting for Spotware to implement this - it's many years overdue now. It will hopefully appear now they have completed their .Net upgrade, but I can't wait any longer.

So I think that the only solution is to backtest with ticks and create your own range bars. It's a simple bar so this is relatively trivial,

Tick resolution backtesting is slow, but has the advantage of accuracy.

Then you have the problem of visualising your bars, indicators and trades. I'm researching this right now.

I don't think you can create a new chart directly in the CTrader API like to can with MT4 or JForex - though I'm going to post a question to double-check this.

So the main options are to export to a JS or Python charting library. This will be tedious but not particularly difficult. The main issue is deciding which library to use.

For example Highstock has a C# wrapper, though it's pretty limited from what I can see.

For Python you might want to check out Mplfinance: https://medium.com/codex/creating-advanced-financial-charts-with-python-in-one-line-of-code-79f87ed482e8

Python may be the easiest way to go, because there are wrappers that enable you to call Python code directly.

Once I've got something up and running, I'll try to share it with the community.

scotpip
23 May 2022, 17:36

Now that the big upgrade to .Net Core is pretty much done, it would be great to see some progress here and a public commitment to a release date.

Hopefully you will also add the ability to backtest Range and Tick while you're at it? 

The addition of more Renko options would also be welcome - with Median Renko at the least as offered in JForex. Even better would be a configurable Renko, as offered by Sierra, TradingView and a number of MT plugins.

scotpip
06 Jan 2022, 02:14

RE: RE: Backtesting features

amusleh said:,

The Renko/Range back testing will be added in next upcoming versions.

You can calculate any metric  programmatically by using the cBot history and generate a report based on your needs, then you can write it on a CSV or Excel file on your disk.

Regarding your other mentioned points, please open a thread under suggestions section of forum, if your suggested features got enough vote from community then we will consider adding them.

Thanks. 

I'll wait till the port to .Net Core is released - I know that's what's been tying up the development team. Then I'll push for you guys to add full-featured backtesting.

I think it's a genuine commercial opportunity for Spotware. You already have the best FX trading venue and the best API for automated execution. But the backtesting is lacking.

I know from painful experience that there is a HUGE gap in the market for a modern, professional-level end-to-end development environment for retail FX traders. MT5 has major weaknesses as you will know better than most, and the smaller players are little better and offer a very limited choice of brokers. If you need leverage, none of them integrate with offshore operations like IC Markets in the Seychelles. And no-one offers a good solution for Renko and Range. It's all very frustrating.

My current idea is to use AmiBroker for idea exploration and testing, then port the whole thing over to CTrader for execution. Hardly ideal, to put it mildly, but after some pretty active research it seems to be the least bad solution.

Fill the gap with full-featured backtesting and robustness testing and many of the most serious and committed retail traders would find the platform irresistible. You would beat MT5 on every point. And the sophisticated algo traders are often the most serious independents with he highest volumes, generating income for your brokers.

That's the question really. Do you want the casual traders playing around manually on their phones - generating high customer churn and low trading volume? Or do you want the serious systematic traders looking to build big accounts and develop long-term relationships with their brokers?

My proposal will never get many votes in a popularity contest, because few casual traders are prepared to put in the hard yards to develop pro-level algo skills. But it is a big deal for the traders that you really want to attract.

The foundation is there - but you need to fill the gaps first!

scotpip
04 Jan 2022, 17:10

Backtesting features

Hi,

Why do you think cTrader back tester is a toy back tester?

Well - if I'm missing something please tell me, but on a first view:

- The reporting is minimal. Many important metrics missing. No analysis by month, day, hour, session-hour, for example - which is vital stuff.

- There is an optimiser, but it's very hard to review the results for robustness and there's no export, so I'd have to hack something.

- No Monte Carlo robustness testing.

- No walk-forward testing - which is extremely important for avoiding over-fitting

- Doesn't work with Renko and Range bars

- Pretty cludgy to test multiple pairs on the same strategy

- Looks virtually impossible to do real portfolio testing and optimisation

Etc.

Not anywhere approaching a professional backtesting tool, IMHO. Or am I wrong?

scotpip
03 Jan 2022, 12:40

RE:

amusleh said:

Hi,

If you use bars data then the back tester will only have access to each bar OHLC prices, and it will iterate over them one by one.

If you have a pending order at 1.02 and the next bar close/high price was above it then the order will be considered triggered at 1.02 not at the bar close/high price.

To get more accurate results you can use tick data for back testing instead of bars data.

Thanks - that's what I was hoping. It's surprising how many backtesters can't do this and only recognise pending orders at the price on bar close.

My preferred workflow is to do initial research with bars for speed, and verification at tick resolution for accuracy. But if you're ever tested a 20 pair algo with ticks you'll realise how glacially slow that can be!

So now I can put cTrader on the list, I have to decide whether to go with the nice CTrader API and live with the toy backtester, or go with the better backtester on MT5 and live with the horrible API and strange proprietary language...

Can I ask why you decided to go with CTrader?

scotpip
04 Nov 2021, 01:34

Hi

Thanks for responding! I was hoping I'd missed something but it seems not...

I've posted a feature request as you suggested.

Yes, it's possible to make the bars from ticks. I actually have the code which I use to generate backtest data.

But it consumes a lot of CPU cycles and there's a risk of lag in newsy markets. My own system will be trading a wide range of symbols on a wide range of bar sizes to this is a concern.

An API bar feed would offer accurate bars in the most efficient way.

scotpip
17 Mar 2020, 18:51

PanagiotisCharalampous said:

Thanks, I managed to reproduce it. We will check

Any progress on this? Can't really commit to the platform while there's a bug that will limit my ability to backtest...

scotpip
10 Mar 2020, 16:41

RE:

Nope - crashed again at a similar but different date and time on EURUSD t1.

10/03/2020 14:39:28.082 | cBot crashed: Error #62081310
 

scotpip
10 Mar 2020, 16:05

Thanks for the quick response.

Clearing the cache made no difference I'm afraid. Crashing after around 20 months, as before on 3 runs. I'm running the test on an empty bot for GBPUSD t1.

10/03/2020 13:16:41.356 | cBot crashed: Error #62081310

But I've noticed on each case the crash was at 10/10/2016 00:00:00. Maybe a data glitch? I'll try a different pair and report back - but the ticks are VERY slow to download from ICM, so may take some time.

It's not an OS-level resource issue. I have 90% of CPU and 60% of memory unused.

scotpip
09 Mar 2020, 18:59

I was planning to do something like this myself so it's good to see a starting point. I'd be looking to add other Renko bar types and perhaps some JS charting.

If a community member can get this working, it does make you wonder why the promised native feature from the cTrader team has been delayed for so long...

I'm new to cTrader, so before I invest time with this, do people find it runs over a decent period of data? I've hit a crashing bug simply trying to backtest tick data against an empty algo, which has rather shaken my faith in the platform...

scotpip
24 Jul 2018, 16:56

RE:

Thanks for the quick reply.

The corporate website is misleading - it lists Renko and a number of other charting features are available - it would be more honest if they said they are forthcoming. Not helpful - cost me a fair bit of time.

https://www.spotware.com/ctrader/trading/ctrader-charting

Welcome news that it's on the way, though.