Memory Manager Bot free

by moneybiz in category Other at 27/08/2015
Description

The memory manager when attached as robot periodically (configurable) frees the memory used by cAlgo thus keeping it down pressured.

Don't set the "Reclaim Period" to very low values which may effect the performance of cAlgo.

On each tick received it displays the current working set (memory used by cAlgo) and the last memory reclaimed.
Since it's checking to free or not on tick received event on slow trading hours the text will be updated less frequent.

The bot needs "FullAccess" permission to work, so you need to allow them when asked.

 

Warning! Executing the following cBot may result in loss of funds. Use it at your own risk.
Notification Publishing copyrighted material is strictly prohibited. If you believe there is copyrighted material in this section you may use the Copyright Infringement Notification form to submit a claim.
Formula / Source Code
Language: C#
Trading Platform: cAlgo
using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
using cAlgo.API;

namespace MoneyBiz.TraderBots.Bots
{
    [Robot(TimeZone = TimeZones.UTC, AccessRights = AccessRights.FullAccess)]
    public class MemoryManager : Robot
    {
        private long _lastFreeTime;
        private long _lastReclaimed;

        [Parameter("Reclaim Period (seconds)", DefaultValue = 60, MinValue = 10, Step = 10)]
        public int ReclaimPeriod { get; set; }

        protected override void OnStart()
        {
            if (Environment.OSVersion.Platform != PlatformID.Win32NT)
            {
                Print("Memory Manager: Platform not supported.");
                Stop();

                return;
            }

            var stats = Free();

            DrawStats(stats);
        }

        protected override void OnTick()
        {
            var stats = Free();

            DrawStats(stats);
        }

        public void DrawStats(Tuple<long, long> stats)
        {
            if (stats != null)
            {
                _lastReclaimed = stats.Item1 - stats.Item2;
            }

            var currentWorkingSet = BytesToMegabytes(Environment.WorkingSet);
            var lastReclaimed = BytesToMegabytes(_lastReclaimed);
            var timeSpan = TimeSpan.FromTicks(DateTime.Now.Ticks - _lastFreeTime);

            var text = string.Format("Working Set: {1:N2} MB{0}Last Reclaimed: {2:N2} MB ({3:N0} s)", Environment.NewLine, currentWorkingSet, lastReclaimed, timeSpan.TotalSeconds);

            ChartObjects.DrawText("MemStats", text, StaticPosition.TopLeft, Colors.Orange);
        }

        private static double BytesToMegabytes(long value)
        {
            return (double)value / (1024 * 1024);
        }

        private Tuple<long, long> Free()
        {
            if (DateTime.Now.Ticks - _lastFreeTime <= TimeSpan.FromSeconds(ReclaimPeriod).Ticks)
            {
                return null;
            }

            Tuple<long, long> workingSets = null;

            try
            {
                using (var process = Process.GetCurrentProcess())
                {
                    var before = Environment.WorkingSet;

                    if (Environment.Is64BitProcess)
                    {
                        SetProcessWorkingSetSize64(process.Handle, -1, -1);
                    }
                    else
                    {
                        SetProcessWorkingSetSize32(process.Handle, -1, -1);
                    }

                    var after = Environment.WorkingSet;

                    workingSets = Tuple.Create(before, after);
                }

                _lastFreeTime = DateTime.Now.Ticks;

                Print("Memory Manager: Reclaimed {0:N0} bytes.", workingSets.Item1 - workingSets.Item2);
            } catch (Exception ex)
            {
                Print("Memory Manager: " + ex);
            }

            return workingSets;
        }

        [DllImport("KERNEL32.DLL", EntryPoint = "SetProcessWorkingSetSize", SetLastError = true, CallingConvention = CallingConvention.StdCall)]
        static internal extern bool SetProcessWorkingSetSize32(IntPtr pProcess, int dwMinimumWorkingSetSize, int dwMaximumWorkingSetSize);

        [DllImport("KERNEL32.DLL", EntryPoint = "SetProcessWorkingSetSize", SetLastError = true, CallingConvention = CallingConvention.StdCall)]
        static internal extern bool SetProcessWorkingSetSize64(IntPtr pProcess, long dwMinimumWorkingSetSize, long dwMaximumWorkingSetSize);
    }
}
Comments

ClickAlgo - August 28, 2015 @ 12:16

This is maybe best executing in hours/days and not seconds as each time it clears the memory, the platform has to get the data again, I am sure Spotware cache their data to reduce round-trip calls.

There could be a memory leak with the cAlgo platform or the sand-boxed cBots are not releasing the memory they use even with .NET's garbage collector, I don't know, but what i do know is every-time I stop and restart an instance to a cBot the memory increases.

This tool is useful for now and as I host my cBots on a VM with only 2 Gigs, i will set the memory manager  to 3600 seconds minimum or 1 hr or I will probably just clear the memory every 24hrs, which will save me restarting the cAlgo platform on the Virtual server at regular intervals.

anyway thank you very much :-)

Roni Zbaida - August 31, 2015 @ 02:02

Thank you :)

tvman777 - September 18, 2015 @ 17:21

Thank you for making this!! :D

ekwam - November 10, 2015 @ 13:30

MoneBiz,

 

Thanks for your memory management algo .. i am using it on my VPS server.

 

Bravo..

ChasBrownTH - January 22, 2016 @ 17:56

Fantastic, this has saved me so much trouble! I was getting memory problems on a couple of 'cheap' VPS instances, as well as one damned expensive one, plus on local Hyper-V instances. This solved it all!

THANK YOU so much Moneybiz, this solution is very inspiring.

PS - I can understand why you stopped trying to Win with Forex, but for myself I am still hooked after 11 years ;)

 

ironmine - January 27, 2016 @ 04:28

If I have, let's say, 10 charts opened, do I need to attach this Memory Manager cBot to each chart?

moneybiz - March 02, 2018 @ 12:11

@ironmine: No, just once instance is enough.

firemyst - July 29, 2019 @ 09:08

Thank you for this.

I find that even with forcing GC.Collect() or GC.GetTotalMemory(true) in bot code still doesn't seem to reclaim everything, and was wondering what else could be done short of having to constantly stop/restart bots as @Paul Hayes alluded to above.

ClickAlgo - August 06, 2019 @ 18:02

Hi, FireMyst,

You are better off posting on the forum, posting on these threads do not alert anyone, I just found this by chance, the product below will free the memory used by the cTrader platform only, it was created to help users with a VPS and where they have backtested which allocates a large amount of memory.

https://clickalgo.com/ctrader-cbot-vps-memory-manager

5