With Genetic Programming it's always good to have a flexible set of functions which one can plug in and out of the framework in order to test out different solution hypothesis. After asking around what might be a good way to tackle this problem I heard about Dynamic Method Invocation via Reflection. With reflection it's extremely easy to find all the methods of a given class, so if we want to add a new method then all we have to do is write it and let the GP will automatically pick it up. I wrote a little program to test this out...
First I wrote a class containing some functions:
class Functions
{
private readonly String _symbol;
public Functions()
{
_symbol = "INVALID";
}
public Functions(String symbol)
{
_symbol = symbol;
}
#region ConstNumbers
public double GPConstNumZero()
{
LongOperation();
return 0.0;
}
public double GPConstNumPI()
{
LongOperation();
return Math.PI;
}
#endregion
#region OneParamFunctions
public double GPSin(double val)
{
LongOperation();
return Math.Sin(val);
}
public double GPMACD(double period)
{
LongOperation();
// Do the MACD calculations here
return Double.MaxValue;
}
#endregion
#region TwoParameterFunctions
// N/A
#endregion
private void LongOperation()
{
// I'll explain what goes here a little later
}
}
Then I wrote my test program:
class Program
{
// A dictionary that will map functions to the number of parameters
// where the key is the number of parameters and the functions
// that have the same number of parameters are added in the list
// corresponding to the key.
static Dictionary> _functions =
new Dictionary>();
// A stopwatch for performance analysis
static Stopwatch _stopwatch = new Stopwatch();
static void Main(string[] args)
{
// Start the Dynamic Method Invocation Test
DyanmicInvocationTest();
Console.WriteLine("Press any key to exit!");
Console.ReadKey();
}
static void DyanmicInvocationTest()
{
// Get all the method information
Functions fn = new Functions();
Type typeFn = fn.GetType();
MethodInfo[] methods = typeFn.GetMethods();
foreach (MethodInfo method in methods)
{
if (method.Name.StartsWith("GP"))
{
ParameterInfo[] pi = method.GetParameters();
if (!_functions.ContainsKey(pi.Length))
{
_functions.Add(pi.Length, new List());
}
_functions[pi.Length].Add(method.Name);
}
}
// Setup the performance analysis
int numRuns = 100 * 1000;
long time = 0;
// Run the dynamic invocation performance test
Console.WriteLine("Running dynamic invocation performance test");
for (int i = 0; i < numRuns; ++i)
{
time += DynamicInvoke(fn);
_stopwatch.Reset();
}
Console.WriteLine("Average time dynamic = " + (double)time/(double)numRuns);
time = 0;
// Run the normal invocation performance test
Console.WriteLine("Running normal invocation performance test");
for (int i = 0; i < numRuns; ++i)
{
time += NormalInvoke(fn);
_stopwatch.Reset();
}
Console.WriteLine("Average time normal = " + (double)time / (double)numRuns);
}
static Int64 DynamicInvoke(Functions fn)
{
Type typeFn = fn.GetType();
foreach (int key in _functions.Keys)
{
//Console.WriteLine(
// String.Format("num param {0}, num fn {1}",
// key, _functions[key].Count));
foreach (String function in _functions[key])
{
_stopwatch.Start();
switch (key)
{
case 0:
typeFn.InvokeMember(function,
BindingFlags.Default | BindingFlags.InvokeMethod,
null,
fn,
new object[0],
null);
break;
case 1:
typeFn.InvokeMember(function,
BindingFlags.Default | BindingFlags.InvokeMethod,
null,
fn,
new object[1] { 1.0 },
null);
break;
default:
break;
}
_stopwatch.Stop();
}
}
return _stopwatch.ElapsedTicks;
}
static long NormalInvoke(Functions fn)
{
_stopwatch.Start();
fn.GPConstNumPI();
fn.GPConstNumZero();
fn.GPMACD(1.0);
fn.GPSin(1.0);
_stopwatch.Stop();
return _stopwatch.ElapsedTicks;
}
}
Earlier I didn't show you the LongOperation method of the Functions class because I wanted to explain what I was testing. Since remote method invocation can be expensive, I wanted to see when I was going to be hit the hardest if I'm going to do dynamic method invocation and I contrived two test cases:
- Invoke all the functions, but only return the constants inside them (e.g. when a lot of the data is pre-calculated or it already exists as constants).
- Invoke the methods dynamically.
- Invoke the methods directly.
- Invoke all the functions, do some calculations and return the constants inside them (e.g. when a lot of the data is being computed at run-time).
- Invoke the methods dynamically.
- Invoke the methods directly.
So for the 1st run with no calculations I just ran the code as you see it, but for the second test I made a small change and let the LongOperation do some mathematical calculations (take the square root of a number):
private void LongOperation()
{
for (double i = 0.0; i < 1000.0; ++i)
{
Math.Sqrt(i);
}
}
When running the functions with the calculations I found that the performance hit was about 50% (i.e. methods invoked dynamically through reflection were about 50% slower than the methods invoked directly).
Table 1.0 Avg Ticks: dynamic (315.11694) normal (211.98494)
The truly horrifying part was when I ran the functions without any calculations and the performance hit was 3,967%... yes, you read it right! Invoking methods through reflection is nearly a 4,000% slower if you're not doing any calculations, but simply returning constants. See the graph below:
Table 2.0 Avg Ticks: dynamic (102.74611) normal (2.52593)
So naturally those numbers are mortifying, but there is some hope on the horizon: I found a blog by Gerhard Stephan (no clue who he is) where he discusses how to get a
2000% performance increase over the methods invoked with reflection. I'm still trying to understand what he's doing, but for now it's not looking very promising.
Conclusion:
I try to have as much of the data pre-calculated as possible which reduces my calculation time during the evolution of the strategies and maximizes the evolution iterations, so reflection is not looking very promising for me. If I had a lot more calculations in my functions, then I might consider reflection as a decent alternative. For now I'll keep looking for any ways to get that performance increase and if I can't figure out something within the next couple of days I'll have to drop this idea.
