Month: June 2018

As indicated, for the next release (v0.0.13) I will be adding code to Campy to compile callvirt and box/unbox CIL instructions. Right now, compilation of callvirt actually calls only the base class method. For virtual functions to work (and, actually, a computed “late-bound method on an object”), the JITed code must be stored in the BCL meta. Then, for a callvirt, a pointer corresponding to the virtual function is loaded and called for the object. For box and unbox, I currently have an implementation of box for Int32, but will add in the other basic value types. Unbox will not be in this release–too much other work to do. It will probably take a couple weeks to get all this working.

But, with these changes, an ever larger amount of C# and NET framework should start to work as expected on a GPU. However, I have noticed many functions in the DNA runtime that are attributed “[MethodImpl(MethodImplOptions.InternalCall)]” which do not have a C++/C implementation. Clearly, DNA has some shortcomings that need to be fixed. I will deal with these missing methods first on a case-by-case basis. Then, at some point, a methodical check must be done to verify that there is an implementation for all such methods.

This next release fixes a number of problems with Campy for a more complex example: steepest descent. This example encompasses a number of advanced capabilities of Campy and C#, which is best explained with the implementation shown below. In this example, you will note use of value types, reference types, generics, and multiple Parallel.For() calls.

using System;
using System.Text;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace test
{
    class Program
    {
        static void Main(string[] args)
        {
            var A = new SquareMatrix(new Collection<double>() { 3, 2, 2, 6 });
            var b = new Vector(new Collection<double>() {2, -8});
            var x = new Vector(new Collection<double>() {-2, -2});
	    var r = SD.SteepestDescent(A, b, x);
            System.Console.WriteLine(r.ToString());
        }
    }

    class SquareMatrix
    {
        public int N { get; private set; }
        private List<double> data;
        public SquareMatrix(int n)
        {
            N = n;
            data = new List<double>();
            for (int i = 0; i < n*n; ++i) data.Add(0);
        }

        public SquareMatrix(Collection<double> c)
        {
            data = new List<double>(c);
            var s = Math.Sqrt(c.Count);
            N = (int)Math.Floor(s);
            if (s != (double)N)
            {
                throw new Exception("Need to provide square matrix sized initializer.");
            }
        }

        public static Vector operator *(SquareMatrix a, Vector b)
        {
            Vector result = new Vector(a.N);
            Campy.Parallel.For(result.N, i =>
            {
                for (int j = 0; j < result.N; ++j)
                    result[i] += a.data[i * result.N + j] * b[j];
            });
            return result;
        }
    }

    class Vector
    {
        public int N { get; private set; }
        private List<double> data;

        public Vector(int n)
        {
            N = n;
            data = new List<double>();
            for (int i = 0; i < n; ++i) data.Add(0);
        }

        public double this[int i]
        {
            get
            {
                return data[i];
            }
            set
            {
                data[i] = value;
            }
        }

        public Vector(Collection<double> c)
        {
            data = new List<double>(c);
            N = c.Count;
        }

        public static double operator *(Vector a, Vector b)
        {
            double result = 0;
            for (int i = 0; i < a.N; ++i) result += a[i] * b[i]; return result; } public static Vector operator *(double a, Vector b) { Vector result = new Vector(b.N); Campy.Parallel.For(b.N, i => { result[i] = a * b[i]; });
            return result;
        }

        public static Vector operator -(Vector a, Vector b)
        {
            Vector result = new Vector(a.N);
            Campy.Parallel.For(a.N, i => { result[i] = a[i] - b[i]; });
            return result;
        }

        public static Vector operator +(Vector a, Vector b)
        {
            Vector result = new Vector(a.N);
            Campy.Parallel.For(a.N, i => { result[i] = a[i] + b[i]; });
            return result;
        }

        public override string ToString()
        {
            StringBuilder sb = new StringBuilder();
            for (int i = 0; i < data.Count; ++i)
            {
                sb.Append(data[i] + " ");
            }
            return sb.ToString();
        }
    }

    class SD
    {
        public static Vector SteepestDescent(SquareMatrix A, Vector b, Vector x)
        {
            // Similar to http://ta.twi.tudelft.nl/nw/users/mmbaumann/projects/Projekte/MPI2_slides.pdf
            for (;;)
            {
                Vector r = b - A * x;
                double rr = r * r;
                double rAr = r * (A * r);
		if (Math.Abs(rAr) <= 1.0e-10) break;
                double a = (double) rr / (double) rAr;
                x = x + (a * r);
            }
            return x;
        }

        // https://www.coursera.org/learn/predictive-analytics/lecture/RhkFB/parallelizing-gradient-descent
        // "Hogwild! A lock-free approach to parallelizing stochastic gradient descent"
        // https://arxiv.org/abs/1106.5730

        // Parallelize vector and matrix operations
        // http://www.dcs.warwick.ac.uk/pmbs/pmbs14/PMBS14/Workshop_Schedule_files/8-CUDAHPCG.pdf

        // An introduction to the conjugate gradient method without the agonizing pain
        // https://www.cs.cmu.edu/~quake-papers/painless-conjugate-gradient.pdf

        // https://github.com/gmarkall/cuda_cg/blob/master/gpu_solve.cu
    }
}

 

After a considerable amount of hacking, I’ve released v0.0.11 of Campy. This version fixes a number of problems with reading PE files. Again, most of the problems I have been encountering go back to the old DotNetAnywhere code and its lack of support for anything that has happened in .NET over the last 10+ years, e.g., additional metadata type tables, 64-bit code targets, type and assembly resolution, low-level metadata access, etc. Some changes are for undocumented Net Core hacks, such as a PE machine version 0xFD1D. A search in all of Github.com indicates that this occurs for native libraries, such as System.Collections.dll on Ubuntu 16.04. However, even though it is x64 native code, and you may think the assembly useless, it seems to contain metadata type information that is crucial in the analysis for type/assembly resolution, which you can verify using DotPeek. Incidentally, assembly resolution–the steps used by Net to figure out where and what assembly to load for a program–is somewhat fixed in Campy with the addition of lots of probing of the standard locations for assemblies, and checking “public key” for the correct version. Generics and String finally work again with changes to rewrite the stack types during compilation of a method. I fear, however, that it may be inadequate, for generics are quite complicated. There was a problem with Swigged.CUDA on Ubuntu, but that is now fixed.

So, slowly, Campy is coming up to speed with respect to being platform independent and able to work with a lot of C# (value types, reference types, generics). But, it still has a way to go: boxing, virtual methods, IOS, Mono assemblies, etc. And, it still has a number of bugs with C# generics, which can make it unstable, if not impossible, to use.

Note: While I appreciate why Steve Sanderson et al. switched from DotNetAnywhere to Mono with Blazor in late 2017, I am glad I chose DNA for Campy. Trading DNA for Mono is just trading one set of problems for another. If you’ve been in the business as long as I have (30+ years), you realize that you may think you and your code hot stuff, but someone can always improve on it–or rewrite it completely. That old programmer who wrote that original crapy code that you improved may come back and bite your ass off. That’s the nature of software.

Well, I finally have generics working again…sort of, and currently only in Net Framework apps. Unfortunately, I’m back in dll/assembly hell. When I try to find List<> in Net Core’s System.Collections.dll, it isn’t there. Where is it? And, why am I even looking in the file if Campy BCL has a replacement?

Starting with a specific example, and using DotPeek of the Campy Net Core test program ConsoleApp1/bin/Debug/netcoreapp2.0/win-x64/publish/ConsoleApp1.dll, this is what I can figure out:

• The metadata for TypeRef table containing List`1 in “ConsoleApp1.dll” says the type is in AssemblyRef 0x23000004, major version 4, minor version 1, name “System.Collections”. Since “publish” is a self-contained app, I open “System.Collections.dll” in the same directory.
• DotPeek of “System.Collections.dll” indicates it does not define a type “List`1”. So, looking at the TypeRef’s and AssemblyRef’s in the metadata, it should be in AsmRef 0x23000001, System.Private.CoreLib, major version 4, minor version 0.
• DotPeek of “System.Private.CoreLib.dll” indicates it defines a type “List`1”, and this is in fact where implementation for List<> lives. (Note, List`1 is way down in the TypeDef’s table with id 0x0200040e. So using DotPeek to find the type is basically impossible, because the “find” function in DotPeek crashes with this file. I’m writing a program called Campy.Find that will help with these kinds of queries.)

In fact, there’s already a bit of kludgy code in Campy that does something like this, between “public static IntPtr GetBclType(Type type)” in C# code that looks up the type hierarchy to load in specific files and types, and “function_space_specifier tMetaData* CLIFile_GetMetaDataForAssembly(char * fileName)” in C code within DNA that loads an assembly, probing if needed. But, clearly, it should all be done in DNA. Fortunately, DNA can load files from the host OS file system since I do provide a wrapper for DNA that is called from C#.

Once I have this search coded up in DNA–and patching up DNA to read FieldMarshal tables, which are in System.Private.CoreLib.dll–I’m hoping generics will generally start to work, and the code a little cleaner to boot.