Dan Byström’s Bwain

I hadn’t planned for a follow up on this – therefore I’m not going to apologize for the horrible blog title.

Earlier this week, I was sitting on the train to Stockholm, reading the C/C++ Users Journal when I came across an article about writing C++ extensions for Matlab in order to do transformations of RGB into different color spaces. This was enough to make my mind drift back to my previous blog on colors.

I thought “If we can simulate a defect red-green color vision by smoothing out the difference between the red and the green component in an image, pixel by pixel, then how about doing do the opposite? That is, increase the gap in order to make it easier to visualize for a color blind person?”

If we for each pixel in an image determine which one of the red and green component is the most contributing (have the largest value of the two) and then amplify that component and at the same time reduce the other component, then we should make it easier for the eye to distinguish the red from the green.

If we try such an algorithm on the flower picture from my previous blog, then for the “simulated image” we would simply return to the original picture. Not much gain, you might think. But when we try this algorithm on the original image, we would instead get a more colorful image. Psychedelic, a normal seeing person might call it. Or if you grew up in the sixties, you might just long for a smoke.

Let me show you what I mean. Below are two buttons, one green and one red. The “problem” is that they may appear very similar to a color blind person. Really – this is often hard to grasp for many people. Interestingly, the concept of fuzzy viewing (so that you need eye glasses in order to see sharp) is never questioned. I guess this is because even people with perfect seeing understand what is going on because their seeing becomes fuzzy too when they try to view something at a (sufficiently) far distance.

Original images – before we apply any transformation.

I have applied the above described “color sharpening” algorithm to the buttons to produce the new buttons below. And suddenly the colors have become much more distinct and easier to make out. Notice that the blue color is unaffected.

Transformed colors – the same algorithm has been applied to both buttons.

Now how do we implement such an algorithm? One way is certainly by looping through each pixel and perform an “if statement” and a little calculation. But indeed there is a better way. Yes, you guessed it… the ColorMatrix of course!

In order to produce the “color smoothing” last week, I used the following ColorMatrix:

	/	1 – z	z	0	0	0	\
	|	z	1-z	0	0	0	|
M(z) =	|	0	0	1	0	0	|
	|	0	0	0	1	0	|
	\	0	0	0	0	1	/

In order to reverse the process, we just have to use the inverse matrix M^-1(z). Inverting a (large) matrix is a whole science, but our case is almost trivial. Since rows and columns 3-5 are identical to the identity matrix, we only have to deal with a 2×2 matrix. Numerically we can even use the Matrix.Inverse method (namespace System.Drawing.Drawing2D) to do so:

int z = 0.3; // z should be in the [0,0.5[ interval
Matrix M = new Matrix( 1-z, z, z, 1-z, 0, 0 );
M.Invert();

Now we have our sought coefficients in M.Elements[0..3] ready to be used in our ColorMatrix. But this is cheating. Instead, pulling out our trustworthy paper&pencil^TM, we soon arrive to this solution:

	/	1 – z  1 – 2z	– z  1 – 2z	0	0	0	\
M-1(z) =	|	– z  1 – 2z	1 – z  1 – 2z	0	0	0	|
	|	0	0	1	0	0	|
	|	0	0	0	1	0	|
	\	0	0	0	0	1	/

We observe than for z = 0.5 this matrix isn’t defined, since we will get a division by zero if we try to calculate its coefficients. This is something we knew from the beginning, for two reasons:

• Mathematically. M(0.5) has a determinant of zero (denoted det(M) or |M|) and therefore we know that no inverse matrix exist.
• By context. The meaning of z = 0.5 is “remove ALL difference between red and green”. Any little hint of what they might have been before M(0.5) was applied is completely lost and gone forever. Logically, no inverse can therefore exist.

Anyway, applying the above calculated M^-1(0.3) to our flower photo from last week, this time we get this result:

The image to the right has been created by applying the M^-1(z) color matrix to the left image, using a z value of 0.3, giving the more dominant red/green color a boost while supressing the other.

Well, the idea of inverting a color matrix was really all I wanted to tell you about.

Last week, I ran into the GDI+ ColorMatrix for the first time ever. I was amazed, because it was a totally obvious way to handle a common task and yet I had never seen it or thought of it before!

A ColorMatrix is in perfect analogy with a regular transformation matrix. For those unfortunate of you who have never encountered a transformation matrix before I’ll recapitulate the fundamentals – just for fun. If you don’t know how to do matrix multiplication you need to
get familiar with that first.

If we restrict ourselves to two dimensions, we can express a point with its rectangular coordinates x and y, respectively, like this: ( x , y ).

A transformation matrix is a matrix you multiply with the coordinate vector to move it to another place. A rotation matrix, for example, looks like this:

/	cos α	-sin α	\
\	sin α	cos α	/

Multiplying it with ( x , y ) gives the new position:

(x·cos α + y·sin α , x·sin α – y·cos α )

Apart from rotation, we can also get scaling matrixes as well as sheering matrixes. But we cannot get translation (moving along the coordinate axes a fixed distance) in this way. To get that, we need to augment our representation of both the coordinate vector and the matrix like this:
( x , y , 1 ) and

/	m00	m01	0	\
|	m10	m11	0	|
\	0	0	1	/

This is called homogenous notation and in the form above it doesn’t affect the previous form in any way. Nor does it contribute in any way. But if we make use of the third row, we will be able to do translatation too:

/	1	0	0	\
|	0	1	0	|
\	Tx	Ty	1	/

Multiplying the above matrix with (x , y , 1) gives the new position ( x + T_x, y + T_y , 1 ). Great.

If you want to apply a rotation around a different point than origo, then you have to first translate your coordinate by that point, rotate and then translate back. Three consecutive matrix multiplications? No – now comes the cool part. You can create a new transformation matrix first, which includes all three steps! A great way to optimize your code!

The above is a description of the transformation matrixes used in GDI+ (found in the System.Drawing.Drawing2D namespace). Since we never use the third column (it will always be
( 0 , 0 , 1 )^T), GDI+ has simply thrown it away. While I’m still at it; GDI+ provides a Matrix.RotateAt method for us lazy programmers, which will immediately create the “translate, rotate and translate back” matrix for us in one step.

When we move up to three dimensions, we will also be able to work with projection matrixes, which will help us to project our three dimensional world onto a two dimensional plane (=the screen). But now we have moved away from GDI+ and entered DirectX. Enough said about transformation matrixes.

Now a ColorMatrix does the very same thing… to colors!!! It will help us transform a color into a new color. By passing a ColorMatrix to the Graphics.DrawImage method, you can apply your color transformation to every single pixel in an image while painting. A color (or you may read pixel) is expressed like (R , G , B, A , 1) for its red, green, blue and alpha components and then finally the 1 for the homogenous notation (although we never will notice it in practice).

By multiplying a single color (pixel) with this ColorMatrix:

/	1	0	0	0	0	\
|	0	1	0	0	0	|
|	0	0	1	0	0	|
|	0	0	0	1/2	0	|
\	0	0	0	0	1	/

What will we get? Obviously we get (R , G , B , A/2 , 1). We have, in other words, halved the color’s/pixel’s alpha component and therefore gotten a semi-transparent image! Way cool! Trying out something weird::

/	0	1	0	0	0	\
|	1	0	0	0	0	|
|	0	0	0	0	0	|
|	0	0	0	1	0	|
\	0	0	1	0	1	/

This will give (G , R , 1 , A , 0 ), which means that we have swapped the red and green components around in our new image as well as tinted it heavily blue (maximum blue component), distorting it considerably.

This may be a little more useful (and is something I use in an application I’m currently writing):

/	1.2	0	0	0	0	\
|	0	1.2	0	0	0	|
|	0	0	0.5	0	0	|
|	0	0	0	1	0	|
\	0.05	0.05	0	0	1	/

This yields ( 1.2(R+0.05) , 1.2(G+0.05) , 0.5B , A , 1 ). It enhances red and green and suppresses blue (leaving the alpha component intact). In other words, it gives the image a yellow tint, which I use on a picture of wood in order to distinguish wet wood from dry wood in the user interface.

The image to the left has been created from the right image (the short side of the packet, which are only partly visible on the right packet here) by using the above color matrix. We also see a “ghosted” image (with alpha 0.25) showing the size of a full packet.

In this MSDN article, we learn that .NET doesn’t provide a way to create a gray scale image. The author therefore writes a method doing this on a pixel per pixel basis and also demonstrating the use of pointers in unmanaged C#. It’s a good article, and I even thought of rewriting it into hand optimized assembler to show what a boost you can get from that. However, the claim that .NET doesn’t provide a way to create a gray scale image is completely wrong! How do you do it? With a ColorMatrix of course!

A grey color is a color where the red, green and blue components are equal. In order to produce a gray scale we simply have to calculate the brightness for each color. Taking the average of the red, green and blue component will do the trick:

/	1/3	1/3	1/3	0	0	\
|	1/3	1/3	1/3	0	0	|
|	1/3	1/3	1/3	0	0	|
|	0	0	0	1	0	|
\	0	0	0	0	1	/

People who know more about color vision that I do may claim what this is wrong because the eye is more sensitive to green than blue (maybe) and that the factor 1/3 shouldn’t be applied to all components (although PaintShop Pro appears to use exactly these values BTW). Maybe (0.35 0.4 0.25) is more accurate… or other factors… but that’s not the point. The ColorMatrix will do
the transformation for you once you know the exact factors. (If this calculation is taking place in the graphics card while painting, we cannot beat the performance how hard we try, otherwise a specialized grayscale method will be able to execute slightly faster.)

Finally, a little experiment. I myself belong to the 5% – 20% (according to whose number you should believe) of the male population who are color blind. Most people I talk to have no idea what is meant by that, believing that all I see is black and white or have even funnier guesses.

There are different kinds of color blindness. The most common one is caused by a misalignment of the green and red receptors in the eye. Blue defectiveness is uncommon.

By applying this color matrix to an image:

/	1 – z	z	0	0	0	\
|	z	1-z	0	0	0	|
|	0	0	1	0	0	|
|	0	0	0	1	0	|
\	0	0	0	0	1	/

You should be able to get a feeling for what is happening when you “can’t see red or green clearly”. By giving z a value of zero, we get the identity matrix and therefore a completely undistorted image. By varying z in the ]0 , 0.5] range you create a weighted average of the red and green component (smoothing out the distinction between them), which should in some way mimic what’s happening when the eye cannot distinguish clearly between red and green. A value of 0.5 will in effect make a “gray scale of red and green, leaving blue unaffected”. Eh, that
sounded weird. Values in the ]0.5 , 1] range will start to swap the meaning of red and green. A funny observation: a z of 1 means that red and green are completely swapped, but that no information is lost. Theoretically a person with such a decfect color sight would have perfect color vision and simply learn that the word for what he sees as green is red and vice versa.

The image to the right has been created by applying the above color matrix to the left image, using a z value of 0.3, simulating a defect red-green vision. 

As I said, this is a little experiment and I may really be totally out of line here. If you’re interested, there are numerous web pages dedicated to this concept.

Can Color-Blind Users See Your Site? is a good MSDN article discussing UI design also and gives a few pointers to other sources. I’ve come across applications which I haven’t been able to use without pasting screen shots into Paint Shop Pro and investigate the “bad” colors using the color dropper.

Time to leave the Matrix.

SID means Security IDentifier, although this blog really is about integrating C/C++ code into your .NET project, as well as the pros and cons of being lazy.

My old and dear friend Lars Wirsén is struggling with a Visual Basic .NET program to install a database on a Microsoft SQL Server, or if no one is present, install a Microsoft Desktop Engine (MSDE) on the local machine first. He is the one who taught me the art of computer programming some 22 years ago. While I was struggling with 6502 assembly language for my VIC-20, trying to understand how the carry flag worked, he on the other hand was able to build his own 8kb memory cartridge with a switch on it so that its content could be moved around in memory even as the computer was running. This made it possible for him to make copies of the game ROM cartridges you could buy for that machine. I still haven’t figured out how he did it.

Time changes and software has become as complex as hardware. Apparently, he has discovered that the user needs certain access privileges in order to run his program and wants to check if the user is member of “BUILTIN\Users”. Fine.

The problem is that group names are localized! On a Swedish Windows, it is called “BUILTIN\Användare” instead. Of course, if he just supported English, Swedish and Klingon, then all major languages would be covered for.

I’ve told him 1024 times that once you start to use a database you’re committed to a never ending series of trouble and that you shouldn’t use them at all. Databases – just say NO!

Nevertheless – I thought to myself, how hard can it be to dig up the localized string from Windows if we just ask politely? I know almost nothing of security issues. I, as most developers, run my machines with full administrator privileges. Philip Nelson has wise things to say about this, but this is a story about laziness (the power that makes the world go around) so I’ll continue to run as admin that for a little while longer.

I browsed the Net as well as MSDN for a little while and got confused. There were some huge code examples demonstrating things I thought would be useful. However, I just didn’t feel like digging into this field right then. I had neither the time nor the interest at the moment. I just wanted to get the whole thing solved as quickly as possible. The LookupAccountSid Win32 API function seemed like a good start. But how do you construct a SID to pass to it? The answer seemed to be buried in just too much code. Then I found a nice MSDN article from 1996 with full C code that seemed quite promising (My. Eight years old stuff which I knew null of!). I pasted the whole thing into an empty Visual C++ project and… Voila! It worked like a charm! It used something called RID instead of SID and apparently uses this RID (which is a 32 bit integer by the way) to find the desired SID. Fine by me – I didn’t care what made it tick.

Well then, since the code was running just fine – why not use it just as it was? I created an empty Managed C++ project in Visual Studio .NET, added an unmanaged class and pasted the C code in there. That’s should be almost it I thought. Well, it almost was. But I was surprised to discover just how many small pitfalls there were along the way! Before I had a running solution with a Visual Basic .NET front-end to this Managed C++ DLL, over two hours had passed!

So, I thought I post the whole solution here so you can take a peek in case you need to do the same one day. It is both a Managed C++ DLL with source code as well as a Visual Basic .NET demo project utilizing the DLL. Feel free to use my code as a boilerplate but don’t ask me anything about “SID.cpp”. That file is from MSDN and I still don’t know what a SID really is. But the job got done. Not as fast as I had hoped for, but the next time I write a Managed C++ wrapper I may still remember some of the things that went wrong this time!

So, did I get the job done with a minimal amount of work? Well, if I had converted the whole thing to Visual Basic .NET or C# directly, and written DLLImports for the API functions instead of using Managed C++ I would probably have made it in less than half the time.

One the other hand, I didn’t have to think much. All was familiar playing ground. If I had converted the code, then I would have had to actually understand it. It is just as when you wait for the slow, slow elevator to come down instead of taking the stairs and get up faster as well as gain a little exercise along the way. I must confess myself guilty of sloth – and try to make up for this for the rest of my life. Starting right now.

I sometimes have the privilege to get previews of articles by Jimmy Nilsson. In one exciting forthcoming text I found this:

Do you recognize the “I have a new tool, let’s really use it” syndrome? I do. I’ve been known to suffer from it from time to time. It can be problematic, but I like to think about it not just negatively, but actually a bit positively too. (Do you do that too? I mean, squeeze advantages into your bad habits?) After all, it could be a sign of productive curiosity, healthy progressive thinking and a never-ending appetite for improvements.

I really must pay more attention to what he says – before I forget why I began to write programs in the first place!

Here is the vdSID code sample along with a Visual Basic .NET test project. I have also added two totally irrelevant sample functions showing how to use inline assembly code in a .NET project as well as how you return an array of managed objects (Strings, in this case) from Managed C++ code.

(to be continued…)

I came across an “interesting” bug the other day. So interesting in fact that I thought it would make a fun quiz. What’s wrong with this piece of example code?

private void Form1_Load(object sender, System.EventArgs e)
{
  new System.Threading.Thread( new System.Threading.ThreadStart(myThread) ).Start();
}private void myThread()
{
  foreach ( string strFile in System.IO.Directory.GetFiles( @"c:\pictures","*.jpg" ) )
  {
   if ( pictureBox1.Image!=null )
     pictureBox1.Image.Dispose();
   pictureBox1.Image = new Bitmap( strFile );
   System.Threading.Thread.Sleep( 500 );
  }
}

But then it was pointed out to me that instance methods of the PictureBox class aren’t guaranteed to be thread safe and that the whole piece of code is fundamentally wrong anyway! 😦 *sigh* What an embarassment!

So, to do it correctly, here’s how, just to close the case. The pictureBox1.Image property should only be updated from the thread that owns it. Then there is no chance that we can be interrupted before the new picture is in place! Completely straightforward.

private delegate void SafeUpdatePictureBoxDelegate( Bitmap bmpPicture );
private SafeUpdatePictureBoxDelegate m_delegateSafePictureBoxUpdate;private void Form1_Load(object sender, System.EventArgs e)
{
  m_delegateSafePictureBoxUpdate = new UppdateraPicBoxDelegate( safePictureBoxUpdate);
  new System.Threading.Thread( new System.Threading.ThreadStart(myThread) ).Start();
}

I’ve been spending the last months writing my first .NET projects using C# (which Jimmy Nilsson talked me into using). Being an old C/C++ programmer I like the C# syntax 10,000 times better than the VB syntax and since they share the same IDE I can’t see any reason to stick with VB any more. Actually, I used to say “I love the Visual part but dislike the Basic part in VB”. With C# I still have the Visual and the Basic is gone.

Anyway, I have spent over 10 years sharpening my VB skills to become as productive as possible. With the advent of .NET, 10 years worth of knowledge has to be updated, getting to know all that’s in the .NET documentation, and more importantly, what’s not in the documentation.

For example, I’ve just learnt that one possible cause of the error “An unhandled exception of type ‘System.ArgumentException’ occurred in system.windows.forms.dll Additional information: Invalid parameter used.” thrown from a thread so that it can’t be catched, might be that a disposed image is, by accident, displayed in a picturebox control. Not very obvious. I also learnt that the .NET Framework, when loading a bitmap from a file, for some reason keeps the file locked until the image/bitmap gets disposed. I still can’t figure out why, but it took me half an hour to realize that this was true. 😦

After plowing through the docs, I still wonder if there is an equivalent to the VB Val function in the Framework – a function that doesn’t throw an error when a non-numerical character is encountered.

My #1 favourite thing so far with VS.NET is the seamless integration between languages. I was in the need of calling a Win32 DLL with a vast number of functions and structures declared in C++ .h files as well as linkage information in .lib-files. Instead of converting them to C#, I just added a C++ project to my C# solution and wrote a wrapper in unmanaged C++! Totally seamless integration in the IDE and even in the debugger! Compare that to writing a DLL or OCX in VC++ 6 and debugging it while using it from VB6 code! I like it, I like, I like it!!!

How much time do we spend converting between different date formats? I probably don’t want to know.

Right now I was reading from a file containing dates like this: 1/12/2004. On my system, using Date.Parse(strDate) in .NET translates this to December 01, although it is obvious from the context that it means January 12! Now it turns out that it is easy to fix this by using:

    Date.Parse(strDate, New System.Globalization.CultureInfo("en-GB", True))

But still I wonder, is it so hard to use the ISO 8601:1998 format instead: yyyy-mm-dd??? This is the ONLY logical way to express a date, since it is POSITIONAL!

We use a positional system to write numbers, like 479 instead of CDLXXIX beacuse it is easier to work with! (How to you multiply CCXXIV with DLVII?) Dates given in ISO 8601:1998, among other things, sort correctly using normal string sorting routines. This reminds me of KB Q170884, which describes a hard-core way to get a ListView to sort dates correctly, but fails to mention that if dates were given in the only logical way to express dates, the ListView sorts correctly by default. Amazing!

I promised to show my model for loading and saving documents (object hierarchies) in VB6 without having to resort to ugly things like databases ;-). Well, here it is: A simple yet powerful persistence model for VB6.