C# Nuggets

Concurrency bugs in the CLR JIT

2009-01-10T09:08:00Z

A recent paper by Microsoft research has discovered a concurrency bug in the .Net JIT. They used the F# theorem prover to analyze the JIT il->x86 transformations which were previously thought to comply with the .Net memory model. These bugs show that writing correct concurrent (multi-threaded) code is hard - and in a multi-core world we need to find a way to provably write correct code.

"This discovery means that the current CLR JIT compiler for the x86 platform is not correct; it will be ﬁxed in the future by strictly emitting locked instructions for all volatile stores"

- http://research.microsoft.com/apps/pubs/default.aspx?id=76524

via LtU

JavaScript performance

2008-09-10T19:27:00Z

Recently, the browser vendors have been making serious gains in JavaScript performance. A week ago, Google launched their web browser called Chrome with much fanfare. Chrome is based on the WebKit rendering engine which underpins Apple's Safari browser and the iPhone browser, but using a brand new Just-in-time (JIT) compiling JavaScript virtual machine. This JIT virtual machine is called V8 and was created over the past 18 months by a team led by Lars Bak who created the Java Hotspot virtual machine. The Chrome browser is Open Source in the form of code going by the name Chromium.

Interestingly, it turns out that Apple and Mozilla have spent the past few months building JITs for their JavaScript engines. Apple have an engine called JavaScriptCore, while Mozilla have an engine called SpiderMonkey. The JITs they have built are based on these existing engines but have been given new names. Apple's is called Squirrelfish-Extreme (SFX) while Mozilla's is called TraceMonkey.

Each of these engines seem to have unique features. V8 has what are called 'hidden class transitions' which are simply class definitions inferred at runtime. As class definitions change infrequently, this means that V8 has very fast class member lookup. TraceMonkey uses a novel technique called Tracing. Instead of compiling a method 'just in time' and then executing the compiled machine code, TraceMonkey interprets everything up until it detects a loop. At this point, it will compile the loop. So whereas traditional JIT compiling will compile code which potentially never gets run, Tracing will only ever compile code that gets run. Webkit's SFX has borrowed a trick from .Net and will compile any regular expressions into machine code. Their Webkit Regular Expression Compiler (WREC) is why SFX beats everyone else hands down on regular expression performance.

It has been a hectic 2 weeks. In that time, there have been various claims that first TraceMonkey was the fastest. Then V8 claimed they were the fastest. And over the past couple of days there have been claims that Webkit's SFX is the fastest. So who is right? Well, many factors are involved, and all three engines are still very much in active development. So the lead could shift around yet again.

To try and get a handle on the current state, I ran a benchmark called SunSpider. This is a JavaScript benchmark which does not test HTML DOM performance, therefore giving a true picture of pure JavaScript virtual machine speed.

The headline figure is:

Webkit (ie Safari) nightly builds are 46% faster than Firefox nightly builds. Chrome is 7% faster than Firefox.

However, the above statement does not give the whole picture. It appears that Safari's SFX JIT is not fully enabled on Windows. Recent Webkit checkins indicate that the Windows JIT is still being worked on. As Chrome is currently a Windows-only browser, it is impossible to compare Webkit-SFX with Chrome. Therefore, the 46% figure advantage Webkit has over firefox was obtained on OSX 10.5.

Microsoft have been working on a JavaScript JIT for a couple of years now. It's called Managed JScript and runs on top of the Dynamic Language Runtime (DLR). However, there has been no news from this project for a long while and it certainly won't be included in IE8. The old ActiveScripting-based JScript engine in IE8 has been tweaked a little for performance but lags a long way behind the other vendors. This is a real shame as Microsoft have some world-class virtual machine architects.

Benchmark: Sunspider 0.9

Windows XP SP2 results (Core2 Duo, T7300/2Ghz)
Chromium: 2747.8ms +/- 2%
Firefox: 2941.4ms +/- 4.3% (1.07x as slow)
webkit: 3124.6ms +/-3.3% (1.14x as slow)

OSX 10.5.4 results (Core2 Duo, 2.8Ghz)
Firefox: 1361ms +/- 2.7%
webkit: 931ms +/- 2.3% (1.46x as fast)

Browsers used were the latest nightlies:

Chromium 0.2.152.0
Win32 Firefox 3.1b1pre Gecko/20080910043000 (Content JIT=on)
OSX Firefox 3.1b1pre Gecko/20080910020330 (Content JIT=on)
Webkit r36309 using Squirrelfish-Extreme (SFX)

Fixup candidates for .Net v4

2008-08-08T08:05:00Z

The C# team has been fairly candid in the features they are considering for C# v4. You just have to look hard enough. We'll know what they decide on soon enough as the PDC is just around the corner. Declarative side-effect free methods, Contra/covariance of arguments/return values, Declarative dynamic dispatch (VB-style object access). These all sound great. Perhaps I can add a few suggestions of my own not just for C# but for .Net in general:

A few ideas for .Net vnext:

String.Join should work on IEnumerable<string> not just arrays.
C# enums should be OO just like Java 5.0.
Forget about the ThreadPool. We need a resource-management WorkerPool. Something that manages a task based not just on availability of threads, but of memory and IO. There's no point in throttling threads if you're task/agent is memory or IO bound.
C# null propagation (lifted member access). Ian Griffiths has by far the best description I've ever seen. I'd love to use something like var x = foo()..bar()..baz()..quux() where the double period (..) means propagate nulls.
Server-side CSS parsing, optimization, minimizing.

As for Visual Studio:

Collaborative Editing (cross network pair-programming). Eclipse does it. SubEthaEdit does it.
Much better JavaScript type inferrence in the IDE.
MSTest is too tightly coupled to the IDE. And it shows. MSTest is needy in many areas. There are 2 .Net Unit Testing pioneers at Microsoft. It's a wonder they got it so wrong.
Sort out Edit n Continue so it actually works when I modify an iterator, anonymous method etc. We're still a long long way from the glorious REPL in VB1-6.

When does FlagsAttribute not mean Flags?

2008-06-19T09:26:00Z

In the System.Data namespace, there's a ConnectionState enumeration. This has values like Open, Closed, Connecting, Fetching to represent the state of a database connection.

I pondered on whether a connection which is "Fetching" is also "Open". It turns out that ConnectionState is declared with the FlagsAttribute - The mono DocComment for this enumeration says it best:

/// This enumeration has a FlagsAttribute that allows a bitwise combination of its member values.

 Great - so now I can do something like:

if((connection.State & ConectionState.Open)==ConnectionState.Open) ...

Wrong! Further down the MSDN documentation for ConnectionState, tucked away in the Remarks section is the following paragraph:

The values in this enumeration are not designed to be used as a set of flags.

Great. So the question is, which of the database providers follow the FlagsAttribute, and which follow the documentation comment? Does SQL Server treat ConnectionState as flags? Does the managed Oracle provider? Does Oracle's ODP.Net provider treat it as flags? etc.

Certainly there are many people who treat it like flags. There are equally many people who ignore it.

I think I know this history behind this debacle - if anyone knows for sure, I'd like to know. If you actually look at the ConnectionState definition, you will see that ConnectionState.Closed has a value of 0. This flouts the .Net Design Guidelines and I expect it to be raised by FxCop/CodeAnalysis. Go ahead and read that link if you're not sure why bitwise comparison with 0 is a bad idea. This means there are a lot of bugs about due to the use of Bitwise comparison of ConnectionState.Closed. So I suspect that the System.Data team realized they'd made a mistake and was unable to retroactively remove the FlagsAttribute - so their approach was to document the issue in an obscure remark.

The CLR garbage collector is in need of help

2008-06-12T10:54:00Z

Back in 2000 when the CLR was first shown it's generational garbage collector was fairly cutting edge. But it's weaknesses show it's age - and it desperately needs updating.

Here's an example of just one of the ways the current GC (.Net 3.5) doesn't work:

A .Net process with a number of threads has just saved a large amount of data to a database. All of these threads could potentially allocate memory which causes a GC collection. However, saving the data to the database is a fairly long operation and thus the data gets promoted into the GC's generation 2.

After saving the data, the .Net process waits until it's instructed to do some more work. However, because a large amount of data has been promoted into Gen2, it just sits there until sufficient new allocations cause a Gen2 GC collection. The problem is that if no new work happens for a long time (hours), these unreachable objects take up (virtual) address space. Yes, NT will swap it out to disk as virtual memory. But it's still there.

The GC really needs a timer, which periodically checks if no GCs have happened recently and the process has consumed very little CPU, then perform a Gen2 collection.

Aside: Does anyone know how to trigger a GC.Collect() on a process from a different process, or from inside WinDbg? One of the problems with WinDbg/SOS is figuring out just which objects are reachable - you would have to !gcroot every possible object just to figure out which are the reachable ones.

Windows Geniune Disadvantage

2008-06-10T09:51:00Z

---------------------------
Windows Genuine Advantage
---------------------------
Code not available. The validation code could not be obtained. This may be due to technical difficulties, or you may be running an unsupported operating system. Please close this window and attempt the validation process again, or use the Back button in your Web browser to return to the download details page. [Error code: 0x80072ee7]
---------------------------
OK
---------------------------

Why does Microsoft insist on making my life difficult?

Refactoring

2008-06-03T13:25:00Z

I'm doing a lot of porting and refactoring at the moment. I've come to the conclusion that porting from a dynamically typed language to a statically typed language is an order of magnitude more difficult than doing the reverse.

Refactoring is an interesting game. There are some very easy refactorings - especially with the toolsets available to us nowadays. However, there are some extremely difficult refactorings. I've found the most difficult to refactor methods have one or more of these 3 aspects:

1) It has long methods (>40 lines)
2) The method shares a variable for different things
3) Loops do more than 1 thing

1) causes 2) - so eliminating 1) means that 2) doesn't happen.

I sure wish the NextBigLanguage has an upper bound on the lines/method it can parse.

Code Smells in C#

2008-05-12T16:11:00Z

A lot of my development is done with Code Smells. Whether I'm working on my own code or refactoring someone elses code, if I see some code and it smells funny, then I investigate further.

Recently I've come to realize there's a code smell which we can blame Microsoft (Anders) for!

C# Code Regions.

Back in the early days of .Net (ie 2000-2003), I used to moderately region my code. But since reading Martin Fowler's seminal Refactoring book where the concept of Code Smells was introduced, I found that I didn't need C# Regions.

So why are regions a Code Smell?

There are two types of regioning I've seen in code. Regions that group methods, properties, member variables, constructors etc together, and those which collapse a block of code. I'll deal with each separately.

Using Regions to group members.

In many companies, its commonplace (and even enforced practice) to group similar class members together and place a region block around them. I've even seen this as a coding doctrine inside Microsoft. The problem is that Visual Studio already has a facility to do this automatically. And it's been around in VS since it was called VJ++ and before that in VC. It's called the Class View window, accessed with the shift-ctrl-C shortcut. So why anyone would care about building a class "DOM" into their code using Regions, rather than rely on the Class View window I have no idea. Perhaps they're using Notepad?

Using Regions to hide blocks within a method.

This is when it gets really nasty. I've seen multi-hundred line long methods containing regions, and subregions and sub-subregions. I've seen switch() statements wrapped in a region, then each individual case have a region, then subregions inside of the region inside of the case. Switch statements are already a Code Smell, but having region after region after region is a sure-fire way of telling you the method is too long.

So, please, can we all resolve to stop using Regions?

Interestingly, there are many features that Java has started copying from C# - but regions were not one of those. Perhaps in C# v4 Microsoft can right the wrong and deprecate regions?

Caching and SOAP

2008-03-27T11:12:00Z

If you're making a SOAP request (in this case with the SOAP toolkit), and you get:

"WSDLReader:Loading of the WSDL file failed HRESULT=0x80070057: The parameter is incorrect."

during the MSSoapInit() call, check that you haven't added a cache-control:no-cache custom header to IIS, or that HttpExpires isn't set to zero days.

Silverlight Sudoku with LINQ

2007-08-23T07:30:00Z

A couple of months ago I came across a very clever Sudoku solver written by Peter Norvig. Peter had written the solver using Python. The solver was special as it solved by constraint propagation using list comprehensions and generator expressions making it extremely fast and very memory frugal. It turns out that the new LINQ feature of C# v3 is basically list comprehensions and generator expressions by another name.

Brendan Eich, the creator of JavaScript had ported the original Python implementation to JavaScript and I thought it would be a good idea to try to port it to C# v3. You can check out the code here.

I always had a vague plan to wrap the solver in a Silverlight front-end for added WOW factor, but it never really materialized. Fast forward two months and David Anson posts a Silverlight-based Sudoku game on his blog. This gave me the perfect excuse to integrate the LINQ-based solver into his codebase. I extended David's code to respond to the Escape key and complete the board with the first solution found. You can play with it online here, or download the code here. Have fun!

To solve a game, give the Silverlight board the input focus and hit the escape key.

Silverlight crashing

2007-04-30T17:14:00Z

So I'm sat here watching the Mix07 keynote, 45 minutes after Ray Ozzie first got to the stage. I download the Silverlight Beta for Mac and install it.

Firefox has crashed 4 times while trying to view the Silverlight demos on microsoft.com/silverlight. This is getting really frustrating trying to review the new features.

Silverlight

2007-04-16T21:06:00Z

As has been blogged all over the place, WPF/E has been renamed Silverlight (Note: lower case 'L').

Some people say there's more to come at MIX07. Well, I think I've discovered what the secret is. It's hidden inside of the Silverlight promo video. Silverlight is written in Java! Don't believe me? Take a look at a frame grab from the promo video, clearly showing Silverlight's use of SAX, the javax namespace and use of Java API's like vector.addElement.

The trouble with dynamic languages

2007-01-22T19:51:00Z

Javascript is a dynamic language. And that's a problem. On the one hand it allows you virtually unlimited ability to play with data structures and mold them to your own design. On the other, it presents the virtually insurmountable problem of testability.

You see, a dynamic language delays the checks which are performed by a static language during compilation until runtime. So you have no idea whether your code is correct until it is executed. The usual answer to this is to write unit tests. But the people who say this tend to gloss over the difficulty of getting 100% unit test coverage. 70% coverage is usually considered a very high test rate. And then there's the difficulty of getting the coverage metrics themselves. At the time of writing, there is only one Javascript/browser code coverage tool around. Hopefully all the hype surrounding AJAX will encourage more of these tools to be written.

One tool that is available now is JavascriptLint. Lint was the tool you may have used many years ago to statically analyze your C source code for common programming errors not picked up by the compiler. So JavascriptLint does the same but for Javascript. It's based on SpiderMonkey, the C-based Javascript engine in Firefox. So you can be sure it's rock solid with zero false positives. It's a fantastic tool to help AJAX programming as it seamlessly integrates into your favorite IDE as an external tool. Some of the errors it picks up on are:

Missing semicolons.
Unintended use of octal where decimal was intended
Automatic type coercion
Undeclared variables
Redeclared variables

Some of these are only available with the tool in Option Explicit mode. Even if it doesn't spot outright bugs in your code, the checks for type coercion and missing semicolons are performance bugs in your code.

The benefits of JavascriptLint are so conclusive it gets run over my code several times per day.

What language was ASP.Net originally written in?

2006-11-28T10:16:00Z

In this video, Peter Hallam, one of the original C# designers talks about a new C# v3 feature, but also says that WinForms was the original consumer of the C# language.

This totally contradicts my understanding. I thought that ASP.Net (XSP) was the original team using C# and that WinForms (WFC as it was) was only ported to C# from Java over the winter of 1999/2000 by Chris Anderson.

Am I wrong?

Client-side validation in Firefox

2006-11-01T07:49:00Z

Recently, I was faced with a problem whereby client-side validation of controls failed to work in Firefox. Only server-side validation was working. Client side validation was working fine in IE.

This was puzzling as everything I read said that ASP.Net v2 supported client-side validation in Firefox. No amount of Googling helped to identify the problem, so I had to track it down from scratch - and the results were quite interesting. If you have

set in your web.config, then client-side validation is disabled in firefox.

To see why, we have to jump down into the validation framework that ASP.Net sends to the web browser. The actual Javascript validation function looks like:

function ValidatorValidate(val) {

val.isvalid = true;

if (val.enabled != false) {

if (typeof(val.evaluationfunction) == "function") {

val.isvalid = val.evaluationfunction(val);

}

ValidatorUpdateDisplay(val);

}

Using Venkman, I was able to see that val.evaluationfunction wasn't a function at all, therefore the page wasn't getting validated because of this. A bit more searching found that this evaluationfunction is an expando attribute. In ASP.Net v1, this expando is an IE-only HTML expando, defined directly in the HTML markup. In ASP.Net v2, this is a Javascript expando attribute, defined on the DOM via Javascript, and therefore compatible with both IE and Firefox.

For example, the RequiredFieldValidator calls RegisterExpandoAttribute. You can call this method yourself like:

Page.ClientScript.RegisterExpandoAttribute(TextBox1.ClientID, "value", TextBox1.Text);

This produces a block of Javascript just before the closing </form> tag like:

<!--

var TextBox1 = document.all ? document.all["TextBox1"] : document.getElementById("TextBox1");

TextBox1.value = "";

// -->

</script>

where "value" has become the new expando attribute. In the case of the RequiredFieldValidator, the expando name is" evaluationfunction" and it is added to the <span> representing the control at runtime.

It turned out that this block of Javascript wasn't being sent to Firefox in my situation, and I didn't know why. Instead, the evaluationfunction expando attribute was being rendered directly on the <span> element in HTML.

So after good deal of decompilation using Reflector, and debugging using Reflection Invokeing of private methods, I discovered the problem was that RegisterExpandoAttribute checks the XHTML Rendering Mode prior to generating the Javascript and if that rendering mode is set to Legacy, then the expando is rendered directly onto the <span> element and no Javascript code is rendered. The result of this is to prevent Firefox from performing client-side validation when Legacy XHTML mode is set.

I have no idea why the ASP.Net team chose to do this for validation. Perhaps the Legacy XHTML setting should have been called "ASPNetv1Conformance" because that's actually what it does. The docs for Legacy mode state:

"Reverts a number of rendering changes made for conformance to the v1.1 rendering behavior."