I often have to send large files around by email. For example, I had to email a colleague a 10 MB file. It seems a little rude to hit him out of the blue with a 10 MB email. I wanted something cleaner and less intrusive.

So I created a simple utility I called "Big Mailer". I figured I'd blog about it and share it with the world. Feel free to use it as you see fit. This program consists of both a client and server piece that simplifies uploading content to your web site. Then the program gives you a regular web link you can send via email.

Here's a screen shot:

You can download and install it via ClickOnce here:

Install Big Mailer (700 KB)

You must have the following to use this program:

1. .NET 3.5 Framework installed on the client
2. .NET 3.5 Framework on the server
3. An ASP.NET web site to host the WCF service

After you install the client, click "Host Service" and you'll get the server side code to drop onto your ASP.NET web site. There is a test web site with instructions in that code.

If you don't have an ASP.NET web site to host the service at you can always use public services like Drop Boks (a great service!) or other file sharing sites. The benefit of this program / service is that you retain control of the files and you can conceivably send more sensitive content.

You can also just use FTP if you have that for your web site, but I hate FTP personally. I don't like fighting the firewall issues and I don't want it running on my servers.

So this program allows you to upload content of unlimited size, without FTP, without sending your files to a third party. Also, it sends everything in 16KB blocks, rather than one giant http message. So you get the benefit of both being able to send huge files (say 1 GB) and you get progress / cancel support.

One feature that's notably lacking is authentication. I'll probably release an update with security built-in. For now, take that into consideration before using it.

Enjoy!

PS - This project is now Open Source and is host on CodePlex.

Here's a followup post on the .NET ThreadPool bug that I described here:

Breaking Changes in the ThreadPool: The Movie

I have been in touch with the guys who are in charge of the ThreadPool and they have both confirmed that this is a bug and that they are planning on fixing it in .NET 2.0 SP2 - but they are not sure of the timeline for its release.

In the meantime, Vance Morrison, a .NET Runtime Performance Architect at Microsoft, has given me this work-around.

Take this "broken" code:

	private static void UseThreadPool(int count)
        {
            for ( int i = 0; i < count; i++ )
            {
                ThreadPool.QueueUserWorkItem(
                    delegate { SlowMethod(); } );
            }
        }

And add a strategic Thread.Sleep and it's fixed:

	private static void UseThreadPool(int count)
        {
            for ( int i = 0; i < count; i++ )
            {
                ThreadPool.QueueUserWorkItem(
                    delegate { SlowMethod(); } );
		Thread.Sleep(1);
            }
        }

Well, my recent post on .NET 3.5 Brings BREAKING Changes to ThreadPool sparked quite a bit of interest in the .NET community.

But this is also something difficult to convince people of because it depends so heavily on configuration. The source code doesn't change, the environment does.

So I've put together a screencast demonstrating the problem and elaborating further. If you doubt the validity of the previous post, or can't reproduce the problem, please watch the video:




                     Download the video (approx 18 MB)

Kick it:

In the video I work with a modified version of the program. Here's that for your enjoyment:

The application: NetThreading.exe (From Video).zip (2.59 KB)

The source code:

using System;
using System.Threading;

namespace NewThreadPoolBehavior
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Console.WriteLine( "Running on " + Environment.Version );
            int w, c;
            ThreadPool.GetMaxThreads( out w, out c );
            Console.WriteLine( "Max Currently: " + w + ", " + c );
            ThreadPool.GetMinThreads( out w, out c );
            Console.WriteLine( "Min Currently: " + w + ", " + c );

            Console.WriteLine( "Set min thread count 20? (y/n) " );
            string txt = Console.ReadLine();
            if ( txt == "y" )
            {
                Console.WriteLine( "Setting min to 20" );
                ThreadPool.SetMinThreads( 20, 100 );
                ThreadPool.GetMinThreads( out w, out c );
                Console.WriteLine( "Min Currently: " + w + ", " + c );
            }
            UseThreadPool( 200 );

            Console.ReadLine();
        }

        private static DateTime startTime;

        private static void UseThreadPool(int count)
        {
            startTime = DateTime.Now;
            for ( int i = 0; i < count; i++ )
            {
                ThreadPool.QueueUserWorkItem(
                    delegate { SlowMethod(); } );
            }
        }

        private static int concurrent = 0;

        private static void SlowMethod()
        {
            TimeSpan dt = DateTime.Now - startTime;
            concurrent++;
            Console.WriteLine( "Starting ops (" + concurrent + " concurrent, elapsed=" +
               dt.TotalSeconds.ToString( "N3" ) + " sec.) " );
            Thread.Sleep( 20000 );
            Console.WriteLine( "Finished ops (" + concurrent + " concurrent)" );
            concurrent--;
        }
    }
}

Holy smokes! I thought we had figured out something significant when I posted .NET 3.5 Brings Major (Undocumented) Changes to ThreadPool where we discovered that the .NET 3.5 ThreadPool changed the allocation algorithm for adding threads from linear to logarithmic.

This is bigger. [Recently updated see note below]

Here's the scenario. I have a server - say in the financial sector - that must process many requests and it must get up to speed immediately. I can't pay the 500 ms warm up time for the ThreadPool (.NET 2.0) or the even slower model in .NET 2.0 SP1. What do I do? I call ThreadPool.SetMinThreads(x, x) where x < (the current max), but much higher than 2 (the default). So I might call ThreadPool.SetMinThreads(100, 100) or something like that.

In .NET 1.0 - .NET 2.0 (without SP1) you would go from the warm up time model:


                 warm up time model (.NET 2.0)

To this immediate "ready" threading model.

                 ready model (.NET 2.0)

Notice that we immediately have 100 threads available in this case. BUT, and here's the but: .NET 3.5 (read .NET 2.0 SP1) *ignores* SetMinThreads. Oh, it claims to respect SetMinThreads:

   Running on 2.0.50727.1433
   Max Currently: 500, 1000
   Min Currently: 2, 2
   Set min thread count 100 (y/n)? y
   Setting min to 100
   Min Currently: 100, 100

But, look at the thread graph that results!


           ready model (.NET 2.0 SP1  -- i.e. .NET 3.5)

I don't know about you, but that doesn't look any different AT ALL than if we do not call SetMinThreads:


         warm up time model (.NET 2.0 SP1  -- i.e. .NET 3.5)

What do we conclude from this? .NET 2.0 SP1 does not respect ThreadPool.SetMinThreads. To me, that's a big breaking change. My financial app just stopped working. Yikes! So what happened to my thread pool and SetMinThreads in .NET 2.0 SP1? Anyone?

Kick it:

Don't take my word for it. Run this program on both .NET 2.0 (SP0) and .NET 2.0 SP1 and you'll see for yourself.

Here's the EXE:

   NetThreading.zip (2.3 KB)

And the source code is below:

using System;
using System.Threading;

namespace NewThreadPoolBehavior
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Console.WriteLine( "Running on " + Environment.Version );
            int w, c;
            ThreadPool.GetMaxThreads( out w, out c );
            Console.WriteLine( "Max Currently: " + w + ", " + c );
            ThreadPool.GetMinThreads( out w, out c );
            Console.WriteLine( "Min Currently: " + w + ", " + c );

            Console.WriteLine( "Set min thread count 100? (y/n) " );
            string txt = Console.ReadLine();
            if ( txt == "y" )
            {
                Console.WriteLine( "Setting min to 100" );
                ThreadPool.SetMinThreads( 100, 100 );
                ThreadPool.GetMinThreads( out w, out c );
                Console.WriteLine( "Min Currently: " + w + ", " + c );
            }
            UseThreadPool( 200 );

            Console.ReadLine();
        }

        private static void UseThreadPool(int count)
        {
            for ( int i = 0; i < count; i++ )
            {
                ThreadPool.QueueUserWorkItem(
                    delegate { SlowMethod(); } );
            }
        }

        private static int concurrent = 0;

        private static void SlowMethod()
        {
            concurrent++;
            Console.WriteLine( "Starting ops (" + concurrent + " concurrent)" );
            Thread.Sleep( 20000 );
            Console.WriteLine( "Finished ops (" + concurrent + " concurrent)" );
            concurrent--;
        }
    }
}

[Update: 2/25/2008 - You may be thinking so what's the big deal? Most applications don't directly program against the ThreadPool so this is just some edge case. To make this more real for everyone, this applies to you if you use any of the following: ASP.NET, WCF, .NET Remoting, Delegate.BeginInvoke, SqlCommand.BeginExecute*, Windows Workflow, and more. Sounds serious now right?]

[Update: 5/12/2008 - Please see my lastest followup on this topic: More on the ThreadPool Bug in .NET 2.0 SP1.]

It was all going so smoothly. Jason Whittington, Mark Smith and I were teaching the big DevelopMentor event here in Los Angeles (Guerrilla.NET) when my presentation on the ThreadPool took a nose dive. It started with a great joke involving Wilson (the volleyball from Cast Away).

Wilson and I built an application to compute a multiplication table where each computation was (artificially) slow. To speed it up we threw it at the thread pool using delegate.BeginInvoke. We figured that the ThreadPool would allocate 25 or so threads and the table would display quickly. Here's the expected output - pretty much the same thing we've seen since about .NET 1.0:

    

Each color represents the thread that did that computation.

For the last 7 years, the behavior has been that as the ThreadPool was overloaded, it would steadily start up new threads at the rate of one every 500 milliseconds until it hits its upper limit (typically). Using Performance Monitor (perfmon) we can watch the thread pool adding threads. It usually looks something like this:

    

Much to our surprise we saw completely different behavior. The thread pool added the first 15 or so threads quickly (as expected) but then stalled. New threads were not created every 500ms, instead they were added at increasingly long intervals. My demo took almost twice as long to run as it had the last time I did this demo a few months ago.

Jason, Mark, and I took this code, ported it back to .NET 1.1 and ran it side-by-side with .NET 3.5 and here's what we saw (blue = 3.5, red = 1.1):

    

As of .NET 3.5 the upper limit of the ThreadPool was increased: Knew that.

But, it appears that v3.5 of the CLR changes the policy for adding threads to the thread pool. Rather than adding threads regularly when under load the thread pool uses a logarithmic backoff. My colleague Jason Whittington remarked that this behavior looked similar to the behavior of the thread pool in "Rotor" (the shared-source version of the CLR). We speculated that this backoff algorithm makes sense given the new 250-thread per CPU maximum - it would take a long time to reach that if the runtime waits longer and longer to start a new thread. The 250-thread limits makes it less likely that your application will deadlock the thread pool, and the exponential backoff algorithm keeps the thread pool from creating too many threads too quickly.

Why should you care? Usually you won't, but it could have dramatic impact if you count on that behavior. For example, in our contrived case, .NET 1.1 ran about twice as fast as .NET 3.5 ( ! ):

    

Here's the program and source code (trimmed down to run in both .NET 1.1 and 3.5).

    Math.zip (6.38 KB)

Needless to say, Wilson and I felt kinda stood up.

     

Please see the follow up post on Breaking changes and now there is screencast, movie version as well.

Thanks to everyone who attended Guerrilla .NET last week. I had a great time meeting all of you. Here are the slides and demos from all our presentations.

     Demos.rar (22 MB)

    Challenge.rar (8 MB)

- Michael

I recently wrote up an overview of the new ASP.NET MVC Framework for the Developments newsletter.

I encourage you to read it on the DevelopMentor website. It's an interesting programming model.

I've been playing with my fresh copy of Vista Ultimate - which I am surprised to find that I absolutely love.

Being a big fan of System.Transactions, I naturally wanted to use it with Vista's TxF (Transactional NTFS) file system. But unlike the data libraries, the file APIs don't auto-enlist in the transaction. In fact, there are only COM / PInvoke APIs currently.

There is a nice article about how to work with these APIs in the MSDN article: "NTFS: Enhance Your Apps With File System Transactions". But I was unimpressed with the managed wrapper they created there. In particular, I don't like that the lifetime of the file stream is not forced to be part of a client initiated transaction scope. So I built my own transactional file stream in C#. With this TxFileStream class, you can write succinct code like this:

[Test]
public void ContentAddedDoesNotPersistsAfterRollbackTest()
{
    string fileName = "file3.txt";
    string originalContents = "First contents";
   
    using (TransactionScope scope = new TransactionScope())
    {
        string newContents = "Hello transacted NTFS.";
        using (StreamWriter sw = TxFileStream.CreateWriter(fileName))
        {
            sw.Write(newContents);
        }

        using (StreamReader sr = TxFileStream.CreateReader(fileName))
        {
            string text = sr.ReadToEnd();
            Assert.That(text, Is.EqualTo(newContents));
        }
        // no call to scope.Complete() forces a rollback.
    }

    Assert.That(File.Exists(fileName));
    Assert.That(File.ReadAllText(fileName),
        Is.EqualTo(originalContents));
}

Feel free to download the code and give it a spin!

    Kennedy.TxFiles.zip (36 KB)

Of course, my library comes with comprehensive unit tests. Look here first to figure out how out use the library.

Note: I fixed a bug in creating transactional StreamWriter's in append mode. Previously they partially overwrote the existing content.

Welcome to my third Visual Studio tricks post. This time it's more of a utility, than a tip. I want to talk about managing the recently projects list. If you're like me, then you work with many different projects (especially after I teach a class) and your recent project list becomes polluted with projects you don't care about.

     


In this post, "Recent Projects in Visual Studio 2005", .net DEvHammer discusses how to access the registry to alter that list.

Well, I didn't feel like going to the registry everytime I wanted to clean that list. So I whipped up a simple UI to manage that list (basically manage that registry list).




You're welcome to download this program if it looks useful to you. I decided to publish it via ClickOnce so it will always be up-to-date. If you're using FireFox, you'll need to FFClickOnce add-on to make this work.

    Install Visual Studio Recent Files Utility (approx 200 KB)



I hope you enjoy it!

This project is now hosted on CodePlex and is Open Source.

Welcome to the next installment of my Visual Studio Tricks series. Continuing on from last time when I discussed how to quickly switch between startup projects using hot-keys, we'll cover another startup project trick.

Here's the scenario. Suppose you're working on a brand new WCF application that has both a client and server piece.

        

You need to start both the client and server to accomplish anything interesting. Most people fumble around starting the server, then the client.

Did you know that you can tell VS to run both at once. You just have to know where to look. The place to look is the properties of the solution. i.e. Right-click on the solution and choose properties and you'll get this dialog:

                                   Click for full image

By default this is set to "Single startup project". But you just have to select "Multiple startup projects" and you're on your way to smooth sailing with your client / server application. You even get debugging of both the client and server when you press F5.

Enjoy!

Welcome to my Visual Studio Tricks series. Here I’ll give you some quick tips for saving you lots of time when working with Visual Studio.

In this first installment, I'll show you how to switch between projects more quickly. Any time you work on a large project, you’ll have several projects that you might want to launch from within a solution. This includes at least an EXE and a unit test project right? Hint, hint, nudge, nudge.

Typically you do this by right-clicking on the project and say “Set as Startup Project”. But did you know you can set a hot key for this? Just go to the keyboard options in VS 2005 and type “SetAsStar” and you’ll see this screen:

If you select the startup project option, you can now add a keyboard shortcut. Sounds simple but it saves lots of time and fumbling with the mouse. There are no hotkeys assigned by default. Note that I chose <ctrl>-<shift>+P. That seems like a good choice for me.

I'll be speaking tonight at The Central New Jersey .NET User Group (http://www.njdotnet.net) on one of my favorite topics: Five Fundamental Object Oriented Design Principles for Agile Development.

Thanks to Jason Beres (http://www.geekswithblogs.net/jberes) for inviting me to speak at his user group and welcoming me to New Jersey.

Once you've attended the talk, you might be interested in downloading the slides and sample code:

    http://www.MichaelCKennedy.net/Talks/Downloads/NJdotNet/AgileDesign.zip

You also might want to look into some of the tools I was using in Visual Studio. See an older post on my tricked out Visual Studio.

Thanks to Reza Madani and Mike Vincent for having me speak at the combined meeting of OC VBUG and OC C# last night. It was a packed house which was wonderful, but I felt bad for folks who had to stand in the back for 2 hours.

The audience participation was great. Thanks to everyone who asked questions or had comments.

You can download the slides and samples from my website here:

http://www.michaelckennedy.net/Talks/Downloads/OCVBandCSharp/AgileDesign.zip

Many attendees also noticed the tricked-out Visual Studio I was using during the presentation. That was from CodeRush and Refactor! both highly recommended. You can check those out here:

http://www.devexpress.com/Products/NET/CodeRush/
http://www.devexpress.com/Products/NET/Refactor/

and DevExpress has some very interesting screen-casts demos here:

http://www.devexpress.com/Products/NET/CodeRush/Training.xml

Also I was using the new Vista programming font Consolas, which you can download here:

Consolas Font Pack for Microsoft Visual Studio 2005

Thanks to David McCater for hosting my Agile Object Oriented Design talk at San Diego .NET Developers Group on May 2nd. I met some great guys and thoroughly enjoyed talking about agile and OOD topics with everyone.

You can download the slides at:

http://www.michaelckennedy.net/talks/downloads/sddotnetdev/agiledesign.zip

Dave, over at extremeplanner.com, wrote a nice summary of the talk I gave at XPSD. I completely agree with him that "a review of these fundamentals [is] refreshing." As you can see, I'm doing my part to spread the word:

• Upcoming Talk in Orange County
• Upcoming Talk in San Diego (Tentative)
  
• The Rock & Roll Code Camp (details upcoming)
  
  
• XPSD Object Oriented Design Talk (past presentation)
• January SoCal Code Camp Talk (past presentation)

Many pundits and authors in the software industry are busy promoting the next big product/feature/etc that they seem to forget that these things only provide second order benefits as compared to the more "fundamental" ideas such as good object oriented design. For example, looking through some of the user groups' upcoming meetings you'll find topics like:

• Looking at some Cool New Features of ASP.NET 2.0: Managing Membership and Role Management
• Leveraging the New Windows Mobile APIs
• Visual Studio Team System
• Closer to the Data: Using Managed Code in the Database with SQL Server 2005

Lots of neat, whiz-bang features, but I would contend that 90% of the audience would be much better off if they had heard a good talk about solid OOD or TDD or this list goes on. It's been my experience that much of these old (5-15 years), tried and true fundamentals are unfamiliar to most developers.

So here is my call to action:

If you are a speaker or author, reach for something big for your next topic. Yes it's easier to write/talk about the details of the next version of feature X. But you'll do your audience a service if you reach higher, think bigger, and go for something "fundamental."

If you are attending user groups and code camps, ask for and expect something more than a talk on the of latest version of product X.

I am tenatively scheduled to speak in San Diego, CA at the “San Diego .NET Developers Group” (http://sddotnetdg.org/). We haven’t arranged a date yet, but I think it will be around late spring / early summer.

This time I will again be presenting a talk on object oriented design and agile development, probably a slight variation on the general topics from my other talks.

I’ll provide more details as they become available.

You often hear it stated by very bright people that they thought they understood object oriented design until they began practicing Test Driven Development (TDD). I definitely include myself in that group (the misunderstanding OOD part anyway!).

Here is a list of the very best books I have found that helps bridge the divide between OOD and Agile for those of you who are currently making that transition. They are highly recommended.

Agile Software Development, Principles, Patterns, and Practices
by Robert C. Martin

Refactoring: Improving the Design of Existing Code
by Martin Fowler

Working Effectively with Legacy Code
by Michael Feathers

I’m very pleased to announce that I’ll be speaking in Orange County, CA on May 9th from 6pm onward.

I’ll will be presenting a talk on object oriented design and agile development at a joint meeting of the “Orange Country C# Developers Group” (http://sddotnetdg.org/) and the “Orange County VB.NET User Group” (http://www.ocvbug.org/).

I encourage anyone in the Anaheim / L.A. area interested in the interplay between object oriented design and agile software development to attend.

[NOTE: This recent post refers to a much older (March 3, 2006) event.]

I want thank Carlton Nettleton and June Clarke for inviting me to present my “Five Fundamental Object Oriented Design Principles for Agile Development” talk at XPSD this past March. The audience participation was great and I really learned a lot myself!

You can download the slides and code samples here:

http://michaelckennedy.net/Talks/Downloads/XPSD/AgileDesign.zip

 

[NOTE: This recent post refers to a much older (Jan 21, 2006) event.]

I want to thank everyone who attended my talk at the first Southern California Code Camp. I really enjoyed speaking there and I got the sense that many of you got something from the talk. In case you weren’t there and are interested, here’s the abstract:

Five Fundamental Object Oriented Design Principles for Agile Development

This session will present five object oriented design principles that facilitate agile software development. These general design principles promote the creation of testable, maintainable, and reusable software. The principles include the Open Closed Principle and the Liskov Substitution Principle. The interaction between Agile Development and these principles will be demonstrated using several code samples.

 You can download the slides and code samples here:

http://michaelckennedy.net/Talks/Downloads/SoCalCodeCamp/AgileDesign.zip