Unobserved Task Exceptions

Wherein we examine a delayed crash due to an unobserved exception in a System.Threading.Task.

Recently we experienced a service crash when a Task we had spun up to do some work threw an exception and no one was there to catch it.  Normally we’d expect an immediate crash when an exception escapes from a ThreadStart routine because the AppDomain immediately unloads, but that’s not how Tasks work.  Contrary to expectations, not only will the crash almost certainly be delayed, it may not happen at all.  To demonstrate this behavior we have a simple application below which creates a task that waits a few seconds then tries to crash the application by throwing an unobserved exception.  Doubtless the task is disappointed to find that the expected crash does not materialize.

What’s going on?  Clearly the process is having a crisis of identity.  The reality of the situation is that the framework has caught the exception and will hold on to it until the task is finalized.  Luckily the prompt informs us that we can still crash if only we’ll press any key.

Pressing a key as instructed triggers a full GC collection and crashes the process as expected.  The reason is that the collection forced the Task finalizer to run, indeed it was the only object in the finalization queue, here.  Task’s finalizer throws if there was an unobserved exception on the task, which is the whole point of the exercise.

Let’s pretend that we’re debugging this issue postmortem.  With a dump file from the suicidal process we’ve got to get to the bottom of what went wrong.  Dumping all the threads with !Threads (always a good first step) shows us that there is an exception on the finalizer thread.  This is a big neon “DANGER” sign waving a red flag.  Unhandled exceptions on the finalizer thread spell disaster for the process, so we’ll investigate that one right away.

Switching over to the finalizer thread and running !ClrStack shows us we are indeed finalizing a Task object.

Let’s print out the exception that’s causing us so much grief.  It turns out to be an AggregateException which contains the originating exception inside it.  Tasks always wrap the original exception in an AggregateException.  !PrintException is kind enough to point out that there is an inner exception and even gives the command to dump it.

Once we’ve printed out the inner exception things are much more clear.  The AggregateException contains the real culprit, which was the intentional InvalidOperation thrown by a lambda inside of Program.Main.  Who would write code like this?

We’ve seen above via a contrived example how an unobserved exception in a task can bring down an entire process.  Since the timing of the crash is subject to the whims of the finalizer, there’s no way to know exactly when or if it will crash.  However, with diligence we can reconstruct what went wrong in the Task and hopefully fix it.