MacBook Rev. F Audio Skipping in Vista Analisys and Solution – Part 1

Last week I got the latest Apple’s 13.3″ MacBook Rev. F. (aluminum unibody model)

This is an amazing looking laptop, and is one of the fastest performing laptops out there. I decided to have dual boot OS and installed Vista and Apple’s BootCamp.
Here is MacBook’s performance index in Vista:

As you can see, the overall rating is pretty high and the graphics card has pretty high rating (5.4) as well. One nice thing is that at the back Apple gave us easy access to the hard drive on the back. This makes a very easy upgrade. The thing that I found is that this is the only laptop that can fit new 500GB SATA laptop drives which are thicker (12.5mm) than standard 250GB laptop drive (9.5mm). So I managed to easily upgrade mine to 500GB 5400rpm Hitachi laptop drive.

The Problem

After playing with MacBook for a while on Vista, I noticed that when playing music, frequently there are audible clicks and audio skipping. This was even more audible when I was listening streamed radio stations over shoutcast network. On top of this, I think that there is significant reduction of the MacBook battery run time as well (comparing to OSX). I decided to investigate this more closely.

In short, I found, the reason why this happens is because of the exceptionally high DPC latency.

Why Audio Drop-outs Occur?

(Taken from http://www.thesycon.de/deu/latency_check.shtml)

Processing of streaming data in real-time is a very challenging task for Windows based applications and device drivers. This is because by design Windows is not a real-time operating system. There is no guarantee that certain (periodic) actions can be executed in a timely manner.

Audio or video data streams transferred from or to an external device are typically handled by a kernel-mode device driver. Data processing in such device drivers is interrupt-driven. Typically, the external hardware periodically issues interrupts to request the driver to transfer the next block of data. In Windows NT based systems (Windows 2000 and better) there is a specific interrupt handling mechanism. A device driver cannot process data immediately in its interrupt routine. It has to schedule a Deferred Procedure Call (DPC) which basically is a callback routine that will be called by the operating system as soon as possible. Any data transfer performed by the device driver takes place in the context of this callback routine, named DPC for short.

The operating system maintains DPCs scheduled by device drivers in a queue. There is one DPC queue per CPU available in the system. At certain points the kernel checks the DPC queue and if no interrupt is to be processed and no DPC is currently running the first DPC will be un-queued and executed. DPC queue processing happens before the dispatcher selects a thread and assigns the CPU to it. So, a Deferred Procedure Call has a higher priority than any thread in the system.

Note that the Deferred Procedure Call concept exists in kernel mode only. Any user-mode code (Windows applications) runs in the context of a thread. Threads are managed and scheduled for execution by the dispatcher.

While there is a pre-emptive multitasking for threads, DPCs are executed sequentially according to the first in, first out nature of a DPC queue. Thus, a sort of cooperative multitasking scheme exists for Deferred Procedure Calls. If any DPC runs for an excessive amount of time then other DPCs will be delayed by that amount of time. Consequently, the latency of a particular DPC is defined as the sum of the execution time of all DPCs queued in front of that DPC. In order to achieve reasonable DPC latencies, in the Windows Device Driver Kit (DDK) documentation Microsoft recommends to return from a DPC routine as quick as possible. Any lengthy operation and specifically loops that wait for a hardware state change (polling) are strongly discouraged.

Unfortunately, many existing device drivers do not conform to this advice. Such drivers spend an excessive amount of time in their DPC routines, causing an exceptional large latency for any other driver’s DPCs. For a device driver that handles data streams in real-time it is crucial that a DPC scheduled from its interrupt routine is executed before the hardware issues the next interrupt. If the DPC is delayed and runs after the next interrupt occurred, typically a hardware buffer overrun occurs and the flow of data is interrupted. A drop-out occurs.

The Solution

After running the DPC Latency Checker program, I found that the excessive DPC latency was indeed the cause of all the audio skipping. 

Here is the DPC Latency with my MacBook idling:

At moments the latency was as high as 30.5ms!

After running the Process Explorer program I was able to track down the program that was causing the problem. It was the Apple’s KbdMgr.exe. This program is responsible for managing the keyboard, handling the special keys and displaying the OSD (on-screen-display) when you, for example, change screen brightness. 

For a program that is mostly idle the KbdMgr.exe consumes quite a lot of CPU cycles.

You can see that KbdMgr has eight threads, but only one of them constantly consumes high number of CPU cycles. After inspecting the API calls that thread (TID 1352) makes, I found that the main cause for the high CPU cycles is the bad KbdMgr’s multiprocessor implementation (deadlock situation betwen CPU cores in User/Kernel mode). In another words, by simply limiting KbdMgr to a CPU core #1 (CPU affinity) and optionally setting KdbMgr’s priority to Idle, the DPC latency of my MacBook improved dramatically:

As you can see the latency went from whopping high 30.5ms down to 1ms. The result was smooth audio playback with no skipping, much more responsive machine and improved laptop’s battery life.

Here is the fix: KbdMgr.rar
The new fixed version will automatically run on CPU #1 with low priority and will reduce the DPC latency of MacBook.

Installation:
1. Go to Task Manager and kill KbdMgr.exe process
2. Download and extract the contents of the KbdMgr.rar file to a temporary directory
3. Backup the original KbdMgr.exe file located in C:\Program Files\Boot Camp directory
4. Copy new KbdMgr.exe file to C:\Program Files\Boot Camp directory
5. Restart your machine

~ by Alexander Popovich on October 30, 2008.