Monday, November 19, 2007

VMware - using named pipes to redirect serial console output

Most kernel debuggers (and some headless operating systems) require the use of a serial console. When running inside a VMware virtual machine, this could complicate things.

A neat way to solve this problem (and get your self a home made "LOM") is to use named pipes (well, it's actually a Unix Domain Socket) to redirect serial output to a TCP port.

Just add a Serial Port to your Virtual Machine, set the Virtual Machine end as the client, and the other end to an application.

Now we'll need to make use of a Named Pipe TCP Proxy Utility.

Windows users will probably pick NPTP (it has an easy to use GUI) and UNIX users (well, Linux, since VMware supports that) will pick something like Socat (an extended netcat that supports pipes, etc).

Some Windows user may prefer a command line alternative (with source code available). VMwareGateway also supports running as a service. It listens on localhost on port 567. The name of the pipe is \\.\pipe\vmwaredebug.

So, what about those who run VMware on Linux?

Well, we'll create a serial port in VMware, same as above, and point it to a file.
Then, we'll use socat:

% socat -d -d -d /path/to/port tcp4-listen:6666

See the socat manpage and examples for more info.

What about people who can't run VMware (eg: not running Linux or Windows)?

Well, qemu supports serial port redirection! See the '-nographic' option.