Comparison-based File Server Verification
Yuen-Lin Tan��, Terrence Wong, John D. Strunk, Gregory R. Ganger
Carnegie Mellon University
Comparison-based server verification involves testing a
server by comparing its responses to those of a reference
server. An intermediary, called a “server Tee,” interposes
between clients and the reference server, synchronizes
the system-under-test (SUT) to match the reference
server’s state, duplicates each request for the SUT,
and compares each pair of responses to identify any discrepancies.
The result is a detailed view into any differences
in how the SUT satisfies the client-server protocol
specification, which can be invaluable in debugging
servers, achieving bug compatibility, and isolating performance
differences. This paper introduces, develops,
and illustrates the use of comparison-based server verification.
As a concrete example, it describes a NFSv3
Tee and reports on its use in identifying interesting differences
in several production NFS servers and in debugging
a prototype NFS server. These experiences confirm
that comparison-based server verification can be a useful
tool for server implementors.
1 Introduction
Debugging servers is tough. Although the client-server
interface is usually documented in a specification, there
are often vague or unspecified aspects. Isolating specification
interpretation flaws in request processing and in
responses can be a painful activity. Worse, a server that
works with one type of client may not work with another,
and testing with all possible clients is not easy.
The most common testing practices are RPC-level test
suites and benchmarking with one or more clients. With
enough effort, one can construct a suite of tests that exercises
each RPC in a variety of cases and verifies that each
response conforms to what the specification dictates.
This is a very useful approach, though time-consuming
to develop and difficult to perfect in the face of specification
vagueness. Popular benchmark programs, such as
SPEC SFS [15] for NFS servers, are often used to stresstest
servers and verify that they work for the clients used
in the benchmark runs.
This paper proposes an additional tool for server testing:
comparison-based server verification. The idea is sim-
��Currently works for VMware.
RPC Requests
Figure 1: Using a server Tee for comparison-based verification.
The server Tee is interposed between unmodified clients and the unmodified
reference server, relaying requests and responses between
them. The Tee also sends the same requests to the system-under-test
and compares the responses to those from the reference server. With
the exception of performance interference, this latter activity should be
invisible to the clients.
ple: each request is sent to both the system-under-test
(SUT) and a reference server, and the two responses are
compared. This can even be done in a live environment
with real clients to produce scenarios that artificial test
suites may miss. The reference server is chosen based on
the belief that it is a valid implementation of the relevant
interface specification. For example, it might be a server
that has been used for some time by many user communities.
The reference server thus becomes a “gold standard”
against which the SUT’s conformity can be evaluated.
Given a good reference server, comparison-based
server verification can assist with debugging infrequent
problems, achieving “bug compatibility,” and isolating
performance differences.
This paper specifically develops the concept of
comparison-based verification of file servers via use of
a file server Tee (See Figure 1).1 A file server Tee interposes
on communication between clients and the reference
server. The Tee automatically sets and maintains
SUT state (i.e., directories, files, etc.) to match the reference
server’s state, forwards client requests to the reference
server, duplicates client requests for the SUT, and
compares the two responses for each request. Only the
reference server’s responses are sent to clients, which
1The name, “server Tee,” was inspired by the UNIX tee command,
which reads data from standard input and writes it to both standard
output and one or more output files.

RAID data recovery, Mac data recovery, Unix data recovery, Linux data recovery, Oracle data recovery, CD data recovery, Zip data recovery, DVD data recovery , Flash data recovery, Laptop data recovery, PDA data recovery, Ipaq data recovery, Maxtor HDD, Hitachi HDD, Fujitsi HDD, Seagate HDD, Hewlett-Packard HDD, HP HDD, IBM HDD, MP3 data recovery, DVD data recovery, CD-RW data recovery, DAT data recovery, Smartmedia data recovery, Network data recovery, Lost data recovery, Back-up expert data recovery, Tape data recovery, NTFS data recovery, FAT 16 data recovery, FAT 32 data recovery, Novell data recovery, Recovery tool data recovery, Compact flash data recovery, Hard drive data recovery, IDE data recovery, SCSI data recovery, Deskstar data recovery, Maxtor data recovery, Fujitsu HDD data recovery, Samsung data recovery, IBM data recovery, Seagate data recovery, Hitachi data recovery, Western Digital data recovery, Quantum data recovery, Microdrives data recovery, Easy Recovery, Recover deleted data , Data Recovery, Data Recovery Software, Undelete data, Recover, Recovery, Restore data, Unerase deleted data, unformat, Deleted, Data Destorer, fat recovery, Data, Recovery Software, File recovery, Drive Recovery, Recovery Disk , Easy data recovery, Partition recovery, Data Recovery Program, File Recovery, Disaster Recovery, Undelete File, Hard Disk Rrecovery, Win95 Data Recovery, Win98 Data Recovery, WinME data recovery, WinNT 4.x data recovery, WinXP data recovery, Windows2000 data recovery, System Utilities data recovery, File data recovery, Disk Management recovery, BitMart 2000 data recovery, Hard Drive Data Recovery, CompactFlash I, CompactFlash II, CF Compact Flash Type I Card,CF Compact Flash Type II Card, MD Micro Drive Card, XD Picture Card, SM Smart Media Card, MMC I Multi Media Type I Card, MMC II Multi Media Type II Card, RS-MMC Reduced Size Multi Media Card, SD Secure Digital Card, Mini SD Mini Secure Digital Card, TFlash T-Flash Card, MS Memory Stick Card, MS DUO Memory Stick Duo Card, MS PRO Memory Stick PRO Card, MS PRO DUO Memory Stick PRO Duo Card, MS Memory Stick Card MagicGate, MS DUO Memory Stick Duo Card MagicGate, MS PRO Memory Stick PRO Card MagicGate, MS PRO DUO Memory Stick PRO Duo Card MagicGate, MicroDrive Card and TFlash Memory Cards, Digital Camera Memory Card, RS-MMC, ATAPI Drive, JVC JY-HD10U, Secured Data Deletion, IT Security Firewall & Antiviruses, PocketPC Recocery, System File Recovery , RAID