PERFORCE/CVS benchmark - not extremely scientific.
An experiment.
I just took the time to try out CVS and PERFORCE over the following
links:
28.8 kbit/sec modem,
64 kbit/sec ISDN,
10 megabit/sec ethernet
locally
The 28.8 link has ping times of 260ms, markedly slower than most
56kb sites on the Internet.
All but the 28.8 test was between FreeBSD machines. The client
in the 28.8 test was a Linux box.
The data is 126 files in 41 directories totalling about 1MB (the
Berkeley dbopen source).
Raw data (min:sec):
28.8Kb 64Kb 10Mb local Notes
------ ------ ------ ------ ------
first co CVS 3:34 1:34 1:01 :50 1,2
of all files PERFORCE 3:23 1:13 :10 :15
no-op update CVS :39 :31 :35 :13 3
when up-to-date PERFORCE :02 :02 :02 :02
submit 1 file CVS :08 :04 :03 :02
from leaf dir PERFORCE :06 :02 :01 :01
submit 1 file CVS :40 :28 :40 :20 4
from root dir PERFORCE :06 :02 :01 :01
submit 8 files CVS :39 :13 :12 :10 5
from leaf dir PERFORCE :31 :08 :04 :04
label all files CVS :48 6
PERFORCE :00.6
Summary:
Over fast links, PERFORCE is 3-6 times faster when dealing with the whole tree.
Over most links, PERFORCE is 3 times faster when dealing with a single file.
On slow links, PERFORCE and CVS are the same at bulk file movement.
Analysis:
(1) In the bulk transfers, CVS seems to have some fixed cost
that becomes more apparent over fast connections. Over fast
connections the gap between PERFORCE and CVS is remarkable, but at
lower speed (28.8), the gap more or less closes. Both PERFORCE and CVS
have streaming protocols, so they aren't affected by the latency
of the modem line.
PERFORCE's checkout is much faster than CVS for three reasons: First,
it doesn't need to scour the RCS tree to find the head revisions
of the files - it has that in a database. Second, once PERFORCE knows
what files to update it can stream data solidly: it doesn't take
a breath at the beginning of each directory. Third, while CVS
apparently can deliver the head rev without invoking the RCS
'co' program, PERFORCE has a complete RCS implementation linked in,
and can produce any revision directly. It uses a combination
of mmap() and memchr( ptr, '@', len) to make quick work of
parsing even large RCS files. (Sorry if I am drooling here.)
(2) I can't exactly explain why the CVS initial checkout is
faster locally than over the 10Mb ethernet, except to note that
CVS seems to stop and smell the roses at the beginning of each
directory, and there must be more roses on the ethernet.
PERFORCE is slower locally than over the ethernet because of disk
thrashing: while it is updating the client workspace, it is
also modifying server data to record the clients updated state.
This wouldn't happen if I had more than one disk.
(3) CVS seems to have different modes for local vs. remote
operations. In the local mode, it scours the repository and
the working directories at the same time. In the remote case,
it seems to do the remote repository first and then use the
result to drive its subsequent probe of the working directory.
This scouring affects both global and local updates and commits.
The transfer of data affects remote updates and commits. For
some reason, it is a dramatic difference even over the ethernet,
though not much more over the slow net.
PERFORCE's "update" command is substantially faster than CVS because
the server has all relevant information in a database, and
doesn't need to scour the disk or transfer any information.
(4) PERFORCE's "submit" command is faster than CVS for the same reason:
the server has all relevant information and doesn't need to
probe any disks.
(5) PERFORCE's "submit" may gain more ground on CVS because of its
built-in RCS implementation: PERFORCE doesn't need to fork/exec a
RCS ci(1) for each file.
(6) PERFORCE's file labelling is orders of magnitude faster than CVS
because PERFORCE stores the label information in a database, while
CVS puts it into each RCS file.