This is the mail archive of the
gnats-devel@sources.redhat.com
mailing list for the GNATS project.
fix for PR 219 - large PRs or updates take many minutes to process
- To: <gnats-devel at sources dot redhat dot com>
- Subject: fix for PR 219 - large PRs or updates take many minutes to process
- From: "Dirk Bergstrom" <dirk at juniper dot net>
- Date: Mon, 23 Jul 2001 00:11:14 -0700
after i submitted PR 219, it bit us again, locking up the server for a
couple hours. i decided i'd apply my meager C skills, and see if i
couldn't fix it. it seemed clear it was a badly written loop, and it
wasn't too hard to find it. turns out there were two problems, a typo
and a needlessly repeated strlen() call. the patch gives a 1000x
improvement in processing time for large PR updates.
the PR is here:
http://sources.redhat.com/cgi-bin/gnatsweb.pl?cmd=view&pr=219&database=g
nats
and the patch is here (untouched by meddlesome MTAs):
http://otisbean.com/file-pr_patch.txt
and here it is for your reading pleasure:
Index: file-pr.c
===================================================================
RCS file: /cvs/gnats/gnats/gnats/file-pr.c,v
retrieving revision 1.40
diff -u -r1.40 file-pr.c
--- file-pr.c 2001/07/15 17:32:31 1.40
+++ file-pr.c 2001/07/23 07:03:06
@@ -729,6 +729,7 @@
{
char line [1024]; /* This size doesn't matter here */
char *buf;
+ size_t buf_size, len;
const char *from, *to, *subject, *date, *cc;
PR *pr;
@@ -761,17 +762,24 @@
\n",
from, to, cc, subject, date);
- while ((fgets (line, sizeof (buf) - 1, infile)) != NULL)
+ /* copy the message from infile, indenting each line by one character
*/
+ len = buf_size = strlen (buf);
+ while ((fgets (line, sizeof (line) - 1, infile)) != NULL)
{
- size_t len = strlen (buf);
size_t lineLen = strlen (line);
- buf = xrealloc (buf, len + lineLen + 2);
+ if (buf_size < len + lineLen + 2)
+ {
+ buf_size += sizeof(line) << 2;
+ buf = xrealloc (buf, buf_size);
+ }
+
if (buf[len - 1] == '\n')
{
buf[len++] = ' ';
}
memcpy (buf + len, line, lineLen + 1);
+ len += lineLen;
}
fclose (infile);
--------------
two fixes:
1) read sizeof(line)-1 chars from the message (<=1024 chars), instead of
sizeof(buf)-1 chars (2 chars).
2) add code to keep track of buf's length in a variable, instead of
doing a strlen(buf) for every loop iteration.
the first fix gives a 20-30x improvement (from hours to minutes) for a
1.4 MB testcase. the second fix gave a 50x improvement (five minutes to
>10 seconds) for the same testcase.
and some cleanup:
xrealloc()ing buf in 4K chunks only when needed, instead of every loop
iteration. doesn't seem to change speed, but it feels better...
--
Dirk Bergstrom dirk@juniper.net
_____________________________________________
Juniper Networks Inc., Computer Geek
Tel: 707.433.0564 Fax: 707.433.0769