r59718 MediaWiki - Code Review archive

Repository:MediaWiki
Revision:r59717‎ | r59718 | r59719 >
Date:01:55, 4 December 2009
Author:tstarling
Status:deferred
Tags:
Comment:
Fix bug in BacklinkCache: the lack of an ORDER BY clause in getLinks(), combined with the lack of sensible indexes on the categorylinks table, was causing partition() to return starts and ends scaterred randomly across the result set. For large jobs, many partitions end up being large, causing HTMLCacheUpdate::doPartialUpdate() to repartition, thus requeueing jobs in an infinite recursive loop.

The BacklinkCache bug was there since r47317, but was relatively harmless until r54841 introduced the infinite loop issue.
Modified paths:
  • /trunk/phase3/RELEASE-NOTES (modified) (history)
  • /trunk/phase3/includes/BacklinkCache.php (modified) (history)

Diff [purge]

Index: trunk/phase3/includes/BacklinkCache.php
@@ -53,13 +53,13 @@
5454 public function getLinks( $table, $startId = false, $endId = false ) {
5555 wfProfileIn( __METHOD__ );
5656
 57+ $fromField = $this->getPrefix( $table ) . '_from';
5758 if ( $startId || $endId ) {
5859 // Partial range, not cached
5960 wfDebug( __METHOD__.": from DB (uncacheable range)\n" );
6061 $conds = $this->getConditions( $table );
6162 // Use the from field in the condition rather than the joined page_id,
6263 // because databases are stupid and don't necessarily propagate indexes.
63 - $fromField = $this->getPrefix( $table ) . '_from';
6464 if ( $startId ) {
6565 $conds[] = "$fromField >= " . intval( $startId );
6666 }
@@ -71,7 +71,10 @@
7272 array( 'page_namespace', 'page_title', 'page_id'),
7373 $conds,
7474 __METHOD__,
75 - array('STRAIGHT_JOIN') );
 75+ array(
 76+ 'STRAIGHT_JOIN',
 77+ 'ORDER BY' => $fromField
 78+ ) );
7679 $ta = TitleArray::newFromResult( $res );
7780 wfProfileOut( __METHOD__ );
7881 return $ta;
@@ -84,7 +87,10 @@
8588 array( 'page_namespace', 'page_title', 'page_id' ),
8689 $this->getConditions( $table ),
8790 __METHOD__,
88 - array('STRAIGHT_JOIN') );
 91+ array(
 92+ 'STRAIGHT_JOIN',
 93+ 'ORDER BY' => $fromField,
 94+ ) );
8995 $this->fullResultCache[$table] = $res;
9096 }
9197 $ta = TitleArray::newFromResult( $this->fullResultCache[$table] );
@@ -225,6 +231,12 @@
226232 $row = $res->fetchObject();
227233 $end = $row->page_id - 1;
228234 }
 235+
 236+ # Sanity check order
 237+ if ( $start && $end && $start > $end ) {
 238+ throw new MWException( __METHOD__.': Internal error: query result out of order' );
 239+ }
 240+
229241 $batches[] = array( $start, $end );
230242 }
231243 return array( 'numRows' => $numRows, 'batches' => $batches );
Index: trunk/phase3/RELEASE-NOTES
@@ -661,6 +661,8 @@
662662 * (bug 18762) both redirects and links get fixed one after another if
663663 redirects-only switch is not present
664664 * (bug 20159) thumbnails rerendered if older that $wgThumbnailEpoch
 665+* Fixed a bug which in some situations causes the job queue to grow forever,
 666+ due to an infinite loop of job requeues.
665667
666668 == API changes in 1.16 ==
667669

Follow-up revisions

RevisionCommit summaryAuthorDate
r59719Temporary hack to reduce the job queue size. Should be replaced by r59718 onc...tstarling01:58, 4 December 2009
r60962Reverted r59719 and merged r59718 instead: proper fix for job queue infinite ...tstarling05:45, 12 January 2010

Past revisions this follows-up on

RevisionCommit summaryAuthorDate
r47317* Mostly reverted r41081 and related. Although the motivation (to save a quer...tstarling14:26, 16 February 2009
r54841In response to a report from Domas that we are seeing HTMLCacheUpdate::invali...tstarling05:00, 12 August 2009

Status & tagging log