r104049 MediaWiki - Code Review archive

Repository:MediaWiki
Revision:r104048‎ | r104049 | r104050 >
Date:16:37, 23 November 2011
Author:catrope
Status:ok (Comments)
Tags:
Comment:
(bug 32598) Fix OOM errors in GlobalUsage caused by calling findFiles() on an array of several thousand files, which caused several thousand File objects to be in memory simultaneously. Unrolled the findFiles() call to calling findFile() in a loop and destroying the File object after each iteration.
Modified paths:
  • /trunk/extensions/GlobalUsage/GlobalUsageHooks.php (modified) (history)

Diff [purge]

Index: trunk/extensions/GlobalUsage/GlobalUsageHooks.php
@@ -17,7 +17,19 @@
1818
1919 // Create a list of locally existing images
2020 $images = array_keys( $linksUpdater->getImages() );
21 - $localFiles = array_keys( RepoGroup::singleton()->getLocalRepo()->findFiles( $images ) );
 21+
 22+ //$localFiles = array_keys( RepoGroup::singleton()->getLocalRepo()->findFiles( $images ) );
 23+ // Unrolling findFiles() here because pages with thousands of images trigger an OOM
 24+ // error while building an array with thousands of File objects (bug 32598)
 25+ $localFiles = array();
 26+ $repo = RepoGroup::singleton()->getLocalRepo();
 27+ foreach ( $images as $image ) {
 28+ $file = $repo->findFile( $image );
 29+ if ( $file ) {
 30+ $localFiles[] = $file->getTitle()->getDBkey();
 31+ }
 32+ }
 33+
2234 $missingFiles = array_diff( $images, $localFiles );
2335
2436 global $wgUseDumbLinkUpdate;

Sign-offs

UserFlagDate
Bryaninspected17:15, 23 November 2011

Follow-up revisions

RevisionCommit summaryAuthorDate
r1040501.18wmf1: MFT r104049, which had already been tested on the live site prior t...catrope16:49, 23 November 2011
r104178REL1_18 MFT r100516, r103675, r104049, r104162reedy16:20, 24 November 2011

Comments

#Comment by G.Hagedorn (talk | contribs)   21:43, 23 November 2011

Deployed to 1.18wmf1, please consider tagging for 1.18. We have outstanding bugs for GlobalUsage not working on 1.18, not sure this fixes it, but would like to test.

Status & tagging log