r84397 MediaWiki - Code Review archive

Repository:MediaWiki
Revision:r84396‎ | r84397 | r84398 >
Date:16:52, 20 March 2011
Author:catrope
Status:ok (Comments)
Tags:
Comment:
(bug 27336) Add $wgJobTypesExcludedFromDefaultQueue for job types that aren't supposed to be run by 'normal' job runners. Modified patch by Michael Dale
Modified paths:
  • /trunk/phase3/includes/DefaultSettings.php (modified) (history)
  • /trunk/phase3/includes/job/JobQueue.php (modified) (history)

Diff [purge]

Index: trunk/phase3/includes/DefaultSettings.php
@@ -4652,6 +4652,18 @@
46534653 $wgExcludeFromThumbnailPurge = array();
46544654
46554655 /**
 4656+
 4657+ * Jobs that must be explicitly requested, i.e. aren't run by job runners unless special flags are set.
 4658+ *
 4659+ * These can be:
 4660+ * - Very long-running jobs.
 4661+ * - Jobs that you would never want to run as part of a page rendering request.
 4662+ * - Jobs that you want to run on specialized machines ( like transcoding, or a particular
 4663+ * machine on your cluster has 'outside' web access you could restrict uploadFromUrl )
 4664+ */
 4665+$wgJobTypesExcludedFromDefaultQueue = array();
 4666+
 4667+/**
46564668 * Additional functions to be performed with updateSpecialPages.
46574669 * Expensive Querypages are already updated.
46584670 */
Index: trunk/phase3/includes/job/JobQueue.php
@@ -95,6 +95,7 @@
9696 * @return Job or false if there's no jobs
9797 */
9898 static function pop( $offset = 0 ) {
 99+ global $wgJobTypesExcludedFromDefaultQueue;
99100 wfProfileIn( __METHOD__ );
100101
101102 $dbr = wfGetDB( DB_SLAVE );
@@ -105,16 +106,23 @@
106107 NB: If random fetch previously was used, offset
107108 will always be ahead of few entries
108109 */
 110+ $conditions = array();
 111+ if ( count( $wgJobTypesExcludedFromDefaultQueue ) != 0 ) {
 112+ foreach ( $wgJobTypesExcludedFromDefaultQueue as $cmdType ) {
 113+ $conditions[] = "job_cmd != " . $dbr->addQuotes( $cmdType );
 114+ }
 115+ }
 116+ $offset = intval( $offset );
 117+ $row = $dbr->selectRow( 'job', '*', array_merge( $conditions, array( "job_id >= $offset" ) ) , __METHOD__,
 118+ array( 'ORDER BY' => 'job_id', 'LIMIT' => 1 )
 119+ );
109120
110 - $row = $dbr->selectRow( 'job', '*', "job_id >= ${offset}", __METHOD__,
111 - array( 'ORDER BY' => 'job_id', 'LIMIT' => 1 ) );
112 -
113121 // Refetching without offset is needed as some of job IDs could have had delayed commits
114122 // and have lower IDs than jobs already executed, blame concurrency :)
115123 //
116124 if ( $row === false ) {
117125 if ( $offset != 0 ) {
118 - $row = $dbr->selectRow( 'job', '*', '', __METHOD__,
 126+ $row = $dbr->selectRow( 'job', '*', $conditions, __METHOD__,
119127 array( 'ORDER BY' => 'job_id', 'LIMIT' => 1 ) );
120128 }
121129

Sign-offs

UserFlagDate
😂inspected14:42, 19 April 2011

Follow-up revisions

RevisionCommit summaryAuthorDate
r84443TimedMediaHandler: $wgJobExplitRequestTypes --> $wgJobTypesExcludedFromDefaul...catrope11:12, 21 March 2011
r91219* Follow-up r84397: make sure that mysql uses the job_id index even with the ...aaron20:32, 30 June 2011

Past revisions this follows-up on

RevisionCommit summaryAuthorDate
r82001updated ffmpeg mapping for firefogg api...dale23:11, 11 February 2011

Comments

#Comment by Aaron Schulz (talk | contribs)   22:44, 17 June 2011

Make sure this result in a filesort by trying to use the (job_cmd, job_namespace, job_title, job_params) index.

#Comment by Aaron Schulz (talk | contribs)   20:20, 30 June 2011

That should be "*doesn't* result" of course ;)

Status & tagging log