r94592 MediaWiki - Code Review archive

Repository:MediaWiki
Revision:r94591‎ | r94592 | r94593 >
Date:23:40, 15 August 2011
Author:raindrift
Status:ok (Comments)
Tags:todo 
Comment:
Removed complex replag handling, now just query master when record isn't present on slave.
followup to r92009
Modified paths:
  • /trunk/phase3/includes/upload/UploadStash.php (modified) (history)

Diff [purge]

Index: trunk/phase3/includes/upload/UploadStash.php
@@ -22,10 +22,6 @@
2323 // Format of the key for files -- has to be suitable as a filename itself (e.g. ab12cd34ef.jpg)
2424 const KEY_FORMAT_REGEX = '/^[\w-\.]+\.\w*$/';
2525
26 - // When a given stashed file can't be loaded, wait for the slaves to catch up. If they're more than MAX_LAG
27 - // behind, throw an exception instead. (at what point is broken better than slow?)
28 - const MAX_LAG = 30;
29 -
3026 /**
3127 * repository that this uses to store temp files
3228 * public because we sometimes need to get a LocalFile within the same repo.
@@ -98,23 +94,9 @@
9995 $dbr = $this->repo->getSlaveDb();
10096
10197 if ( !isset( $this->fileMetadata[$key] ) ) {
102 - // try this first. if it fails to find the row, check for lag, wait, try again. if its still missing, throw an exception.
103 - // this more complex solution keeps things moving for page loads with many requests
104 - // (ie. validating image ownership) when replag is high
10598 if ( !$this->fetchFileMetadata( $key ) ) {
106 - $lag = $dbr->getLag();
107 - if ( $lag > 0 && $lag <= self::MAX_LAG ) {
108 - // if there's not too much replication lag, just wait for the slave to catch up to our last insert.
109 - sleep( ceil( $lag ) );
110 - } elseif ( $lag > self::MAX_LAG ) {
111 - // that's a lot of lag to introduce into the middle of the UI.
112 - throw new UploadStashMaxLagExceededException(
113 - 'Couldn\'t load stashed file metadata, and replication lag is above threshold: (MAX_LAG=' . self::MAX_LAG . ')'
114 - );
115 - }
116 -
117 - // now that the waiting has happened, try again
118 - $this->fetchFileMetadata( $key );
 99+ // If nothing was received, it's likely due to replication lag. Check the master to see if the record is there.
 100+ $this->fetchFileMetadata( $key, true );
119101 }
120102
121103 if ( !isset( $this->fileMetadata[$key] ) ) {
@@ -470,9 +452,16 @@
471453 * @param $key String: key
472454 * @return boolean
473455 */
474 - protected function fetchFileMetadata( $key ) {
 456+ protected function fetchFileMetadata( $key, $readFromMaster = false ) {
475457 // populate $fileMetadata[$key]
476 - $dbr = $this->repo->getSlaveDb();
 458+ $dbr = null;
 459+ if( $readFromMaster ) {
 460+ // sometimes reading from the master is necessary, if there's replication lag.
 461+ $dbr = $this->repo->getMasterDb();
 462+ } else {
 463+ $dbr = $this->repo->getSlaveDb();
 464+ }
 465+
477466 $row = $dbr->selectRow(
478467 'uploadstash',
479468 '*',
@@ -685,5 +674,4 @@
686675 class UploadStashZeroLengthFileException extends MWException {};
687676 class UploadStashNotLoggedInException extends MWException {};
688677 class UploadStashWrongOwnerException extends MWException {};
689 -class UploadStashMaxLagExceededException extends MWException {};
690678 class UploadStashNoSuchKeyException extends MWException {};

Follow-up revisions

RevisionCommit summaryAuthorDate
r94670Cleaning up little things, updates to code clarity, documentation fixes per C...raindrift17:57, 16 August 2011

Past revisions this follows-up on

RevisionCommit summaryAuthorDate
r92009Refactored UploadStash and related classes to use the database for file metad...raindrift21:11, 12 July 2011

Comments

#Comment by Catrope (talk | contribs)   11:29, 16 August 2011
+	protected function fetchFileMetadata( $key, $readFromMaster = false ) {

Instead of using a meaningless boolean parameter (there are lots of legacy ones in MW, but we've turned against this style lately), please do something like $readFromDBType = DB_SLAVE which the caller then overrides to DB_MASTER. A call to fetchfileMetadata( $key, DB_MASTER ) is more meaningful than a call to fetchFileMetadata( $key, true ) .

(In normal cases, that means you can feed $readFromDBType directly into wfGetDB() as well, but your case is different because of the filerepo stuff.)

Otherwise OK, marking as such.

#Comment by NeilK (talk | contribs)   18:28, 17 August 2011

I still find this to be a strange practice. If lag is high, doesn't that indicate the master is probably overloaded? Shouldn't this be exactly the moment when we shouldn't be bothering the master?

Status & tagging log