r90643 MediaWiki - Code Review archive

Repository:MediaWiki
Revision:< r90642‎ | r90643 | r90644 >
Date:03:14, 23 June 2011
Author:tstarling
Status:resolved (Comments)
Tags:
Comment:
General database-related code cleanup:
* Merged the 4 simulated implementations of Database*::replace(). I took diffs, they were nearly identical. I made one based on the IBM DB2 version, since it used insert() which looked like a nice touch.
* Provided the non-simulated implementation of Database*::replace() via a protected member function, and made DatabaseMysql::replace() and DatabaseSqlite::replace() into a wrapper for it.
* Moved the MySQL-specific functionality from masterPosWait(), getSlavePos() and getMasterPos() from DatabaseBase to DatabaseMysql.
* Renamed getStatus() to getMysqlStatus() and moved it to DatabaseMysql. Removed "unimplemented" errors from two other subclasses. Really there's no way another DBMS could or should implement this function.
* Split the LoadMonitor class into a no-op class and a MySQL-specific class, to avoid fatal errors due to the getMysqlStatus() call if other DBMSs tried to use LoadBalancer with multiple servers. Of course there are lots of other reasons it won't work, I'm just fixing the architecture issue here.

And while I have a replicated test setup handy:
* On MySQL 4.1.9 and later, use SHOW SLAVE STATUS to get the lag instead of SHOW PROCESSLIST. This has the advantage of reading zero when there are no events for a while.
Modified paths:
  • /trunk/phase3/RELEASE-NOTES-1.19 (modified) (history)
  • /trunk/phase3/includes/AutoLoader.php (modified) (history)
  • /trunk/phase3/includes/db/Database.php (modified) (history)
  • /trunk/phase3/includes/db/DatabaseIbm_db2.php (modified) (history)
  • /trunk/phase3/includes/db/DatabaseMssql.php (modified) (history)
  • /trunk/phase3/includes/db/DatabaseMysql.php (modified) (history)
  • /trunk/phase3/includes/db/DatabaseOracle.php (modified) (history)
  • /trunk/phase3/includes/db/DatabasePostgres.php (modified) (history)
  • /trunk/phase3/includes/db/DatabaseSqlite.php (modified) (history)
  • /trunk/phase3/includes/db/DatabaseUtility.php (modified) (history)
  • /trunk/phase3/includes/db/LoadBalancer.php (modified) (history)
  • /trunk/phase3/includes/db/LoadMonitor.php (modified) (history)

Diff [purge]

Index: trunk/phase3/RELEASE-NOTES-1.19
——@@ -54,6 +54,8 @@
5555 * (bug 29441) Expose CapitalLinks config in JS to allow modules to properly
5656 handle titles on case-sensitive wikis.
5757 * (bug 29397) Implement mw.Title module in core.
 58+* In MySQL 4.1.9+ with replication enabled, get the slave lag from SHOW SLAVE
 59+ STATUS instead of SHOW PROCESSLIST.
5860
5961 === Bug fixes in 1.19 ===
6062 * (bug 28868) Show total pages in the subtitle of an image on the
Index: trunk/phase3/includes/db/DatabaseSqlite.php
——@@ -532,12 +532,12 @@
533533 if ( isset( $rows[0] ) && is_array( $rows[0] ) ) {
534534 $ret = true;
535535 foreach ( $rows as $v ) {
536 - if ( !parent::replace( $table, $uniqueIndexes, $v, "$fname/multi-row" ) ) {
 536+ if ( !$this->nativeReplace( $table, $v, "$fname/multi-row" ) ) {
537537 $ret = false;
538538 }
539539 }
540540 } else {
541 - $ret = parent::replace( $table, $uniqueIndexes, $rows, "$fname/single-row" );
 541+ $ret = $this->nativeReplace( $table, $rows, "$fname/single-row" );
542542 }
543543
544544 return $ret;
Index: trunk/phase3/includes/db/LoadBalancer.php
——@@ -50,9 +50,18 @@
5151 $this->mLaggedSlaveMode = false;
5252 $this->mErrorConnection = false;
5353 $this->mAllowLagged = false;
54 - $this->mLoadMonitorClass = isset( $params['loadMonitor'] )
55 - ? $params['loadMonitor'] : 'LoadMonitor_MySQL';
5654
 55+ if ( isset( $params['loadMonitor'] ) ) {
 56+ $this->mLoadMonitorClass = $params['loadMonitor'];
 57+ } else {
 58+ $master = reset( $params['servers'] );
 59+ if ( isset( $master['type'] ) && $master['type'] === 'mysql' ) {
 60+ $this->mLoadMonitorClass = 'LoadMonitor_MySQL';
 61+ } else {
 62+ $this->mLoadMonitorClass = 'LoadMonitor_Null';
 63+ }
 64+ }
 65+
5766 foreach( $params['servers'] as $i => $server ) {
5867 $this->mLoads[$i] = $server['load'];
5968 if ( isset( $server['groupLoads'] ) ) {
Index: trunk/phase3/includes/db/LoadMonitor.php
——@@ -56,7 +56,22 @@
5757 function getLagTimes( $serverIndexes, $wiki );
5858 }
5959
 60+class LoadMonitor_Null implements LoadMonitor {
 61+ function __construct( $parent ) {
 62+ }
6063
 64+ function scaleLoads( &$loads, $group = false, $wiki = false ) {
 65+ }
 66+
 67+ function postConnectionBackoff( $conn, $threshold ) {
 68+ }
 69+
 70+ function getLagTimes( $serverIndexes, $wiki ) {
 71+ return array_fill_keys( $serverIndexes, 0 );
 72+ }
 73+}
 74+
 75+
6176 /**
6277 * Basic MySQL load monitor with no external dependencies
6378 * Uses memcached to cache the replication lag for a short time
——@@ -150,7 +165,7 @@
151166 if ( !$threshold ) {
152167 return 0;
153168 }
154 - $status = $conn->getStatus("Thread%");
 169+ $status = $conn->getMysqlStatus("Thread%");
155170 if ( $status['Threads_running'] > $threshold ) {
156171 return $status['Threads_connected'];
157172 } else {
Index: trunk/phase3/includes/db/DatabaseUtility.php
——@@ -259,3 +259,10 @@
260260 return $this->str;
261261 }
262262 }
 263+
 264+/**
 265+ * An object representing a master or slave position in a replicated setup.
 266+ */
 267+interface DBMasterPos {
 268+}
 269+
Index: trunk/phase3/includes/db/DatabaseMysql.php
——@@ -241,6 +241,10 @@
242242
243243 function affectedRows() { return mysql_affected_rows( $this->mConn ); }
244244
 245+ function replace( $table, $uniqueIndexes, $rows, $fname = 'DatabaseMysql::replace' ) {
 246+ return $this->nativeReplace( $table, $rows, $fname );
 247+ }
 248+
245249 /**
246250 * Estimate rows in dataset
247251 * Returns estimated count, based on EXPLAIN output
——@@ -348,7 +352,11 @@
349353
350354 /**
351355 * Returns slave lag.
352 - * At the moment, this will only work if the DB user has the PROCESS privilege
 356+ *
 357+ * On MySQL 4.1.9 and later, this will do a SHOW SLAVE STATUS. On earlier
 358+ * versions of MySQL, it uses SHOW PROCESSLIST, which requires the PROCESS
 359+ * privilege.
 360+ *
353361 * @result int
354362 */
355363 function getLag() {
——@@ -356,6 +364,31 @@
357365 wfDebug( "getLag: fake slave lagged {$this->mFakeSlaveLag} seconds\n" );
358366 return $this->mFakeSlaveLag;
359367 }
 368+
 369+ if ( version_compare( $this->getServerVersion(), '4.1.9', '>=' ) ) {
 370+ return $this->getLagFromSlaveStatus();
 371+ } else {
 372+ return $this->getLagFromProcesslist();
 373+ }
 374+ }
 375+
 376+ function getLagFromSlaveStatus() {
 377+ $res = $this->query( 'SHOW SLAVE STATUS', __METHOD__ );
 378+ if ( !$res ) {
 379+ return false;
 380+ }
 381+ $row = $res->fetchObject();
 382+ if ( !$row ) {
 383+ return false;
 384+ }
 385+ if ( strval( $row->Seconds_Behind_Master ) === '' ) {
 386+ return false;
 387+ } else {
 388+ return intval( $row->Seconds_Behind_Master );
 389+ }
 390+ }
 391+
 392+ function getLagFromProcesslist() {
360393 $res = $this->query( 'SHOW PROCESSLIST', __METHOD__ );
361394 if( !$res ) {
362395 return false;
——@@ -388,7 +421,84 @@
389422 }
390423 return false;
391424 }
 425+
 426+ /**
 427+ * Wait for the slave to catch up to a given master position.
 428+ *
 429+ * @param $pos DBMasterPos object
 430+ * @param $timeout Integer: the maximum number of seconds to wait for synchronisation
 431+ */
 432+ function masterPosWait( DBMasterPos $pos, $timeout ) {
 433+ $fname = 'DatabaseBase::masterPosWait';
 434+ wfProfileIn( $fname );
392435
 436+ # Commit any open transactions
 437+ if ( $this->mTrxLevel ) {
 438+ $this->commit();
 439+ }
 440+
 441+ if ( !is_null( $this->mFakeSlaveLag ) ) {
 442+ $status = parent::masterPosWait( $pos, $timeout );
 443+ wfProfileOut( $fname );
 444+ return $status;
 445+ }
 446+
 447+ # Call doQuery() directly, to avoid opening a transaction if DBO_TRX is set
 448+ $encFile = $this->addQuotes( $pos->file );
 449+ $encPos = intval( $pos->pos );
 450+ $sql = "SELECT MASTER_POS_WAIT($encFile, $encPos, $timeout)";
 451+ $res = $this->doQuery( $sql );
 452+
 453+ if ( $res && $row = $this->fetchRow( $res ) ) {
 454+ wfProfileOut( $fname );
 455+ return $row[0];
 456+ } else {
 457+ wfProfileOut( $fname );
 458+ return false;
 459+ }
 460+ }
 461+
 462+ /**
 463+ * Get the position of the master from SHOW SLAVE STATUS
 464+ *
 465+ * @return MySQLMasterPos|false
 466+ */
 467+ function getSlavePos() {
 468+ if ( !is_null( $this->mFakeSlaveLag ) ) {
 469+ return parent::getSlavePos();
 470+ }
 471+
 472+ $res = $this->query( 'SHOW SLAVE STATUS', 'DatabaseBase::getSlavePos' );
 473+ $row = $this->fetchObject( $res );
 474+
 475+ if ( $row ) {
 476+ $pos = isset( $row->Exec_master_log_pos ) ? $row->Exec_master_log_pos : $row->Exec_Master_Log_Pos;
 477+ return new MySQLMasterPos( $row->Relay_Master_Log_File, $pos );
 478+ } else {
 479+ return false;
 480+ }
 481+ }
 482+
 483+ /**
 484+ * Get the position of the master from SHOW MASTER STATUS
 485+ *
 486+ * @return MySQLMasterPos|false
 487+ */
 488+ function getMasterPos() {
 489+ if ( $this->mFakeMaster ) {
 490+ return parent::getMasterPos();
 491+ }
 492+
 493+ $res = $this->query( 'SHOW MASTER STATUS', 'DatabaseBase::getMasterPos' );
 494+ $row = $this->fetchObject( $res );
 495+
 496+ if ( $row ) {
 497+ return new MySQLMasterPos( $row->File, $row->Position );
 498+ } else {
 499+ return false;
 500+ }
 501+ }
 502+
393503 function getServerVersion() {
394504 return mysql_get_server_info( $this->mConn );
395505 }
——@@ -587,6 +697,23 @@
588698 $vars['wgDBTableOptions'] = $GLOBALS['wgDBTableOptions'];
589699 return $vars;
590700 }
 701+
 702+ /**
 703+ * Get status information from SHOW STATUS in an associative array
 704+ *
 705+ * @return array
 706+ */
 707+ function getMysqlStatus( $which = "%" ) {
 708+ $res = $this->query( "SHOW STATUS LIKE '{$which}'" );
 709+ $status = array();
 710+
 711+ foreach ( $res as $row ) {
 712+ $status[$row->Variable_name] = $row->Value;
 713+ }
 714+
 715+ return $status;
 716+ }
 717+
591718 }
592719
593720 /**
——@@ -644,7 +771,7 @@
645772 }
646773 }
647774
648 -class MySQLMasterPos {
 775+class MySQLMasterPos implements DBMasterPos {
649776 var $file, $pos;
650777
651778 function __construct( $file, $pos ) {
Index: trunk/phase3/includes/db/DatabaseOracle.php
——@@ -707,66 +707,6 @@
708708 return ( isset( $this->sequenceData[$table] ) ) ? $this->sequenceData[$table] : false;
709709 }
710710
711 - /**
712 - * REPLACE query wrapper
713 - * Oracle simulates this with a DELETE followed by INSERT
714 - * $row is the row to insert, an associative array
715 - * $uniqueIndexes is an array of indexes. Each element may be either a
716 - * field name or an array of field names
717 - *
718 - * It may be more efficient to leave off unique indexes which are unlikely to collide.
719 - * However if you do this, you run the risk of encountering errors which wouldn't have
720 - * occurred in MySQL.
721 - *
722 - * @param $table String: table name
723 - * @param $uniqueIndexes Array: array of indexes. Each element may be
724 - * either a field name or an array of field names
725 - * @param $rows Array: rows to insert to $table
726 - * @param $fname String: function name, you can use __METHOD__ here
727 - */
728 - function replace( $table, $uniqueIndexes, $rows, $fname = 'DatabaseOracle::replace' ) {
729 - $table = $this->tableName( $table );
730 -
731 - if ( count( $rows ) == 0 ) {
732 - return;
733 - }
734 -
735 - # Single row case
736 - if ( !is_array( reset( $rows ) ) ) {
737 - $rows = array( $rows );
738 - }
739 -
740 - $sequenceData = $this->getSequenceData( $table );
741 -
742 - foreach ( $rows as $row ) {
743 - # Delete rows which collide
744 - if ( $uniqueIndexes ) {
745 - $deleteConds = array();
746 - foreach ( $uniqueIndexes as $key=>$index ) {
747 - if ( is_array( $index ) ) {
748 - $deleteConds2 = array();
749 - foreach ( $index as $col ) {
750 - $deleteConds2[$col] = $row[$col];
751 - }
752 - $deleteConds[$key] = $this->makeList( $deleteConds2, LIST_AND );
753 - } else {
754 - $deleteConds[$index] = $row[$index];
755 - }
756 - }
757 - $deleteConds = array( $this->makeList( $deleteConds, LIST_OR ) );
758 - $this->delete( $table, $deleteConds, $fname );
759 - }
760 -
761 -
762 - if ( $sequenceData !== false && !isset( $row[$sequenceData['column']] ) ) {
763 - $row[$sequenceData['column']] = $this->nextSequenceValue( $sequenceData['sequence'] );
764 - }
765 -
766 - # Now insert the row
767 - $this->insert( $table, $row, $fname );
768 - }
769 - }
770 -
771711 # Returns the size of a text field, or -1 for "unlimited"
772712 function textFieldSize( $table, $field ) {
773713 $fieldInfoData = $this->fieldInfo( $table, $field );
Index: trunk/phase3/includes/db/DatabasePostgres.php
——@@ -647,66 +647,6 @@
648648 return $currval;
649649 }
650650
651 - /**
652 - * REPLACE query wrapper
653 - * Postgres simulates this with a DELETE followed by INSERT
654 - * $row is the row to insert, an associative array
655 - * $uniqueIndexes is an array of indexes. Each element may be either a
656 - * field name or an array of field names
657 - *
658 - * It may be more efficient to leave off unique indexes which are unlikely to collide.
659 - * However if you do this, you run the risk of encountering errors which wouldn't have
660 - * occurred in MySQL
661 - */
662 - function replace( $table, $uniqueIndexes, $rows, $fname = 'DatabasePostgres::replace' ) {
663 - $table = $this->tableName( $table );
664 -
665 - if ( count( $rows ) == 0 ) {
666 - return;
667 - }
668 -
669 - # Single row case
670 - if ( !is_array( reset( $rows ) ) ) {
671 - $rows = array( $rows );
672 - }
673 -
674 - foreach( $rows as $row ) {
675 - # Delete rows which collide
676 - if ( $uniqueIndexes ) {
677 - $sql = "DELETE FROM $table WHERE ";
678 - $first = true;
679 - foreach ( $uniqueIndexes as $index ) {
680 - if ( $first ) {
681 - $first = false;
682 - $sql .= '(';
683 - } else {
684 - $sql .= ') OR (';
685 - }
686 - if ( is_array( $index ) ) {
687 - $first2 = true;
688 - foreach ( $index as $col ) {
689 - if ( $first2 ) {
690 - $first2 = false;
691 - } else {
692 - $sql .= ' AND ';
693 - }
694 - $sql .= $col.'=' . $this->addQuotes( $row[$col] );
695 - }
696 - } else {
697 - $sql .= $index.'=' . $this->addQuotes( $row[$index] );
698 - }
699 - }
700 - $sql .= ')';
701 - $this->query( $sql, $fname );
702 - }
703 -
704 - # Now insert the row
705 - $sql = "INSERT INTO $table (" . $this->makeList( array_keys( $row ), LIST_NAMES ) .') VALUES (' .
706 - $this->makeList( $row, LIST_COMMA ) . ')';
707 - $this->query( $sql, $fname );
708 - }
709 - }
710 -
711651 # Returns the size of a text field, or -1 for "unlimited"
712652 function textFieldSize( $table, $field ) {
713653 $table = $this->tableName( $table );
Index: trunk/phase3/includes/db/DatabaseIbm_db2.php
——@@ -555,45 +555,6 @@
556556 }
557557
558558 /**
559 - * Create tables, stored procedures, and so on
560 - */
561 - public function setup_database() {
562 - try {
563 - // TODO: switch to root login if available
564 -
565 - // Switch into the correct namespace
566 - $this->applySchema();
567 - $this->begin();
568 -
569 - $res = $this->sourceFile( "../maintenance/ibm_db2/tables.sql" );
570 - if ( $res !== true ) {
571 - print ' <b>FAILED</b>: ' . htmlspecialchars( $res ) . '</li>';
572 - } else {
573 - print ' done</li>';
574 - }
575 - $res = $this->sourceFile( "../maintenance/ibm_db2/foreignkeys.sql" );
576 - if ( $res !== true ) {
577 - print ' <b>FAILED</b>: ' . htmlspecialchars( $res ) . '</li>';
578 - } else {
579 - print '<li>Foreign keys done</li>';
580 - }
581 -
582 - // TODO: populate interwiki links
583 -
584 - if ( $this->lastError() ) {
585 - $this->installPrint(
586 - 'Errors encountered during table creation -- rolled back' );
587 - $this->installPrint( 'Please install again' );
588 - $this->rollback();
589 - } else {
590 - $this->commit();
591 - }
592 - } catch ( MWException $mwe ) {
593 - print "<br><pre>$mwe</pre><br>";
594 - }
595 - }
596 -
597 - /**
598559 * Escapes strings
599560 * Doesn't escape numbers
600561 *
——@@ -1068,63 +1029,6 @@
10691030 }
10701031
10711032 /**
1072 - * Simulates REPLACE with a DELETE followed by INSERT
1073 - * @param $table Object
1074 - * @param $uniqueIndexes Array consisting of indexes and arrays of indexes
1075 - * @param $rows Array: rows to insert
1076 - * @param $fname String: name of the function for profiling
1077 - * @return nothing
1078 - */
1079 - function replace( $table, $uniqueIndexes, $rows,
1080 - $fname = 'DatabaseIbm_db2::replace' )
1081 - {
1082 - $table = $this->tableName( $table );
1083 -
1084 - if ( count( $rows )==0 ) {
1085 - return;
1086 - }
1087 -
1088 - # Single row case
1089 - if ( !is_array( reset( $rows ) ) ) {
1090 - $rows = array( $rows );
1091 - }
1092 -
1093 - foreach( $rows as $row ) {
1094 - # Delete rows which collide
1095 - if ( $uniqueIndexes ) {
1096 - $sql = "DELETE FROM $table WHERE ";
1097 - $first = true;
1098 - foreach ( $uniqueIndexes as $index ) {
1099 - if ( $first ) {
1100 - $first = false;
1101 - $sql .= '( ';
1102 - } else {
1103 - $sql .= ' ) OR ( ';
1104 - }
1105 - if ( is_array( $index ) ) {
1106 - $first2 = true;
1107 - foreach ( $index as $col ) {
1108 - if ( $first2 ) {
1109 - $first2 = false;
1110 - } else {
1111 - $sql .= ' AND ';
1112 - }
1113 - $sql .= $col . '=' . $this->addQuotes( $row[$col] );
1114 - }
1115 - } else {
1116 - $sql .= $index . '=' . $this->addQuotes( $row[$index] );
1117 - }
1118 - }
1119 - $sql .= ' )';
1120 - $this->query( $sql, $fname );
1121 - }
1122 -
1123 - # Now insert the row
1124 - $this->insert($table, $row);
1125 - }
1126 - }
1127 -
1128 - /**
11291033 * Returns the number of rows in the result set
11301034 * Has to be called right after the corresponding select query
11311035 * @param $res Object result set
——@@ -1347,14 +1251,6 @@
13481252 ######################################
13491253 /**
13501254 * Not implemented
1351 - * @return string ''
1352 - */
1353 - public function getStatus( $which = '%' ) {
1354 - $this->installPrint( 'Not implemented for DB2: getStatus()' );
1355 - return '';
1356 - }
1357 - /**
1358 - * Not implemented
13591255 * @return string $sql
13601256 */
13611257 public function limitResultForUpdate( $sql, $num ) {
Index: trunk/phase3/includes/db/Database.php
——@@ -2215,22 +2215,86 @@
22162216 }
22172217
22182218 /**
2219 - * REPLACE query wrapper
2220 - * PostgreSQL simulates this with a DELETE followed by INSERT
2221 - * $row is the row to insert, an associative array
2222 - * $uniqueIndexes is an array of indexes. Each element may be either a
2223 - * field name or an array of field names
 2219+ * REPLACE query wrapper.
22242220 *
2225 - * It may be more efficient to leave off unique indexes which are unlikely to collide.
2226 - * However if you do this, you run the risk of encountering errors which wouldn't have
2227 - * occurred in MySQL
 2221+ * REPLACE is a very handy MySQL extension, which functions like an INSERT
 2222+ * except that when there is a duplicate key error, the old row is deleted
 2223+ * and the new row is inserted in its place.
22282224 *
 2225+ * We simulate this with standard SQL with a DELETE followed by INSERT. To
 2226+ * perform the delete, we need to know what the unique indexes are so that
 2227+ * we know how to find the conflicting rows.
 2228+ *
 2229+ * It may be more efficient to leave off unique indexes which are unlikely
 2230+ * to collide. However if you do this, you run the risk of encountering
 2231+ * errors which wouldn't have occurred in MySQL.
 2232+ *
 2233+ * @param $rows Can be either a single row to insert, or multiple rows,
 2234+ * in the same format as for DatabaseBase::insert()
 2235+ * @param $uniqueIndexes is an array of indexes. Each element may be either
 2236+ * a field name or an array of field names
 2237+ *
22292238 * @param $table String: The table to replace the row(s) in.
22302239 * @param $uniqueIndexes Array: An associative array of indexes
22312240 * @param $rows Array: Array of rows to replace
22322241 * @param $fname String: Calling function name (use __METHOD__) for logs/profiling
22332242 */
22342243 function replace( $table, $uniqueIndexes, $rows, $fname = 'DatabaseBase::replace' ) {
 2244+ $quotedTable = $this->tableName( $table );
 2245+
 2246+ if ( count( $rows ) == 0 ) {
 2247+ return;
 2248+ }
 2249+
 2250+ # Single row case
 2251+ if ( !is_array( reset( $rows ) ) ) {
 2252+ $rows = array( $rows );
 2253+ }
 2254+
 2255+ foreach( $rows as $row ) {
 2256+ # Delete rows which collide
 2257+ if ( $uniqueIndexes ) {
 2258+ $sql = "DELETE FROM $quotedTable WHERE ";
 2259+ $first = true;
 2260+ foreach ( $uniqueIndexes as $index ) {
 2261+ if ( $first ) {
 2262+ $first = false;
 2263+ $sql .= '( ';
 2264+ } else {
 2265+ $sql .= ' ) OR ( ';
 2266+ }
 2267+ if ( is_array( $index ) ) {
 2268+ $first2 = true;
 2269+ foreach ( $index as $col ) {
 2270+ if ( $first2 ) {
 2271+ $first2 = false;
 2272+ } else {
 2273+ $sql .= ' AND ';
 2274+ }
 2275+ $sql .= $col . '=' . $this->addQuotes( $row[$col] );
 2276+ }
 2277+ } else {
 2278+ $sql .= $index . '=' . $this->addQuotes( $row[$index] );
 2279+ }
 2280+ }
 2281+ $sql .= ' )';
 2282+ $this->query( $sql, $fname );
 2283+ }
 2284+
 2285+ # Now insert the row
 2286+ $this->insert( $table, $row );
 2287+ }
 2288+ }
 2289+
 2290+ /**
 2291+ * REPLACE query wrapper for MySQL and SQLite, which have a native REPLACE
 2292+ * statement.
 2293+ *
 2294+ * @param $table Table name
 2295+ * @param $rows Rows to insert
 2296+ * @param $fname Caller function name
 2297+ */
 2298+ protected function nativeReplace( $table, $rows, $fname ) {
22352299 $table = $this->tableName( $table );
22362300
22372301 # Single row case
——@@ -2584,20 +2648,20 @@
25852649 }
25862650
25872651 /**
2588 - * Do a SELECT MASTER_POS_WAIT()
 2652+ * Wait for the slave to catch up to a given master position.
25892653 *
2590 - * @param $pos MySQLMasterPos object
2591 - * @param $timeout Integer: the maximum number of seconds to wait for synchronisation
 2654+ * @param $pos DBMasterPos object
 2655+ * @param $timeout Integer: the maximum number of seconds to wait for
 2656+ * synchronisation
 2657+ *
 2658+ * @return An integer: zero if the slave was past that position already,
 2659+ * greater than zero if we waited for some period of time, less than
 2660+ * zero if we timed out.
25922661 */
2593 - function masterPosWait( MySQLMasterPos $pos, $timeout ) {
 2662+ function masterPosWait( DBMasterPos $pos, $timeout ) {
25942663 $fname = 'DatabaseBase::masterPosWait';
25952664 wfProfileIn( $fname );
25962665
2597 - # Commit any open transactions
2598 - if ( $this->mTrxLevel ) {
2599 - $this->commit();
2600 - }
2601 -
26022666 if ( !is_null( $this->mFakeSlaveLag ) ) {
26032667 $wait = intval( ( $pos->pos - microtime( true ) + $this->mFakeSlaveLag ) * 1e6 );
26042668
——@@ -2617,59 +2681,36 @@
26182682 }
26192683 }
26202684
2621 - # Call doQuery() directly, to avoid opening a transaction if DBO_TRX is set
2622 - $encFile = $this->addQuotes( $pos->file );
2623 - $encPos = intval( $pos->pos );
2624 - $sql = "SELECT MASTER_POS_WAIT($encFile, $encPos, $timeout)";
2625 - $res = $this->doQuery( $sql );
 2685+ wfProfileOut( $fname );
26262686
2627 - if ( $res && $row = $this->fetchRow( $res ) ) {
2628 - wfProfileOut( $fname );
2629 - return $row[0];
2630 - } else {
2631 - wfProfileOut( $fname );
2632 - return false;
2633 - }
 2687+ # Real waits are implemented in the subclass.
 2688+ return 0;
26342689 }
26352690
26362691 /**
2637 - * Get the position of the master from SHOW SLAVE STATUS
 2692+ * Get the replication position of this slave
26382693 *
2639 - * @return MySQLMasterPos|false
 2694+ * @return DBMasterPos, or false if this is not a slave.
26402695 */
26412696 function getSlavePos() {
26422697 if ( !is_null( $this->mFakeSlaveLag ) ) {
26432698 $pos = new MySQLMasterPos( 'fake', microtime( true ) - $this->mFakeSlaveLag );
26442699 wfDebug( __METHOD__ . ": fake slave pos = $pos\n" );
26452700 return $pos;
2646 - }
2647 -
2648 - $res = $this->query( 'SHOW SLAVE STATUS', 'DatabaseBase::getSlavePos' );
2649 - $row = $this->fetchObject( $res );
2650 -
2651 - if ( $row ) {
2652 - $pos = isset( $row->Exec_master_log_pos ) ? $row->Exec_master_log_pos : $row->Exec_Master_Log_Pos;
2653 - return new MySQLMasterPos( $row->Relay_Master_Log_File, $pos );
26542701 } else {
 2702+ # Stub
26552703 return false;
26562704 }
26572705 }
26582706
26592707 /**
2660 - * Get the position of the master from SHOW MASTER STATUS
 2708+ * Get the position of this master
26612709 *
2662 - * @return MySQLMasterPos|false
 2710+ * @return DBMasterPos, or false if this is not a master
26632711 */
26642712 function getMasterPos() {
26652713 if ( $this->mFakeMaster ) {
26662714 return new MySQLMasterPos( 'fake', microtime( true ) );
2667 - }
2668 -
2669 - $res = $this->query( 'SHOW MASTER STATUS', 'DatabaseBase::getMasterPos' );
2670 - $row = $this->fetchObject( $res );
2671 -
2672 - if ( $row ) {
2673 - return new MySQLMasterPos( $row->File, $row->Position );
26742715 } else {
26752716 return false;
26762717 }
——@@ -2818,22 +2859,6 @@
28192860 }
28202861
28212862 /**
2822 - * Get status information from SHOW STATUS in an associative array
2823 - *
2824 - * @return array
2825 - */
2826 - function getStatus( $which = "%" ) {
2827 - $res = $this->query( "SHOW STATUS LIKE '{$which}'" );
2828 - $status = array();
2829 -
2830 - foreach ( $res as $row ) {
2831 - $status[$row->Variable_name] = $row->Value;
2832 - }
2833 -
2834 - return $status;
2835 - }
2836 -
2837 - /**
28382863 * Return the maximum number of items allowed in a list, or 0 for unlimited.
28392864 *
28402865 * return int
Index: trunk/phase3/includes/db/DatabaseMssql.php
——@@ -539,65 +539,6 @@
540540 }
541541 }
542542
543 -
544 - # REPLACE query wrapper
545 - # MSSQL simulates this with a DELETE followed by INSERT
546 - # $row is the row to insert, an associative array
547 - # $uniqueIndexes is an array of indexes. Each element may be either a
548 - # field name or an array of field names
549 - #
550 - # It may be more efficient to leave off unique indexes which are unlikely to collide.
551 - # However if you do this, you run the risk of encountering errors which wouldn't have
552 - # occurred in MySQL
553 - function replace( $table, $uniqueIndexes, $rows, $fname = 'DatabaseMssql::replace' ) {
554 - $table = $this->tableName( $table );
555 -
556 - if ( count( $rows ) == 0 ) {
557 - return;
558 - }
559 -
560 - # Single row case
561 - if ( !is_array( reset( $rows ) ) ) {
562 - $rows = array( $rows );
563 - }
564 -
565 - foreach ( $rows as $row ) {
566 - # Delete rows which collide
567 - if ( $uniqueIndexes ) {
568 - $sql = "DELETE FROM $table WHERE ";
569 - $first = true;
570 - foreach ( $uniqueIndexes as $index ) {
571 - if ( $first ) {
572 - $first = false;
573 - $sql .= "(";
574 - } else {
575 - $sql .= ') OR (';
576 - }
577 - if ( is_array( $index ) ) {
578 - $first2 = true;
579 - foreach ( $index as $col ) {
580 - if ( $first2 ) {
581 - $first2 = false;
582 - } else {
583 - $sql .= ' AND ';
584 - }
585 - $sql .= $col . '=' . $this->addQuotes( $row[$col] );
586 - }
587 - } else {
588 - $sql .= $index . '=' . $this->addQuotes( $row[$index] );
589 - }
590 - }
591 - $sql .= ')';
592 - $this->query( $sql, $fname );
593 - }
594 -
595 - # Now insert the row
596 - $sql = "INSERT INTO $table (" . $this->makeList( array_keys( $row ), LIST_NAMES ) . ') VALUES (' .
597 - $this->makeList( $row, LIST_COMMA ) . ')';
598 - $this->query( $sql, $fname );
599 - }
600 - }
601 -
602543 # Returns the size of a text field, or -1 for "unlimited"
603544 function textFieldSize( $table, $field ) {
604545 $table = $this->tableName( $table );
——@@ -762,48 +703,6 @@
763704 $this->mTrxLevel = 0;
764705 }
765706
766 - function setup_database() {
767 - global $wgDBuser;
768 -
769 - // Make sure that we can write to the correct schema
770 - $ctest = "mediawiki_test_table";
771 - if ( $this->tableExists( $ctest ) ) {
772 - $this->doQuery( "DROP TABLE $ctest" );
773 - }
774 - $SQL = "CREATE TABLE $ctest (a int)";
775 - $res = $this->doQuery( $SQL );
776 - if ( !$res ) {
777 - print "<b>FAILED</b>. Make sure that the user " . htmlspecialchars( $wgDBuser ) . " can write to the database</li>\n";
778 - die();
779 - }
780 - $this->doQuery( "DROP TABLE $ctest" );
781 -
782 - $res = $this->sourceFile( "../maintenance/mssql/tables.sql" );
783 - if ( $res !== true ) {
784 - echo " <b>FAILED</b></li>";
785 - die( htmlspecialchars( $res ) );
786 - }
787 -
788 - # Avoid the non-standard "REPLACE INTO" syntax
789 - $f = fopen( "../maintenance/interwiki.sql", 'r' );
790 - if ( $f == false ) {
791 - die( "<li>Could not find the interwiki.sql file" );
792 - }
793 - # We simply assume it is already empty as we have just created it
794 - $SQL = "INSERT INTO interwiki(iw_prefix,iw_url,iw_local) VALUES ";
795 - while ( ! feof( $f ) ) {
796 - $line = fgets( $f, 1024 );
797 - $matches = array();
798 - if ( !preg_match( '/^\s*(\(.+?),(\d)\)/', $line, $matches ) ) {
799 - continue;
800 - }
801 - $this->query( "$SQL $matches[1],$matches[2])" );
802 - }
803 - print " (table interwiki successfully populated)...\n";
804 -
805 - $this->commit();
806 - }
807 -
808707 /**
809708 * Escapes a identifier for use inm SQL.
810709 * Throws an exception if it is invalid.
Index: trunk/phase3/includes/AutoLoader.php
——@@ -386,6 +386,7 @@
387387 'DBConnectionError' => 'includes/db/DatabaseError.php',
388388 'DBError' => 'includes/db/DatabaseError.php',
389389 'DBObject' => 'includes/db/DatabaseUtility.php',
 390+ 'DBMasterPos' => 'includes/db/DatabaseUtility.php',
390391 'DBQueryError' => 'includes/db/DatabaseError.php',
391392 'DBUnexpectedError' => 'includes/db/DatabaseError.php',
392393 'FakeResultWrapper' => 'includes/db/DatabaseUtility.php',
——@@ -401,6 +402,7 @@
402403 'LoadBalancer_Single' => 'includes/db/LBFactory_Single.php',
403404 'LoadMonitor' => 'includes/db/LoadMonitor.php',
404405 'LoadMonitor_MySQL' => 'includes/db/LoadMonitor.php',
 406+ 'LoadMonitor_Null' => 'includes/db/LoadMonitor.php',
405407 'MySQLField' => 'includes/db/DatabaseMysql.php',
406408 'MySQLMasterPos' => 'includes/db/DatabaseMysql.php',
407409 'ORAField' => 'includes/db/DatabaseOracle.php',

Follow-up revisions

RevisionCommit summaryAuthorDate
r90644Merged the DatabaseMysql::getLag() patch from trunk r90643 for immediate depl...tstarling03:19, 23 June 2011
r90773Hack followup to r90643...reedy16:30, 25 June 2011
r95647* Fix for r90643: in the case where there is only one server, implying no rep...tstarling04:42, 29 August 2011
r95648Further updates for r90643/r95647 in core and extensions. Fixed all callers o...tstarling05:04, 29 August 2011

Comments

#Comment by Raymond (talk | contribs)   08:13, 23 June 2011

Error on Translatewiki, running MySQL 5.1.41-3ubuntu12.10-log :

Es ist ein Datenbankfehler aufgetreten. Der Grund kann ein Programmierfehler sein. Die letzte Datenbankabfrage lautete:

(SQL-Abfrage versteckt)

aus der Funktion „DatabaseMysql::getLagFromSlaveStatus“. Die Datenbank meldete den Fehler „1227: Access denied; you need the SUPER,REPLICATION CLIENT privilege for this operation (localhost)“.

#Comment by Aaron Schulz (talk | contribs)   22:43, 24 June 2011

Getting the same problem on Contribs/Watchlist.

#Comment by Tim Starling (talk | contribs)   23:06, 24 June 2011

So grant REPLICATION CLIENT.

#Comment by Tim Starling (talk | contribs)   23:07, 24 June 2011

Wait, are you saying this is happening when replication is not in use?

#Comment by Aaron Schulz (talk | contribs)   23:08, 24 June 2011

My test wiki is just a 1 DB set up. I don't know about TWN.

#Comment by Reedy (talk | contribs)   16:21, 25 June 2011

Yup, I'm getting it on my dev wiki.

Translatewiki is just 1 DB 

A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:
SHOW SLAVE STATUS
from within function "DatabaseMysql::getLagFromSlaveStatus". Database returned error "1227: Access denied; you need the SUPER,REPLICATION CLIENT privilege for this operation (localhost)".
#Comment by Tim Starling (talk | contribs)   05:05, 29 August 2011

Should be OK as of r95648.

Status & tagging log