r46845 MediaWiki - Code Review archive

Repository:MediaWiki
Revision:r46844‎ | r46845 | r46846 >
Date:14:30, 5 February 2009
Author:catrope
Status:deferred (Comments)
Tags:
Comment:
* API: BREAKING CHANGE: (bug 11430) Return fewer results than the limit in some cases to prevent running out of memory
* This means queries could possibly return fewer results than the limit and still set a query-continue
* Add iicontinue, rvcontinue, cicontinue, incontinue, amfrom to faciliate query-continue for these modules
* Implemented by blocking additions to the ApiResult object if they would make it too large
** Important things like query-continue values and warnings are exempt from this check
** RSS feeds and exported XML are also exempted (size-checking them would be too messy)
** Result size is checked against $wgAPIMaxResultSize, which defaults to 8 MB

For those who really care, per-file details follow:

ApiResult.php:
* Introduced ApiResult::$mSize which keeps track of the result size.
* Introduced ApiResult::size() which calculates an array's size
(which is the sum of the strlen()s of its elements).
* ApiResult::addValue() now checks that the result size stays below
$wgAPIMaxResultSize. If the item won't fit, it won't be added and addValue()
will return false. Callers should check the return value and set a
query-continue if it's false.
* Closed the back door that is ApiResult::getData(): callers can't manipulate
the data array directly anymore so they can't bypass the result size limit.
* Added ApiResult::setIndexedTagName_internal() which will call
setIndexedTagName() on an array already in the result. This is needed for the
'new' order of adding results, which means addValue()ing one result at a time
until you hit the limit or run out, then calling this function to set the tag
name.
* Added ApiResult::disableSizeCheck() and enableSizeCheck() which disable and
enable size checking in addValue(). This is used for stuff like query-continue
elements and warnings which shouldn't count towards the result size.
* Added ApiResult::unsetValue() which removes an element from the result and
decreases $mSize.

ApiBase.php:
* Like ApiResult::getData(), ApiBase::getResultData() no longer returns a
reference.
* Use ApiResult::disableSizeCheck() in ApiBase::setWarning()

ApiQueryBase.php:
* Added ApiQueryBase::addPageSubItem(), which adds page subitems one item
at a time.
* addPageSubItem() and addPageSubItems() now return whether the subitem
fit in the result.
* Use ApiResult::disableSizeCheck() in setContinueEnumParameter()

ApiMain.php:
* Use ApiResult::disableSizeCheck() in ApiMain::substituteResultWithError()
* Use getParameter() rather than $mRequest to obtain requestid

DefaultSettings.php:
* Added $wgAPIMaxResultSize, with a default value of 8 MB

ApiQuery*.php:
* Added results one at a time, and set a query-continue if the result is full.

ApiQueryLangLinks.php and friends:
* Migrated from addPageSubItems() to addPageSubItem(). This eliminates the
need for $lastId.

ApiQueryAllLinks.php, ApiQueryWatchlist.php, ApiQueryAllimages.php, ApiQuerySearch.php:
* Renamed $data to something more appropriate ($pageids, $ids or $titles)

ApiQuerySiteinfo.php:
* Abuse siprop as a query-continue parameter and set it to all props that
couldn't be processed.

ApiQueryRandom.php:
* Doesn't do continuations, because the result is supposed to be random.
* Be smart enough to not run the second query if the results of the first
didn't fit.

ApiQueryImageInfo.php, ApiQueryRevisions.php, ApiQueryCategoryInfo.php, ApiQueryInfo.php:
* Added continue parameter which basically skips the first so many items

ApiQueryBacklinks.php:
* Throw the result in a big array first and addValue() that one element at a time if necessary
** This is necessary because the results aren't retrieved in order
* Introduced $this->pageMap to map namespace and title to page ID
* Rewritten extractRowInfo() and extractRedirRowInfo() a little
* Declared all private member variables explicitly

ApiQueryDeletedrevs.php:
* Use a pagemap just like in Backlinks
* Introduce fake page IDs and keep track of them so we know where to add what
** This doesn't change the output format, because the fake page IDs start at 0 and are consecutive

ApiQueryAllmessages.php:
* Add amfrom to facilitate query-continue

ApiQueryUsers.php:
* Rewrite: put the getOtherUsersInfo() code in execute()
Modified paths:
  • /trunk/phase3/includes/DefaultSettings.php (modified) (history)
  • /trunk/phase3/includes/api/ApiBase.php (modified) (history)
  • /trunk/phase3/includes/api/ApiFormatBase.php (modified) (history)
  • /trunk/phase3/includes/api/ApiMain.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQuery.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryAllCategories.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryAllLinks.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryAllUsers.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryAllimages.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryAllmessages.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryAllpages.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryBacklinks.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryBase.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryBlocks.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryCategories.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryCategoryInfo.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryCategoryMembers.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryDeletedrevs.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryDuplicateFiles.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryExtLinksUsage.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryExternalLinks.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryImageInfo.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryImages.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryInfo.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryLangLinks.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryLinks.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryLogEvents.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryRandom.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryRecentChanges.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryRevisions.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQuerySearch.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQuerySiteinfo.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryUserContributions.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryUsers.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryWatchlist.php (modified) (history)
  • /trunk/phase3/includes/api/ApiQueryWatchlistRaw.php (modified) (history)
  • /trunk/phase3/includes/api/ApiResult.php (modified) (history)

Diff [purge]

Index: trunk/phase3/includes/api/ApiQueryAllLinks.php
@@ -101,8 +101,9 @@
102102
103103 $res = $this->select(__METHOD__);
104104
105 - $data = array ();
 105+ $pageids = array ();
106106 $count = 0;
 107+ $result = $this->getResult();
107108 while ($row = $db->fetchObject($res)) {
108109 if (++ $count > $limit) {
109110 // We've reached the one extra which shows that there are additional pages to be had. Stop here...
@@ -123,7 +124,13 @@
124125 $vals['ns'] = intval($title->getNamespace());
125126 $vals['title'] = $title->getPrefixedText();
126127 }
127 - $data[] = $vals;
 128+ $fit = $result->addValue(array('query', $this->getModuleName()), null, $vals);
 129+ if(!$fit)
 130+ {
 131+ $this->setContinueEnumParameter('continue',
 132+ $this->keyToTitle($row->pl_title) . "|" . $row->pl_from);
 133+ break;
 134+ }
128135 } else {
129136 $pageids[] = $row->pl_from;
130137 }
@@ -131,9 +138,7 @@
132139 $db->freeResult($res);
133140
134141 if (is_null($resultPageSet)) {
135 - $result = $this->getResult();
136 - $result->setIndexedTagName($data, 'l');
137 - $result->addValue('query', $this->getModuleName(), $data);
 142+ $result->setIndexedTagName_internal(array('query', $this->getModuleName()), 'l');
138143 } else {
139144 $resultPageSet->populateFromPageIDs($pageids);
140145 }
@@ -192,4 +197,4 @@
193198 public function getVersion() {
194199 return __CLASS__ . ': $Id$';
195200 }
196 -}
 201+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryRecentChanges.php
@@ -191,9 +191,7 @@
192192 $this->token = $params['token'];
193193 $this->addOption('LIMIT', $params['limit'] +1);
194194
195 - $data = array ();
196195 $count = 0;
197 -
198196 /* Perform the actual query. */
199197 $db = $this->getDB();
200198 $res = $this->select(__METHOD__);
@@ -210,16 +208,20 @@
211209 $vals = $this->extractRowInfo($row);
212210
213211 /* Add that row's data to our final output. */
214 - if($vals)
215 - $data[] = $vals;
 212+ if(!$vals)
 213+ continue;
 214+ $fit = $this->getResult()->addValue(array('query', $this->getModuleName()), null, $vals);
 215+ if(!$fit)
 216+ {
 217+ $this->setContinueEnumParameter('start', wfTimestamp(TS_ISO_8601, $row->rc_timestamp));
 218+ break;
 219+ }
216220 }
217221
218222 $db->freeResult($res);
219223
220224 /* Format the result */
221 - $result = $this->getResult();
222 - $result->setIndexedTagName($data, 'rc');
223 - $result->addValue('query', $this->getModuleName(), $data);
 225+ $this->getResult()->setIndexedTagName_internal(array('query', $this->getModuleName()), 'rc');
224226 }
225227
226228 /**
@@ -453,4 +455,4 @@
454456 public function getVersion() {
455457 return __CLASS__ . ': $Id$';
456458 }
457 -}
 459+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryAllCategories.php
@@ -110,14 +110,18 @@
111111 }
112112 if( isset( $prop['hidden'] ) && $row->cat_hidden )
113113 $item['hidden'] = '';
114 - $categories[] = $item;
 114+ $fit = $result->addValue(array('query', $this->getModuleName()), null, $item);
 115+ if(!$fit)
 116+ {
 117+ $this->setContinueEnumParameter('from', $this->keyToTitle($row->cat_title));
 118+ break;
 119+ }
115120 }
116121 }
117122 $db->freeResult($res);
118123
119124 if (is_null($resultPageSet)) {
120 - $result->setIndexedTagName($categories, 'c');
121 - $result->addValue('query', $this->getModuleName(), $categories);
 125+ $result->setIndexedTagName_internal(array('query', $this->getModuleName()), 'c');
122126 } else {
123127 $resultPageSet->populateFromTitles($pages);
124128 }
@@ -173,4 +177,4 @@
174178 public function getVersion() {
175179 return __CLASS__ . ': $Id$';
176180 }
177 -}
 181+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryLinks.php
@@ -119,9 +119,6 @@
120120 $res = $this->select(__METHOD__);
121121
122122 if (is_null($resultPageSet)) {
123 -
124 - $data = array();
125 - $lastId = 0; // database has no ID 0
126123 $count = 0;
127124 while ($row = $db->fetchObject($res)) {
128125 if(++$count > $params['limit']) {
@@ -132,23 +129,17 @@
133130 $this->keyToTitle($row->pl_title));
134131 break;
135132 }
136 - if ($lastId != $row->pl_from) {
137 - if($lastId != 0) {
138 - $this->addPageSubItems($lastId, $data);
139 - $data = array();
140 - }
141 - $lastId = $row->pl_from;
142 - }
143 -
144133 $vals = array();
145134 ApiQueryBase :: addTitleInfo($vals, Title :: makeTitle($row->pl_namespace, $row->pl_title));
146 - $data[] = $vals;
 135+ $fit = $this->addPageSubItem($row->pl_from, $vals);
 136+ if(!$fit)
 137+ {
 138+ $this->setContinueEnumParameter('continue',
 139+ "{$row->pl_from}|{$row->pl_namespace}|" .
 140+ $this->keyToTitle($row->pl_title));
 141+ break;
 142+ }
147143 }
148 -
149 - if($lastId != 0) {
150 - $this->addPageSubItems($lastId, $data);
151 - }
152 -
153144 } else {
154145
155146 $titles = array();
@@ -215,4 +206,4 @@
216207 public function getVersion() {
217208 return __CLASS__ . ': $Id$';
218209 }
219 -}
 210+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryRandom.php
@@ -62,7 +62,7 @@
6363 $this->addFields($resultPageSet->getPageTableFields());
6464 }
6565
66 - protected function runQuery(&$data, &$resultPageSet) {
 66+ protected function runQuery(&$resultPageSet) {
6767 $db = $this->getDB();
6868 $res = $this->select(__METHOD__);
6969 $count = 0;
@@ -73,7 +73,14 @@
7474 // Prevent duplicates
7575 if(!in_array($row->page_id, $this->pageIDs))
7676 {
77 - $data[] = $this->extractRowInfo($row);
 77+ $fit = $this->getResult()->addValue(
 78+ array('query', $this->getModuleName()),
 79+ null, $this->extractRowInfo($row));
 80+ if(!$fit)
 81+ # We can't really query-continue a random list.
 82+ # Return an insanely high value so
 83+ # $count < $limit is false
 84+ return 1E9;
7885 $this->pageIDs[] = $row->page_id;
7986 }
8087 }
@@ -87,11 +94,10 @@
8895 public function run($resultPageSet = null) {
8996 $params = $this->extractRequestParams();
9097 $result = $this->getResult();
91 - $data = array();
9298 $this->pageIDs = array();
9399
94100 $this->prepareQuery(wfRandom(), $params['limit'], $params['namespace'], $resultPageSet, $params['redirect']);
95 - $count = $this->runQuery($data, $resultPageSet);
 101+ $count = $this->runQuery($resultPageSet);
96102 if($count < $params['limit'])
97103 {
98104 /* We got too few pages, we probably picked a high value
@@ -99,12 +105,11 @@
100106 * also the comment in Title::getRandomTitle()
101107 */
102108 $this->prepareQuery(0, $params['limit'] - $count, $params['namespace'], $resultPageSet, $params['redirect']);
103 - $this->runQuery($data, $resultPageSet);
 109+ $this->runQuery($resultPageSet);
104110 }
105111
106112 if(is_null($resultPageSet)) {
107 - $result->setIndexedTagName($data, 'page');
108 - $result->addValue('query', $this->getModuleName(), $data);
 113+ $result->setIndexedTagName_internal(array('query', $this->getModuleName()), 'page');
109114 }
110115 }
111116
@@ -157,4 +162,4 @@
158163 public function getVersion() {
159164 return __CLASS__ . ': $Id: ApiQueryRandom.php overlordq$';
160165 }
161 -}
 166+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQuerySiteinfo.php
@@ -41,50 +41,60 @@
4242
4343 public function execute() {
4444 $params = $this->extractRequestParams();
 45+ $done = array();
4546 foreach( $params['prop'] as $p )
4647 {
4748 switch ( $p )
4849 {
4950 case 'general':
50 - $this->appendGeneralInfo( $p );
 51+ $fit = $this->appendGeneralInfo( $p );
5152 break;
5253 case 'namespaces':
53 - $this->appendNamespaces( $p );
 54+ $fit = $this->appendNamespaces( $p );
5455 break;
5556 case 'namespacealiases':
56 - $this->appendNamespaceAliases( $p );
 57+ $fit = $this->appendNamespaceAliases( $p );
5758 break;
5859 case 'specialpagealiases':
59 - $this->appendSpecialPageAliases( $p );
 60+ $fit = $this->appendSpecialPageAliases( $p );
6061 break;
6162 case 'magicwords':
62 - $this->appendMagicWords( $p );
 63+ $fit = $this->appendMagicWords( $p );
6364 break;
6465 case 'interwikimap':
6566 $filteriw = isset( $params['filteriw'] ) ? $params['filteriw'] : false;
66 - $this->appendInterwikiMap( $p, $filteriw );
 67+ $fit = $this->appendInterwikiMap( $p, $filteriw );
6768 break;
6869 case 'dbrepllag':
69 - $this->appendDbReplLagInfo( $p, $params['showalldb'] );
 70+ $fit = $this->appendDbReplLagInfo( $p, $params['showalldb'] );
7071 break;
7172 case 'statistics':
72 - $this->appendStatistics( $p );
 73+ $fit = $this->appendStatistics( $p );
7374 break;
7475 case 'usergroups':
75 - $this->appendUserGroups( $p );
 76+ $fit = $this->appendUserGroups( $p );
7677 break;
7778 case 'extensions':
78 - $this->appendExtensions( $p );
 79+ $fit = $this->appendExtensions( $p );
7980 break;
8081 case 'fileextensions':
81 - $this->appendFileExtensions( $p );
 82+ $fit = $this->appendFileExtensions( $p );
8283 break;
8384 case 'rightsinfo':
84 - $this->appendRightsInfo( $p );
 85+ $fit = $this->appendRightsInfo( $p );
8586 break;
8687 default :
8788 ApiBase :: dieDebug( __METHOD__, "Unknown prop=$p" );
8889 }
 90+ if(!$fit)
 91+ {
 92+ # Abuse siprop as a query-continue parameter
 93+ # and set it to all unprocessed props
 94+ $this->setContinueEnumParameter('prop', implode('|',
 95+ array_diff($params['prop'], $done)));
 96+ break;
 97+ }
 98+ $done[] = $p;
8999 }
90100 }
91101
@@ -129,7 +139,7 @@
130140 $data['timezone'] = $tz;
131141 $data['timeoffset'] = $offset;
132142
133 - $this->getResult()->addValue( 'query', $property, $data );
 143+ return $this->getResult()->addValue( 'query', $property, $data );
134144 }
135145
136146 protected function appendNamespaces( $property ) {
@@ -151,7 +161,7 @@
152162 }
153163
154164 $this->getResult()->setIndexedTagName( $data, 'ns' );
155 - $this->getResult()->addValue( 'query', $property, $data );
 165+ return $this->getResult()->addValue( 'query', $property, $data );
156166 }
157167
158168 protected function appendNamespaceAliases( $property ) {
@@ -168,7 +178,7 @@
169179 }
170180
171181 $this->getResult()->setIndexedTagName( $data, 'ns' );
172 - $this->getResult()->addValue( 'query', $property, $data );
 182+ return $this->getResult()->addValue( 'query', $property, $data );
173183 }
174184
175185 protected function appendSpecialPageAliases( $property ) {
@@ -181,7 +191,7 @@
182192 $data[] = $arr;
183193 }
184194 $this->getResult()->setIndexedTagName( $data, 'specialpage' );
185 - $this->getResult()->addValue( 'query', $property, $data );
 195+ return $this->getResult()->addValue( 'query', $property, $data );
186196 }
187197
188198 protected function appendMagicWords( $property ) {
@@ -197,7 +207,7 @@
198208 $data[] = $arr;
199209 }
200210 $this->getResult()->setIndexedTagName($data, 'magicword');
201 - $this->getResult()->addValue('query', $property, $data);
 211+ return $this->getResult()->addValue( 'query', $property, $data );
202212 }
203213
204214 protected function appendInterwikiMap( $property, $filter ) {
@@ -235,7 +245,7 @@
236246 $db->freeResult( $res );
237247
238248 $this->getResult()->setIndexedTagName( $data, 'iw' );
239 - $this->getResult()->addValue( 'query', $property, $data );
 249+ return $this->getResult()->addValue( 'query', $property, $data );
240250 }
241251
242252 protected function appendDbReplLagInfo( $property, $includeAll ) {
@@ -263,7 +273,7 @@
264274
265275 $result = $this->getResult();
266276 $result->setIndexedTagName( $data, 'db' );
267 - $result->addValue( 'query', $property, $data );
 277+ return $this->getResult()->addValue( 'query', $property, $data );
268278 }
269279
270280 protected function appendStatistics( $property ) {
@@ -277,7 +287,7 @@
278288 $data['activeusers'] = intval( SiteStats::activeUsers() );
279289 $data['admins'] = intval( SiteStats::numberingroup('sysop') );
280290 $data['jobs'] = intval( SiteStats::jobs() );
281 - $this->getResult()->addValue( 'query', $property, $data );
 291+ return $this->getResult()->addValue( 'query', $property, $data );
282292 }
283293
284294 protected function appendUserGroups( $property ) {
@@ -290,7 +300,7 @@
291301 }
292302
293303 $this->getResult()->setIndexedTagName( $data, 'group' );
294 - $this->getResult()->addValue( 'query', $property, $data );
 304+ return $this->getResult()->addValue( 'query', $property, $data );
295305 }
296306
297307 protected function appendFileExtensions( $property ) {
@@ -301,7 +311,7 @@
302312 $data[] = array( 'ext' => $ext );
303313 }
304314 $this->getResult()->setIndexedTagName( $data, 'fe' );
305 - $this->getResult()->addValue( 'query', $property, $data );
 315+ return $this->getResult()->addValue( 'query', $property, $data );
306316 }
307317
308318 protected function appendExtensions( $property ) {
@@ -334,7 +344,7 @@
335345 }
336346
337347 $this->getResult()->setIndexedTagName( $data, 'ext' );
338 - $this->getResult()->addValue( 'query', $property, $data );
 348+ return $this->getResult()->addValue( 'query', $property, $data );
339349 }
340350
341351
@@ -352,7 +362,7 @@
353363 'text' => $text ? $text : ''
354364 );
355365
356 - $this->getResult()->addValue( 'query', $property, $data );
 366+ return $this->getResult()->addValue( 'query', $property, $data );
357367 }
358368
359369
@@ -423,4 +433,4 @@
424434 public function getVersion() {
425435 return __CLASS__ . ': $Id$';
426436 }
427 -}
 437+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryExtLinksUsage.php
@@ -110,7 +110,7 @@
111111
112112 $res = $this->select(__METHOD__);
113113
114 - $data = array ();
 114+ $result = $this->getResult();
115115 $count = 0;
116116 while ($row = $db->fetchObject($res)) {
117117 if (++ $count > $limit) {
@@ -130,7 +130,12 @@
131131 }
132132 if ($fld_url)
133133 $vals['url'] = $row->el_to;
134 - $data[] = $vals;
 134+ $fit = $result->addValue(array('query', $this->getModuleName()), null, $vals);
 135+ if(!$fit)
 136+ {
 137+ $this->setContinueEnumParameter('offset', $offset + $count - 1);
 138+ break;
 139+ }
135140 } else {
136141 $resultPageSet->processDbRow($row);
137142 }
@@ -138,9 +143,8 @@
139144 $db->freeResult($res);
140145
141146 if (is_null($resultPageSet)) {
142 - $result = $this->getResult();
143 - $result->setIndexedTagName($data, $this->getModulePrefix());
144 - $result->addValue('query', $this->getModuleName(), $data);
 147+ $result->setIndexedTagName_internal(array('query', $this->getModuleName()),
 148+ $this->getModulePrefix());
145149 }
146150 }
147151
@@ -208,4 +212,4 @@
209213 public function getVersion() {
210214 return __CLASS__ . ': $Id$';
211215 }
212 -}
 216+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryAllpages.php
@@ -135,8 +135,8 @@
136136 $this->addOption('LIMIT', $limit+1);
137137 $res = $this->select(__METHOD__);
138138
139 - $data = array ();
140139 $count = 0;
 140+ $result = $this->getResult();
141141 while ($row = $db->fetchObject($res)) {
142142 if (++ $count > $limit) {
143143 // We've reached the one extra which shows that there are additional pages to be had. Stop here...
@@ -147,10 +147,16 @@
148148
149149 if (is_null($resultPageSet)) {
150150 $title = Title :: makeTitle($row->page_namespace, $row->page_title);
151 - $data[] = array(
 151+ $vals = array(
152152 'pageid' => intval($row->page_id),
153153 'ns' => intval($title->getNamespace()),
154154 'title' => $title->getPrefixedText());
 155+ $fit = $result->addValue(array('query', $this->getModuleName()), null, $vals);
 156+ if(!$fit)
 157+ {
 158+ $this->setContinueEnumParameter('from', $this->keyToTitle($row->page_title));
 159+ break;
 160+ }
155161 } else {
156162 $resultPageSet->processDbRow($row);
157163 }
@@ -158,9 +164,7 @@
159165 $db->freeResult($res);
160166
161167 if (is_null($resultPageSet)) {
162 - $result = $this->getResult();
163 - $result->setIndexedTagName($data, 'p');
164 - $result->addValue('query', $this->getModuleName(), $data);
 168+ $result->setIndexedTagName_internal(array('query', $this->getModuleName()), 'p');
165169 }
166170 }
167171
@@ -266,4 +270,4 @@
267271 public function getVersion() {
268272 return __CLASS__ . ': $Id$';
269273 }
270 -}
 274+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryBacklinks.php
@@ -38,7 +38,9 @@
3939 */
4040 class ApiQueryBacklinks extends ApiQueryGeneratorBase {
4141
42 - private $params, $rootTitle, $contRedirs, $contLevel, $contTitle, $contID, $redirID;
 42+ private $params, $rootTitle, $contRedirs, $contLevel, $contTitle, $contID, $redirID, $redirect;
 43+ private $bl_ns, $bl_from, $bl_table, $bl_code, $bl_title, $bl_sort, $bl_fields, $hasNS;
 44+ private $pageMap, $resultArr;
4345
4446 // output element name, database column field prefix, database table
4547 private $backlinksSettings = array (
@@ -61,6 +63,7 @@
6264
6365 public function __construct($query, $moduleName) {
6466 extract($this->backlinksSettings[$moduleName]);
 67+ $this->resultArr = array();
6568
6669 parent :: __construct($query, $moduleName, $code);
6770 $this->bl_ns = $prefix . '_namespace';
@@ -188,7 +191,7 @@
189192 $res = $this->select(__METHOD__.'::firstQuery');
190193
191194 $count = 0;
192 - $this->data = array ();
 195+ $this->pageMap = array(); // Maps ns and title to pageid
193196 $this->continueStr = null;
194197 $this->redirTitles = array();
195198 while ($row = $db->fetchObject($res)) {
@@ -223,10 +226,10 @@
224227 // We've reached the one extra which shows that there are additional pages to be had. Stop here...
225228 // We need to keep the parent page of this redir in
226229 if($this->hasNS)
227 - $contTitle = Title::makeTitle($row->{$this->bl_ns}, $row->{$this->bl_title});
 230+ $parentID = $this->pageMap[$row->{$this->bl_ns}][$row->{$this->bl_title}];
228231 else
229 - $contTitle = Title::makeTitle(NS_FILE, $row->{$this->bl_title});
230 - $this->continueStr = $this->getContinueRedirStr($contTitle->getArticleID(), $row->page_id);
 232+ $parentID = $this->pageMap[NS_IMAGE][$row->{$this->bl_title}];
 233+ $this->continueStr = $this->getContinueRedirStr($parentID, $row->page_id);
231234 break;
232235 }
233236
@@ -237,30 +240,67 @@
238241 }
239242 $db->freeResult($res);
240243 }
 244+ // Try to add the result data in one go and pray that it fits
 245+ $fit = $this->getResult()->addValue('query', $this->getModuleName(), $this->resultArr);
 246+ if(!$fit)
 247+ {
 248+ // It didn't fit. Add elements one by one until the
 249+ // result is full.
 250+ foreach($this->resultArr as $pageID => $arr)
 251+ {
 252+ // Add the basic entry without redirlinks first
 253+ $fit = $this->getResult()->addValue(
 254+ array('query', $this->getModuleName()),
 255+ $pageID, array_diff_key($arr, array('redirlinks' => '')));
 256+ if(!$fit)
 257+ {
 258+ $this->continueStr = $this->getContinueStr($pageID);
 259+ break;
 260+ }
 261+
 262+ $hasRedirs = false;
 263+ foreach((array)@$arr['redirlinks'] as $key => $redir)
 264+ {
 265+ $fit = $this->getResult()->addValue(
 266+ array('query', $this->getModuleName(), $pageID, 'redirlinks'),
 267+ $key, $redir);
 268+ if(!$fit)
 269+ {
 270+ $this->continueStr = $this->getContinueRedirStr($pageID, $redir['pageid']);
 271+ break;
 272+ }
 273+ $hasRedirs = true;
 274+ }
 275+ if($hasRedirs)
 276+ $this->getResult()->setIndexedTagName_internal(
 277+ array('query', $this->getModuleName(), $pageID, 'redirlinks'),
 278+ $this->bl_code);
 279+ if(!$fit)
 280+ break;
 281+ }
 282+ }
241283 if(!is_null($this->continueStr))
242284 $this->setContinueEnumParameter('continue', $this->continueStr);
243285
244286 if (is_null($resultPageSet)) {
245 - $resultData = array();
246 - foreach($this->data as $ns => $a)
247 - foreach($a as $title => $arr)
248 - $resultData[] = $arr;
249 - $result = $this->getResult();
250 - $result->setIndexedTagName($resultData, $this->bl_code);
251 - $result->addValue('query', $this->getModuleName(), $resultData);
 287+ $this->getResult()->setIndexedTagName_internal(
 288+ array('query', $this->getModuleName()),
 289+ $this->bl_code);
252290 }
253291 }
254292
255293 private function extractRowInfo($row) {
256 - if(!isset($this->data[$row->page_namespace][$row->page_title])) {
257 - $this->data[$row->page_namespace][$row->page_title]['pageid'] = $row->page_id;
258 - ApiQueryBase::addTitleInfo($this->data[$row->page_namespace][$row->page_title], Title::makeTitle($row->page_namespace, $row->page_title));
259 - if($row->page_is_redirect)
260 - {
261 - $this->data[$row->page_namespace][$row->page_title]['redirect'] = '';
262 - $this->redirTitles[] = Title::makeTitle($row->page_namespace, $row->page_title);
263 - }
 294+ $this->pageMap[$row->page_namespace][$row->page_title] = $row->page_id;
 295+ $t = Title::makeTitle($row->page_namespace, $row->page_title);
 296+ $a = array('pageid' => $row->page_id);
 297+ ApiQueryBase::addTitleInfo($a, $t);
 298+ if($row->page_is_redirect)
 299+ {
 300+ $a['redirect'] = '';
 301+ $this->redirTitles[] = $t;
264302 }
 303+ // Put all the results in an array first
 304+ $this->resultArr[$a['pageid']] = $a;
265305 }
266306
267307 private function extractRedirRowInfo($row)
@@ -270,8 +310,10 @@
271311 if($row->page_is_redirect)
272312 $a['redirect'] = '';
273313 $ns = $this->hasNS ? $row->{$this->bl_ns} : NS_FILE;
274 - $this->data[$ns][$row->{$this->bl_title}]['redirlinks'][] = $a;
275 - $this->getResult()->setIndexedTagName($this->data[$ns][$row->{$this->bl_title}]['redirlinks'], $this->bl_code);
 314+ $parentID = $this->pageMap[$ns][$row->{$this->bl_title}];
 315+ // Put all the results in an array first
 316+ $this->resultArr[$parentID]['redirlinks'][] = $a;
 317+ $this->getResult()->setIndexedTagName($this->resultArr[$parentID]['redirlinks'], $this->bl_code);
276318 }
277319
278320 protected function processContinue() {
Index: trunk/phase3/includes/api/ApiQueryDeletedrevs.php
@@ -173,9 +173,9 @@
174174 $this->addWhereRange('ar_timestamp', $params['dir'], $params['start'], $params['end']);
175175 }
176176 $res = $this->select(__METHOD__);
177 - $pages = array();
 177+ $pageMap = array(); // Maps ns&title to (fake) pageid
178178 $count = 0;
179 - // First populate the $pages array
 179+ $newPageID = 0;
180180 while($row = $db->fetchObject($res))
181181 {
182182 if(++$count > $limit)
@@ -205,32 +205,38 @@
206206 if($fld_content)
207207 ApiResult::setContent($rev, Revision::getRevisionText($row));
208208
209 - $t = Title::makeTitle($row->ar_namespace, $row->ar_title);
210 - if(!isset($pages[$t->getPrefixedText()]))
 209+ if(!isset($pageMap[$row->ar_namespace][$row->ar_title]))
211210 {
212 - $pages[$t->getPrefixedText()] = array(
 211+ $pageID = $newPageID++;
 212+ $pageMap[$row->ar_namespace][$row->ar_title] = $pageID;
 213+ $t = Title::makeTitle($row->ar_namespace, $row->ar_title);
 214+ $a = array(
213215 'title' => $t->getPrefixedText(),
214216 'ns' => intval($row->ar_namespace),
215217 'revisions' => array($rev)
216218 );
 219+ $result->setIndexedTagName($a['revisions'], 'rev');
217220 if($fld_token)
218 - $pages[$t->getPrefixedText()]['token'] = $token;
 221+ $a['token'] = $token;
 222+ $fit = $result->addValue(array('query', $this->getModuleName()), $pageID, $a);
219223 }
220224 else
221 - $pages[$t->getPrefixedText()]['revisions'][] = $rev;
 225+ {
 226+ $pageID = $pageMap[$row->ar_namespace][$row->ar_title];
 227+ $fit = $result->addValue(
 228+ array('query', $this->getModuleName(), $pageID, 'revisions'),
 229+ null, $rev);
 230+ }
 231+ if(!$fit)
 232+ {
 233+ $this->setContinueEnumParameter('start', wfTimestamp(TS_ISO_8601, $row->ar_timestamp));
 234+ break;
 235+ }
222236 }
223237 $db->freeResult($res);
 238+ $result->setIndexedTagName_internal(array('query', $this->getModuleName()), 'page');
 239+ }
224240
225 - // We don't want entire pagenames as keys, so let's make this array indexed
226 - foreach($pages as $page)
227 - {
228 - $result->setIndexedTagName($page['revisions'], 'rev');
229 - $data[] = $page;
230 - }
231 - $result->setIndexedTagName($data, 'page');
232 - $result->addValue('query', $this->getModuleName(), $data);
233 - }
234 -
235241 public function getAllowedParams() {
236242 return array (
237243 'start' => array(
@@ -325,4 +331,4 @@
326332 public function getVersion() {
327333 return __CLASS__ . ': $Id$';
328334 }
329 -}
 335+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryBase.php
@@ -272,15 +272,37 @@
273273 /**
274274 * Add a sub-element under the page element with the given page ID
275275 * @param int $pageId Page ID
276 - * @param array $data Data array à la ApiResult
 276+ * @param array $data Data array � la ApiResult
 277+ * @return bool Whether the element fit in the result
277278 */
278279 protected function addPageSubItems($pageId, $data) {
279280 $result = $this->getResult();
280281 $result->setIndexedTagName($data, $this->getModulePrefix());
281 - $result->addValue(array ('query', 'pages', intval($pageId)),
 282+ return $result->addValue(array('query', 'pages', intval($pageId)),
282283 $this->getModuleName(),
283284 $data);
284285 }
 286+
 287+ /**
 288+ * Same as addPageSubItems(), but one element of $data
 289+ * at a time
 290+ * @param int $pageId Page ID
 291+ * @param array $data Data array � la ApiResult
 292+ * @param string $elemname XML element name. If null, getModuleName() is used
 293+ * @return bool Whether the element fit in the result
 294+ */
 295+ protected function addPageSubItem($pageId, $item, $elemname = null) {
 296+ if(is_null($elemname))
 297+ $elemname = $this->getModulePrefix();
 298+ $result = $this->getResult();
 299+ $fit = $result->addValue(array('query', 'pages', $pageId,
 300+ $this->getModuleName()), null, $item);
 301+ if(!$fit)
 302+ return false;
 303+ $result->setIndexedTagName_internal(array('query', 'pages', $pageId,
 304+ $this->getModuleName()), $elemname);
 305+ return true;
 306+ }
285307
286308 /**
287309 * Set a query-continue value
@@ -288,10 +310,11 @@
289311 * @param $paramValue Parameter value
290312 */
291313 protected function setContinueEnumParameter($paramName, $paramValue) {
292 -
293314 $paramName = $this->encodeParamName($paramName);
294315 $msg = array( $paramName => $paramValue );
 316+ $this->getResult()->disableSizeCheck();
295317 $this->getResult()->addValue('query-continue', $this->getModuleName(), $msg);
 318+ $this->getResult()->enableSizeCheck();
296319 }
297320
298321 /**
Index: trunk/phase3/includes/api/ApiQueryBlocks.php
@@ -169,10 +169,14 @@
170170 if($row->ipb_allow_usertalk)
171171 $block['allowusertalk'] = '';
172172 }
173 - $data[] = $block;
 173+ $fit = $result->addValue(array('query', $this->getModuleName()), null, $block);
 174+ if(!$fit)
 175+ {
 176+ $this->setContinueEnumParameter('start', wfTimestamp(TS_ISO_8601, $row->ipb_timestamp));
 177+ break;
 178+ }
174179 }
175 - $result->setIndexedTagName($data, 'block');
176 - $result->addValue('query', $this->getModuleName(), $data);
 180+ $result->setIndexedTagName_internal(array('query', $this->getModuleName()), 'block');
177181 }
178182
179183 protected function prepareUsername($user)
@@ -261,4 +265,4 @@
262266 public function getVersion() {
263267 return __CLASS__ . ': $Id$';
264268 }
265 -}
 269+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryCategoryMembers.php
@@ -112,8 +112,6 @@
113113 break;
114114 }
115115
116 - $lastSortKey = $row->cl_sortkey; // detect duplicate sortkeys
117 -
118116 if (is_null($resultPageSet)) {
119117 $vals = array();
120118 if ($fld_ids)
@@ -127,16 +125,26 @@
128126 $vals['sortkey'] = $row->cl_sortkey;
129127 if ($fld_timestamp)
130128 $vals['timestamp'] = wfTimestamp(TS_ISO_8601, $row->cl_timestamp);
131 - $data[] = $vals;
 129+ $fit = $this->getResult()->addValue(array('query', $this->getModuleName()),
 130+ null, $vals);
 131+ if(!$fit)
 132+ {
 133+ if ($params['sort'] == 'timestamp')
 134+ $this->setContinueEnumParameter('start', wfTimestamp(TS_ISO_8601, $row->cl_timestamp));
 135+ else
 136+ $this->setContinueEnumParameter('continue', $this->getContinueStr($row, $lastSortKey));
 137+ break;
 138+ }
132139 } else {
133140 $resultPageSet->processDbRow($row);
134141 }
 142+ $lastSortKey = $row->cl_sortkey; // detect duplicate sortkeys
135143 }
136144 $db->freeResult($res);
137145
138146 if (is_null($resultPageSet)) {
139 - $this->getResult()->setIndexedTagName($data, 'cm');
140 - $this->getResult()->addValue('query', $this->getModuleName(), $data);
 147+ $this->getResult()->setIndexedTagName_internal(
 148+ array('query', $this->getModuleName()), 'cm');
141149 }
142150 }
143151
@@ -257,4 +265,4 @@
258266 public function getVersion() {
259267 return __CLASS__ . ': $Id$';
260268 }
261 -}
 269+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiResult.php
@@ -47,7 +47,7 @@
4848 */
4949 class ApiResult extends ApiBase {
5050
51 - private $mData, $mIsRawMode;
 51+ private $mData, $mIsRawMode, $mSize, $mCheckingSize;
5252
5353 /**
5454 * Constructor
@@ -55,6 +55,7 @@
5656 public function __construct($main) {
5757 parent :: __construct($main, 'result');
5858 $this->mIsRawMode = false;
 59+ $this->mCheckingSize = true;
5960 $this->reset();
6061 }
6162
@@ -63,6 +64,7 @@
6465 */
6566 public function reset() {
6667 $this->mData = array ();
 68+ $this->mSize = 0;
6769 }
6870
6971 /**
@@ -81,13 +83,54 @@
8284 }
8385
8486 /**
85 - * Get result's internal data array
 87+ * Get the result's internal data array (read-only)
8688 */
87 - public function & getData() {
 89+ public function getData() {
8890 return $this->mData;
8991 }
 92+
 93+ /**
 94+ * Get the 'real' size of a result item. This means the strlen() of the item,
 95+ * or the sum of the strlen()s of the elements if the item is an array.
 96+ * @param mixed $value
 97+ * @return int
 98+ */
 99+ public static function size($value) {
 100+ $s = 0;
 101+ if(is_array($value))
 102+ foreach($value as $v)
 103+ $s += self::size($v);
 104+ else if(!is_object($value))
 105+ // Objects can't always be cast to string
 106+ $s = strlen($value);
 107+ return $s;
 108+ }
90109
91110 /**
 111+ * Get the size of the result, i.e. the amount of bytes in it
 112+ * @return int
 113+ */
 114+ public function getSize() {
 115+ return $this->mSize;
 116+ }
 117+
 118+ /**
 119+ * Disable size checking in addValue(). Don't use this unless you
 120+ * REALLY know what you're doing. Values added while size checking
 121+ * was disabled will not be counted (ever)
 122+ */
 123+ public function disableSizeCheck() {
 124+ $this->mCheckingSize = false;
 125+ }
 126+
 127+ /**
 128+ * Re-enable size checking in addValue()
 129+ */
 130+ public function enableSizeCheck() {
 131+ $this->mCheckingSize = true;
 132+ }
 133+
 134+ /**
92135 * Add an output value to the array by name.
93136 * Verifies that value with the same name has not been added before.
94137 */
@@ -157,15 +200,38 @@
158201 }
159202
160203 /**
 204+ * Calls setIndexedTagName() on an array already in the result.
 205+ * Don't specify a path to a value that's not in the result, or
 206+ * you'll get nasty errors.
 207+ * @param array $path Path to the array, like addValue()'s path
 208+ * @param string $tag
 209+ */
 210+ public function setIndexedTagName_internal( $path, $tag ) {
 211+ $data = & $this->mData;
 212+ foreach((array)$path as $p)
 213+ $data = & $data[$p];
 214+ if(is_null($data))
 215+ return;
 216+ $this->setIndexedTagName($data, $tag);
 217+ }
 218+
 219+ /**
161220 * Add value to the output data at the given path.
162221 * Path is an indexed array, each element specifing the branch at which to add the new value
163222 * Setting $path to array('a','b','c') is equivalent to data['a']['b']['c'] = $value
164223 * If $name is empty, the $value is added as a next list element data[] = $value
 224+ * @return bool True if $value fits in the result, false if not
165225 */
166226 public function addValue($path, $name, $value) {
 227+ global $wgAPIMaxResultSize;
 228+ $data = & $this->mData;
 229+ if( $this->mCheckingSize ) {
 230+ $newsize = $this->mSize + self::size($value);
 231+ if($newsize > $wgAPIMaxResultSize)
 232+ return false;
 233+ $this->mSize = $newsize;
 234+ }
167235
168 - $data = & $this->getData();
169 -
170236 if (!is_null($path)) {
171237 if (is_array($path)) {
172238 foreach ($path as $p) {
@@ -184,9 +250,27 @@
185251 $data[] = $value; // Add list element
186252 else
187253 ApiResult :: setElement($data, $name, $value); // Add named element
 254+ return true;
188255 }
189256
190257 /**
 258+ * Unset a value previously added to the result set.
 259+ * Fails silently if the value isn't found.
 260+ * For parameters, see addValue()
 261+ */
 262+ public function unsetValue($path, $name) {
 263+ $data = & $this->mData;
 264+ if(!is_null($path))
 265+ foreach((array)$path as $p) {
 266+ if(!isset($data[$p]))
 267+ return;
 268+ $data = & $data[$p];
 269+ }
 270+ $this->mSize -= self::size($data[$name]);
 271+ unset($data[$name]);
 272+ }
 273+
 274+ /**
191275 * Ensure all values in this result are valid UTF-8.
192276 */
193277 public function cleanUpUTF8()
Index: trunk/phase3/includes/api/ApiQueryCategoryInfo.php
@@ -29,7 +29,7 @@
3030 }
3131
3232 /**
33 - * This query adds <categories> subelement to all pages with the list of images embedded into those pages.
 33+ * This query adds the <categories> subelement to all pages with the list of categories the page is in
3434 *
3535 * @ingroup API
3636 */
@@ -39,7 +39,8 @@
4040 parent :: __construct($query, $moduleName, 'ci');
4141 }
4242
43 - public function execute() {
 43+ public function execute() {
 44+ $params = $this->extractRequestParams();
4445 $alltitles = $this->getPageSet()->getAllTitlesByNamespace();
4546 if ( empty( $alltitles[NS_CATEGORY] ) ) {
4647 return;
@@ -65,13 +66,20 @@
6667 'pp_propname' => 'hiddencat')),
6768 ));
6869 $this->addFields(array('cat_title', 'cat_pages', 'cat_subcats', 'cat_files', 'pp_propname AS cat_hidden'));
69 - $this->addWhere(array('cat_title' => $cattitles));
 70+ $this->addWhere(array('cat_title' => $cattitles));
 71+ if(!is_null($params['continue']))
 72+ {
 73+ // We need to set a LIMIT in order to be able to set
 74+ // an OFFSET
 75+ $this->addOption('LIMIT', count($titles));
 76+ $this->addOption('OFFSET', $params['continue']);
 77+ }
7078
7179 $db = $this->getDB();
7280 $res = $this->select(__METHOD__);
7381
74 - $data = array();
7582 $catids = array_flip($cattitles);
 83+ $count = (int)@$params['continue'];
7684 while($row = $db->fetchObject($res))
7785 {
7886 $vals = array();
@@ -81,11 +89,29 @@
8290 $vals['subcats'] = $row->cat_subcats;
8391 if($row->cat_hidden)
8492 $vals['hidden'] = '';
85 - $this->addPageSubItems($catids[$row->cat_title], $vals);
 93+ $fit = $this->addPageSubItems($catids[$row->cat_title], $vals);
 94+ if(!$fit)
 95+ {
 96+ $this->setContinueEnumParameter('continue', $count);
 97+ break;
 98+ }
 99+ $count++;
86100 }
87101 $db->freeResult($res);
88102 }
89103
 104+ public function getAllowedParams() {
 105+ return array (
 106+ 'continue' => null,
 107+ );
 108+ }
 109+
 110+ public function getParamDescription() {
 111+ return array (
 112+ 'continue' => 'When more results are available, use this to continue',
 113+ );
 114+ }
 115+
90116 public function getDescription() {
91117 return 'Returns information about the given categories';
92118 }
@@ -97,4 +123,4 @@
98124 public function getVersion() {
99125 return __CLASS__ . ': $Id$';
100126 }
101 -}
 127+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiFormatBase.php
@@ -261,9 +261,13 @@
262262 public static function setResult($result, $feed, $feedItems) {
263263 // Store output in the Result data.
264264 // This way we can check during execution if any error has occured
265 - $data = & $result->getData();
266 - $data['_feed'] = $feed;
267 - $data['_feeditems'] = $feedItems;
 265+ // Disable size checking for this because we can't continue
 266+ // cleanly; size checking would cause more problems than it'd
 267+ // solve
 268+ $result->disableSizeCheck();
 269+ $result->addValue(null, '_feed', $feed);
 270+ $result->addValue(null, '_feeditems', $feedItems);
 271+ $result->enableSizeCheck();
268272 }
269273
270274 /**
@@ -282,8 +286,8 @@
283287
284288 /**
285289 * This class expects the result data to be in a custom format set by self::setResult()
286 - * $result['_feed'] - an instance of one of the $wgFeedClasses classes
287 - * $result['_feeditems'] - an array of FeedItem instances
 290+ * $result['_feed'] - an instance of one of the $wgFeedClasses classes
 291+ * $result['_feeditems'] - an array of FeedItem instances
288292 */
289293 public function execute() {
290294 $data = $this->getResultData();
@@ -304,4 +308,4 @@
305309 public function getVersion() {
306310 return __CLASS__ . ': $Id$';
307311 }
308 -}
 312+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryAllimages.php
@@ -97,7 +97,7 @@
9898
9999 $res = $this->select(__METHOD__);
100100
101 - $data = array ();
 101+ $titles = array();
102102 $count = 0;
103103 $result = $this->getResult();
104104 while ($row = $db->fetchObject($res)) {
@@ -110,20 +110,23 @@
111111
112112 if (is_null($resultPageSet)) {
113113 $file = $repo->newFileFromRow( $row );
114 - $data[] = array_merge(array('name' => $row->img_name),
 114+ $info = array_merge(array('name' => $row->img_name),
115115 ApiQueryImageInfo::getInfo($file, $prop, $result));
 116+ $fit = $result->addValue(array('query', $this->getModuleName()), null, $info);
 117+ if( !$fit ) {
 118+ $this->setContinueEnumParameter('from', $this->keyToTitle($row->img_name));
 119+ break;
 120+ }
116121 } else {
117 - $data[] = Title::makeTitle(NS_FILE, $row->img_name);
 122+ $titles[] = Title::makeTitle(NS_IMAGE, $row->img_name);
118123 }
119124 }
120125 $db->freeResult($res);
121126
122127 if (is_null($resultPageSet)) {
123 - $result = $this->getResult();
124 - $result->setIndexedTagName($data, 'img');
125 - $result->addValue('query', $this->getModuleName(), $data);
 128+ $result->setIndexedTagName_internal(array('query', $this->getModuleName()), 'img');
126129 } else {
127 - $resultPageSet->populateFromTitles( $data );
 130+ $resultPageSet->populateFromTitles($titles);
128131 }
129132 }
130133
Index: trunk/phase3/includes/api/ApiQueryAllmessages.php
@@ -75,6 +75,9 @@
7676 //Get all requested messages
7777 $messages = array();
7878 foreach( $messages_target as $message ) {
 79+ if(!is_null($params['from']))
 80+ if($message < $params['from'])
 81+ continue;
7982 $messages[$message] = wfMsg( $message );
8083 }
8184
@@ -89,10 +92,14 @@
9093 } else {
9194 $result->setContent( $message, $value );
9295 }
93 - $messages_out[] = $message;
 96+ $fit = $result->addValue(array('query', $this->getModuleName()), null, $message);
 97+ if(!$fit)
 98+ {
 99+ $this->setContinueEnumParameter('from', $name);
 100+ break;
 101+ }
94102 }
95 - $result->setIndexedTagName( $messages_out, 'message' );
96 - $result->addValue( 'query', $this->getModuleName(), $messages_out );
 103+ $result->setIndexedTagName_internal(array('query', $this->getModuleName()), 'message');
97104 }
98105
99106 public function getAllowedParams() {
@@ -102,6 +109,7 @@
103110 ),
104111 'filter' => array(),
105112 'lang' => null,
 113+ 'from' => null,
106114 );
107115 }
108116
@@ -110,6 +118,7 @@
111119 'messages' => 'Which messages to output. "*" means all messages',
112120 'filter' => 'Return only messages that contain this string',
113121 'lang' => 'Return messages in this language',
 122+ 'from' => 'Return messages starting at this message',
114123 );
115124 }
116125
@@ -127,4 +136,4 @@
128137 public function getVersion() {
129138 return __CLASS__ . ': $Id$';
130139 }
131 -}
 140+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQuerySearch.php
@@ -87,7 +87,7 @@
8888 $this->dieUsage("{$what} search is disabled",
8989 "search-{$what}-disabled");
9090
91 - $data = array ();
 91+ $titles = array ();
9292 $count = 0;
9393 while( $result = $matches->next() ) {
9494 if (++ $count > $limit) {
@@ -102,20 +102,24 @@
103103
104104 $title = $result->getTitle();
105105 if (is_null($resultPageSet)) {
106 - $data[] = array(
 106+ $vals = array(
107107 'ns' => intval($title->getNamespace()),
108108 'title' => $title->getPrefixedText());
 109+ $fit = $this->getResult()->addValue(array('query', $this->getModuleName()), null, $vals);
 110+ if(!$fit)
 111+ {
 112+ $this->setContinueEnumParameter('offset', $params['offset'] + $count - 1);
 113+ break;
 114+ }
109115 } else {
110 - $data[] = $title;
 116+ $titles[] = $title;
111117 }
112118 }
113119
114120 if (is_null($resultPageSet)) {
115 - $result = $this->getResult();
116 - $result->setIndexedTagName($data, 'p');
117 - $result->addValue('query', $this->getModuleName(), $data);
 121+ $this->getResult()->setIndexedTagName_internal(array('query', $this->getModuleName()), 'p');
118122 } else {
119 - $resultPageSet->populateFromTitles($data);
 123+ $resultPageSet->populateFromTitles($titles);
120124 }
121125 }
122126
@@ -172,4 +176,4 @@
173177 public function getVersion() {
174178 return __CLASS__ . ': $Id$';
175179 }
176 -}
 180+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryLogEvents.php
@@ -125,7 +125,6 @@
126126 $this->addWhere('log_deleted & ' . LogPage::DELETED_USER . ' = 0');
127127 }
128128
129 - $data = array ();
130129 $count = 0;
131130 $res = $this->select(__METHOD__);
132131 while ($row = $db->fetchObject($res)) {
@@ -136,13 +135,18 @@
137136 }
138137
139138 $vals = $this->extractRowInfo($row);
140 - if($vals)
141 - $data[] = $vals;
 139+ if(!$vals)
 140+ continue;
 141+ $fit = $this->getResult()->addValue(array('query', $this->getModuleName()), null, $vals);
 142+ if(!$fit)
 143+ {
 144+ $this->setContinueEnumParameter('start', wfTimestamp(TS_ISO_8601, $row->log_timestamp));
 145+ break;
 146+ }
142147 }
143148 $db->freeResult($res);
144149
145 - $this->getResult()->setIndexedTagName($data, 'item');
146 - $this->getResult()->addValue('query', $this->getModuleName(), $data);
 150+ $this->getResult()->setIndexedTagName_internal(array('query', $this->getModuleName()), 'item');
147151 }
148152
149153 public static function addLogParams($result, &$vals, $params, $type, $ts) {
Index: trunk/phase3/includes/api/ApiQueryWatchlistRaw.php
@@ -89,7 +89,6 @@
9090 $res = $this->select(__METHOD__);
9191
9292 $db = $this->getDB();
93 - $data = array();
9493 $titles = array();
9594 $count = 0;
9695 while($row = $db->fetchObject($res))
@@ -108,16 +107,19 @@
109108 ApiQueryBase::addTitleInfo($vals, $t);
110109 if(isset($prop['changed']) && !is_null($row->wl_notificationtimestamp))
111110 $vals['changed'] = wfTimestamp(TS_ISO_8601, $row->wl_notificationtimestamp);
112 - $data[] = $vals;
 111+ $fit = $this->getResult()->addValue($this->getModuleName(), null, $vals);
 112+ if(!$fit)
 113+ {
 114+ $this->setContinueEnumParameter('continue', $row->wl_namespace . '|' .
 115+ $this->keyToTitle($row->wl_title));
 116+ break;
 117+ }
113118 }
114119 else
115120 $titles[] = $t;
116121 }
117122 if(is_null($resultPageSet))
118 - {
119 - $this->getResult()->setIndexedTagName($data, 'wr');
120 - $this->getResult()->addValue(null, $this->getModuleName(), $data);
121 - }
 123+ $this->getResult()->setIndexedTagName_internal($this->getModuleName(), 'wr');
122124 else
123125 $resultPageSet->populateFromTitles($titles);
124126 }
@@ -176,4 +178,4 @@
177179 public function getVersion() {
178180 return __CLASS__ . ': $Id$';
179181 }
180 -}
 182+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQuery.php
@@ -255,6 +255,10 @@
256256 $pageSet = $this->getPageSet();
257257 $result = $this->getResult();
258258
 259+ # We don't check for a full result set here because we can't be adding
 260+ # more than 380K. The maximum revision size is in the megabyte range,
 261+ # and the maximum result size must be even higher than that.
 262+
259263 // Title normalizations
260264 $normValues = array ();
261265 foreach ($pageSet->getNormalizedTitles() as $rawTitleStr => $titleStr) {
@@ -368,6 +372,10 @@
369373 $exporter->closeStream();
370374 $exportxml = ob_get_contents();
371375 ob_end_clean();
 376+ // Don't check the size of exported stuff
 377+ // It's not continuable, so it would cause more
 378+ // problems than it'd solve
 379+ $result->disableSizeCheck();
372380 if ($this->params['exportnowrap']) {
373381 $result->reset();
374382 // Raw formatter will handle this
@@ -378,6 +386,7 @@
379387 ApiResult::setContent($r, $exportxml);
380388 $result->addValue('query', 'export', $r);
381389 }
 390+ $result->enableSizeCheck();
382391 }
383392 }
384393 }
@@ -552,4 +561,4 @@
553562 $vers[] = $psModule->getVersion();
554563 return $vers;
555564 }
556 -}
 565+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryLangLinks.php
@@ -71,8 +71,6 @@
7272 $this->addOption('LIMIT', $params['limit'] + 1);
7373 $res = $this->select(__METHOD__);
7474
75 - $data = array();
76 - $lastId = 0; // database has no ID 0
7775 $count = 0;
7876 $db = $this->getDB();
7977 while ($row = $db->fetchObject($res)) {
@@ -82,23 +80,15 @@
8381 $this->setContinueEnumParameter('continue', "{$row->ll_from}|{$row->ll_lang}");
8482 break;
8583 }
86 - if ($lastId != $row->ll_from) {
87 - if($lastId != 0) {
88 - $this->addPageSubItems($lastId, $data);
89 - $data = array();
90 - }
91 - $lastId = $row->ll_from;
92 - }
93 -
9484 $entry = array('lang' => $row->ll_lang);
9585 ApiResult :: setContent($entry, $row->ll_title);
96 - $data[] = $entry;
 86+ $fit = $this->addPageSubItem($row->ll_from, $entry);
 87+ if(!$fit)
 88+ {
 89+ $this->setContinueEnumParameter('continue', "{$row->ll_from}|{$row->ll_lang}");
 90+ break;
 91+ }
9792 }
98 -
99 - if($lastId != 0) {
100 - $this->addPageSubItems($lastId, $data);
101 - }
102 -
10393 $db->freeResult($res);
10494 }
10595
@@ -136,4 +126,4 @@
137127 public function getVersion() {
138128 return __CLASS__ . ': $Id$';
139129 }
140 -}
 130+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiMain.php
@@ -361,9 +361,11 @@
362362 }
363363
364364 $this->getResult()->reset();
 365+ $this->getResult()->disableSizeCheck();
365366 // Re-add the id
366 - if($this->mRequest->getCheck('requestid'))
367 - $this->getResult()->addValue(null, 'requestid', $this->mRequest->getVal('requestid'));
 367+ $requestid = $this->getParameter('requestid');
 368+ if(!is_null($requestid))
 369+ $this->getResult()->addValue(null, 'requestid', $requestid);
368370 $this->getResult()->addValue(null, 'error', $errMessage);
369371
370372 return $errMessage['code'];
@@ -374,8 +376,9 @@
375377 */
376378 protected function executeAction() {
377379 // First add the id to the top element
378 - if($this->mRequest->getCheck('requestid'))
379 - $this->getResult()->addValue(null, 'requestid', $this->mRequest->getVal('requestid'));
 380+ $requestid = $this->getParameter('requestid');
 381+ if(!is_null($requestid))
 382+ $this->getResult()->addValue(null, 'requestid', $requestid);
380383
381384 $params = $this->extractRequestParams();
382385
@@ -719,4 +722,4 @@
720723 public function __toString() {
721724 return "{$this->getCodeString()}: {$this->getMessage()}";
722725 }
723 -}
 726+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryImageInfo.php
@@ -56,49 +56,97 @@
5757 }
5858
5959 $pageIds = $this->getPageSet()->getAllTitlesByNamespace();
 60+ $cnt = 0;
6061 if (!empty($pageIds[NS_FILE])) {
61 -
6262 $result = $this->getResult();
6363 $images = RepoGroup::singleton()->findFiles( array_keys( $pageIds[NS_FILE] ) );
6464 foreach ( $images as $img ) {
65 - $data = array();
66 -
 65+ $cnt++;
 66+ if(!is_null($params['continue']))
 67+ if($cnt < $params['continue'])
 68+ continue;
 69+ $pageId = $pageIds[NS_IMAGE][ $img->getOriginalTitle()->getDBkey() ];
 70+
 71+ $fit = $result->addValue(
 72+ array('query', 'pages', intval($pageId)),
 73+ 'imagerepository', $img->getRepoName()
 74+ );
 75+ if(!$fit)
 76+ {
 77+ if(count($pageIds[NS_IMAGE]) == 1)
 78+ # The user is screwed. imageinfo can't be solely
 79+ # responsible for exceeding the limit in this case,
 80+ # so set a query-continue that just returns the same
 81+ # thing again. When the violating queries have been
 82+ # out-continued, the result will get through
 83+ $this->setContinueEnumParameter('start',
 84+ wfTimestamp(TS_ISO_8601, $img->getTimestamp()));
 85+ else
 86+ $this->setContinueEnumParameter('continue', $cnt);
 87+ break;
 88+ }
 89+
6790 // Get information about the current version first
6891 // Check that the current version is within the start-end boundaries
 92+ $gotOne = false;
6993 if((is_null($params['start']) || $img->getTimestamp() <= $params['start']) &&
7094 (is_null($params['end']) || $img->getTimestamp() >= $params['end'])) {
71 - $data[] = self::getInfo( $img, $prop, $result, $scale );
 95+ $gotOne = true;
 96+ $fit = $this->addPageSubItem($pageId,
 97+ self::getInfo( $img, $prop, $result, $scale));
 98+ if(!$fit)
 99+ {
 100+ if(count($pageIds[NS_IMAGE]) == 1)
 101+ # See the 'the user is screwed' comment above
 102+ $this->setContinueEnumParameter('start',
 103+ wfTimestamp(TS_ISO_8601, $img->getTimestamp()));
 104+ else
 105+ $this->setContinueEnumParameter('continue', $cnt);
 106+ break;
 107+ }
72108 }
73109
74110 // Now get the old revisions
75111 // Get one more to facilitate query-continue functionality
76 - $count = count($data);
 112+ $count = ($gotOne ? 1 : 0);
77113 $oldies = $img->getHistory($params['limit'] - $count + 1, $params['start'], $params['end']);
78114 foreach($oldies as $oldie) {
79115 if(++$count > $params['limit']) {
80116 // We've reached the extra one which shows that there are additional pages to be had. Stop here...
81117 // Only set a query-continue if there was only one title
82118 if(count($pageIds[NS_FILE]) == 1)
83 - $this->setContinueEnumParameter('start', $oldie->getTimestamp());
 119+ $this->setContinueEnumParameter('start', wfTimestamp(TS_ISO_8601, $oldie->getTimestamp()));
84120 break;
85121 }
86 - $data[] = self::getInfo( $oldie, $prop, $result );
 122+ $fit = $this->addPageSubItem($pageId,
 123+ self::getInfo($oldie, $prop, $result));
 124+ if(!$fit)
 125+ {
 126+ if(count($pageIds[NS_IMAGE]) == 1)
 127+ $this->setContinueEnumParameter('start',
 128+ wfTimestamp(TS_ISO_8601, $oldie->getTimestamp()));
 129+ else
 130+ $this->setContinueEnumParameter('continue', $cnt);
 131+ break;
 132+ }
87133 }
88 -
89 - $pageId = $pageIds[NS_FILE][ $img->getOriginalTitle()->getDBkey() ];
90 - $result->addValue(
91 - array( 'query', 'pages', intval( $pageId ) ),
92 - 'imagerepository', $img->getRepoName()
93 - );
94 - $this->addPageSubItems($pageId, $data);
 134+ if(!$fit)
 135+ break;
95136 }
96137
97138 $missing = array_diff( array_keys( $pageIds[NS_FILE] ), array_keys( $images ) );
98 - foreach ( $missing as $title )
99 - $result->addValue(
100 - array( 'query', 'pages', intval( $pageIds[NS_FILE][$title] ) ),
 139+ foreach ($missing as $title) {
 140+ $cnt++;
 141+ if(!is_null($params['continue']))
 142+ if($count < $params['continue'])
 143+ continue;
 144+ $fit = $result->addValue(
 145+ array('query', 'pages', intval($pageIds[NS_FILE][$title])),
101146 'imagerepository', ''
102147 );
 148+ if(!$fit)
 149+ $this->setContinueEnumParameter('continue', $cnt);
 150+ }
103151 }
104152 }
105153
@@ -208,7 +256,8 @@
209257 'urlheight' => array(
210258 ApiBase :: PARAM_TYPE => 'integer',
211259 ApiBase :: PARAM_DFLT => -1
212 - )
 260+ ),
 261+ 'continue' => null,
213262 );
214263 }
215264
@@ -221,6 +270,7 @@
222271 'urlwidth' => array('If iiprop=url is set, a URL to an image scaled to this width will be returned.',
223272 'Only the current version of the image can be scaled.'),
224273 'urlheight' => 'Similar to iiurlwidth. Cannot be used without iiurlwidth',
 274+ 'continue' => 'When more results are available, use this to continue',
225275 );
226276 }
227277
Index: trunk/phase3/includes/api/ApiQueryWatchlist.php
@@ -168,7 +168,7 @@
169169
170170 $this->addOption('LIMIT', $params['limit'] +1);
171171
172 - $data = array ();
 172+ $ids = array ();
173173 $count = 0;
174174 $res = $this->select(__METHOD__);
175175
@@ -182,13 +182,18 @@
183183
184184 if (is_null($resultPageSet)) {
185185 $vals = $this->extractRowInfo($row);
186 - if ($vals)
187 - $data[] = $vals;
 186+ $fit = $this->getResult()->addValue(array('query', $this->getModuleName()), null, $vals);
 187+ if(!$fit)
 188+ {
 189+ $this->setContinueEnumParameter('start',
 190+ wfTimestamp(TS_ISO_8601, $row->rc_timestamp));
 191+ break;
 192+ }
188193 } else {
189194 if ($params['allrev']) {
190 - $data[] = intval($row->rc_this_oldid);
 195+ $ids[] = intval($row->rc_this_oldid);
191196 } else {
192 - $data[] = intval($row->rc_cur_id);
 197+ $ids[] = intval($row->rc_cur_id);
193198 }
194199 }
195200 }
@@ -196,13 +201,12 @@
197202 $db->freeResult($res);
198203
199204 if (is_null($resultPageSet)) {
200 - $this->getResult()->setIndexedTagName($data, 'item');
201 - $this->getResult()->addValue('query', $this->getModuleName(), $data);
 205+ $this->getResult()->setIndexedTagName_internal(array('query', $this->getModuleName()), 'item');
202206 }
203207 elseif ($params['allrev']) {
204 - $resultPageSet->populateFromRevisionIDs($data);
 208+ $resultPageSet->populateFromRevisionIDs($ids);
205209 } else {
206 - $resultPageSet->populateFromPageIDs($data);
 210+ $resultPageSet->populateFromPageIDs($ids);
207211 }
208212 }
209213
@@ -340,4 +344,4 @@
341345 public function getVersion() {
342346 return __CLASS__ . ': $Id$';
343347 }
344 -}
 348+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryRevisions.php
@@ -231,6 +231,8 @@
232232 ApiBase :: dieDebug(__METHOD__, 'param validation?');
233233
234234 $this->addOption('LIMIT', $limit +1);
 235+ if(!is_null($continue))
 236+ $this->addOption('OFFSET', $continue);
235237
236238 $data = array ();
237239 $count = 0;
@@ -246,29 +248,19 @@
247249 $this->setContinueEnumParameter('startid', intval($row->rev_id));
248250 break;
249251 }
250 -
251252 $revision = new Revision( $row );
252 - $this->getResult()->addValue(
253 - array (
254 - 'query',
255 - 'pages',
256 - $revision->getPage(),
257 - 'revisions'),
258 - null,
259 - $this->extractRowInfo( $revision ));
260 - }
261 - $db->freeResult($res);
262 -
263 - // Ensure that all revisions are shown as '<rev>' elements
264 - $result = $this->getResult();
265 - if ($result->getIsRawMode()) {
266 - $data =& $result->getData();
267 - foreach ($data['query']['pages'] as & $page) {
268 - if (is_array($page) && array_key_exists('revisions', $page)) {
269 - $result->setIndexedTagName($page['revisions'], 'rev');
270 - }
 253+ //
 254+ $fit = $this->addPageSubItem($revision->getPage(), $this->extractRowInfo($revision), 'rev');
 255+ if(!$fit)
 256+ {
 257+ if($enumRevMode)
 258+ $this->setContinueEnumParameter('startid', intval($row->rev_id));
 259+ else
 260+ $this->setContinueEnumParameter('continue', $continue + $count - 1);
 261+ break;
271262 }
272263 }
 264+ $db->freeResult($res);
273265 }
274266
275267 private function extractRowInfo( $revision ) {
@@ -412,6 +404,7 @@
413405 ApiBase :: PARAM_TYPE => array_keys($this->getTokenFunctions()),
414406 ApiBase :: PARAM_ISMULTI => true
415407 ),
 408+ 'continue' => null,
416409 );
417410 }
418411
@@ -430,6 +423,7 @@
431424 'generatexml' => 'generate XML parse tree for revision content',
432425 'section' => 'only retrieve the content of this section',
433426 'token' => 'Which tokens to obtain for each revision',
 427+ 'continue' => 'When more results are available, use this to continue',
434428 );
435429 }
436430
Index: trunk/phase3/includes/api/ApiQueryExternalLinks.php
@@ -61,8 +61,6 @@
6262 $db = $this->getDB();
6363 $res = $this->select(__METHOD__);
6464
65 - $data = array();
66 - $lastId = 0; // database has no ID 0
6765 $count = 0;
6866 while ($row = $db->fetchObject($res)) {
6967 if (++$count > $params['limit']) {
@@ -71,23 +69,15 @@
7270 $this->setContinueEnumParameter('offset', @$params['offset'] + $params['limit']);
7371 break;
7472 }
75 - if ($lastId != $row->el_from) {
76 - if($lastId != 0) {
77 - $this->addPageSubItems($lastId, $data);
78 - $data = array();
79 - }
80 - $lastId = $row->el_from;
81 - }
82 -
8373 $entry = array();
8474 ApiResult :: setContent($entry, $row->el_to);
85 - $data[] = $entry;
 75+ $fit = $this->addPageSubItem($row->el_from, $entry);
 76+ if(!$fit)
 77+ {
 78+ $this->setContinueEnumParameter('offset', @$params['offset'] + $count - 1);
 79+ break;
 80+ }
8681 }
87 -
88 - if($lastId != 0) {
89 - $this->addPageSubItems($lastId, $data);
90 - }
91 -
9282 $db->freeResult($res);
9383 }
9484
@@ -125,4 +115,4 @@
126116 public function getVersion() {
127117 return __CLASS__ . ': $Id$';
128118 }
129 -}
 119+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiBase.php
@@ -147,7 +147,7 @@
148148 /**
149149 * Get the result data array
150150 */
151 - public function & getResultData() {
 151+ public function getResultData() {
152152 return $this->getResult()->getData();
153153 }
154154
@@ -156,20 +156,23 @@
157157 * notice any changes in API.
158158 */
159159 public function setWarning($warning) {
160 - # If there is a warning already, append it to the existing one
161 - $data =& $this->getResult()->getData();
 160+ $data = $this->getResult()->getData();
162161 if(isset($data['warnings'][$this->getModuleName()]))
163162 {
164163 # Don't add duplicate warnings
165164 $warn_regex = preg_quote($warning, '/');
166165 if(preg_match("/{$warn_regex}(\\n|$)/", $data['warnings'][$this->getModuleName()]['*']))
167166 return;
168 - $warning = "{$data['warnings'][$this->getModuleName()]['*']}\n$warning";
169 - unset($data['warnings'][$this->getModuleName()]);
 167+ $oldwarning = $data['warnings'][$this->getModuleName()]['*'];
 168+ # If there is a warning already, append it to the existing one
 169+ $warning = "$oldwarning\n$warning";
 170+ $this->getResult()->unsetValue('warnings', $this->getModuleName());
170171 }
171172 $msg = array();
172173 ApiResult :: setContent($msg, $warning);
 174+ $this->getResult()->disableSizeCheck();
173175 $this->getResult()->addValue('warnings', $this->getModuleName(), $msg);
 176+ $this->getResult()->enableSizeCheck();
174177 }
175178
176179 /**
@@ -905,4 +908,4 @@
906909 public static function getBaseVersion() {
907910 return __CLASS__ . ': $Id$';
908911 }
909 -}
 912+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryCategories.php
@@ -137,8 +137,6 @@
138138
139139 if (is_null($resultPageSet)) {
140140
141 - $data = array();
142 - $lastId = 0; // database has no ID 0
143141 $count = 0;
144142 while ($row = $db->fetchObject($res)) {
145143 if (++$count > $params['limit']) {
@@ -148,16 +146,8 @@
149147 '|' . $this->keyToTitle($row->cl_to));
150148 break;
151149 }
152 - if ($lastId != $row->cl_from) {
153 - if($lastId != 0) {
154 - $this->addPageSubItems($lastId, $data);
155 - $data = array();
156 - }
157 - $lastId = $row->cl_from;
158 - }
159150
160151 $title = Title :: makeTitle(NS_CATEGORY, $row->cl_to);
161 -
162152 $vals = array();
163153 ApiQueryBase :: addTitleInfo($vals, $title);
164154 if ($fld_sortkey)
@@ -165,13 +155,14 @@
166156 if ($fld_timestamp)
167157 $vals['timestamp'] = wfTimestamp(TS_ISO_8601, $row->cl_timestamp);
168158
169 - $data[] = $vals;
 159+ $fit = $this->addPageSubItem($row->cl_from, $vals);
 160+ if(!$fit)
 161+ {
 162+ $this->setContinueEnumParameter('continue', $row->cl_from .
 163+ '|' . $this->keyToTitle($row->cl_to));
 164+ break;
 165+ }
170166 }
171 -
172 - if($lastId != 0) {
173 - $this->addPageSubItems($lastId, $data);
174 - }
175 -
176167 } else {
177168
178169 $titles = array();
@@ -248,4 +239,4 @@
249240 public function getVersion() {
250241 return __CLASS__ . ': $Id$';
251242 }
252 -}
 243+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryAllUsers.php
@@ -127,7 +127,16 @@
128128 if (!$row || $lastUser !== $row->user_name) {
129129 // Save the last pass's user data
130130 if (is_array($lastUserData))
131 - $data[] = $lastUserData;
 131+ {
 132+ $fit = $result->addValue(array('query', $this->getModuleName()),
 133+ null, $lastUserData);
 134+ if(!$fit)
 135+ {
 136+ $this->setContinueEnumParameter('from',
 137+ $this->keyToTitle($lastUserData['name']));
 138+ break;
 139+ }
 140+ }
132141
133142 // No more rows left
134143 if (!$row)
@@ -169,8 +178,7 @@
170179
171180 $db->freeResult($res);
172181
173 - $result->setIndexedTagName($data, 'u');
174 - $result->addValue('query', $this->getModuleName(), $data);
 182+ $result->setIndexedTagName_internal(array('query', $this->getModuleName()), 'u');
175183 }
176184
177185 public function getAllowedParams() {
@@ -226,4 +234,4 @@
227235 public function getVersion() {
228236 return __CLASS__ . ': $Id$';
229237 }
230 -}
 238+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryInfo.php
@@ -434,7 +434,12 @@
435435 }
436436 }
437437
 438+ $count = 0;
438439 foreach ( $titles as $pageid => $title ) {
 440+ $count++;
 441+ if(!is_null($params['continue']))
 442+ if($count < $params['continue'])
 443+ continue;
439444 $pageInfo = array (
440445 'touched' => wfTimestamp(TS_ISO_8601, $pageTouched[$pageid]),
441446 'lastrevid' => intval($pageLatest[$pageid]),
@@ -480,18 +485,27 @@
481486 if($title->userCanRead())
482487 $pageInfo['readable'] = '';
483488
484 - $result->addValue(array (
 489+ $fit = $result->addValue(array (
485490 'query',
486491 'pages'
487492 ), $pageid, $pageInfo);
 493+ if(!$fit)
 494+ {
 495+ $this->setContinueEnumParameter('continue', $count);
 496+ break;
 497+ }
488498 }
489499
490500 // Get properties for missing titles if requested
491501 if(!is_null($params['token']) || $fld_protection || $fld_talkid || $fld_subjectid ||
492502 $fld_url || $fld_readable)
493503 {
494 - $res = &$result->getData();
495504 foreach($missing as $pageid => $title) {
 505+ $count++;
 506+ if(!is_null($params['continue']))
 507+ if($count < $params['continue'])
 508+ continue;
 509+ $fit = true;
496510 if(!is_null($params['token']))
497511 {
498512 $tokenFunctions = $this->getTokenFunctions();
@@ -502,30 +516,45 @@
503517 if($val === false)
504518 $this->setWarning("Action '$t' is not allowed for the current user");
505519 else
506 - $res['query']['pages'][$pageid][$t . 'token'] = $val;
 520+ $fit = $result->addValue(
 521+ array('query', 'pages', $pageid),
 522+ $t . 'token', $val);
507523 }
508524 }
509 - if($fld_protection)
 525+ if($fld_protection && $fit)
510526 {
511527 // Apparently the XML formatting code doesn't like array(null)
512528 // This is painful to fix, so we'll just work around it
513529 if(isset($prottitles[$title->getNamespace()][$title->getDBkey()]))
514 - $res['query']['pages'][$pageid]['protection'] = $prottitles[$title->getNamespace()][$title->getDBkey()];
 530+ $val = $prottitles[$title->getNamespace()][$title->getDBkey()];
515531 else
516 - $res['query']['pages'][$pageid]['protection'] = array();
517 - $result->setIndexedTagName($res['query']['pages'][$pageid]['protection'], 'pr');
 532+ $val = array();
 533+ $result->setIndexedTagName($val, 'pr');
 534+ $fit = $result->addValue(
 535+ array('query', 'pages', $pageid),
 536+ 'protection', $val);
518537 }
519 - if($fld_talkid && isset($talkids[$title->getNamespace()][$title->getDBKey()]))
520 - $res['query']['pages'][$pageid]['talkid'] = $talkids[$title->getNamespace()][$title->getDBKey()];
521 - if($fld_subjectid && isset($subjectids[$title->getNamespace()][$title->getDBKey()]))
522 - $res['query']['pages'][$pageid]['subjectid'] = $subjectids[$title->getNamespace()][$title->getDBKey()];
523 - if($fld_url) {
524 - $res['query']['pages'][$pageid]['fullurl'] = $title->getFullURL();
525 - $res['query']['pages'][$pageid]['editurl'] = $title->getFullURL('action=edit');
 538+ if($fld_talkid && isset($talkids[$title->getNamespace()][$title->getDbKey()]) && $fit)
 539+ $fit = $result->addValue(array('query', 'pages', $pageid), 'talkid',
 540+ $talkids[$title->getNamespace()][$title->getDbKey()]);
 541+ if($fld_subjectid && isset($subjectids[$title->getNamespace()][$title->getDbKey()]) && $fit)
 542+ $fit = $result->addValue(array('query', 'pages', $pageid), 'subjectid',
 543+ $subjectids[$title->getNamespace()][$title->getDbKey()]);
 544+ if($fld_url && $fit) {
 545+ $fit = $result->addValue(array('query', 'pages', $pageid), 'fullurl',
 546+ $title->getFullURL());
 547+ if($fit)
 548+ $fit = $result->addValue(array('query', 'pages', $pageid), 'editurl',
 549+ $title->getFullURL('action=edit'));
526550 }
527 - if($fld_readable)
 551+ if($fld_readable && $fit)
528552 if($title->userCanRead())
529 - $res['query']['pages'][$pageid]['readable'] = '';
 553+ $fit = $result->addValue(array('query', 'pages', $pageid), 'readable', '');
 554+ if(!$fit)
 555+ {
 556+ $this->setContinueEnumParameter('continue', $count);
 557+ break;
 558+ }
530559 }
531560 }
532561 }
@@ -546,7 +575,8 @@
547576 ApiBase :: PARAM_DFLT => NULL,
548577 ApiBase :: PARAM_ISMULTI => true,
549578 ApiBase :: PARAM_TYPE => array_keys($this->getTokenFunctions())
550 - )
 579+ ),
 580+ 'continue' => null,
551581 );
552582 }
553583
@@ -559,6 +589,7 @@
560590 ' subjectid - The page ID of the parent page for each talk page'
561591 ),
562592 'token' => 'Request a token to perform a data-modifying action on a page',
 593+ 'continue' => 'When more results are available, use this to continue',
563594 );
564595 }
565596
Index: trunk/phase3/includes/api/ApiQueryDuplicateFiles.php
@@ -86,9 +86,7 @@
8787 $res = $this->select(__METHOD__);
8888 $db = $this->getDB();
8989 $count = 0;
90 - $data = array();
9190 $titles = array();
92 - $lastName = '';
9391 while($row = $db->fetchObject($res))
9492 {
9593 if(++$count > $params['limit'])
@@ -104,27 +102,23 @@
105103 $titles[] = Title::makeTitle(NS_FILE, $row->dup_name);
106104 else
107105 {
108 - if($row->orig_name != $lastName)
109 - {
110 - if($lastName != '')
111 - {
112 - $this->addPageSubItems($images[$lastName], $data);
113 - $data = array();
114 - }
115 - $lastName = $row->orig_name;
116 - }
117 -
118 - $data[] = array(
 106+ $r = array(
119107 'name' => $row->dup_name,
120108 'user' => $row->dup_user_text,
121109 'timestamp' => wfTimestamp(TS_ISO_8601, $row->dup_timestamp)
122110 );
 111+ $fit = $this->addPageSubItem($images[$row->orig_name], $r);
 112+ if(!$fit)
 113+ {
 114+ $this->setContinueEnumParameter('continue',
 115+ $this->keyToTitle($row->orig_name) . '|' .
 116+ $this->keyToTitle($row->dup_name));
 117+ break;
 118+ }
123119 }
124120 }
125121 if(!is_null($resultPageSet))
126122 $resultPageSet->populateFromTitles($titles);
127 - else if($lastName != '')
128 - $this->addPageSubItems($images[$lastName], $data);
129123 $db->freeResult($res);
130124 }
131125
Index: trunk/phase3/includes/api/ApiQueryUsers.php
@@ -51,71 +51,72 @@
5252 $this->prop = array();
5353 }
5454
55 - if(is_array($params['users'])) {
56 - $r = $this->getOtherUsersInfo($params['users']);
57 - $result->setIndexedTagName($r, 'user');
58 - }
59 - $result->addValue("query", $this->getModuleName(), $r);
60 - }
61 -
62 - protected function getOtherUsersInfo($users) {
63 - $goodNames = $retval = array();
 55+ $users = (array)$params['users'];
 56+ $goodNames = $done = array();
 57+ $result = $this->getResult();
6458 // Canonicalize user names
6559 foreach($users as $u) {
6660 $n = User::getCanonicalName($u);
6761 if($n === false || $n === '')
68 - $retval[] = array('name' => $u, 'invalid' => '');
 62+ {
 63+ $vals = array('name' => $u, 'invalid' => '');
 64+ $fit = $result->addValue(array('query', $this->getModuleName()),
 65+ null, $vals);
 66+ if(!$fit)
 67+ {
 68+ $this->setContinueEnumParameter('users',
 69+ implode('|', array_diff($users, $done)));
 70+ $goodNames = array();
 71+ break;
 72+ }
 73+ $done[] = $u;
 74+ }
6975 else
7076 $goodNames[] = $n;
7177 }
72 - if(!count($goodNames))
73 - return $retval;
 78+ if(count($goodNames))
 79+ {
 80+ $db = $this->getDb();
 81+ $this->addTables('user', 'u1');
 82+ $this->addFields('u1.user_name');
 83+ $this->addWhereFld('u1.user_name', $goodNames);
 84+ $this->addFieldsIf('u1.user_editcount', isset($this->prop['editcount']));
 85+ $this->addFieldsIf('u1.user_registration', isset($this->prop['registration']));
7486
75 - $db = $this->getDB();
76 - $this->addTables('user', 'u1');
77 - $this->addFields('u1.*');
78 - $this->addWhereFld('u1.user_name', $goodNames);
 87+ if(isset($this->prop['groups'])) {
 88+ $this->addTables('user_groups');
 89+ $this->addJoinConds(array('user_groups' => array('LEFT JOIN', 'ug_user=u1.user_id')));
 90+ $this->addFields('ug_group');
 91+ }
 92+ if(isset($this->prop['blockinfo'])) {
 93+ $this->addTables('ipblocks');
 94+ $this->addTables('user', 'u2');
 95+ $u2 = $this->getAliasedName('user', 'u2');
 96+ $this->addJoinConds(array(
 97+ 'ipblocks' => array('LEFT JOIN', 'ipb_user=u1.user_id'),
 98+ $u2 => array('LEFT JOIN', 'ipb_by=u2.user_id')));
 99+ $this->addFields(array('ipb_reason', 'u2.user_name blocker_name'));
 100+ }
79101
80 - if(isset($this->prop['groups'])) {
81 - $this->addTables('user_groups');
82 - $this->addJoinConds(array('user_groups' => array('LEFT JOIN', 'ug_user=u1.user_id')));
83 - $this->addFields('ug_group');
 102+ $data = array();
 103+ $res = $this->select(__METHOD__);
 104+ while(($r = $db->fetchObject($res))) {
 105+ $data[$r->user_name]['name'] = $r->user_name;
 106+ if(isset($this->prop['editcount']))
 107+ $data[$r->user_name]['editcount'] = $r->user_editcount;
 108+ if(isset($this->prop['registration']))
 109+ $data[$r->user_name]['registration'] = wfTimestampOrNull(TS_ISO_8601, $r->user_registration);
 110+ if(isset($this->prop['groups']))
 111+ // This row contains only one group, others will be added from other rows
 112+ if(!is_null($r->ug_group))
 113+ $data[$r->user_name]['groups'][] = $r->ug_group;
 114+ if(isset($this->prop['blockinfo']))
 115+ if(!is_null($r->blocker_name)) {
 116+ $data[$r->user_name]['blockedby'] = $r->blocker_name;
 117+ $data[$r->user_name]['blockreason'] = $r->ipb_reason;
 118+ }
 119+ }
84120 }
85 - if(isset($this->prop['blockinfo'])) {
86 - $this->addTables('ipblocks');
87 - $this->addTables('user', 'u2');
88 - $u2 = $this->getAliasedName('user', 'u2');
89 - $this->addJoinConds(array(
90 - 'ipblocks' => array('LEFT JOIN', 'ipb_user=u1.user_id'),
91 - $u2 => array('LEFT JOIN', 'ipb_by=u2.user_id')));
92 - $this->addFields(array('ipb_reason', 'u2.user_name blocker_name'));
93 - }
94 -
95 - $data = array();
96 - $res = $this->select(__METHOD__);
97 - while(($r = $db->fetchObject($res))) {
98 - $user = User::newFromRow($r);
99 - $name = $user->getName();
100 - $data[$name]['name'] = $name;
101 - if(isset($this->prop['editcount']))
102 - // No proper member function in User class for this
103 - $data[$name]['editcount'] = $r->user_editcount;
104 - if(isset($this->prop['registration']))
105 - // Nor for this one
106 - $data[$name]['registration'] = wfTimestampOrNull(TS_ISO_8601, $r->user_registration);
107 - if(isset($this->prop['groups']))
108 - // This row contains only one group, others will be added from other rows
109 - if(!is_null($r->ug_group))
110 - $data[$name]['groups'][] = $r->ug_group;
111 - if(isset($this->prop['blockinfo']))
112 - if(!is_null($r->blocker_name)) {
113 - $data[$name]['blockedby'] = $r->blocker_name;
114 - $data[$name]['blockreason'] = $r->ipb_reason;
115 - }
116 - if(isset($this->prop['emailable']) && $user->canReceiveEmail())
117 - $data[$name]['emailable'] = '';
118 - }
119 -
120121 // Second pass: add result data to $retval
121122 foreach($goodNames as $u) {
122123 if(!isset($data[$u]))
@@ -125,8 +126,9 @@
126127 $this->getResult()->setIndexedTagName($data[$u]['groups'], 'g');
127128 $retval[] = $data[$u];
128129 }
 130+ $done[] = $u;
129131 }
130 - return $retval;
 132+ return $this->getResult()->setIndexedTagName_internal(array('query', $this->getModuleName()), 'user');
131133 }
132134
133135 public function getAllowedParams() {
Index: trunk/phase3/includes/api/ApiQueryUserContributions.php
@@ -83,7 +83,6 @@
8484 $res = $this->select( __METHOD__ );
8585
8686 //Initialise some variables
87 - $data = array ();
8887 $count = 0;
8988 $limit = $this->params['limit'];
9089
@@ -99,16 +98,21 @@
10099 }
101100
102101 $vals = $this->extractRowInfo($row);
103 - if ($vals)
104 - $data[] = $vals;
 102+ $fit = $this->getResult()->addValue(array('query', $this->getModuleName()), null, $vals);
 103+ if(!$fit)
 104+ {
 105+ if($this->multiUserMode)
 106+ $this->setContinueEnumParameter('continue', $this->continueStr($row));
 107+ else
 108+ $this->setContinueEnumParameter('start', wfTimestamp(TS_ISO_8601, $row->rev_timestamp));
 109+ break;
 110+ }
105111 }
106112
107113 //Free the database record so the connection can get on with other stuff
108114 $db->freeResult($res);
109115
110 - //And send the whole shebang out as output.
111 - $this->getResult()->setIndexedTagName($data, 'item');
112 - $this->getResult()->addValue('query', $this->getModuleName(), $data);
 116+ $this->getResult()->setIndexedTagName_internal(array('query', $this->getModuleName()), 'item');
113117 }
114118
115119 /**
@@ -358,4 +362,4 @@
359363 public function getVersion() {
360364 return __CLASS__ . ': $Id$';
361365 }
362 -}
 366+}
\ No newline at end of file
Index: trunk/phase3/includes/api/ApiQueryImages.php
@@ -82,9 +82,6 @@
8383 $res = $this->select(__METHOD__);
8484
8585 if (is_null($resultPageSet)) {
86 -
87 - $data = array();
88 - $lastId = 0; // database has no ID 0
8986 $count = 0;
9087 while ($row = $db->fetchObject($res)) {
9188 if (++$count > $params['limit']) {
@@ -94,23 +91,16 @@
9592 '|' . $this->keyToTitle($row->il_to));
9693 break;
9794 }
98 - if ($lastId != $row->il_from) {
99 - if($lastId != 0) {
100 - $this->addPageSubItems($lastId, $data);
101 - $data = array();
102 - }
103 - $lastId = $row->il_from;
104 - }
105 -
10695 $vals = array();
10796 ApiQueryBase :: addTitleInfo($vals, Title :: makeTitle(NS_FILE, $row->il_to));
108 - $data[] = $vals;
 97+ $fit = $this->addPageSubItem($row->il_from, $vals);
 98+ if(!$fit)
 99+ {
 100+ $this->setContinueEnumParameter('continue', $row->il_from .
 101+ '|' . $this->keyToTitle($row->il_to));
 102+ break;
 103+ }
109104 }
110 -
111 - if($lastId != 0) {
112 - $this->addPageSubItems($lastId, $data);
113 - }
114 -
115105 } else {
116106
117107 $titles = array();
@@ -167,4 +157,4 @@
168158 public function getVersion() {
169159 return __CLASS__ . ': $Id$';
170160 }
171 -}
 161+}
\ No newline at end of file
Index: trunk/phase3/includes/DefaultSettings.php
@@ -3519,6 +3519,12 @@
35203520 $wgAPIMaxDBRows = 5000;
35213521
35223522 /**
 3523+ * The maximum size (in bytes) of an API result.
 3524+ * Don't set this lower than $wgMaxArticleSize*1024
 3525+ */
 3526+$wgAPIMaxResultSize = 8388608;
 3527+
 3528+/**
35233529 * Parser test suite files to be run by parserTests.php when no specific
35243530 * filename is passed to it.
35253531 *

Follow-up revisions

RevisionCommit summaryAuthorDate
r46847RELEASE-NOTES for r46845catrope15:25, 5 February 2009
r46849Fix regression from r46845 which broke ApiResult::cleanUpUTF8() and caused an...catrope15:46, 5 February 2009
r47036Fix some more regressions from r46845 reported by Brad Jorsch on the mediawik...catrope13:28, 9 February 2009
r47048Fix up r47037, which was itself a fix-up of r46845. Change suggested by Brad ...catrope19:24, 9 February 2009
r47324API: Fix yet another regression from r46845 that completely broke list=users....catrope17:47, 16 February 2009
r47470API: (bug 17561) Recommit r44231 ("Added usprop=emailable to list=users"), wh...catrope22:31, 18 February 2009
r47514API: (bug 17563) Fix regression from r46845 that changed the output for list=...catrope21:24, 19 February 2009

Comments

#Comment by Catrope (talk | contribs)   17:52, 16 February 2009

Regression fixes in r46847, r46848, r46849, r47036, r47048, r47324

#Comment by Catrope (talk | contribs)   22:32, 18 February 2009

and r47470

Status & tagging log