r65152 MediaWiki - Code Review archive

Repository:MediaWiki
Revision:r65151‎ | r65152 | r65153 >
Date:02:43, 17 April 2010
Author:mah
Status:resolved (Comments)
Tags:
Comment:
* New UploadFromUrlJob class to handle Upload-by-Copy
* Define variable for ApiUserrights.php that wasn't defined before.
* Add convertVerifyErrorToStatus and getVerificationErrorCode to
UploadBase to translate error consts since UploadFromUrl will
need a message to display to end-users.
* refactor mime-checking out of UploadBase::verifyFile into
UploadBase::verifyMimeType
* Make UploadBase::verifyFile always return arrays for errors
* Use HttpFunctions instead of custom curl handler for async downloading
* TODO: Need a way to feed errors back to the requestor
* TODO: Need to add watchlist param handling and warnings checks.
Modified paths:
  • /trunk/phase3/includes/AutoLoader.php (modified) (history)
  • /trunk/phase3/includes/DefaultSettings.php (modified) (history)
  • /trunk/phase3/includes/UploadFromUrlJob.php (added) (history)
  • /trunk/phase3/includes/api/ApiBase.php (modified) (history)
  • /trunk/phase3/includes/api/ApiUpload.php (modified) (history)
  • /trunk/phase3/includes/api/ApiUserrights.php (modified) (history)
  • /trunk/phase3/includes/upload/UploadBase.php (modified) (history)
  • /trunk/phase3/includes/upload/UploadFromUrl.php (modified) (history)
  • /trunk/phase3/languages/messages/MessagesEn.php (modified) (history)
  • /trunk/phase3/maintenance/language/messages.inc (modified) (history)
  • /trunk/phase3/maintenance/tests/MediaWikiParserTest.php (modified) (history)
  • /trunk/phase3/maintenance/tests/UploadFromUrlTest.php (added) (history)

Diff [purge]

Index: trunk/phase3/languages/messages/MessagesEn.php
@@ -2055,6 +2055,17 @@
20562056 'filetype-banned-type' => "'''\".\$1\"''' is not a permitted file type.
20572057 Permitted {{PLURAL:\$3|file type is|file types are}} \$2.",
20582058 'filetype-missing' => 'The file has no extension (like ".jpg").',
 2059+'empty-file' => 'The file you submitted was empty',
 2060+'file-too-large' => 'The file you submitted was too large',
 2061+'filename-tooshort' => 'The filename is too short',
 2062+'filetype-banned' => 'This type of file is banned',
 2063+'verification-error' => 'This file did not pass file verification',
 2064+'hookaborted' => 'The modification you tried to make was aborted by an extension hook',
 2065+'illegal-filename' => 'The filename is not allowed',
 2066+'overwrite' => 'Overwriting an existing file is not allowed',
 2067+'unknown-error' => 'An unknown error occured',
 2068+'tmp-create-error' => 'Couldn\'t create temporary file',
 2069+'tmp-write-error' => 'Error writing temporary file',
20592070 'large-file' => 'It is recommended that files are no larger than $1;
20602071 this file is $2.',
20612072 'largefileserver' => 'This file is bigger than the server is configured to allow.',
@@ -2094,6 +2105,7 @@
20952106 'uploadedimage' => 'uploaded "[[$1]]"',
20962107 'overwroteimage' => 'uploaded a new version of "[[$1]]"',
20972108 'uploaddisabled' => 'Uploads disabled',
 2109+'copyuploaddisabled' => 'Upload by URL disabled',
20982110 'uploaddisabledtext' => 'File uploads are disabled.',
20992111 'php-uploaddisabledtext' => 'File uploads are disabled in PHP.
21002112 Please check the file_uploads setting.',
Index: trunk/phase3/includes/DefaultSettings.php
@@ -1935,6 +1935,7 @@
19361936 'sendMail' => 'EmaillingJob',
19371937 'enotifNotify' => 'EnotifNotifyJob',
19381938 'fixDoubleRedirect' => 'DoubleRedirectJob',
 1939+ 'uploadFromUrl' => 'UploadFromUrlJob',
19391940 );
19401941
19411942 /**
Index: trunk/phase3/includes/AutoLoader.php
@@ -237,6 +237,7 @@
238238 'UploadFromStash' => 'includes/upload/UploadFromStash.php',
239239 'UploadFromFile' => 'includes/upload/UploadFromFile.php',
240240 'UploadFromUrl' => 'includes/upload/UploadFromUrl.php',
 241+ 'UploadFromUrlJob' => 'includes/UploadFromUrlJob.php',
241242 'User' => 'includes/User.php',
242243 'UserArray' => 'includes/UserArray.php',
243244 'UserArrayFromResult' => 'includes/UserArray.php',
Index: trunk/phase3/includes/upload/UploadFromUrl.php
@@ -10,6 +10,7 @@
1111 */
1212 class UploadFromUrl extends UploadBase {
1313 protected $mTempDownloadPath;
 14+ protected $comment, $watchList;
1415
1516 /**
1617 * Checks if the user is allowed to use the upload-by-URL feature. If the
@@ -32,16 +33,56 @@
3334 /**
3435 * Entry point for API upload
3536 */
36 - public function initialize( $name, $url, $na, $nb = false ) {
37 - global $wgTmpDirectory;
 37+ public function initialize( $name, $url, $comment, $watchlist ) {
 38+ global $wgUser;
3839
39 - $localFile = tempnam( $wgTmpDirectory, 'WEBUPLOAD' );
40 - $this->initializePathInfo( $name, $localFile, 0, true );
 40+ if( !Http::isValidURI( $url ) ) {
 41+ return Status::newFatal( 'http-invalid-url' );
 42+ }
 43+ $params = array(
 44+ "userName" => $wgUser->getName(),
 45+ "userID" => $wgUser->getID(),
 46+ "url" => trim( $url ),
 47+ "timestamp" => wfTimestampNow(),
 48+ "comment" => $comment,
 49+ "watchlist" => $watchlist);
4150
42 - $this->mUrl = trim( $url );
 51+ $title = Title::newFromText( $name );
 52+ /* // Check whether the user has the appropriate permissions to upload anyway */
 53+ /* $permission = $this->isAllowed( $wgUser ); */
 54+
 55+ /* if ( $permission !== true ) { */
 56+ /* if ( !$wgUser->isLoggedIn() ) { */
 57+ /* return Status::newFatal( 'uploadnologintext' ); */
 58+ /* } else { */
 59+ /* return Status::newFatal( 'badaccess-groups' ); */
 60+ /* } */
 61+ /* } */
 62+
 63+ /* $permErrors = $this->verifyPermissions( $wgUser ); */
 64+ /* if ( $permErrors !== true ) { */
 65+ /* return Status::newFatal( 'badaccess-groups' ); */
 66+ /* } */
 67+
 68+
 69+ $job = new UploadFromUrlJob( $title, $params );
 70+ $job->insert();
4371 }
4472
4573 /**
 74+ * Initialize a queued download
 75+ * @param $job Job
 76+ */
 77+ public function initializeFromJob( $job ) {
 78+ $this->mUrl = $job->params['url'];
 79+ $this->mTempPath = tempnam( $wgTmpDirectory, 'COPYUPLOAD' );
 80+ $this->mDesiredDestName = $job->title;
 81+ $this->comment = $job->params['comment'];
 82+ $this->watchList = $job->params['watchlist'];
 83+ $this->getTitle();
 84+ }
 85+
 86+ /**
4687 * Entry point for SpecialUpload
4788 * @param $request Object: WebRequest object
4889 */
@@ -66,60 +107,63 @@
67108 return self::isValidUrl( $request->getVal( 'wpUploadFileURL' ) );
68109 }
69110
70 - public static function isValidUrl( $url ) {
71 - // Only allow HTTP or FTP for now
72 - return (bool)preg_match( '!^(http://|ftp://)!', $url );
 111+ private function saveTempFile( $req ) {
 112+ $filename = tempnam( wfTempDir(), 'URL' );
 113+ if ( $filename === false ) {
 114+ return Status::newFatal( 'tmp-create-error' );
 115+ }
 116+ if ( file_put_contents( $filename, $req->getContent() ) === false ) {
 117+ return Status::newFatal( 'tmp-write-error' );
 118+ }
 119+
 120+ $this->mTempPath = $filename;
 121+ $this->mFileSize = filesize( $filename );
 122+
 123+ return Status::newGood();
73124 }
74125
75 - /**
76 - * Do the real fetching stuff
77 - */
78 - function fetchFile() {
79 - if( !self::isValidUrl( $this->mUrl ) ) {
80 - return Status::newFatal( 'upload-proto-error' );
 126+ public function doUpload() {
 127+ global $wgUser;
 128+
 129+ $req = HttpRequest::factory($this->mUrl);
 130+ $status = $req->execute();
 131+
 132+ if( !$status->isOk() ) {
 133+ return $status;
81134 }
82135
83 - # Open temporary file
84 - $this->mCurlDestHandle = @fopen( $this->mTempPath, "wb" );
85 - if( $this->mCurlDestHandle === false ) {
86 - # Could not open temporary file to write in
87 - return Status::newFatal( 'upload-file-error' );
 136+ $status = $this->saveTempFile( $req );
 137+ $this->mRemoveTempFile = true;
 138+
 139+ if( !$status->isOk() ) {
 140+ return $status;
88141 }
89 -
90 - $options = array(
91 - 'method' => 'GET',
92 - 'timeout' => 10,
93 - );
94 - $req = HttpRequest::factory( $this->mUrl, $options );
95 - $req->setCallback( array( $this, 'uploadCurlCallback' ) );
96 - $status = $req->execute();
97 - fclose( $this->mCurlDestHandle );
98 - unset( $this->mCurlDestHandle );
99142
100 - global $wgMaxUploadSize;
101 - if ( $this->mFileSize > $wgMaxUploadSize ) {
102 - # Just return an ok, so that the regular verifications can handle
103 - # the file-too-large error
104 - return Status::newGood();
 143+ $v = $this->verifyUpload();
 144+ if( $v['status'] !== UploadBase::OK ) {
 145+ return $this->convertVerifyErrorToStatus( $v['status'], $v['details'] );
105146 }
106147
107 - return $status;
108 - }
 148+ // This has to come from API
 149+ /* $warnings = $this->checkForWarnings(); */
 150+ /* if( isset($warnings) ) return $warnings; */
109151
110 - /**
111 - * Callback function for CURL-based web transfer
112 - * Write data to file unless we've passed the length limit;
113 - * if so, abort immediately.
114 - * @access private
115 - */
116 - function uploadCurlCallback( $ch, $data ) {
117 - global $wgMaxUploadSize;
118 - $length = strlen( $data );
119 - $this->mFileSize += $length;
120 - if( $this->mFileSize > $wgMaxUploadSize ) {
121 - return 0;
 152+ // Use comment as initial page text by default
 153+ if ( is_null( $this->mParams['text'] ) ) {
 154+ $this->mParams['text'] = $this->mParams['comment'];
122155 }
123 - fwrite( $this->mCurlDestHandle, $data );
124 - return $length;
 156+
 157+ $file = $this->getLocalFile();
 158+ // This comes from ApiBase
 159+ /* $watch = $this->getWatchlistValue( $this->mParams['watchlist'], $file->getTitle() ); */
 160+
 161+ if ( !$status->isGood() ) {
 162+ return $status;
 163+ }
 164+
 165+ $status = $this->getLocalFile()->upload( $this->mTempPath, $this->comment,
 166+ $this->pageText, File::DELETE_SOURCE, $this->mFileProps, false, $wgUser );
 167+
 168+ return $status;
125169 }
126170 }
Index: trunk/phase3/includes/upload/UploadBase.php
@@ -29,6 +29,7 @@
3030 const FILETYPE_MISSING = 8;
3131 const FILETYPE_BADTYPE = 9;
3232 const VERIFICATION_ERROR = 10;
 33+
3334 # HOOK_ABORTED is the new name of UPLOAD_VERIFICATION_ERROR
3435 const UPLOAD_VERIFICATION_ERROR = 11;
3536 const HOOK_ABORTED = 11;
@@ -41,6 +42,24 @@
4243 return self::SESSION_KEYNAME;
4344 }
4445
 46+ public function getVerificationErrorCode( $error ) {
 47+ $code_to_status = array(self::EMPTY_FILE => 'empty-file',
 48+ self::FILE_TOO_LARGE => 'file-too-large',
 49+ self::FILETYPE_MISSING => 'filetype-missing',
 50+ self::FILETYPE_BADTYPE => 'filetype-banned',
 51+ self::MIN_LENGTH_PARTNAME => 'filename-tooshort',
 52+ self::ILLEGAL_FILENAME => 'illegal-filename',
 53+ self::OVERWRITE_EXISTING_FILE => 'overwrite',
 54+ self::VERIFICATION_ERROR => 'verification-error',
 55+ self::HOOK_ABORTED => 'hookaborted',
 56+ );
 57+ if( isset( $code_to_status[$error] ) ) {
 58+ return $code_to_status[$error];
 59+ }
 60+
 61+ return 'unknown-error';
 62+ }
 63+
4564 /**
4665 * Returns true if uploads are enabled.
4766 * Can be override by subclasses.
@@ -201,7 +220,7 @@
202221 if( $this->isEmptyFile() ) {
203222 return array( 'status' => self::EMPTY_FILE );
204223 }
205 -
 224+
206225 /**
207226 * Honor $wgMaxUploadSize
208227 */
@@ -217,9 +236,6 @@
218237 */
219238 $verification = $this->verifyFile();
220239 if( $verification !== true ) {
221 - if( !is_array( $verification ) ) {
222 - $verification = array( $verification );
223 - }
224240 return array(
225241 'status' => self::VERIFICATION_ERROR,
226242 'details' => $verification
@@ -278,19 +294,12 @@
279295 }
280296
281297 /**
282 - * Verifies that it's ok to include the uploaded file
283 - *
284 - * @return mixed true of the file is verified, a string or array otherwise.
 298+ * Verify the mime type
 299+ * @param $magic MagicMime object
 300+ * @param $mime string representing the mime
 301+ * @return mixed true if the file is verified, an array otherwise
285302 */
286 - protected function verifyFile() {
287 - $this->mFileProps = File::getPropsFromPath( $this->mTempPath, $this->mFinalExtension );
288 - $this->checkMacBinary();
289 -
290 - # magically determine mime type
291 - $magic = MimeMagic::singleton();
292 - $mime = $magic->guessMimeType( $this->mTempPath, false );
293 -
294 - # check mime type, if desired
 303+ protected function verifyMimeType( $magic, $mime ) {
295304 global $wgVerifyMimeType;
296305 if ( $wgVerifyMimeType ) {
297306 wfDebug ( "\n\nmime: <$mime> extension: <{$this->mFinalExtension}>\n\n");
@@ -316,13 +325,35 @@
317326 }
318327 }
319328
 329+ return true;
 330+ }
 331+
 332+ /**
 333+ * Verifies that it's ok to include the uploaded file
 334+ *
 335+ * @return mixed true of the file is verified, array otherwise.
 336+ */
 337+ protected function verifyFile() {
 338+ $this->mFileProps = File::getPropsFromPath( $this->mTempPath, $this->mFinalExtension );
 339+ $this->checkMacBinary();
 340+
 341+ # magically determine mime type
 342+ $magic = MimeMagic::singleton();
 343+ $mime = $magic->guessMimeType( $this->mTempPath, false );
 344+
 345+ # check mime type, if desired
 346+ $status = $this->verifyMimeType( $magic, $mime );
 347+ if ( $status !== true ) {
 348+ return $status;
 349+ }
 350+
320351 # check for htmlish code and javascript
321352 if( self::detectScript( $this->mTempPath, $mime, $this->mFinalExtension ) ) {
322 - return 'uploadscripted';
 353+ return array( 'uploadscripted' );
323354 }
324355 if( $this->mFinalExtension == 'svg' || $mime == 'image/svg+xml' ) {
325356 if( self::detectScriptInSvg( $this->mTempPath ) ) {
326 - return 'uploadscripted';
 357+ return array( 'uploadscripted' );
327358 }
328359 }
329360
@@ -354,7 +385,6 @@
355386 return true;
356387 }
357388 $permErrors = $nt->getUserPermissionsErrors( 'edit', $user );
358 - $permErrorsUpload = $nt->getUserPermissionsErrors( 'upload', $user );
359389 $permErrorsCreate = ( $nt->exists() ? array() : $nt->getUserPermissionsErrors( 'create', $user ) );
360390 if( $permErrors || $permErrorsUpload || $permErrorsCreate ) {
361391 $permErrors = array_merge( $permErrors, wfArrayDiff2( $permErrorsUpload, $permErrors ) );
@@ -440,7 +470,8 @@
441471 * @return mixed Status indicating the whether the upload succeeded.
442472 */
443473 public function performUpload( $comment, $pageText, $watch, $user ) {
444 - wfDebug( "\n\n\performUpload: sum:" . $comment . ' c: ' . $pageText . ' w:' . $watch );
 474+ wfDebug( "\n\n\performUpload: sum: " . $comment . ' c: ' . $pageText .
 475+ ' w: ' . $watch );
445476 $status = $this->getLocalFile()->upload( $this->mTempPath, $comment, $pageText,
446477 File::DELETE_SOURCE, $this->mFileProps, false, $user );
447478
@@ -575,7 +606,8 @@
576607 }
577608
578609 /**
579 - * Generate a random session key from stash in cases where we want to start an upload without much information
 610+ * Generate a random session key from stash in cases where we want
 611+ * to start an upload without much information
580612 */
581613 protected function getSessionKey() {
582614 $key = mt_rand( 0, 0x7fffffff );
@@ -850,7 +882,8 @@
851883
852884 if ( !$wgAntivirusSetup[$wgAntivirus] ) {
853885 wfDebug( __METHOD__ . ": unknown virus scanner: $wgAntivirus\n" );
854 - $wgOut->wrapWikiMsg( "<div class=\"error\">\n$1</div>", array( 'virus-badscanner', $wgAntivirus ) );
 886+ $wgOut->wrapWikiMsg( "<div class=\"error\">\n$1</div>",
 887+ array( 'virus-badscanner', $wgAntivirus ) );
855888 return wfMsg( 'virus-unknownscanner' ) . " $wgAntivirus";
856889 }
857890
@@ -1133,4 +1166,9 @@
11341167 return ApiQueryImageInfo::getInfo( $file, array_flip( $imParam ), $result );
11351168 }
11361169
 1170+ public function convertVerifyErrorToStatus( $error ) {
 1171+ $args = func_get_args();
 1172+ array_shift($args);
 1173+ return Status::newFatal( $this->getVerificationErrorCode( $error ), $args );
 1174+ }
11371175 }
Index: trunk/phase3/includes/api/ApiUpload.php
@@ -83,7 +83,7 @@
8484 } elseif ( isset( $this->mParams['url'] ) ) {
8585 // make sure upload by URL is enabled:
8686 if ( !$wgAllowCopyUploads ) {
87 - $this->dieUsageMsg( array( 'uploaddisabled' ) );
 87+ $this->dieUsageMsg( array( 'copyuploaddisabled' ) );
8888 }
8989
9090 // make sure the current user can upload
@@ -91,14 +91,12 @@
9292 $this->dieUsageMsg( array( 'badaccess-groups' ) );
9393 }
9494
95 - $this->mUpload = new UploadFromUrl();
96 - $this->mUpload->initialize( $this->mParams['filename'],
97 - $this->mParams['url'] );
 95+ $this->mUpload = new UploadFromUrl;
 96+ $this->mUpload->initialize( $this->mParams['filename'], $this->mParams['url'],
 97+ $this->mParams['comment'] );
9898
99 - $status = $this->mUpload->fetchFile();
100 - if ( !$status->isOK() ) {
101 - $this->dieUsage( $status->getWikiText(), 'fetchfileerror' );
102 - }
 99+ $this->getResult()->addValue( null, $this->getModuleName(), Status::newGood() );
 100+ return;
103101 }
104102 } else {
105103 $this->dieUsageMsg( array( 'missingparam', 'filename' ) );
Index: trunk/phase3/includes/api/ApiUserrights.php
@@ -43,6 +43,7 @@
4444
4545 $user = $this->getUser();
4646
 47+ $form = new UserrightsPage;
4748 $r['user'] = $user->getName();
4849 list( $r['added'], $r['removed'] ) =
4950 $form->doSaveUserGroups(
Index: trunk/phase3/includes/api/ApiBase.php
@@ -989,6 +989,7 @@
990990 'invalid-session-key' => array( 'code' => 'invalid-session-key', 'info' => 'Not a valid session key' ),
991991 'nouploadmodule' => array( 'code' => 'nouploadmodule', 'info' => 'No upload module set' ),
992992 'uploaddisabled' => array( 'code' => 'uploaddisabled', 'info' => 'Uploads are not enabled. Make sure $wgEnableUploads is set to true in LocalSettings.php and the PHP ini setting file_uploads is true' ),
 993+ 'copyuploaddisabled' => array( 'code' => 'copyuploaddisabled', 'info' => 'Uploads by URL is not enabled. Make sure $wgAllowCopyUploads is set to true in LocalSettings.php.' ),
993994 );
994995
995996 /**
Index: trunk/phase3/includes/UploadFromUrlJob.php
@@ -0,0 +1,30 @@
 2+<?php
 3+
 4+/**
 5+ * Job for email notification mails
 6+ *
 7+ * @ingroup JobQueue
 8+ */
 9+class UploadFromUrlJob extends Job {
 10+
 11+ public function __construct( $title, $params, $id = 0 ) {
 12+ parent::__construct( 'uploadFromUrl', $title, $params, $id );
 13+ }
 14+
 15+ public function run() {
 16+ global $wgUser;
 17+
 18+ if ( $this->params['userID'] ) {
 19+ $wgUser = User::newFromId( $this->params['userID'] );
 20+ } else {
 21+ $wgUser = new User;
 22+ }
 23+ $wgUser->mEffectiveGroups[] = 'sysop';
 24+ $wgUser->mRights = null;
 25+
 26+ $upload = new UploadFromUrl();
 27+ $upload->initializeFromJob( $this );
 28+
 29+ return $upload->doUpload();
 30+ }
 31+}
Property changes on: trunk/phase3/includes/UploadFromUrlJob.php
___________________________________________________________________
Added: svn:eol-syle
132 + native
Index: trunk/phase3/maintenance/language/messages.inc
@@ -1233,6 +1233,7 @@
12341234 'uploadedimage',
12351235 'overwroteimage',
12361236 'uploaddisabled',
 1237+ 'copyuploaddisabled',
12371238 'uploaddisabledtext',
12381239 'php-uploaddisabledtext',
12391240 'uploadscripted',
Index: trunk/phase3/maintenance/tests/MediaWikiParserTest.php
@@ -27,6 +27,7 @@
2828 $tables[] = 'filearchive';
2929 $tables[] = 'logging';
3030 $tables[] = 'updatelog';
 31+ $tables[] = 'iwlinks';
3132 return true;
3233 }
3334
Index: trunk/phase3/maintenance/tests/UploadFromUrlTest.php
@@ -0,0 +1,191 @@
 2+<?php
 3+
 4+global $IP;
 5+require_once( "ApiSetup.php" );
 6+require_once( dirname( dirname( __FILE__ ) ) . "/deleteArchivedFiles.inc" );
 7+require_once( dirname( dirname( __FILE__ ) ) . "/deleteArchivedRevisions.inc" );
 8+
 9+class nullClass {
 10+ public function handleOutput(){}
 11+ public function purgeRedundantText(){}
 12+}
 13+
 14+class UploadFromUrlTest extends ApiSetup {
 15+
 16+ function setUp() {
 17+ global $wgEnableUploads, $wgLocalFileRepo;
 18+
 19+ $wgEnableUploads = true;
 20+ parent::setup();
 21+ $wgLocalFileRepo = array(
 22+ 'class' => 'LocalRepo',
 23+ 'name' => 'local',
 24+ 'directory' => 'test-repo',
 25+ 'url' => 'http://example.com/images',
 26+ 'hashLevels' => 2,
 27+ 'transformVia404' => false,
 28+ );
 29+
 30+ ini_set( 'log_errors', 1 );
 31+ ini_set( 'error_reporting', 1 );
 32+ ini_set( 'display_errors', 1 );
 33+ }
 34+
 35+ function doApiRequest( $params, $data = null ) {
 36+ $session = isset( $data[2] ) ? $data[2] : array();
 37+ $_SESSION = $session;
 38+
 39+ $req = new FauxRequest( $params, true, $session );
 40+ $module = new ApiMain( $req, true );
 41+ $module->execute();
 42+
 43+ return array( $module->getResultData(), $req, $_SESSION );
 44+ }
 45+
 46+ function testClearQueue() {
 47+ while ( $job = Job::pop() ) {}
 48+ $this->assertFalse($job);
 49+ }
 50+
 51+ function testLogin() {
 52+ $data = $this->doApiRequest( array(
 53+ 'action' => 'login',
 54+ 'lgname' => self::$userName,
 55+ 'lgpassword' => self::$passWord ) );
 56+ $this->assertArrayHasKey( "login", $data[0] );
 57+ $this->assertArrayHasKey( "result", $data[0]['login'] );
 58+ $this->assertEquals( "NeedToken", $data[0]['login']['result'] );
 59+ $token = $data[0]['login']['token'];
 60+
 61+ $data = $this->doApiRequest( array(
 62+ 'action' => 'login',
 63+ "lgtoken" => $token,
 64+ "lgname" => self::$userName,
 65+ "lgpassword" => self::$passWord ) );
 66+
 67+ $this->assertArrayHasKey( "login", $data[0] );
 68+ $this->assertArrayHasKey( "result", $data[0]['login'] );
 69+ $this->assertEquals( "Success", $data[0]['login']['result'] );
 70+ $this->assertArrayHasKey( 'lgtoken', $data[0]['login'] );
 71+
 72+ return $data;
 73+ }
 74+
 75+ /**
 76+ * @depends testLogin
 77+ */
 78+ function testSetupUrlDownload( $data ) {
 79+ global $wgUser;
 80+ $wgUser = User::newFromName( self::$userName );
 81+ $wgUser->load();
 82+ $data[2]['wsEditToken'] = $data[2]['wsToken'];
 83+ $token = md5( $data[2]['wsToken'] ) . EDIT_TOKEN_SUFFIX;
 84+ $exception = false;
 85+
 86+ try {
 87+ $this->doApiRequest( array(
 88+ 'action' => 'upload',
 89+ ), $data );
 90+ } catch ( UsageException $e ) {
 91+ $exception = true;
 92+ $this->assertEquals( "The token parameter must be set", $e->getMessage() );
 93+ }
 94+ $this->assertTrue( $exception, "Got exception" );
 95+
 96+ $exception = false;
 97+ try {
 98+ $this->doApiRequest( array(
 99+ 'action' => 'upload',
 100+ 'token' => $token,
 101+ ), $data );
 102+ } catch ( UsageException $e ) {
 103+ $exception = true;
 104+ $this->assertEquals( "One of the parameters sessionkey, file, url is required",
 105+ $e->getMessage() );
 106+ }
 107+ $this->assertTrue( $exception, "Got exception" );
 108+
 109+ $exception = false;
 110+ try {
 111+ $this->doApiRequest( array(
 112+ 'action' => 'upload',
 113+ 'url' => 'http://www.example.com/test.png',
 114+ 'token' => $token,
 115+ ), $data );
 116+ } catch ( UsageException $e ) {
 117+ $exception = true;
 118+ $this->assertEquals( "The filename parameter must be set", $e->getMessage() );
 119+ }
 120+ $this->assertTrue( $exception, "Got exception" );
 121+
 122+ $wgUser->removeGroup('sysop');
 123+ $exception = false;
 124+ try {
 125+ $this->doApiRequest( array(
 126+ 'action' => 'upload',
 127+ 'url' => 'http://www.example.com/test.png',
 128+ 'filename' => 'Test.png',
 129+ 'token' => $token,
 130+ ), $data );
 131+ } catch ( UsageException $e ) {
 132+ $exception = true;
 133+ $this->assertEquals( "Permission denied", $e->getMessage() );
 134+ }
 135+ $this->assertTrue( $exception, "Got exception" );
 136+
 137+ $wgUser->addGroup('*');
 138+ $wgUser->addGroup('sysop');
 139+ $exception = false;
 140+ $data = $this->doApiRequest( array(
 141+ 'action' => 'upload',
 142+ 'url' => 'http://bits.wikimedia.org/skins-1.5/common/images/poweredby_mediawiki_88x31.png',
 143+ 'filename' => 'Test.png',
 144+ 'token' => $token,
 145+ ), $data );
 146+
 147+ $this->assertThat( $data[0]['upload'], $this->isInstanceOf( 'Status' ),
 148+ "Got Status Object" );
 149+ $this->assertTrue( $data[0]['upload']->isOk(), 'Job added');
 150+
 151+ $job = Job::pop();
 152+ $this->assertThat( $job, $this->isInstanceOf( 'UploadFromUrlJob' ),
 153+ "Got Job Object" );
 154+
 155+ $job = Job::pop_type( 'upload' );
 156+ $this->assertFalse( $job );
 157+ }
 158+
 159+ /**
 160+ * @depends testLogin
 161+ */
 162+ function testDoDownload( $data ) {
 163+ global $wgUser;
 164+ $data[2]['wsEditToken'] = $data[2]['wsToken'];
 165+ $token = md5( $data[2]['wsToken'] ) . EDIT_TOKEN_SUFFIX;
 166+
 167+ $wgUser->addGroup('users');
 168+ $data = $this->doApiRequest( array(
 169+ 'action' => 'upload',
 170+ 'filename' => 'Test.png',
 171+ 'url' => 'http://bits.wikimedia.org/skins-1.5/common/images/poweredby_mediawiki_88x31.png',
 172+ 'token' => $token,
 173+ ), $data );
 174+
 175+ $job = Job::pop();
 176+ $this->assertEquals( 'UploadFromUrlJob', get_class($job) );
 177+
 178+ $status = $job->run();
 179+ $this->assertTrue( $status->isOk() );
 180+
 181+ return $data;
 182+ }
 183+
 184+ /**
 185+ * @depends testDoDownload
 186+ */
 187+ function testVerifyDownload( $data ) {
 188+ $t = Title::newFromText("Test.png", NS_FILE);
 189+
 190+ $this->assertTrue($t->exists());
 191+ }
 192+}
Property changes on: trunk/phase3/maintenance/tests/UploadFromUrlTest.php
___________________________________________________________________
Added: svn:eol-syle
1193 + native

Follow-up revisions

RevisionCommit summaryAuthorDate
r65161Tweaks for messages added in r65152.siebrand15:44, 17 April 2010
r65162re: r65152 fix translatewiki problems...mah16:36, 17 April 2010
r65165re: r65152...mah16:49, 17 April 2010
r65191for r65152: fixed web ui for UploadFromUrl...mah21:44, 17 April 2010
r67885* re r65152 add back async option for uploadByURL API call...mah03:58, 12 June 2010
r89003* (bug 29174) Fix regression in upload-by-URL: files larger than PHP memory l...brion22:31, 27 May 2011

Comments

#Comment by Catrope (talk | contribs)   12:20, 17 April 2010
+				$this->getResult()->addValue( null, $this->getModuleName(), Status::newGood() );

You can't add objects to API results, the formatter will yell at you.

#Comment by MarkAHershberger (talk | contribs)   19:10, 17 April 2010

see r65165

#Comment by Raymond (talk | contribs)   14:17, 17 April 2010

Seen on translatewiki:

PHP Fatal error: Call to undefined method UploadFromUrl::isvalidurl() in /www/w/includes/upload/UploadFromUrl.php on line 107

#Comment by Raymond (talk | contribs)   14:19, 17 April 2010

Seen on translatewiki during upload a file from disk:

PHP Notice: Undefined variable: permErrorsUpload in /www/w/includes/upload/UploadBase.php on line 389

#Comment by MarkAHershberger (talk | contribs)   19:10, 17 April 2010

see r65162

#Comment by Raymond (talk | contribs)   19:57, 17 April 2010

Next try on translatewiki: Upload by URL from http://www.spekking.de/pix/OperKoeln.jpg results into:

Error on Wiki: 'emptyfile' => 'The file you uploaded seems to be empty. ...'

plus

  • PHP Warning: Missing argument 4 for UploadFromUrl::initialize(), called in /www/w/includes/upload/UploadFromUrl.php on line 99 and defined in /www/w/includes/upload/UploadFromUrl.php on line 38
  • PHP Notice: Undefined variable: wgTmpDirectory in /www/w/includes/upload/UploadFromUrl.php on line 80
#Comment by MarkAHershberger (talk | contribs)   21:46, 17 April 2010

Sorry about that, see r65191

#Comment by Bryan (talk | contribs)   19:20, 29 April 2010

upload-by-url via the API is only possible async. I think that be default the upload must be synchronous (easier for clients and no b/w compat breaking changes) and have a parameter to make it async.

More review to come, once I have time.

#Comment by MarkAHershberger (talk | contribs)   04:01, 12 June 2010

see r67885

#Comment by Bryan (talk | contribs)   19:42, 29 July 2010

All issues resolved now.

#Comment by Brion VIBBER (talk | contribs)   21:59, 27 May 2011

This did actually cause a regression for large files, as MwHttpRequest normally buffers content in memory: bug 29174

It should be an easy fix.

#Comment by Brion VIBBER (talk | contribs)   22:34, 27 May 2011

Fixed on trunk in r89003; needs merging to 1.17, 1.17-wmf, 1.18.

Status & tagging log