r87942 MediaWiki - Code Review archive

Repository:	MediaWiki
Revision:	< r87941‎ \| r87942 \| r87943 >
Date:	01:55, 13 May 2011
Author:	nelson
Status:	deferred (Comments)
Tags:
Comment:	First checkin
Modified paths:	/trunk/extensions/SwiftMedia (added) (history) /trunk/extensions/SwiftMedia/LocalSettings.php (added) (history) /trunk/extensions/SwiftMedia/README (added) (history) /trunk/extensions/SwiftMedia/SwiftMedia.body.php (added) (history) /trunk/extensions/SwiftMedia/SwiftMedia.i18n.php (added) (history) /trunk/extensions/SwiftMedia/SwiftMedia.php (added) (history) /trunk/extensions/SwiftMedia/proxy-server.conf (added) (history) /trunk/extensions/SwiftMedia/wmf (added) (history) /trunk/extensions/SwiftMedia/wmf/__init__.py (added) (history) /trunk/extensions/SwiftMedia/wmf/client.py (added) (history) /trunk/extensions/SwiftMedia/wmf/rewrite.py (added) (history)

Diff [purge]

Index: trunk/extensions/SwiftMedia/SwiftMedia.i18n.php
—	—	@@ -0,0 +1,6 @@
	2	+<?php
	3	+$messages = array();
	4	+$messages['en'] = array(
	5	+ 'swiftmedia' => 'Openstack\'s Swift is a very large scale reliable object store. It will serve multiple petabytes of media files.',
	6	+);
	7	+
Index: trunk/extensions/SwiftMedia/LocalSettings.php
—	—	@@ -0,0 +1,158 @@
	2	+<?php
	3	+# This file was automatically generated by the MediaWiki 1.18alpha
	4	+# installer. If you make manual changes, please keep track in case you
	5	+# need to recreate them later.
	6	+#
	7	+# See includes/DefaultSettings.php for all configurable settings
	8	+# and their default values, but don't forget to make changes in _this_
	9	+# file, not there.
	10	+#
	11	+# Further documentation for configuration settings may be found at:
	12	+# http://www.mediawiki.org/wiki/Manual:Configuration_settings
	13	+
	14	+# Protect against web entry
	15	+if ( !defined( 'MEDIAWIKI' ) ) {
	16	+ exit;
	17	+}
	18	+
	19	+## Uncomment this to disable output compression
	20	+# $wgDisableOutputCompression = true;
	21	+
	22	+$wgSitename = "mediastore";
	23	+$wgMetaNamespace = "Mediastore";
	24	+
	25	+## The URL base path to the directory containing the wiki;
	26	+## defaults for all runtime URL paths are based off of this.
	27	+## For more information on customizing the URLs please see:
	28	+## http://www.mediawiki.org/wiki/Manual:Short_URL
	29	+$wgScriptPath = "";
	30	+$wgScriptExtension = ".php";
	31	+
	32	+## The relative URL path to the skins directory
	33	+$wgStylePath = "$wgScriptPath/skins";
	34	+
	35	+## The relative URL path to the logo. Make sure you change this from the default,
	36	+## or else you'll overwrite your logo when you upgrade!
	37	+$wgLogo = "$wgStylePath/common/images/wiki.png";
	38	+
	39	+## UPO means: this is also a user preference option
	40	+
	41	+$wgEnableEmail = true;
	42	+$wgEnableUserEmail = true; # UPO
	43	+
	44	+$wgEmergencyContact = "apache@ersch.wikimedia.org";
	45	+$wgPasswordSender = "apache@ersch.wikimedia.org";
	46	+
	47	+$wgEnotifUserTalk = false; # UPO
	48	+$wgEnotifWatchlist = false; # UPO
	49	+$wgEmailAuthentication = true;
	50	+
	51	+## Database settings
	52	+$wgDBtype = "mysql";
	53	+$wgDBserver = "localhost";
	54	+$wgDBname = "my_wiki";
	55	+$wgDBuser = "root";
	56	+$wgDBpassword = "leakywiks";
	57	+
	58	+# MySQL specific settings
	59	+$wgDBprefix = "";
	60	+
	61	+# MySQL table options to use during installation or update
	62	+$wgDBTableOptions = "ENGINE=InnoDB, DEFAULT CHARSET=binary";
	63	+
	64	+# Experimental charset support for MySQL 4.1/5.0.
	65	+$wgDBmysql5 = false;
	66	+
	67	+## Shared memory settings
	68	+$wgMainCacheType = CACHE_NONE;
	69	+$wgMemCachedServers = array();
	70	+
	71	+## To enable image uploads, make sure the 'images' directory
	72	+## is writable, then set this to true:
	73	+$wgEnableUploads = true;
	74	+#$wgUseImageMagick = true;
	75	+#$wgImageMagickConvertCommand = "/usr/bin/convert";
	76	+
	77	+# InstantCommons allows wiki to use images from http://commons.wikimedia.org
	78	+$wgUseInstantCommons = false;
	79	+
	80	+## If you use ImageMagick (or any other shell command) on a
	81	+## Linux server, this will need to be set to the name of an
	82	+## available UTF-8 locale
	83	+$wgShellLocale = "en_US.utf8";
	84	+
	85	+## If you want to use image uploads under safe mode,
	86	+## create the directories images/archive, images/thumb and
	87	+## images/temp, and make them all writable. Then uncomment
	88	+## this, if it's not already uncommented:
	89	+$wgHashedUploadDirectory = true;
	90	+
	91	+## If you have the appropriate support software installed
	92	+## you can enable inline LaTeX equations:
	93	+$wgUseTeX = false;
	94	+
	95	+## Set $wgCacheDirectory to a writable directory on the web server
	96	+## to make your wiki go slightly faster. The directory should not
	97	+## be publically accessible from the web.
	98	+#$wgCacheDirectory = "$IP/cache";
	99	+
	100	+# Site language code, should be one of ./languages/Language(.*).php
	101	+$wgLanguageCode = "en";
	102	+
	103	+$wgSecretKey = "790d7ee286c2fb4187b434ceb0fef12b45f30023a3d4613af8cddb164d7705aa";
	104	+
	105	+# Site upgrade key. Must be set to a string (default provided) to turn on the
	106	+# web installer while LocalSettings.php is in place
	107	+$wgUpgradeKey = "ab66efc4edb006fd";
	108	+
	109	+## Default skin: you can change the default skin. Use the internal symbolic
	110	+## names, ie 'standard', 'nostalgia', 'cologneblue', 'monobook', 'vector':
	111	+$wgDefaultSkin = "vector";
	112	+
	113	+## For attaching licensing metadata to pages, and displaying an
	114	+## appropriate copyright notice / icon. GNU Free Documentation
	115	+## License and Creative Commons licenses are supported so far.
	116	+#$wgEnableCreativeCommonsRdf = true;
	117	+$wgRightsPage = ""; # Set to the title of a wiki page that describes your license/copyright
	118	+$wgRightsUrl = "";
	119	+$wgRightsText = "";
	120	+$wgRightsIcon = "";
	121	+# $wgRightsCode = ""; # Not yet used
	122	+
	123	+# Path to the GNU diff3 utility. Used for conflict resolution.
	124	+$wgDiff3 = "/usr/bin/diff3";
	125	+
	126	+
	127	+# End of automatically generated settings.
	128	+# Add more configuration options below.
	129	+
	130	+if (1) {
	131	+
	132	+require_once( "$IP/extensions/SwiftMedia/SwiftMedia.php" );
	133	+
	134	+$wgUploadDirectory = "$IP/images/swift";
	135	+$wgDeletedDirectory = "{$wgUploadDirectory}/deleted";
	136	+$wgUploadPath = "http://alsted.wikimedia.org/images/swift";
	137	+
	138	+$wgLocalFileRepo = array(
	139	+ 'class' => 'SwiftRepo',
	140	+ 'name' => 'swift',
	141	+ #'directory' => 'http://alsted.wikimedia.org/images', #$wgUploadDirectory,
	142	+ 'directory' => $wgUploadDirectory,
	143	+ 'user' => 'system:media',
	144	+ 'key' => '8lksg0p',
	145	+ 'authurl' => 'http://alsted.wikimedia.org/auth/v1.0',
	146	+ 'container' => 'images%2Fswift',
	147	+ 'scriptDirUrl' => $wgScriptPath,
	148	+ 'scriptExtension' => $wgScriptExtension,
	149	+ 'url' => $wgUploadBaseUrl ? $wgUploadBaseUrl . $wgUploadPath : $wgUploadPath,
	150	+ 'hashLevels' => $wgHashedUploadDirectory ? 2 : 0,
	151	+ 'thumbScriptUrl' => $wgThumbnailScriptPath,
	152	+ 'transformVia404' => !$wgGenerateThumbnailOnParse,
	153	+ 'deletedDir' => $wgDeletedDirectory,
	154	+ 'deletedHashLevels' => 3
	155	+);
	156	+}
	157	+
	158	+$wgDebugLogFile = "/var/www/debug/abcd";
	159	+
Index: trunk/extensions/SwiftMedia/SwiftMedia.php
—	—	@@ -0,0 +1,18 @@
	2	+<?php
	3	+$wgExtensionCredits['other'][] = array(
	4	+ 'path' => __FILE__, // File name for the extension itself, required for getting the revision number from SVN - string, adding in 1.15
	5	+ 'name' => "SwiftMedia", // Name of extension - string
	6	+ 'descriptionmsg' => "swiftmedia", // Same as above but name of a message, for i18n - string, added in 1.12.0
	7	+ 'version' => 0, // Version number of extension - number or string
	8	+ 'author' => "Russ Nelson", // The extension author's name - string or array for multiple
	9	+ 'url' => "http://www.mediawiki.org/wiki/Extension:SwiftMedia", // URL of extension (usually instructions) - string
	10	+);
	11	+
	12	+$wgAutoloadClasses['SwiftFile'] =
	13	+$wgAutoloadClasses['SwiftRepo'] = dirname(__FILE__) . '/SwiftMedia.body.php';
	14	+$wgAutoloadClasses['CF_Authentication'] =
	15	+$wgAutoloadClasses['CF_Connection'] =
	16	+$wgAutoloadClasses['CF_Container'] =
	17	+$wgAutoloadClasses['CF_Object'] = '/usr/share/php-cloudfiles/cloudfiles.php';
	18	+
	19	+$wgExtensionMessagesFiles['swiftmedia'] = dirname( __FILE__ ) . '/SwiftMedia.i18n.php';
Index: trunk/extensions/SwiftMedia/proxy-server.conf
—	—	@@ -0,0 +1,39 @@
	2	+[DEFAULT]
	3	+#cert_file = /etc/swift/cert.crt
	4	+#key_file = /etc/swift/cert.key
	5	+bind_port = 80
	6	+workers = 8
	7	+user = swift
	8	+
	9	+[pipeline:main]
	10	+pipeline = rewrite healthcheck cache swauth proxy-server
	11	+#pipeline = healthcheck cache swauth proxy-server
	12	+
	13	+[app:proxy-server]
	14	+use = egg:swift#proxy
	15	+allow_account_management = true
	16	+
	17	+#[filter:auth]
	18	+#use = egg:swift#auth
	19	+#ssl = true
	20	+
	21	+[filter:swauth]
	22	+use = egg:swift#swauth
	23	+#default_swift_cluster = local#https://$PROXY_LOCAL_NET_IP:8080/v1
	24	+default_swift_cluster = local#http://alsted.wikimedia.org/v1
	25	+# Highly recommended to change this key to something else!
	26	+super_admin_key = j0kans
	27	+
	28	+[filter:healthcheck]
	29	+use = egg:swift#healthcheck
	30	+
	31	+[filter:cache]
	32	+use = egg:swift#memcache
	33	+memcache_servers = 127.0.0.1:11211
	34	+
	35	+[filter:rewrite]
	36	+url = http://127.0.0.1/auth/v1.0
	37	+login = system:media
	38	+key = 8lksg0p
	39	+paste.filter_factory = wmf.rewrite:filter_factory
	40	+
Index: trunk/extensions/SwiftMedia/wmf/client.py
—	—	@@ -0,0 +1,883 @@
	2	+# Copyright (c) 2010 OpenStack, LLC.
	3	+#
	4	+# Licensed under the Apache License, Version 2.0 (the "License");
	5	+# you may not use this file except in compliance with the License.
	6	+# You may obtain a copy of the License at
	7	+#
	8	+# http://www.apache.org/licenses/LICENSE-2.0
	9	+#
	10	+# Unless required by applicable law or agreed to in writing, software
	11	+# distributed under the License is distributed on an "AS IS" BASIS,
	12	+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
	13	+# implied.
	14	+# See the License for the specific language governing permissions and
	15	+# limitations under the License.
	16	+
	17	+"""
	18	+Cloud Files client library used internally
	19	+"""
	20	+import socket
	21	+from cStringIO import StringIO
	22	+from httplib import HTTPException, HTTPSConnection
	23	+from re import compile, DOTALL
	24	+from tokenize import generate_tokens, STRING, NAME, OP
	25	+from urllib import quote as _quote, unquote
	26	+from urlparse import urlparse, urlunparse
	27	+
	28	+try:
	29	+ from eventlet import sleep
	30	+except:
	31	+ from time import sleep
	32	+
	33	+from swift.common.bufferedhttp \
	34	+ import BufferedHTTPConnection as HTTPConnection
	35	+
	36	+def quote(value, safe='/'):
	37	+ """
	38	+ Patched version of urllib.quote that encodes utf8 strings before quoting
	39	+ """
	40	+ if isinstance(value, unicode):
	41	+ value = value.encode('utf8')
	42	+ return _quote(value, safe)
	43	+
	44	+
	45	+# look for a real json parser first
	46	+try:
	47	+ # simplejson is popular and pretty good
	48	+ from simplejson import loads as json_loads
	49	+except ImportError:
	50	+ try:
	51	+ # 2.6 will have a json module in the stdlib
	52	+ from json import loads as json_loads
	53	+ except ImportError:
	54	+ # fall back on local parser otherwise
	55	+ comments = compile(r'/\.\/\|//[^\r\n]', DOTALL)
	56	+
	57	+ def json_loads(string):
	58	+ '''
	59	+ Fairly competent json parser exploiting the python tokenizer and
	60	+ eval(). -- From python-cloudfiles
	61	+
	62	+ _loads(serialized_json) -> object
	63	+ '''
	64	+ try:
	65	+ res = []
	66	+ consts = {'true': True, 'false': False, 'null': None}
	67	+ string = '(' + comments.sub('', string) + ')'
	68	+ for type, val, _, _, _ in \
	69	+ generate_tokens(StringIO(string).readline):
	70	+ if (type == OP and val not in '[]{}:,()-') or \
	71	+ (type == NAME and val not in consts):
	72	+ raise AttributeError()
	73	+ elif type == STRING:
	74	+ res.append('u')
	75	+ res.append(val.replace('\\/', '/'))
	76	+ else:
	77	+ res.append(val)
	78	+ return eval(''.join(res), {}, consts)
	79	+ except:
	80	+ raise AttributeError()
	81	+
	82	+
	83	+class ClientException(Exception):
	84	+
	85	+ def __init__(self, msg, http_scheme='', http_host='', http_port='',
	86	+ http_path='', http_query='', http_status=0, http_reason='',
	87	+ http_device=''):
	88	+ Exception.__init__(self, msg)
	89	+ self.msg = msg
	90	+ self.http_scheme = http_scheme
	91	+ self.http_host = http_host
	92	+ self.http_port = http_port
	93	+ self.http_path = http_path
	94	+ self.http_query = http_query
	95	+ self.http_status = http_status
	96	+ self.http_reason = http_reason
	97	+ self.http_device = http_device
	98	+
	99	+ def __str__(self):
	100	+ a = self.msg
	101	+ b = ''
	102	+ if self.http_scheme:
	103	+ b += '%s://' % self.http_scheme
	104	+ if self.http_host:
	105	+ b += self.http_host
	106	+ if self.http_port:
	107	+ b += ':%s' % self.http_port
	108	+ if self.http_path:
	109	+ b += self.http_path
	110	+ if self.http_query:
	111	+ b += '?%s' % self.http_query
	112	+ if self.http_status:
	113	+ if b:
	114	+ b = '%s %s' % (b, self.http_status)
	115	+ else:
	116	+ b = str(self.http_status)
	117	+ if self.http_reason:
	118	+ if b:
	119	+ b = '%s %s' % (b, self.http_reason)
	120	+ else:
	121	+ b = '- %s' % self.http_reason
	122	+ if self.http_device:
	123	+ if b:
	124	+ b = '%s: device %s' % (b, self.http_device)
	125	+ else:
	126	+ b = 'device %s' % self.http_device
	127	+ return b and '%s: %s' % (a, b) or a
	128	+
	129	+
	130	+def http_connection(url):
	131	+ """
	132	+ Make an HTTPConnection or HTTPSConnection
	133	+
	134	+ :param url: url to connect to
	135	+ :returns: tuple of (parsed url, connection object)
	136	+ :raises ClientException: Unable to handle protocol scheme
	137	+ """
	138	+ parsed = urlparse(url)
	139	+ if parsed.scheme == 'http':
	140	+ conn = HTTPConnection(parsed.netloc)
	141	+ elif parsed.scheme == 'https':
	142	+ conn = HTTPSConnection(parsed.netloc)
	143	+ else:
	144	+ raise ClientException('Cannot handle protocol scheme %s for url %s' %
	145	+ (parsed.scheme, repr(url)))
	146	+ return parsed, conn
	147	+
	148	+
	149	+def get_auth(url, user, key, snet=False):
	150	+ """
	151	+ Get authentication/authorization credentials.
	152	+
	153	+ The snet parameter is used for Rackspace's ServiceNet internal network
	154	+ implementation. In this function, it simply adds snet- to the beginning
	155	+ of the host name for the returned storage URL. With Rackspace Cloud Files,
	156	+ use of this network path causes no bandwidth charges but requires the
	157	+ client to be running on Rackspace's ServiceNet network.
	158	+
	159	+ :param url: authentication/authorization URL
	160	+ :param user: user to authenticate as
	161	+ :param key: key or password for authorization
	162	+ :param snet: use SERVICENET internal network (see above), default is False
	163	+ :returns: tuple of (storage URL, auth token)
	164	+ :raises ClientException: HTTP GET request to auth URL failed
	165	+ """
	166	+ parsed, conn = http_connection(url)
	167	+ conn.request('GET', parsed.path, '',
	168	+ {'X-Auth-User': user, 'X-Auth-Key': key})
	169	+ resp = conn.getresponse()
	170	+ resp.read()
	171	+ if resp.status < 200 or resp.status >= 300:
	172	+ raise ClientException('Auth GET failed', http_scheme=parsed.scheme,
	173	+ http_host=conn.host, http_port=conn.port,
	174	+ http_path=parsed.path, http_status=resp.status,
	175	+ http_reason=resp.reason)
	176	+ url = resp.getheader('x-storage-url')
	177	+ if snet:
	178	+ parsed = list(urlparse(url))
	179	+ # Second item in the list is the netloc
	180	+ parsed[1] = 'snet-' + parsed[1]
	181	+ url = urlunparse(parsed)
	182	+ return url, resp.getheader('x-storage-token',
	183	+ resp.getheader('x-auth-token'))
	184	+
	185	+
	186	+def get_account(url, token, marker=None, limit=None, prefix=None,
	187	+ http_conn=None, full_listing=False):
	188	+ """
	189	+ Get a listing of containers for the account.
	190	+
	191	+ :param url: storage URL
	192	+ :param token: auth token
	193	+ :param marker: marker query
	194	+ :param limit: limit query
	195	+ :param prefix: prefix query
	196	+ :param http_conn: HTTP connection object (If None, it will create the
	197	+ conn object)
	198	+ :param full_listing: if True, return a full listing, else returns a max
	199	+ of 10000 listings
	200	+ :returns: a tuple of (response headers, a list of containers) The response
	201	+ headers will be a dict and all header names will be lowercase.
	202	+ :raises ClientException: HTTP GET request failed
	203	+ """
	204	+ if not http_conn:
	205	+ http_conn = http_connection(url)
	206	+ if full_listing:
	207	+ rv = get_account(url, token, marker, limit, prefix, http_conn)
	208	+ listing = rv[1]
	209	+ while listing:
	210	+ marker = listing[-1]['name']
	211	+ listing = \
	212	+ get_account(url, token, marker, limit, prefix, http_conn)[1]
	213	+ if listing:
	214	+ rv.extend(listing)
	215	+ return rv
	216	+ parsed, conn = http_conn
	217	+ qs = 'format=json'
	218	+ if marker:
	219	+ qs += '&marker=%s' % quote(marker)
	220	+ if limit:
	221	+ qs += '&limit=%d' % limit
	222	+ if prefix:
	223	+ qs += '&prefix=%s' % quote(prefix)
	224	+ conn.request('GET', '%s?%s' % (parsed.path, qs), '',
	225	+ {'X-Auth-Token': token})
	226	+ resp = conn.getresponse()
	227	+ resp_headers = {}
	228	+ for header, value in resp.getheaders():
	229	+ resp_headers[header.lower()] = value
	230	+ if resp.status < 200 or resp.status >= 300:
	231	+ resp.read()
	232	+ raise ClientException('Account GET failed', http_scheme=parsed.scheme,
	233	+ http_host=conn.host, http_port=conn.port,
	234	+ http_path=parsed.path, http_query=qs, http_status=resp.status,
	235	+ http_reason=resp.reason)
	236	+ if resp.status == 204:
	237	+ resp.read()
	238	+ return resp_headers, []
	239	+ return resp_headers, json_loads(resp.read())
	240	+
	241	+
	242	+def head_account(url, token, http_conn=None):
	243	+ """
	244	+ Get account stats.
	245	+
	246	+ :param url: storage URL
	247	+ :param token: auth token
	248	+ :param http_conn: HTTP connection object (If None, it will create the
	249	+ conn object)
	250	+ :returns: a dict containing the response's headers (all header names will
	251	+ be lowercase)
	252	+ :raises ClientException: HTTP HEAD request failed
	253	+ """
	254	+ if http_conn:
	255	+ parsed, conn = http_conn
	256	+ else:
	257	+ parsed, conn = http_connection(url)
	258	+ conn.request('HEAD', parsed.path, '', {'X-Auth-Token': token})
	259	+ resp = conn.getresponse()
	260	+ resp.read()
	261	+ if resp.status < 200 or resp.status >= 300:
	262	+ raise ClientException('Account HEAD failed', http_scheme=parsed.scheme,
	263	+ http_host=conn.host, http_port=conn.port,
	264	+ http_path=parsed.path, http_status=resp.status,
	265	+ http_reason=resp.reason)
	266	+ resp_headers = {}
	267	+ for header, value in resp.getheaders():
	268	+ resp_headers[header.lower()] = value
	269	+ return resp_headers
	270	+
	271	+
	272	+def post_account(url, token, headers, http_conn=None):
	273	+ """
	274	+ Update an account's metadata.
	275	+
	276	+ :param url: storage URL
	277	+ :param token: auth token
	278	+ :param headers: additional headers to include in the request
	279	+ :param http_conn: HTTP connection object (If None, it will create the
	280	+ conn object)
	281	+ :raises ClientException: HTTP POST request failed
	282	+ """
	283	+ if http_conn:
	284	+ parsed, conn = http_conn
	285	+ else:
	286	+ parsed, conn = http_connection(url)
	287	+ headers['X-Auth-Token'] = token
	288	+ conn.request('POST', parsed.path, '', headers)
	289	+ resp = conn.getresponse()
	290	+ resp.read()
	291	+ if resp.status < 200 or resp.status >= 300:
	292	+ raise ClientException('Account POST failed',
	293	+ http_scheme=parsed.scheme, http_host=conn.host,
	294	+ http_port=conn.port, http_path=path, http_status=resp.status,
	295	+ http_reason=resp.reason)
	296	+
	297	+
	298	+def get_container(url, token, container, marker=None, limit=None,
	299	+ prefix=None, delimiter=None, http_conn=None,
	300	+ full_listing=False):
	301	+ """
	302	+ Get a listing of objects for the container.
	303	+
	304	+ :param url: storage URL
	305	+ :param token: auth token
	306	+ :param container: container name to get a listing for
	307	+ :param marker: marker query
	308	+ :param limit: limit query
	309	+ :param prefix: prefix query
	310	+ :param delimeter: string to delimit the queries on
	311	+ :param http_conn: HTTP connection object (If None, it will create the
	312	+ conn object)
	313	+ :param full_listing: if True, return a full listing, else returns a max
	314	+ of 10000 listings
	315	+ :returns: a tuple of (response headers, a list of objects) The response
	316	+ headers will be a dict and all header names will be lowercase.
	317	+ :raises ClientException: HTTP GET request failed
	318	+ """
	319	+ if not http_conn:
	320	+ http_conn = http_connection(url)
	321	+ if full_listing:
	322	+ rv = get_container(url, token, container, marker, limit, prefix,
	323	+ delimiter, http_conn)
	324	+ listing = rv[1]
	325	+ while listing:
	326	+ if not delimiter:
	327	+ marker = listing[-1]['name']
	328	+ else:
	329	+ marker = listing[-1].get('name', listing[-1].get('subdir'))
	330	+ listing = get_container(url, token, container, marker, limit,
	331	+ prefix, delimiter, http_conn)[1]
	332	+ if listing:
	333	+ rv[1].extend(listing)
	334	+ return rv
	335	+ parsed, conn = http_conn
	336	+ path = '%s/%s' % (parsed.path, quote(container))
	337	+ qs = 'format=json'
	338	+ if marker:
	339	+ qs += '&marker=%s' % quote(marker)
	340	+ if limit:
	341	+ qs += '&limit=%d' % limit
	342	+ if prefix:
	343	+ qs += '&prefix=%s' % quote(prefix)
	344	+ if delimiter:
	345	+ qs += '&delimiter=%s' % quote(delimiter)
	346	+ conn.request('GET', '%s?%s' % (path, qs), '', {'X-Auth-Token': token})
	347	+ resp = conn.getresponse()
	348	+ if resp.status < 200 or resp.status >= 300:
	349	+ resp.read()
	350	+ raise ClientException('Container GET failed',
	351	+ http_scheme=parsed.scheme, http_host=conn.host,
	352	+ http_port=conn.port, http_path=path, http_query=qs,
	353	+ http_status=resp.status, http_reason=resp.reason)
	354	+ resp_headers = {}
	355	+ for header, value in resp.getheaders():
	356	+ resp_headers[header.lower()] = value
	357	+ if resp.status == 204:
	358	+ resp.read()
	359	+ return resp_headers, []
	360	+ return resp_headers, json_loads(resp.read())
	361	+
	362	+
	363	+def head_container(url, token, container, http_conn=None):
	364	+ """
	365	+ Get container stats.
	366	+
	367	+ :param url: storage URL
	368	+ :param token: auth token
	369	+ :param container: container name to get stats for
	370	+ :param http_conn: HTTP connection object (If None, it will create the
	371	+ conn object)
	372	+ :returns: a dict containing the response's headers (all header names will
	373	+ be lowercase)
	374	+ :raises ClientException: HTTP HEAD request failed
	375	+ """
	376	+ if http_conn:
	377	+ parsed, conn = http_conn
	378	+ else:
	379	+ parsed, conn = http_connection(url)
	380	+ path = '%s/%s' % (parsed.path, quote(container))
	381	+ conn.request('HEAD', path, '', {'X-Auth-Token': token})
	382	+ resp = conn.getresponse()
	383	+ resp.read()
	384	+ if resp.status < 200 or resp.status >= 300:
	385	+ raise ClientException('Container HEAD failed',
	386	+ http_scheme=parsed.scheme, http_host=conn.host,
	387	+ http_port=conn.port, http_path=path, http_status=resp.status,
	388	+ http_reason=resp.reason)
	389	+ resp_headers = {}
	390	+ for header, value in resp.getheaders():
	391	+ resp_headers[header.lower()] = value
	392	+ return resp_headers
	393	+
	394	+
	395	+def put_container(url, token, container, headers=None, http_conn=None):
	396	+ """
	397	+ Create a container
	398	+
	399	+ :param url: storage URL
	400	+ :param token: auth token
	401	+ :param container: container name to create
	402	+ :param headers: additional headers to include in the request
	403	+ :param http_conn: HTTP connection object (If None, it will create the
	404	+ conn object)
	405	+ :raises ClientException: HTTP PUT request failed
	406	+ """
	407	+ if http_conn:
	408	+ parsed, conn = http_conn
	409	+ else:
	410	+ parsed, conn = http_connection(url)
	411	+ path = '%s/%s' % (parsed.path, quote(container))
	412	+ if not headers:
	413	+ headers = {}
	414	+ headers['X-Auth-Token'] = token
	415	+ conn.request('PUT', path, '', headers)
	416	+ resp = conn.getresponse()
	417	+ resp.read()
	418	+ if resp.status < 200 or resp.status >= 300:
	419	+ raise ClientException('Container PUT failed',
	420	+ http_scheme=parsed.scheme, http_host=conn.host,
	421	+ http_port=conn.port, http_path=path, http_status=resp.status,
	422	+ http_reason=resp.reason)
	423	+
	424	+
	425	+def post_container(url, token, container, headers, http_conn=None):
	426	+ """
	427	+ Update a container's metadata.
	428	+
	429	+ :param url: storage URL
	430	+ :param token: auth token
	431	+ :param container: container name to update
	432	+ :param headers: additional headers to include in the request
	433	+ :param http_conn: HTTP connection object (If None, it will create the
	434	+ conn object)
	435	+ :raises ClientException: HTTP POST request failed
	436	+ """
	437	+ if http_conn:
	438	+ parsed, conn = http_conn
	439	+ else:
	440	+ parsed, conn = http_connection(url)
	441	+ path = '%s/%s' % (parsed.path, quote(container))
	442	+ headers['X-Auth-Token'] = token
	443	+ conn.request('POST', path, '', headers)
	444	+ resp = conn.getresponse()
	445	+ resp.read()
	446	+ if resp.status < 200 or resp.status >= 300:
	447	+ raise ClientException('Container POST failed',
	448	+ http_scheme=parsed.scheme, http_host=conn.host,
	449	+ http_port=conn.port, http_path=path, http_status=resp.status,
	450	+ http_reason=resp.reason)
	451	+
	452	+
	453	+def delete_container(url, token, container, http_conn=None):
	454	+ """
	455	+ Delete a container
	456	+
	457	+ :param url: storage URL
	458	+ :param token: auth token
	459	+ :param container: container name to delete
	460	+ :param http_conn: HTTP connection object (If None, it will create the
	461	+ conn object)
	462	+ :raises ClientException: HTTP DELETE request failed
	463	+ """
	464	+ if http_conn:
	465	+ parsed, conn = http_conn
	466	+ else:
	467	+ parsed, conn = http_connection(url)
	468	+ path = '%s/%s' % (parsed.path, quote(container))
	469	+ conn.request('DELETE', path, '', {'X-Auth-Token': token})
	470	+ resp = conn.getresponse()
	471	+ resp.read()
	472	+ if resp.status < 200 or resp.status >= 300:
	473	+ raise ClientException('Container DELETE failed',
	474	+ http_scheme=parsed.scheme, http_host=conn.host,
	475	+ http_port=conn.port, http_path=path, http_status=resp.status,
	476	+ http_reason=resp.reason)
	477	+
	478	+
	479	+def get_object(url, token, container, name, http_conn=None,
	480	+ resp_chunk_size=None):
	481	+ """
	482	+ Get an object
	483	+
	484	+ :param url: storage URL
	485	+ :param token: auth token
	486	+ :param container: container name that the object is in
	487	+ :param name: object name to get
	488	+ :param http_conn: HTTP connection object (If None, it will create the
	489	+ conn object)
	490	+ :param resp_chunk_size: if defined, chunk size of data to read. NOTE: If
	491	+ you specify a resp_chunk_size you must fully read
	492	+ the object's contents before making another
	493	+ request.
	494	+ :returns: a tuple of (response headers, the object's contents) The response
	495	+ headers will be a dict and all header names will be lowercase.
	496	+ :raises ClientException: HTTP GET request failed
	497	+ """
	498	+ if http_conn:
	499	+ parsed, conn = http_conn
	500	+ else:
	501	+ parsed, conn = http_connection(url)
	502	+ path = '%s/%s/%s' % (parsed.path, quote(container), quote(name))
	503	+ conn.request('GET', path, '', {'X-Auth-Token': token})
	504	+ resp = conn.getresponse()
	505	+ if resp.status < 200 or resp.status >= 300:
	506	+ resp.read()
	507	+ raise ClientException('Object GET failed', http_scheme=parsed.scheme,
	508	+ http_host=conn.host, http_port=conn.port, http_path=path,
	509	+ http_status=resp.status, http_reason=resp.reason)
	510	+ if resp_chunk_size:
	511	+
	512	+ def _object_body():
	513	+ buf = resp.read(resp_chunk_size)
	514	+ while buf:
	515	+ yield buf
	516	+ buf = resp.read(resp_chunk_size)
	517	+ object_body = _object_body()
	518	+ else:
	519	+ object_body = resp.read()
	520	+ resp_headers = {}
	521	+ for header, value in resp.getheaders():
	522	+ resp_headers[header.lower()] = value
	523	+ return resp_headers, object_body
	524	+
	525	+
	526	+def head_object(url, token, container, name, http_conn=None):
	527	+ """
	528	+ Get object info
	529	+
	530	+ :param url: storage URL
	531	+ :param token: auth token
	532	+ :param container: container name that the object is in
	533	+ :param name: object name to get info for
	534	+ :param http_conn: HTTP connection object (If None, it will create the
	535	+ conn object)
	536	+ :returns: a dict containing the response's headers (all header names will
	537	+ be lowercase)
	538	+ :raises ClientException: HTTP HEAD request failed
	539	+ """
	540	+ if http_conn:
	541	+ parsed, conn = http_conn
	542	+ else:
	543	+ parsed, conn = http_connection(url)
	544	+ path = '%s/%s/%s' % (parsed.path, quote(container), quote(name))
	545	+ conn.request('HEAD', path, '', {'X-Auth-Token': token})
	546	+ resp = conn.getresponse()
	547	+ resp.read()
	548	+ if resp.status < 200 or resp.status >= 300:
	549	+ raise ClientException('Object HEAD failed', http_scheme=parsed.scheme,
	550	+ http_host=conn.host, http_port=conn.port, http_path=path,
	551	+ http_status=resp.status, http_reason=resp.reason)
	552	+ resp_headers = {}
	553	+ for header, value in resp.getheaders():
	554	+ resp_headers[header.lower()] = value
	555	+ return resp_headers
	556	+
	557	+class Put_object_chunked(object):
	558	+ """
	559	+ The standard put_object() requires either the data itself, or a function
	560	+ with an 'read()', which it calls to fetch data. Unfortunately, that
	561	+ doesn't work if you have your own loop which generates the data, or if
	562	+ you're writing the data from an iterator (think "writing the same data
	563	+ to two locations.")
	564	+ """
	565	+ def __init__(self, url, token, container, name,
	566	+ etag=None, content_type=None, headers=None,
	567	+ http_conn=None):
	568	+ """
	569	+ Open an object for writing.
	570	+
	571	+ :param url: storage URL
	572	+ :param token: auth token
	573	+ :param container: container name that the object is in
	574	+ :param name: object name to put
	575	+ :param etag: etag of contents
	576	+ :param content_type: value to send as content-type header
	577	+ :param headers: additional headers to include in the request
	578	+ :param http_conn: HTTP connection object (If None, it will create the
	579	+ conn object)
	580	+ """
	581	+ if http_conn:
	582	+ parsed, conn = http_conn
	583	+ else:
	584	+ parsed, conn = http_connection(url)
	585	+ path = '%s/%s/%s' % (parsed.path, container, name)
	586	+ if not headers:
	587	+ headers = {}
	588	+ headers['X-Auth-Token'] = token
	589	+ if etag:
	590	+ headers['ETag'] = etag.strip('"')
	591	+ if content_type is not None:
	592	+ headers['Content-Type'] = content_type
	593	+ # http://en.wikipedia.org/wiki/Chunked_transfer_encoding
	594	+ headers['Transfer-Encoding'] = 'chunked'
	595	+ conn.putrequest('PUT', path)
	596	+ for header, value in headers.iteritems():
	597	+ conn.putheader(header, value)
	598	+ conn.endheaders()
	599	+ self.path = path
	600	+ self.parsed = parsed
	601	+ self.conn = conn
	602	+
	603	+ def write(self, chunk):
	604	+ """
	605	+ Write a chunk of data.
	606	+
	607	+ :param chunk: a chunk of data.
	608	+ """
	609	+ size = len(chunk)
	610	+ if size:
	611	+ # only send non-null data, and besides, 0 length is EOF
	612	+ self.conn.send('%x\r\n%s\r\n' % (size, chunk))
	613	+
	614	+ def close(self):
	615	+ """
	616	+ Write the last chunk and return the response.
	617	+
	618	+ :returns The response from the swift proxy
	619	+ :raises ClientException: HTTP PUT request failed
	620	+ """
	621	+ # finish off the last chunk, get the response, and return the etag.
	622	+ if not self.conn: return ''
	623	+ self.conn.send('0\r\n\r\n')
	624	+ resp = self.conn.getresponse()
	625	+ resp.read()
	626	+ if resp.status < 200 or resp.status >= 300:
	627	+ raise ClientException('Object PUT failed',
	628	+ http_scheme=self.parsed.scheme, http_host=self.conn.host,
	629	+ http_port=self.conn.port, http_path=self.path,
	630	+ http_status=resp.status, http_reason=resp.reason)
	631	+ self.conn = None
	632	+ return resp.getheader('etag').strip('"')
	633	+
	634	+ __del__ = close # in case they forget
	635	+
	636	+def put_object(url, token, container, name, contents, content_length=None,
	637	+ etag=None, chunk_size=65536, content_type=None, headers=None,
	638	+ http_conn=None):
	639	+ """
	640	+ Put an object
	641	+
	642	+ :param url: storage URL
	643	+ :param token: auth token
	644	+ :param container: container name that the object is in
	645	+ :param name: object name to put
	646	+ :param contents: a string or a file like object to read object data from
	647	+ :param content_length: value to send as content-length header
	648	+ :param etag: etag of contents
	649	+ :param chunk_size: chunk size of data to write
	650	+ :param content_type: value to send as content-type header
	651	+ :param headers: additional headers to include in the request
	652	+ :param http_conn: HTTP connection object (If None, it will create the
	653	+ conn object)
	654	+ :returns: etag from server response
	655	+ :raises ClientException: HTTP PUT request failed
	656	+ """
	657	+ if http_conn:
	658	+ parsed, conn = http_conn
	659	+ else:
	660	+ parsed, conn = http_connection(url)
	661	+ path = '%s/%s/%s' % (parsed.path, quote(container), quote(name))
	662	+ if not headers:
	663	+ headers = {}
	664	+ headers['X-Auth-Token'] = token
	665	+ if etag:
	666	+ headers['ETag'] = etag.strip('"')
	667	+ if content_length is not None:
	668	+ headers['Content-Length'] = str(content_length)
	669	+ if content_type is not None:
	670	+ headers['Content-Type'] = content_type
	671	+ if not contents:
	672	+ headers['Content-Length'] = '0'
	673	+ if hasattr(contents, 'read'):
	674	+ conn.putrequest('PUT', path)
	675	+ for header, value in headers.iteritems():
	676	+ conn.putheader(header, value)
	677	+ if not content_length:
	678	+ conn.putheader('Transfer-Encoding', 'chunked')
	679	+ conn.endheaders()
	680	+ chunk = contents.read(chunk_size)
	681	+ while chunk:
	682	+ if not content_length:
	683	+ conn.send('%x\r\n%s\r\n' % (len(chunk), chunk))
	684	+ else:
	685	+ conn.send(chunk)
	686	+ chunk = contents.read(chunk_size)
	687	+ if not content_length:
	688	+ conn.send('0\r\n\r\n')
	689	+ else:
	690	+ conn.request('PUT', path, contents, headers)
	691	+ resp = conn.getresponse()
	692	+ resp.read()
	693	+ if resp.status < 200 or resp.status >= 300:
	694	+ raise ClientException('Object PUT failed', http_scheme=parsed.scheme,
	695	+ http_host=conn.host, http_port=conn.port, http_path=path,
	696	+ http_status=resp.status, http_reason=resp.reason)
	697	+ return resp.getheader('etag').strip('"')
	698	+
	699	+
	700	+def post_object(url, token, container, name, headers, http_conn=None):
	701	+ """
	702	+ Update object metadata
	703	+
	704	+ :param url: storage URL
	705	+ :param token: auth token
	706	+ :param container: container name that the object is in
	707	+ :param name: name of the object to update
	708	+ :param headers: additional headers to include in the request
	709	+ :param http_conn: HTTP connection object (If None, it will create the
	710	+ conn object)
	711	+ :raises ClientException: HTTP POST request failed
	712	+ """
	713	+ if http_conn:
	714	+ parsed, conn = http_conn
	715	+ else:
	716	+ parsed, conn = http_connection(url)
	717	+ path = '%s/%s/%s' % (parsed.path, quote(container), quote(name))
	718	+ headers['X-Auth-Token'] = token
	719	+ conn.request('POST', path, '', headers)
	720	+ resp = conn.getresponse()
	721	+ resp.read()
	722	+ if resp.status < 200 or resp.status >= 300:
	723	+ raise ClientException('Object POST failed', http_scheme=parsed.scheme,
	724	+ http_host=conn.host, http_port=conn.port, http_path=path,
	725	+ http_status=resp.status, http_reason=resp.reason)
	726	+
	727	+
	728	+def delete_object(url, token, container, name, http_conn=None):
	729	+ """
	730	+ Delete object
	731	+
	732	+ :param url: storage URL
	733	+ :param token: auth token
	734	+ :param container: container name that the object is in
	735	+ :param name: object name to delete
	736	+ :param http_conn: HTTP connection object (If None, it will create the
	737	+ conn object)
	738	+ :raises ClientException: HTTP DELETE request failed
	739	+ """
	740	+ if http_conn:
	741	+ parsed, conn = http_conn
	742	+ else:
	743	+ parsed, conn = http_connection(url)
	744	+ path = '%s/%s/%s' % (parsed.path, quote(container), quote(name))
	745	+ conn.request('DELETE', path, '', {'X-Auth-Token': token})
	746	+ resp = conn.getresponse()
	747	+ resp.read()
	748	+ if resp.status < 200 or resp.status >= 300:
	749	+ raise ClientException('Object DELETE failed',
	750	+ http_scheme=parsed.scheme, http_host=conn.host,
	751	+ http_port=conn.port, http_path=path, http_status=resp.status,
	752	+ http_reason=resp.reason)
	753	+
	754	+
	755	+class Connection(object):
	756	+ """Convenience class to make requests that will also retry the request"""
	757	+
	758	+ def __init__(self, authurl, user, key, retries=5, preauthurl=None,
	759	+ preauthtoken=None, snet=False):
	760	+ """
	761	+ :param authurl: authenitcation URL
	762	+ :param user: user name to authenticate as
	763	+ :param key: key/password to authenticate with
	764	+ :param retries: Number of times to retry the request before failing
	765	+ :param preauthurl: storage URL (if you have already authenticated)
	766	+ :param preauthtoken: authentication token (if you have already
	767	+ authenticated)
	768	+ :param snet: use SERVICENET internal network default is False
	769	+ """
	770	+ self.authurl = authurl
	771	+ self.user = user
	772	+ self.key = key
	773	+ self.retries = retries
	774	+ self.http_conn = None
	775	+ self.url = preauthurl
	776	+ self.token = preauthtoken
	777	+ self.attempts = 0
	778	+ self.snet = snet
	779	+
	780	+ def get_auth(self):
	781	+ return get_auth(self.authurl, self.user, self.key, snet=self.snet)
	782	+
	783	+ def http_connection(self):
	784	+ return http_connection(self.url)
	785	+
	786	+ def _retry(self, func, args, *kwargs):
	787	+ self.attempts = 0
	788	+ backoff = 1
	789	+ while self.attempts <= self.retries:
	790	+ self.attempts += 1
	791	+ try:
	792	+ if not self.url or not self.token:
	793	+ self.url, self.token = self.get_auth()
	794	+ self.http_conn = None
	795	+ if not self.http_conn:
	796	+ self.http_conn = self.http_connection()
	797	+ kwargs['http_conn'] = self.http_conn
	798	+ rv = func(self.url, self.token, args, *kwargs)
	799	+ return rv
	800	+ except (socket.error, HTTPException):
	801	+ if self.attempts > self.retries:
	802	+ raise
	803	+ self.http_conn = None
	804	+ except ClientException, err:
	805	+ if self.attempts > self.retries:
	806	+ raise
	807	+ if err.http_status == 401:
	808	+ self.url = self.token = None
	809	+ if self.attempts > 1:
	810	+ raise
	811	+ elif 500 <= err.http_status <= 599:
	812	+ pass
	813	+ else:
	814	+ raise
	815	+ sleep(backoff)
	816	+ backoff *= 2
	817	+
	818	+ def head_account(self):
	819	+ """Wrapper for :func:`head_account`"""
	820	+ return self._retry(head_account)
	821	+
	822	+ def get_account(self, marker=None, limit=None, prefix=None,
	823	+ full_listing=False):
	824	+ """Wrapper for :func:`get_account`"""
	825	+ # TODO(unknown): With full_listing=True this will restart the entire
	826	+ # listing with each retry. Need to make a better version that just
	827	+ # retries where it left off.
	828	+ return self._retry(get_account, marker=marker, limit=limit,
	829	+ prefix=prefix, full_listing=full_listing)
	830	+
	831	+ def post_account(self, headers):
	832	+ """Wrapper for :func:`post_account`"""
	833	+ return self._retry(post_account, headers)
	834	+
	835	+ def head_container(self, container):
	836	+ """Wrapper for :func:`head_container`"""
	837	+ return self._retry(head_container, container)
	838	+
	839	+ def get_container(self, container, marker=None, limit=None, prefix=None,
	840	+ delimiter=None, full_listing=False):
	841	+ """Wrapper for :func:`get_container`"""
	842	+ # TODO(unknown): With full_listing=True this will restart the entire
	843	+ # listing with each retry. Need to make a better version that just
	844	+ # retries where it left off.
	845	+ return self._retry(get_container, container, marker=marker,
	846	+ limit=limit, prefix=prefix, delimiter=delimiter,
	847	+ full_listing=full_listing)
	848	+
	849	+ def put_container(self, container, headers=None):
	850	+ """Wrapper for :func:`put_container`"""
	851	+ return self._retry(put_container, container, headers=headers)
	852	+
	853	+ def post_container(self, container, headers):
	854	+ """Wrapper for :func:`post_container`"""
	855	+ return self._retry(post_container, container, headers)
	856	+
	857	+ def delete_container(self, container):
	858	+ """Wrapper for :func:`delete_container`"""
	859	+ return self._retry(delete_container, container)
	860	+
	861	+ def head_object(self, container, obj):
	862	+ """Wrapper for :func:`head_object`"""
	863	+ return self._retry(head_object, container, obj)
	864	+
	865	+ def get_object(self, container, obj, resp_chunk_size=None):
	866	+ """Wrapper for :func:`get_object`"""
	867	+ return self._retry(get_object, container, obj,
	868	+ resp_chunk_size=resp_chunk_size)
	869	+
	870	+ def put_object(self, container, obj, contents, content_length=None,
	871	+ etag=None, chunk_size=65536, content_type=None,
	872	+ headers=None):
	873	+ """Wrapper for :func:`put_object`"""
	874	+ return self._retry(put_object, container, obj, contents,
	875	+ content_length=content_length, etag=etag, chunk_size=chunk_size,
	876	+ content_type=content_type, headers=headers)
	877	+
	878	+ def post_object(self, container, obj, headers):
	879	+ """Wrapper for :func:`post_object`"""
	880	+ return self._retry(post_object, container, obj, headers)
	881	+
	882	+ def delete_object(self, container, obj):
	883	+ """Wrapper for :func:`delete_object`"""
	884	+ return self._retry(delete_object, container, obj)
Property changes on: trunk/extensions/SwiftMedia/wmf/client.py
___________________________________________________________________
Added: svn:eol-style
1	885	+ native
Index: trunk/extensions/SwiftMedia/wmf/__init__.py
—	—	@@ -0,0 +1 @@
	2	+__version__ = '1.1.0'
Property changes on: trunk/extensions/SwiftMedia/wmf/__init__.py
___________________________________________________________________
Added: svn:eol-style
1	3	+ native
Index: trunk/extensions/SwiftMedia/wmf/rewrite.py
—	—	@@ -0,0 +1,162 @@
	2	+# Portions Copyright (c) 2010 OpenStack, LLC.
	3	+# Everything else Copyright (c) 2011 Wikimedia Foundation, Inc.
	4	+# all of it licensed under the Apache Software License, included by reference.
	5	+
	6	+# unit test is in test_rewrite.py. Tests are referenced by numbered comments.
	7	+
	8	+import webob
	9	+import webob.exc
	10	+import re
	11	+from eventlet.green import urllib2
	12	+import wmf.client
	13	+import time
	14	+
	15	+# the auth system turns our login and key into an account / token pair.
	16	+# the account remains valid forever, but the token times out.
	17	+account = 'AUTH_dea4a45c-a80b-43b5-8e8b-e452f0dc778f'
	18	+
	19	+class Copy2(object):
	20	+ """
	21	+ Given an open file and a Swift object, we hand back an iterator which reads from the file,
	22	+ writes a copy into a Swift object, and returns what it read.
	23	+ """
	24	+ token = 'AUTH_tk95804b3cb6a44cfd994c28742cd3333f'
	25	+ token = None
	26	+
	27	+ def __init__(self, conn, app, url, container, obj, authurl, login, key, content_type=None, modified=None):
	28	+ self.app = app
	29	+ self.conn = conn
	30	+ if self.token is None:
	31	+ (account, self.token) = wmf.client.get_auth(authurl, login, key)
	32	+ if modified is not None:
	33	+ h = {'!Migration-Timestamp!': '%s' % modified}
	34	+ else:
	35	+ h = {}
	36	+ self.copyconn = wmf.client.Put_object_chunked(url, self.token,
	37	+ container, obj, content_type=content_type, headers=h)
	38	+
	39	+ def __iter__(self):
	40	+ return self
	41	+
	42	+ def next(self):
	43	+ data = self.conn.read(4096)
	44	+ if not data:
	45	+ # if we get a 401 error, it's okay, but we should re-auth.
	46	+ try:
	47	+ self.copyconn.close() #06 or #04 if it fails.
	48	+ except wmf.client.ClientException,err:
	49	+ self.app.logger.warn( "PUT Status: %d" % err.http_status)
	50	+ if err.http_status == 401:
	51	+ # not worth retrying the write.
	52	+ self.token = None
	53	+ else:
	54	+ raise
	55	+ raise StopIteration
	56	+ self.copyconn.write(data)
	57	+ return data
	58	+
	59	+class ObjectController(object):
	60	+
	61	+ def __init__(self):
	62	+ self.response_args = []
	63	+
	64	+ def do_start_response(self, *args):
	65	+ """ Remember our arguments but do nothing with them """
	66	+ self.response_args.extend(args)
	67	+
	68	+class WMFRewrite(object):
	69	+ """
	70	+ Rewrite Media Store URLs so that swift knows how to deal.
	71	+
	72	+ Mostly it's a question of inserting the AUTH_ string, and escaping the %2F's in the container section.
	73	+ """
	74	+
	75	+ def __init__(self, app, conf):
	76	+ self.app = app
	77	+ self.account = account
	78	+ self.authurl = conf['url'].strip()
	79	+ self.login = conf['login'].strip()
	80	+ self.key = conf['key'].strip()
	81	+
	82	+ def __call__(self, env, start_response):
	83	+ #try:
	84	+ req = webob.Request(env)
	85	+ if req.method == 'PUT':
	86	+ return self.app(env, start_response)
	87	+ reqorig = req.copy()
	88	+ # http://upload.wikimedia.org/wikipedia/commons/a/aa/000_Finlanda_harta.PNG
	89	+ # http://upload.wikimedia.org/wikipedia/commons/thumb/a/aa/000_Finlanda_harta.PNG/75px-000_Finlanda_harta.PNG
	90	+ match = re.match(r'/(.?)/(.?)/(.*)', req.path)
	91	+ if req.path.startswith('/auth') or req.path.find('AUTH') >= 0:
	92	+ # if it already has AUTH, presume that it's good. #07
	93	+ return self.app(env, start_response)
	94	+ elif match:
	95	+ # https://alsted.wikimedia.org:8080/v1/AUTH_6790933748e741268babd69804c6298b/wikipedia%252Fen/2/25/Machinesmith.png
	96	+ container = match.group(2) #02
	97	+ obj = match.group(3)
	98	+ # include the thumb in the container.
	99	+ if obj.startswith("thumb/"): #03
	100	+ container += "%2Fthumb"
	101	+ obj = obj[len("thumb/"):]
	102	+ # quote slashes in the container name
	103	+ container = "%s%%2F%s" % (match.group(1), container)
	104	+
	105	+ if not obj:
	106	+ # don't let them list the container (it's really FRICKING huge) #08
	107	+ return webob.exc.HTTPForbidden('No container listing')(env, start_response)
	108	+
	109	+ # save a url with just the account name in it.
	110	+ req.path_info = "/v1/%s" % (self.account)
	111	+ req.host = '127.0.0.1'
	112	+ url = req.url[:]
	113	+ # Create a path to our object's name.
	114	+ req.path_info = "/v1/%s/%s/%s" % (self.account, container, obj)
	115	+
	116	+ controller = ObjectController()
	117	+ # do_start_response just remembers what it got called with, because we may want to generate a different response.
	118	+ app_iter = self.app(env, controller.do_start_response) #01
	119	+ status = int(controller.response_args[0].split()[0])
	120	+ headers = dict(controller.response_args[1])
	121	+ #self.app.logger.warn( "Status: %d" % status)
	122	+
	123	+ if 200 <= status < 300 or status == 304:
	124	+ # We have it! Just return it as usual.
	125	+ #headers['X-Swift-Proxy']= `headers`
	126	+ if 'etag' in headers: del headers['etag']
	127	+ return webob.Response(status=status, headers=headers ,
	128	+ app_iter=app_iter)(env, start_response) #01a
	129	+ elif status == 404: #4
	130	+ # go to the thumb media store for unknown files
	131	+ reqorig.host = 'ms5.pmtpa.wmnet'
	132	+ # upload doesn't like our User-agent, otherwise we could call it using urllib2.url()
	133	+ opener = urllib2.build_opener()
	134	+ opener.addheaders = [('User-agent', 'Mozilla/5.0')]
	135	+ upcopy = opener.open(reqorig.url)
	136	+ # get the Content-Type.
	137	+ uinfo = upcopy.info()
	138	+ c_t = uinfo.gettype()
	139	+ last_modified = time.mktime(uinfo.getdate('Last-Modified'))
	140	+ # Fetch from upload, write into the cluster, and return it to them.
	141	+ resp = webob.Response(app_iter=Copy2(upcopy, self.app, url,
	142	+ urllib2.quote(container), obj, self.authurl, self.login,
	143	+ self.key, content_type=c_t, modified=last_modified), content_type=c_t)
	144	+ resp.headers.add('Things', "%s %s %s %s %s %s %s %s" % (url, urllib2.quote(container), obj, self.authurl, self.login, self.key, c_t, last_modified))
	145	+ resp.headers.add('Last-Modified', uinfo.getheader('Last-Modified'))
	146	+ return resp(env, start_response)
	147	+ elif status == 401:
	148	+ # if the Storage URL is invalid or has expired we'll get this error.
	149	+ return webob.exc.HTTPUnauthorized('Token may have timed out')(env, start_response) #05
	150	+ else:
	151	+ return webob.exc.HTTPNotImplemented('Unknown Status: %s' % (status))(env, start_response) #10
	152	+ else:
	153	+ return webob.exc.HTTPBadRequest('Regexp failed: "%s"' % (req.path))(env, start_response) #11
	154	+ #except:
	155	+ #return webob.exc.HTTPNotFound('Internal error')(env, start_response)
	156	+
	157	+def filter_factory(global_conf, **local_conf):
	158	+ conf = global_conf.copy()
	159	+ conf.update(local_conf)
	160	+ def wmfrewrite_filter(app):
	161	+ return WMFRewrite(app, conf)
	162	+ return wmfrewrite_filter
	163	+
Property changes on: trunk/extensions/SwiftMedia/wmf/rewrite.py
___________________________________________________________________
Added: svn:eol-style
1	164	+ native
Index: trunk/extensions/SwiftMedia/README
—	—	@@ -0,0 +1,2 @@
	2	+Please see
	3	+http://www.mediawiki.org/wiki/Extension:SwiftMedia
Index: trunk/extensions/SwiftMedia/SwiftMedia.body.php
—	—	@@ -0,0 +1,2977 @@
	2	+<?php
	3	+/**
	4	+ * Local file in the wiki's own database, only stored in Swift
	5	+ *
	6	+ * @file
	7	+ * @ingroup FileRepo
	8	+ */
	9	+
	10	+/**
	11	+ * Class to represent a local file in the wiki's own database, only stored in Swift
	12	+ *
	13	+ * Provides methods to retrieve paths (physical, logical, URL),
	14	+ * to generate image thumbnails or for uploading.
	15	+ *
	16	+ * Note that only the repo object knows what its file class is called. You should
	17	+ * never name a file class explictly outside of the repo class. Instead use the
	18	+ * repo's factory functions to generate file objects, for example:
	19	+ *
	20	+ * RepoGroup::singleton()->getLocalRepo()->newFile($title);
	21	+ *
	22	+ * The convenience functions wfLocalFile() and wfFindFile() should be sufficient
	23	+ * in most cases.
	24	+ *
	25	+ * @ingroup FileRepo
	26	+ */
	27	+class SwiftFile extends LocalFile {
	28	+ /**#@+
	29	+ * @private
	30	+ */
	31	+ var
	32	+ $conn; # our connection to the Swift proxy.
	33	+ #$fileExists, # does the file file exist on disk? (loadFromXxx)
	34	+ #$historyLine, # Number of line to return by nextHistoryLine() (constructor)
	35	+ #$historyRes, # result of the query for the file's history (nextHistoryLine)
	36	+ #$width, # \
	37	+ #$height, # \|
	38	+ #$bits, # --- returned by getimagesize (loadFromXxx)
	39	+ #$attr, # /
	40	+ #$media_type, # MEDIATYPE_xxx (bitmap, drawing, audio...)
	41	+ #$mime, # MIME type, determined by MimeMagic::guessMimeType
	42	+ #$major_mime, # Major mime type
	43	+ #$minor_mime, # Minor mime type
	44	+ #$size, # Size in bytes (loadFromXxx)
	45	+ #$metadata, # Handler-specific metadata
	46	+ #$timestamp, # Upload timestamp
	47	+ #$sha1, # SHA-1 base 36 content hash
	48	+ #$user, $user_text, # User, who uploaded the file
	49	+ #$description, # Description of current revision of the file
	50	+ #$dataLoaded, # Whether or not all this has been loaded from the database (loadFromXxx)
	51	+ #$upgraded, # Whether the row was upgraded on load
	52	+ #$locked, # True if the image row is locked
	53	+ #$missing, # True if file is not present in file system. Not to be cached in memcached
	54	+ #$deleted; # Bitfield akin to rev_deleted
	55	+ /*#@-/
	56	+
	57	+ /**
	58	+ * Create a LocalFile from a title
	59	+ * Do not call this except from inside a repo class.
	60	+ *
	61	+ * Note: $unused param is only here to avoid an E_STRICT
	62	+ */
	63	+ static function newFromTitle( $title, $repo, $unused = null ) {
	64	+ if ( empty($title) ) { return null; }
	65	+ return new self( $title, $repo );
	66	+ }
	67	+
	68	+ /**
	69	+ * Create a LocalFile from a title
	70	+ * Do not call this except from inside a repo class.
	71	+ */
	72	+ static function newFromRow( $row, $repo ) {
	73	+ $title = Title::makeTitle( NS_FILE, $row->img_name );
	74	+ $file = new self( $title, $repo );
	75	+ $file->loadFromRow( $row );
	76	+
	77	+ return $file;
	78	+ }
	79	+
	80	+ /**
	81	+ * Constructor.
	82	+ * Do not call this except from inside a repo class.
	83	+ */
	84	+ function __construct( $title, $repo ) {
	85	+ if ( !is_object( $title ) ) {
	86	+ throw new MWException( __CLASS__ . ' constructor given bogus title.' );
	87	+ }
	88	+
	89	+ parent::__construct( $title, $repo );
	90	+
	91	+ $this->metadata = '';
	92	+ $this->historyLine = 0;
	93	+ $this->historyRes = null;
	94	+ $this->dataLoaded = false;
	95	+ }
	96	+
	97	+ /**
	98	+ * Get the memcached key for the main data for this file, or false if
	99	+ * there is no access to the shared cache.
	100	+ */
	101	+ function getCacheKey() {
	102	+ $hashedName = md5( $this->getName() );
	103	+
	104	+ return $this->repo->getSharedCacheKey( 'file', $hashedName );
	105	+ }
	106	+
	107	+ /**
	108	+ * Try to load file metadata from memcached. Returns true on success.
	109	+ */
	110	+ function loadFromCache() {
	111	+ global $wgMemc;
	112	+
	113	+ wfProfileIn( __METHOD__ );
	114	+ $this->dataLoaded = false;
	115	+ $key = $this->getCacheKey();
	116	+
	117	+ if ( !$key ) {
	118	+ wfProfileOut( __METHOD__ );
	119	+ return false;
	120	+ }
	121	+
	122	+ $cachedValues = $wgMemc->get( $key );
	123	+
	124	+ // Check if the key existed and belongs to this version of MediaWiki
	125	+ if ( isset( $cachedValues['version'] ) && ( $cachedValues['version'] == MW_FILE_VERSION ) ) {
	126	+ wfDebug( "Pulling file metadata from cache key $key\n" );
	127	+ $this->fileExists = $cachedValues['fileExists'];
	128	+ if ( $this->fileExists ) {
	129	+ $this->setProps( $cachedValues );
	130	+ }
	131	+ $this->dataLoaded = true;
	132	+ }
	133	+
	134	+ if ( $this->dataLoaded ) {
	135	+ wfIncrStats( 'image_cache_hit' );
	136	+ } else {
	137	+ wfIncrStats( 'image_cache_miss' );
	138	+ }
	139	+
	140	+ wfProfileOut( __METHOD__ );
	141	+ return $this->dataLoaded;
	142	+ }
	143	+
	144	+ /**
	145	+ * Save the file metadata to memcached
	146	+ */
	147	+ function saveToCache() {
	148	+ global $wgMemc;
	149	+
	150	+ $this->load();
	151	+ $key = $this->getCacheKey();
	152	+
	153	+ if ( !$key ) {
	154	+ return;
	155	+ }
	156	+
	157	+ $fields = $this->getCacheFields( '' );
	158	+ $cache = array( 'version' => MW_FILE_VERSION );
	159	+ $cache['fileExists'] = $this->fileExists;
	160	+
	161	+ if ( $this->fileExists ) {
	162	+ foreach ( $fields as $field ) {
	163	+ $cache[$field] = $this->$field;
	164	+ }
	165	+ }
	166	+
	167	+ $wgMemc->set( $key, $cache, 60 * 60 * 24 * 7 ); // A week
	168	+ }
	169	+
	170	+ /**
	171	+ * Load metadata from the file itself
	172	+ */
	173	+ function loadFromFile() {
	174	+ wfDebug( __METHOD__ . $this->getPath() . "\n" );
	175	+ $this->setProps( self::getPropsFromPath( $this->getPath() ) );
	176	+ }
	177	+
	178	+ function getCacheFields( $prefix = 'img_' ) {
	179	+ static $fields = array( 'size', 'width', 'height', 'bits', 'media_type',
	180	+ 'major_mime', 'minor_mime', 'metadata', 'timestamp', 'sha1', 'user', 'user_text', 'description' );
	181	+ static $results = array();
	182	+
	183	+ if ( $prefix == '' ) {
	184	+ return $fields;
	185	+ }
	186	+
	187	+ if ( !isset( $results[$prefix] ) ) {
	188	+ $prefixedFields = array();
	189	+ foreach ( $fields as $field ) {
	190	+ $prefixedFields[] = $prefix . $field;
	191	+ }
	192	+ $results[$prefix] = $prefixedFields;
	193	+ }
	194	+
	195	+ return $results[$prefix];
	196	+ }
	197	+
	198	+ /**
	199	+ * Load file metadata from the DB
	200	+ */
	201	+ function loadFromDB() {
	202	+ # Polymorphic function name to distinguish foreign and local fetches
	203	+ $fname = get_class( $this ) . '::' . __FUNCTION__;
	204	+ wfProfileIn( $fname );
	205	+
	206	+ # Unconditionally set loaded=true, we don't want the accessors constantly rechecking
	207	+ $this->dataLoaded = true;
	208	+
	209	+ $dbr = $this->repo->getMasterDB();
	210	+
	211	+ $row = $dbr->selectRow( 'image', $this->getCacheFields( 'img_' ),
	212	+ array( 'img_name' => $this->getName() ), $fname );
	213	+
	214	+ if ( $row ) {
	215	+ $this->loadFromRow( $row );
	216	+ } else {
	217	+ $this->fileExists = false;
	218	+ }
	219	+
	220	+ wfProfileOut( $fname );
	221	+ }
	222	+
	223	+ /**
	224	+ * Decode a row from the database (either object or array) to an array
	225	+ * with timestamps and MIME types decoded, and the field prefix removed.
	226	+ */
	227	+ function decodeRow( $row, $prefix = 'img_' ) {
	228	+ $array = (array)$row;
	229	+ $prefixLength = strlen( $prefix );
	230	+
	231	+ // Sanity check prefix once
	232	+ if ( substr( key( $array ), 0, $prefixLength ) !== $prefix ) {
	233	+ throw new MWException( __METHOD__ . ': incorrect $prefix parameter' );
	234	+ }
	235	+
	236	+ $decoded = array();
	237	+
	238	+ foreach ( $array as $name => $value ) {
	239	+ $decoded[substr( $name, $prefixLength )] = $value;
	240	+ }
	241	+
	242	+ $decoded['timestamp'] = wfTimestamp( TS_MW, $decoded['timestamp'] );
	243	+
	244	+ if ( empty( $decoded['major_mime'] ) ) {
	245	+ $decoded['mime'] = 'unknown/unknown';
	246	+ } else {
	247	+ if ( !$decoded['minor_mime'] ) {
	248	+ $decoded['minor_mime'] = 'unknown';
	249	+ }
	250	+ $decoded['mime'] = $decoded['major_mime'] . '/' . $decoded['minor_mime'];
	251	+ }
	252	+
	253	+ # Trim zero padding from char/binary field
	254	+ $decoded['sha1'] = rtrim( $decoded['sha1'], "\0" );
	255	+
	256	+ return $decoded;
	257	+ }
	258	+
	259	+ /**
	260	+ * Load file metadata from a DB result row
	261	+ */
	262	+ function loadFromRow( $row, $prefix = 'img_' ) {
	263	+ $this->dataLoaded = true;
	264	+ $array = $this->decodeRow( $row, $prefix );
	265	+
	266	+ foreach ( $array as $name => $value ) {
	267	+ $this->$name = $value;
	268	+ }
	269	+
	270	+ $this->fileExists = true;
	271	+ $this->maybeUpgradeRow();
	272	+ }
	273	+
	274	+ /**
	275	+ * Load file metadata from cache or DB, unless already loaded
	276	+ */
	277	+ function load() {
	278	+ if ( !$this->dataLoaded ) {
	279	+ if ( !$this->loadFromCache() ) {
	280	+ $this->loadFromDB();
	281	+ $this->saveToCache();
	282	+ }
	283	+ $this->dataLoaded = true;
	284	+ }
	285	+ }
	286	+
	287	+ /**
	288	+ * Upgrade a row if it needs it
	289	+ */
	290	+ function maybeUpgradeRow() {
	291	+ if ( wfReadOnly() ) {
	292	+ return;
	293	+ }
	294	+
	295	+ if ( is_null( $this->media_type ) \|\|
	296	+ $this->mime == 'image/svg'
	297	+ ) {
	298	+ $this->upgradeRow();
	299	+ $this->upgraded = true;
	300	+ } else {
	301	+ $handler = $this->getHandler();
	302	+ if ( $handler && !$handler->isMetadataValid( $this, $this->metadata ) ) {
	303	+ $this->upgradeRow();
	304	+ $this->upgraded = true;
	305	+ }
	306	+ }
	307	+ }
	308	+
	309	+ function getUpgraded() {
	310	+ return $this->upgraded;
	311	+ }
	312	+
	313	+ /**
	314	+ * Fix assorted version-related problems with the image row by reloading it from the file
	315	+ */
	316	+ function upgradeRow() {
	317	+ wfProfileIn( __METHOD__ );
	318	+
	319	+ $this->loadFromFile();
	320	+
	321	+ # Don't destroy file info of missing files
	322	+ if ( !$this->fileExists ) {
	323	+ wfDebug( __METHOD__ . ": file does not exist, aborting\n" );
	324	+ wfProfileOut( __METHOD__ );
	325	+ return;
	326	+ }
	327	+
	328	+ $dbw = $this->repo->getMasterDB();
	329	+ list( $major, $minor ) = self::splitMime( $this->mime );
	330	+
	331	+ if ( wfReadOnly() ) {
	332	+ wfProfileOut( __METHOD__ );
	333	+ return;
	334	+ }
	335	+ wfDebug( __METHOD__ . ': upgrading ' . $this->getName() . " to the current schema\n" );
	336	+
	337	+ $dbw->update( 'image',
	338	+ array(
	339	+ 'img_width' => $this->width,
	340	+ 'img_height' => $this->height,
	341	+ 'img_bits' => $this->bits,
	342	+ 'img_media_type' => $this->media_type,
	343	+ 'img_major_mime' => $major,
	344	+ 'img_minor_mime' => $minor,
	345	+ 'img_metadata' => $this->metadata,
	346	+ 'img_sha1' => $this->sha1,
	347	+ ), array( 'img_name' => $this->getName() ),
	348	+ __METHOD__
	349	+ );
	350	+
	351	+ $this->saveToCache();
	352	+ wfProfileOut( __METHOD__ );
	353	+ }
	354	+
	355	+ /**
	356	+ * Set properties in this object to be equal to those given in the
	357	+ * associative array $info. Only cacheable fields can be set.
	358	+ *
	359	+ * If 'mime' is given, it will be split into major_mime/minor_mime.
	360	+ * If major_mime/minor_mime are given, $this->mime will also be set.
	361	+ */
	362	+ function setProps( $info ) {
	363	+ $this->dataLoaded = true;
	364	+ $fields = $this->getCacheFields( '' );
	365	+ $fields[] = 'fileExists';
	366	+
	367	+ foreach ( $fields as $field ) {
	368	+ if ( isset( $info[$field] ) ) {
	369	+ $this->$field = $info[$field];
	370	+ }
	371	+ }
	372	+
	373	+ // Fix up mime fields
	374	+ if ( isset( $info['major_mime'] ) ) {
	375	+ $this->mime = "{$info['major_mime']}/{$info['minor_mime']}";
	376	+ } elseif ( isset( $info['mime'] ) ) {
	377	+ $this->mime = $info['mime'];
	378	+ list( $this->major_mime, $this->minor_mime ) = self::splitMime( $this->mime );
	379	+ }
	380	+ }
	381	+
	382	+ /** splitMime inherited */
	383	+ /** getName inherited */
	384	+ /** getTitle inherited */
	385	+ /** getURL inherited */
	386	+ /** getViewURL inherited */
	387	+ /** isVisible inhereted */
	388	+
	389	+ function getPath() {
	390	+ // kinda kludgey, but we'll leave it this way until we're sure that we never need the "real" path.
	391	+ if ($this->temp_path) { return $this->temp_path; }
	392	+ else { return parent::getPath(); }
	393	+ }
	394	+
	395	+ function isMissing() {
	396	+ if ( $this->missing === null ) {
	397	+ list( $fileExists ) = $this->repo->fileExistsBatch( array( $this->getVirtualUrl() ), FileRepo::FILES_ONLY );
	398	+ $this->missing = !$fileExists;
	399	+ }
	400	+ return $this->missing;
	401	+ }
	402	+
	403	+ /**
	404	+ * Return the width of the image
	405	+ *
	406	+ * Returns false on error
	407	+ */
	408	+ public function getWidth( $page = 1 ) {
	409	+ $this->load();
	410	+
	411	+ if ( $this->isMultipage() ) {
	412	+ $dim = $this->getHandler()->getPageDimensions( $this, $page );
	413	+ if ( $dim ) {
	414	+ return $dim['width'];
	415	+ } else {
	416	+ return false;
	417	+ }
	418	+ } else {
	419	+ return $this->width;
	420	+ }
	421	+ }
	422	+
	423	+ /**
	424	+ * Return the height of the image
	425	+ *
	426	+ * Returns false on error
	427	+ */
	428	+ public function getHeight( $page = 1 ) {
	429	+ $this->load();
	430	+
	431	+ if ( $this->isMultipage() ) {
	432	+ $dim = $this->getHandler()->getPageDimensions( $this, $page );
	433	+ if ( $dim ) {
	434	+ return $dim['height'];
	435	+ } else {
	436	+ return false;
	437	+ }
	438	+ } else {
	439	+ return $this->height;
	440	+ }
	441	+ }
	442	+
	443	+ /**
	444	+ * Returns ID or name of user who uploaded the file
	445	+ *
	446	+ * @param $type string 'text' or 'id'
	447	+ */
	448	+ function getUser( $type = 'text' ) {
	449	+ $this->load();
	450	+
	451	+ if ( $type == 'text' ) {
	452	+ return $this->user_text;
	453	+ } elseif ( $type == 'id' ) {
	454	+ return $this->user;
	455	+ }
	456	+ }
	457	+
	458	+ /**
	459	+ * Get handler-specific metadata
	460	+ */
	461	+ function getMetadata() {
	462	+ $this->load();
	463	+ return $this->metadata;
	464	+ }
	465	+
	466	+ function getBitDepth() {
	467	+ $this->load();
	468	+ return $this->bits;
	469	+ }
	470	+
	471	+ /**
	472	+ * Return the size of the image file, in bytes
	473	+ */
	474	+ public function getSize() {
	475	+ $this->load();
	476	+ return $this->size;
	477	+ }
	478	+
	479	+ /**
	480	+ * Returns the mime type of the file.
	481	+ */
	482	+ function getMimeType() {
	483	+ $this->load();
	484	+ return $this->mime;
	485	+ }
	486	+
	487	+ /**
	488	+ * Return the type of the media in the file.
	489	+ * Use the value returned by this function with the MEDIATYPE_xxx constants.
	490	+ */
	491	+ function getMediaType() {
	492	+ $this->load();
	493	+ return $this->media_type;
	494	+ }
	495	+
	496	+ /** canRender inherited */
	497	+ /** mustRender inherited */
	498	+ /** allowInlineDisplay inherited */
	499	+ /** isSafeFile inherited */
	500	+ /** isTrustedFile inherited */
	501	+
	502	+ /**
	503	+ * Returns true if the file file exists on disk.
	504	+ * @return boolean Whether file file exist on disk.
	505	+ */
	506	+ public function exists() {
	507	+ $this->load();
	508	+ return $this->fileExists;
	509	+ }
	510	+
	511	+ /** getTransformScript inherited */
	512	+ /** getUnscaledThumb inherited */
	513	+ /** thumbName inherited */
	514	+ /** createThumb inherited */
	515	+ /** getThumbnail inherited */
	516	+
	517	+ /**
	518	+ * Transform a media file
	519	+ *
	520	+ * @param $params Array: an associative array of handler-specific parameters.
	521	+ * Typical keys are width, height and page.
	522	+ * @param $flags Integer: a bitfield, may contain self::RENDER_NOW to force rendering
	523	+ * @return MediaTransformOutput \| false
	524	+ */
	525	+ function transform( $params, $flags = 0 ) {
	526	+ global $wgUseSquid, $wgIgnoreImageErrors, $wgThumbnailEpoch, $wgServer;
	527	+ global $wgTmpDirectory;
	528	+
	529	+ wfProfileIn( __METHOD__ );
	530	+ do {
	531	+ if ( !$this->canRender() ) {
	532	+ // not a bitmap or renderable image, don't try.
	533	+ $thumb = $this->iconThumb();
	534	+ break;
	535	+ }
	536	+
	537	+ // Get the descriptionUrl to embed it as comment into the thumbnail. Bug 19791.
	538	+ $descriptionUrl = $this->getDescriptionUrl();
	539	+ if ( $descriptionUrl ) {
	540	+ $params['descriptionUrl'] = $wgServer . $descriptionUrl;
	541	+ }
	542	+
	543	+ // make the thumb name and URL out of the normalized parameters.
	544	+ // we only use the thumbTemp for a temporary file.
	545	+ $normalisedParams = $params;
	546	+ $this->handler->normaliseParams( $this, $normalisedParams );
	547	+ $thumbName = $this->thumbName( $normalisedParams );
	548	+ $thumbUrl = $this->getThumbUrl( $thumbName );
	549	+
	550	+ // get a temporary place to put the original.
	551	+ $this->temp_path = tempnam( $wgTmpDirectory, 'transform_in_' );
	552	+ $thumbTemp = tempnam( $wgTmpDirectory, 'transform_out_' );
	553	+
	554	+ /* Fetch the image out of Swift */
	555	+ $auth = new CF_Authentication($this->swiftuser, $this->key, NULL, $this->authurl);
	556	+ wfDebug( __METHOD__ . "Auth ", $this->swiftuser . "," . $this->key . "," . $this->authurl . "\n");
	557	+ $auth->authenticate();
	558	+ $conn = new CF_Connection($auth);
	559	+ $container = $conn->get_container($this->container);
	560	+
	561	+ // how does this return failure??
	562	+ $obj = $container->get_object($this->getRel());
	563	+ // we need to do a try here
	564	+ wfDebug( __METHOD__ . " writing to " . $this->temp_path . "\n");
	565	+ $obj->save_to_filename( $this->temp_path);
	566	+
	567	+ $thumb = $this->handler->doTransform( $this, $thumbTemp, $thumbUrl, $params );
	568	+
	569	+ // Store the thumbnail into Swift, but in the thumb version of the container.
	570	+ $container = $conn->get_container($this->container . "%2Fthumb");
	571	+ wfDebug( __METHOD__ . "Creating thumb " . $this->getRel() . "/" . $thumbName . "\n");
	572	+ $obj = $container->create_object($this->getRel() . "/" . $thumbName);
	573	+ $thumbRel = $obj->load_from_filename($thumbTemp);
	574	+
	575	+ // Clean up temporary data.
	576	+ unlink($this->temp_path);
	577	+ $this->temp_path = null;
	578	+ unlink($thumbTemp);
	579	+
	580	+ } while (false);
	581	+
	582	+ wfProfileOut( __METHOD__ );
	583	+ return is_object( $thumb ) ? $thumb : false;
	584	+ }
	585	+
	586	+
	587	+ /**
	588	+ * Fix thumbnail files from 1.4 or before, with extreme prejudice
	589	+ */
	590	+ function migrateThumbFile( $thumbName ) {
	591	+ $thumbDir = $this->getThumbPath();
	592	+ $thumbPath = "$thumbDir/$thumbName";
	593	+
	594	+ if ( is_dir( $thumbPath ) ) {
	595	+ // Directory where file should be
	596	+ // This happened occasionally due to broken migration code in 1.5
	597	+ // Rename to broken-*
	598	+ for ( $i = 0; $i < 100 ; $i++ ) {
	599	+ $broken = $this->repo->getZonePath( 'public' ) . "/broken-$i-$thumbName";
	600	+ if ( !file_exists( $broken ) ) {
	601	+ rename( $thumbPath, $broken );
	602	+ break;
	603	+ }
	604	+ }
	605	+ // Doesn't exist anymore
	606	+ clearstatcache();
	607	+ }
	608	+
	609	+ if ( is_file( $thumbDir ) ) {
	610	+ // File where directory should be
	611	+ unlink( $thumbDir );
	612	+ // Doesn't exist anymore
	613	+ clearstatcache();
	614	+ }
	615	+ }
	616	+
	617	+ /** getHandler inherited */
	618	+ /** iconThumb inherited */
	619	+ /** getLastError inherited */
	620	+
	621	+ /**
	622	+ * Get all thumbnail names previously generated for this file
	623	+ */
	624	+ function getThumbnails() {
	625	+ $this->load();
	626	+
	627	+ $files = array();
	628	+ $dir = $this->getThumbPath();
	629	+
	630	+ if ( is_dir( $dir ) ) {
	631	+ $handle = opendir( $dir );
	632	+
	633	+ if ( $handle ) {
	634	+ while ( false !== ( $file = readdir( $handle ) ) ) {
	635	+ if ( $file { 0 } != '.' ) {
	636	+ $files[] = $file;
	637	+ }
	638	+ }
	639	+
	640	+ closedir( $handle );
	641	+ }
	642	+ }
	643	+
	644	+ return $files;
	645	+ }
	646	+
	647	+ /**
	648	+ * Refresh metadata in memcached, but don't touch thumbnails or squid
	649	+ */
	650	+ function purgeMetadataCache() {
	651	+ $this->loadFromDB();
	652	+ $this->saveToCache();
	653	+ $this->purgeHistory();
	654	+ }
	655	+
	656	+ /**
	657	+ * Purge the shared history (OldLocalFile) cache
	658	+ */
	659	+ function purgeHistory() {
	660	+ global $wgMemc;
	661	+
	662	+ $hashedName = md5( $this->getName() );
	663	+ $oldKey = $this->repo->getSharedCacheKey( 'oldfile', $hashedName );
	664	+
	665	+ if ( $oldKey ) {
	666	+ $wgMemc->delete( $oldKey );
	667	+ }
	668	+ }
	669	+
	670	+ /**
	671	+ * Delete all previously generated thumbnails, refresh metadata in memcached and purge the squid
	672	+ */
	673	+ function purgeCache() {
	674	+ // Refresh metadata cache
	675	+ $this->purgeMetadataCache();
	676	+
	677	+ // Delete thumbnails
	678	+ $this->purgeThumbnails();
	679	+
	680	+ // Purge squid cache for this file
	681	+ SquidUpdate::purge( array( $this->getURL() ) );
	682	+ }
	683	+
	684	+ /**
	685	+ * Delete cached transformed files
	686	+ */
	687	+ function purgeThumbnails() {
	688	+ global $wgUseSquid, $wgExcludeFromThumbnailPurge;
	689	+
	690	+ // Delete thumbnails
	691	+ $files = $this->getThumbnails();
	692	+ $dir = $this->getThumbPath();
	693	+ $urls = array();
	694	+
	695	+ foreach ( $files as $file ) {
	696	+ // Only remove files not in the $wgExcludeFromThumbnailPurge configuration variable
	697	+ $ext = pathinfo( "$dir/$file", PATHINFO_EXTENSION );
	698	+ if ( in_array( $ext, $wgExcludeFromThumbnailPurge ) ) {
	699	+ continue;
	700	+ }
	701	+
	702	+ # Check that the base file name is part of the thumb name
	703	+ # This is a basic sanity check to avoid erasing unrelated directories
	704	+ if ( strpos( $file, $this->getName() ) !== false ) {
	705	+ $url = $this->getThumbUrl( $file );
	706	+ $urls[] = $url;
	707	+ @unlink( "$dir/$file" );
	708	+ }
	709	+ }
	710	+
	711	+ // Purge the squid
	712	+ if ( $wgUseSquid ) {
	713	+ SquidUpdate::purge( $urls );
	714	+ }
	715	+ }
	716	+
	717	+ /** purgeDescription inherited */
	718	+ /** purgeEverything inherited */
	719	+
	720	+ function getHistory( $limit = null, $start = null, $end = null, $inc = true ) {
	721	+ $dbr = $this->repo->getSlaveDB();
	722	+ $tables = array( 'oldimage' );
	723	+ $fields = OldLocalFile::selectFields();
	724	+ $conds = $opts = $join_conds = array();
	725	+ $eq = $inc ? '=' : '';
	726	+ $conds[] = "oi_name = " . $dbr->addQuotes( $this->title->getDBkey() );
	727	+
	728	+ if ( $start ) {
	729	+ $conds[] = "oi_timestamp <$eq " . $dbr->addQuotes( $dbr->timestamp( $start ) );
	730	+ }
	731	+
	732	+ if ( $end ) {
	733	+ $conds[] = "oi_timestamp >$eq " . $dbr->addQuotes( $dbr->timestamp( $end ) );
	734	+ }
	735	+
	736	+ if ( $limit ) {
	737	+ $opts['LIMIT'] = $limit;
	738	+ }
	739	+
	740	+ // Search backwards for time > x queries
	741	+ $order = ( !$start && $end !== null ) ? 'ASC' : 'DESC';
	742	+ $opts['ORDER BY'] = "oi_timestamp $order";
	743	+ $opts['USE INDEX'] = array( 'oldimage' => 'oi_name_timestamp' );
	744	+
	745	+ wfRunHooks( 'LocalFile::getHistory', array( &$this, &$tables, &$fields,
	746	+ &$conds, &$opts, &$join_conds ) );
	747	+
	748	+ $res = $dbr->select( $tables, $fields, $conds, __METHOD__, $opts, $join_conds );
	749	+ $r = array();
	750	+
	751	+ foreach ( $res as $row ) {
	752	+ if ( $this->repo->oldFileFromRowFactory ) {
	753	+ $r[] = call_user_func( $this->repo->oldFileFromRowFactory, $row, $this->repo );
	754	+ } else {
	755	+ $r[] = OldLocalFile::newFromRow( $row, $this->repo );
	756	+ }
	757	+ }
	758	+
	759	+ if ( $order == 'ASC' ) {
	760	+ $r = array_reverse( $r ); // make sure it ends up descending
	761	+ }
	762	+
	763	+ return $r;
	764	+ }
	765	+
	766	+ /**
	767	+ * Return the history of this file, line by line.
	768	+ * starts with current version, then old versions.
	769	+ * uses $this->historyLine to check which line to return:
	770	+ * 0 return line for current version
	771	+ * 1 query for old versions, return first one
	772	+ * 2, ... return next old version from above query
	773	+ */
	774	+ public function nextHistoryLine() {
	775	+ # Polymorphic function name to distinguish foreign and local fetches
	776	+ $fname = get_class( $this ) . '::' . __FUNCTION__;
	777	+
	778	+ $dbr = $this->repo->getSlaveDB();
	779	+
	780	+ if ( $this->historyLine == 0 ) {// called for the first time, return line from cur
	781	+ $this->historyRes = $dbr->select( 'image',
	782	+ array(
	783	+ '*',
	784	+ "'' AS oi_archive_name",
	785	+ '0 as oi_deleted',
	786	+ 'img_sha1'
	787	+ ),
	788	+ array( 'img_name' => $this->title->getDBkey() ),
	789	+ $fname
	790	+ );
	791	+
	792	+ if ( 0 == $dbr->numRows( $this->historyRes ) ) {
	793	+ $this->historyRes = null;
	794	+ return false;
	795	+ }
	796	+ } elseif ( $this->historyLine == 1 ) {
	797	+ $this->historyRes = $dbr->select( 'oldimage', '*',
	798	+ array( 'oi_name' => $this->title->getDBkey() ),
	799	+ $fname,
	800	+ array( 'ORDER BY' => 'oi_timestamp DESC' )
	801	+ );
	802	+ }
	803	+ $this->historyLine ++;
	804	+
	805	+ return $dbr->fetchObject( $this->historyRes );
	806	+ }
	807	+
	808	+ /**
	809	+ * Reset the history pointer to the first element of the history
	810	+ */
	811	+ public function resetHistory() {
	812	+ $this->historyLine = 0;
	813	+
	814	+ if ( !is_null( $this->historyRes ) ) {
	815	+ $this->historyRes = null;
	816	+ }
	817	+ }
	818	+
	819	+ /** getFullPath inherited */
	820	+ /** getHashPath inherited */
	821	+ /** getRel inherited */
	822	+ /** getUrlRel inherited */
	823	+ /** getArchiveRel inherited */
	824	+ /** getThumbRel inherited */
	825	+ /** getArchivePath inherited */
	826	+ /** getThumbPath inherited */
	827	+ /** getArchiveUrl inherited */
	828	+ /** getThumbUrl inherited */
	829	+ /** getArchiveVirtualUrl inherited */
	830	+ /** getThumbVirtualUrl inherited */
	831	+ /** isHashed inherited */
	832	+
	833	+ /**
	834	+ * Upload a file and record it in the DB
	835	+ * @param $srcPath String: source path or virtual URL
	836	+ * @param $comment String: upload description
	837	+ * @param $pageText String: text to use for the new description page,
	838	+ * if a new description page is created
	839	+ * @param $flags Integer: flags for publish()
	840	+ * @param $props Array: File properties, if known. This can be used to reduce the
	841	+ * upload time when uploading virtual URLs for which the file info
	842	+ * is already known
	843	+ * @param $timestamp String: timestamp for img_timestamp, or false to use the current time
	844	+ * @param $user Mixed: User object or null to use $wgUser
	845	+ *
	846	+ * @return FileRepoStatus object. On success, the value member contains the
	847	+ * archive name, or an empty string if it was a new file.
	848	+ */
	849	+ function upload( $srcPath, $comment, $pageText, $flags = 0, $props = false, $timestamp = false, $user = null ) {
	850	+ $this->lock();
	851	+ $status = $this->publish( $srcPath, $flags );
	852	+
	853	+ if ( $status->ok ) {
	854	+ if ( !$this->recordUpload2( $status->value, $comment, $pageText, $props, $timestamp, $user ) ) {
	855	+ $status->fatal( 'filenotfound', $srcPath );
	856	+ }
	857	+ }
	858	+
	859	+ $this->unlock();
	860	+
	861	+ return $status;
	862	+ }
	863	+
	864	+ /**
	865	+ * Record a file upload in the upload log and the image table
	866	+ * @deprecated use upload()
	867	+ */
	868	+ function recordUpload( $oldver, $desc, $license = '', $copyStatus = '', $source = '',
	869	+ $watch = false, $timestamp = false )
	870	+ {
	871	+ $pageText = SpecialUpload::getInitialPageText( $desc, $license, $copyStatus, $source );
	872	+
	873	+ if ( !$this->recordUpload2( $oldver, $desc, $pageText ) ) {
	874	+ return false;
	875	+ }
	876	+
	877	+ if ( $watch ) {
	878	+ global $wgUser;
	879	+ $wgUser->addWatch( $this->getTitle() );
	880	+ }
	881	+ return true;
	882	+
	883	+ }
	884	+
	885	+ /**
	886	+ * Record a file upload in the upload log and the image table
	887	+ */
	888	+ function recordUpload2( $oldver, $comment, $pageText, $props = false, $timestamp = false, $user = null )
	889	+ {
	890	+ if ( is_null( $user ) ) {
	891	+ global $wgUser;
	892	+ $user = $wgUser;
	893	+ }
	894	+
	895	+ $dbw = $this->repo->getMasterDB();
	896	+ $dbw->begin();
	897	+
	898	+ if ( !$props ) {
	899	+ $props = $this->repo->getFileProps( $this->getVirtualUrl() );
	900	+ }
	901	+
	902	+ $props['description'] = $comment;
	903	+ $props['user'] = $user->getId();
	904	+ $props['user_text'] = $user->getName();
	905	+ $props['timestamp'] = wfTimestamp( TS_MW );
	906	+ $this->setProps( $props );
	907	+
	908	+ # Delete thumbnails
	909	+ $this->purgeThumbnails();
	910	+
	911	+ # The file is already on its final location, remove it from the squid cache
	912	+ SquidUpdate::purge( array( $this->getURL() ) );
	913	+
	914	+ # Fail now if the file isn't there
	915	+ if ( !$this->fileExists ) {
	916	+ wfDebug( __METHOD__ . ": File " . $this->getPath() . " went missing!\n" );
	917	+ return false;
	918	+ }
	919	+
	920	+ $reupload = false;
	921	+
	922	+ if ( $timestamp === false ) {
	923	+ $timestamp = $dbw->timestamp();
	924	+ }
	925	+
	926	+ # Test to see if the row exists using INSERT IGNORE
	927	+ # This avoids race conditions by locking the row until the commit, and also
	928	+ # doesn't deadlock. SELECT FOR UPDATE causes a deadlock for every race condition.
	929	+ $dbw->insert( 'image',
	930	+ array(
	931	+ 'img_name' => $this->getName(),
	932	+ 'img_size' => $this->size,
	933	+ 'img_width' => intval( $this->width ),
	934	+ 'img_height' => intval( $this->height ),
	935	+ 'img_bits' => $this->bits,
	936	+ 'img_media_type' => $this->media_type,
	937	+ 'img_major_mime' => $this->major_mime,
	938	+ 'img_minor_mime' => $this->minor_mime,
	939	+ 'img_timestamp' => $timestamp,
	940	+ 'img_description' => $comment,
	941	+ 'img_user' => $user->getId(),
	942	+ 'img_user_text' => $user->getName(),
	943	+ 'img_metadata' => $this->metadata,
	944	+ 'img_sha1' => $this->sha1
	945	+ ),
	946	+ __METHOD__,
	947	+ 'IGNORE'
	948	+ );
	949	+
	950	+ if ( $dbw->affectedRows() == 0 ) {
	951	+ $reupload = true;
	952	+
	953	+ # Collision, this is an update of a file
	954	+ # Insert previous contents into oldimage
	955	+ $dbw->insertSelect( 'oldimage', 'image',
	956	+ array(
	957	+ 'oi_name' => 'img_name',
	958	+ 'oi_archive_name' => $dbw->addQuotes( $oldver ),
	959	+ 'oi_size' => 'img_size',
	960	+ 'oi_width' => 'img_width',
	961	+ 'oi_height' => 'img_height',
	962	+ 'oi_bits' => 'img_bits',
	963	+ 'oi_timestamp' => 'img_timestamp',
	964	+ 'oi_description' => 'img_description',
	965	+ 'oi_user' => 'img_user',
	966	+ 'oi_user_text' => 'img_user_text',
	967	+ 'oi_metadata' => 'img_metadata',
	968	+ 'oi_media_type' => 'img_media_type',
	969	+ 'oi_major_mime' => 'img_major_mime',
	970	+ 'oi_minor_mime' => 'img_minor_mime',
	971	+ 'oi_sha1' => 'img_sha1'
	972	+ ), array( 'img_name' => $this->getName() ), __METHOD__
	973	+ );
	974	+
	975	+ # Update the current image row
	976	+ $dbw->update( 'image',
	977	+ array( /* SET */
	978	+ 'img_size' => $this->size,
	979	+ 'img_width' => intval( $this->width ),
	980	+ 'img_height' => intval( $this->height ),
	981	+ 'img_bits' => $this->bits,
	982	+ 'img_media_type' => $this->media_type,
	983	+ 'img_major_mime' => $this->major_mime,
	984	+ 'img_minor_mime' => $this->minor_mime,
	985	+ 'img_timestamp' => $timestamp,
	986	+ 'img_description' => $comment,
	987	+ 'img_user' => $user->getId(),
	988	+ 'img_user_text' => $user->getName(),
	989	+ 'img_metadata' => $this->metadata,
	990	+ 'img_sha1' => $this->sha1
	991	+ ), array( /* WHERE */
	992	+ 'img_name' => $this->getName()
	993	+ ), __METHOD__
	994	+ );
	995	+ } else {
	996	+ # This is a new file
	997	+ # Update the image count
	998	+ $site_stats = $dbw->tableName( 'site_stats' );
	999	+ $dbw->query( "UPDATE $site_stats SET ss_images=ss_images+1", __METHOD__ );
	1000	+ }
	1001	+
	1002	+ $descTitle = $this->getTitle();
	1003	+ $article = new ImagePage( $descTitle );
	1004	+ $article->setFile( $this );
	1005	+
	1006	+ # Add the log entry
	1007	+ $log = new LogPage( 'upload' );
	1008	+ $action = $reupload ? 'overwrite' : 'upload';
	1009	+ $log->addEntry( $action, $descTitle, $comment, array(), $user );
	1010	+
	1011	+ if ( $descTitle->exists() ) {
	1012	+ # Create a null revision
	1013	+ $latest = $descTitle->getLatestRevID();
	1014	+ $nullRevision = Revision::newNullRevision(
	1015	+ $dbw,
	1016	+ $descTitle->getArticleId(),
	1017	+ $log->getRcComment(),
	1018	+ false
	1019	+ );
	1020	+ $nullRevision->insertOn( $dbw );
	1021	+
	1022	+ wfRunHooks( 'NewRevisionFromEditComplete', array( $article, $nullRevision, $latest, $user ) );
	1023	+ $article->updateRevisionOn( $dbw, $nullRevision );
	1024	+
	1025	+ # Invalidate the cache for the description page
	1026	+ $descTitle->invalidateCache();
	1027	+ $descTitle->purgeSquid();
	1028	+ } else {
	1029	+ # New file; create the description page.
	1030	+ # There's already a log entry, so don't make a second RC entry
	1031	+ # Squid and file cache for the description page are purged by doEdit.
	1032	+ $article->doEdit( $pageText, $comment, EDIT_NEW \| EDIT_SUPPRESS_RC );
	1033	+ }
	1034	+
	1035	+ # Commit the transaction now, in case something goes wrong later
	1036	+ # The most important thing is that files don't get lost, especially archives
	1037	+ $dbw->commit();
	1038	+
	1039	+ # Save to cache and purge the squid
	1040	+ # We shall not saveToCache before the commit since otherwise
	1041	+ # in case of a rollback there is an usable file from memcached
	1042	+ # which in fact doesn't really exist (bug 24978)
	1043	+ $this->saveToCache();
	1044	+
	1045	+ # Hooks, hooks, the magic of hooks...
	1046	+ wfRunHooks( 'FileUpload', array( $this, $reupload, $descTitle->exists() ) );
	1047	+
	1048	+ # Invalidate cache for all pages using this file
	1049	+ $update = new HTMLCacheUpdate( $this->getTitle(), 'imagelinks' );
	1050	+ $update->doUpdate();
	1051	+
	1052	+ # Invalidate cache for all pages that redirects on this page
	1053	+ $redirs = $this->getTitle()->getRedirectsHere();
	1054	+
	1055	+ foreach ( $redirs as $redir ) {
	1056	+ $update = new HTMLCacheUpdate( $redir, 'imagelinks' );
	1057	+ $update->doUpdate();
	1058	+ }
	1059	+
	1060	+ return true;
	1061	+ }
	1062	+
	1063	+ /**
	1064	+ * Move or copy a file to its public location. If a file exists at the
	1065	+ * destination, move it to an archive. Returns a FileRepoStatus object with
	1066	+ * the archive name in the "value" member on success.
	1067	+ *
	1068	+ * The archive name should be passed through to recordUpload for database
	1069	+ * registration.
	1070	+ *
	1071	+ * @param $srcPath String: local filesystem path to the source image
	1072	+ * @param $flags Integer: a bitwise combination of:
	1073	+ * File::DELETE_SOURCE Delete the source file, i.e. move
	1074	+ * rather than copy
	1075	+ * @return FileRepoStatus object. On success, the value member contains the
	1076	+ * archive name, or an empty string if it was a new file.
	1077	+ */
	1078	+ function publish( $srcPath, $flags = 0 ) {
	1079	+ $this->lock();
	1080	+
	1081	+ $dstRel = $this->getRel();
	1082	+ $archiveName = gmdate( 'YmdHis' ) . '!' . $this->getName();
	1083	+ $archiveRel = 'archive/' . $this->getHashPath() . $archiveName;
	1084	+ $flags = $flags & File::DELETE_SOURCE ? LocalRepo::DELETE_SOURCE : 0;
	1085	+ $status = $this->repo->publish( $srcPath, $dstRel, $archiveRel, $flags );
	1086	+
	1087	+ if ( $status->value == 'new' ) {
	1088	+ $status->value = '';
	1089	+ } else {
	1090	+ $status->value = $archiveName;
	1091	+ }
	1092	+
	1093	+ $this->unlock();
	1094	+
	1095	+ return $status;
	1096	+ }
	1097	+
	1098	+ /** getLinksTo inherited */
	1099	+ /** getExifData inherited */
	1100	+ /** isLocal inherited */
	1101	+ /** wasDeleted inherited */
	1102	+
	1103	+ /**
	1104	+ * Move file to the new title
	1105	+ *
	1106	+ * Move current, old version and all thumbnails
	1107	+ * to the new filename. Old file is deleted.
	1108	+ *
	1109	+ * Cache purging is done; checks for validity
	1110	+ * and logging are caller's responsibility
	1111	+ *
	1112	+ * @param $target Title New file name
	1113	+ * @return FileRepoStatus object.
	1114	+ */
	1115	+ function move( $target ) {
	1116	+ wfDebugLog( 'imagemove', "Got request to move {$this->name} to " . $target->getText() );
	1117	+ $this->lock();
	1118	+
	1119	+ $batch = new SwiftFileMoveBatch( $this, $target );
	1120	+ $batch->addCurrent();
	1121	+ $batch->addOlds();
	1122	+
	1123	+ $status = $batch->execute();
	1124	+ wfDebugLog( 'imagemove', "Finished moving {$this->name}" );
	1125	+
	1126	+ $this->purgeEverything();
	1127	+ $this->unlock();
	1128	+
	1129	+ if ( $status->isOk() ) {
	1130	+ // Now switch the object
	1131	+ $this->title = $target;
	1132	+ // Force regeneration of the name and hashpath
	1133	+ unset( $this->name );
	1134	+ unset( $this->hashPath );
	1135	+ // Purge the new image
	1136	+ $this->purgeEverything();
	1137	+ }
	1138	+
	1139	+ return $status;
	1140	+ }
	1141	+
	1142	+ /**
	1143	+ * Delete all versions of the file.
	1144	+ *
	1145	+ * Moves the files into an archive directory (or deletes them)
	1146	+ * and removes the database rows.
	1147	+ *
	1148	+ * Cache purging is done; logging is caller's responsibility.
	1149	+ *
	1150	+ * @param $reason
	1151	+ * @param $suppress
	1152	+ * @return FileRepoStatus object.
	1153	+ */
	1154	+ function delete( $reason, $suppress = false ) {
	1155	+ $this->lock();
	1156	+
	1157	+ $batch = new SwiftFileDeleteBatch( $this, $reason, $suppress );
	1158	+ $batch->addCurrent();
	1159	+
	1160	+ # Get old version relative paths
	1161	+ $dbw = $this->repo->getMasterDB();
	1162	+ $result = $dbw->select( 'oldimage',
	1163	+ array( 'oi_archive_name' ),
	1164	+ array( 'oi_name' => $this->getName() ) );
	1165	+ foreach ( $result as $row ) {
	1166	+ $batch->addOld( $row->oi_archive_name );
	1167	+ }
	1168	+ $status = $batch->execute();
	1169	+
	1170	+ if ( $status->ok ) {
	1171	+ // Update site_stats
	1172	+ $site_stats = $dbw->tableName( 'site_stats' );
	1173	+ $dbw->query( "UPDATE $site_stats SET ss_images=ss_images-1", __METHOD__ );
	1174	+ $this->purgeEverything();
	1175	+ }
	1176	+
	1177	+ $this->unlock();
	1178	+
	1179	+ return $status;
	1180	+ }
	1181	+
	1182	+ /**
	1183	+ * Delete an old version of the file.
	1184	+ *
	1185	+ * Moves the file into an archive directory (or deletes it)
	1186	+ * and removes the database row.
	1187	+ *
	1188	+ * Cache purging is done; logging is caller's responsibility.
	1189	+ *
	1190	+ * @param $archiveName String
	1191	+ * @param $reason String
	1192	+ * @param $suppress Boolean
	1193	+ * @throws MWException or FSException on database or file store failure
	1194	+ * @return FileRepoStatus object.
	1195	+ */
	1196	+ function deleteOld( $archiveName, $reason, $suppress = false ) {
	1197	+ $this->lock();
	1198	+
	1199	+ $batch = new SwiftFileDeleteBatch( $this, $reason, $suppress );
	1200	+ $batch->addOld( $archiveName );
	1201	+ $status = $batch->execute();
	1202	+
	1203	+ $this->unlock();
	1204	+
	1205	+ if ( $status->ok ) {
	1206	+ $this->purgeDescription();
	1207	+ $this->purgeHistory();
	1208	+ }
	1209	+
	1210	+ return $status;
	1211	+ }
	1212	+
	1213	+ /**
	1214	+ * Restore all or specified deleted revisions to the given file.
	1215	+ * Permissions and logging are left to the caller.
	1216	+ *
	1217	+ * May throw database exceptions on error.
	1218	+ *
	1219	+ * @param $versions set of record ids of deleted items to restore,
	1220	+ * or empty to restore all revisions.
	1221	+ * @param $unsuppress Boolean
	1222	+ * @return FileRepoStatus
	1223	+ */
	1224	+ function restore( $versions = array(), $unsuppress = false ) {
	1225	+ $batch = new SwiftFileRestoreBatch( $this, $unsuppress );
	1226	+
	1227	+ if ( !$versions ) {
	1228	+ $batch->addAll();
	1229	+ } else {
	1230	+ $batch->addIds( $versions );
	1231	+ }
	1232	+
	1233	+ $status = $batch->execute();
	1234	+
	1235	+ if ( !$status->isGood() ) {
	1236	+ return $status;
	1237	+ }
	1238	+
	1239	+ $cleanupStatus = $batch->cleanup();
	1240	+ $cleanupStatus->successCount = 0;
	1241	+ $cleanupStatus->failCount = 0;
	1242	+ $status->merge( $cleanupStatus );
	1243	+
	1244	+ return $status;
	1245	+ }
	1246	+
	1247	+ /** isMultipage inherited */
	1248	+ /** pageCount inherited */
	1249	+ /** scaleHeight inherited */
	1250	+ /** getImageSize inherited */
	1251	+
	1252	+ /**
	1253	+ * Get the URL of the file description page.
	1254	+ */
	1255	+ function getDescriptionUrl() {
	1256	+ return $this->title->getLocalUrl();
	1257	+ }
	1258	+
	1259	+ /**
	1260	+ * Get the HTML text of the description page
	1261	+ * This is not used by ImagePage for local files, since (among other things)
	1262	+ * it skips the parser cache.
	1263	+ */
	1264	+ function getDescriptionText() {
	1265	+ global $wgParser;
	1266	+ $revision = Revision::newFromTitle( $this->title );
	1267	+ if ( !$revision ) return false;
	1268	+ $text = $revision->getText();
	1269	+ if ( !$text ) return false;
	1270	+ $pout = $wgParser->parse( $text, $this->title, new ParserOptions() );
	1271	+ return $pout->getText();
	1272	+ }
	1273	+
	1274	+ function getDescription() {
	1275	+ $this->load();
	1276	+ return $this->description;
	1277	+ }
	1278	+
	1279	+ function getTimestamp() {
	1280	+ $this->load();
	1281	+ return $this->timestamp;
	1282	+ }
	1283	+
	1284	+ function getSha1() {
	1285	+ $this->load();
	1286	+ // Initialise now if necessary
	1287	+ if ( $this->sha1 == '' && $this->fileExists ) {
	1288	+ $this->sha1 = File::sha1Base36( $this->getPath() );
	1289	+ if ( !wfReadOnly() && strval( $this->sha1 ) != '' ) {
	1290	+ $dbw = $this->repo->getMasterDB();
	1291	+ $dbw->update( 'image',
	1292	+ array( 'img_sha1' => $this->sha1 ),
	1293	+ array( 'img_name' => $this->getName() ),
	1294	+ __METHOD__ );
	1295	+ $this->saveToCache();
	1296	+ }
	1297	+ }
	1298	+
	1299	+ return $this->sha1;
	1300	+ }
	1301	+
	1302	+ /**
	1303	+ * Start a transaction and lock the image for update
	1304	+ * Increments a reference counter if the lock is already held
	1305	+ * @return boolean True if the image exists, false otherwise
	1306	+ */
	1307	+ function lock() {
	1308	+ $dbw = $this->repo->getMasterDB();
	1309	+
	1310	+ if ( !$this->locked ) {
	1311	+ $dbw->begin();
	1312	+ $this->locked++;
	1313	+ }
	1314	+
	1315	+ return $dbw->selectField( 'image', '1', array( 'img_name' => $this->getName() ), __METHOD__ );
	1316	+ }
	1317	+
	1318	+ /**
	1319	+ * Decrement the lock reference count. If the reference count is reduced to zero, commits
	1320	+ * the transaction and thereby releases the image lock.
	1321	+ */
	1322	+ function unlock() {
	1323	+ if ( $this->locked ) {
	1324	+ --$this->locked;
	1325	+ if ( !$this->locked ) {
	1326	+ $dbw = $this->repo->getMasterDB();
	1327	+ $dbw->commit();
	1328	+ }
	1329	+ }
	1330	+ }
	1331	+
	1332	+ /**
	1333	+ * Roll back the DB transaction and mark the image unlocked
	1334	+ */
	1335	+ function unlockAndRollback() {
	1336	+ $this->locked = false;
	1337	+ $dbw = $this->repo->getMasterDB();
	1338	+ $dbw->rollback();
	1339	+ }
	1340	+} // SwiftFile class
	1341	+
	1342	+# ------------------------------------------------------------------------------
	1343	+
	1344	+/**
	1345	+ * Helper class for file deletion
	1346	+ * @ingroup FileRepo
	1347	+ */
	1348	+class SwiftFileDeleteBatch {
	1349	+
	1350	+ /**
	1351	+ * @var SwiftFile
	1352	+ */
	1353	+ var $file;
	1354	+
	1355	+ var $reason, $srcRels = array(), $archiveUrls = array(), $deletionBatch, $suppress;
	1356	+ var $status;
	1357	+
	1358	+ function __construct( File $file, $reason = '', $suppress = false ) {
	1359	+ $this->file = $file;
	1360	+ $this->reason = $reason;
	1361	+ $this->suppress = $suppress;
	1362	+ $this->status = $file->repo->newGood();
	1363	+ }
	1364	+
	1365	+ function addCurrent() {
	1366	+ $this->srcRels['.'] = $this->file->getRel();
	1367	+ }
	1368	+
	1369	+ function addOld( $oldName ) {
	1370	+ $this->srcRels[$oldName] = $this->file->getArchiveRel( $oldName );
	1371	+ $this->archiveUrls[] = $this->file->getArchiveUrl( $oldName );
	1372	+ }
	1373	+
	1374	+ function getOldRels() {
	1375	+ if ( !isset( $this->srcRels['.'] ) ) {
	1376	+ $oldRels =& $this->srcRels;
	1377	+ $deleteCurrent = false;
	1378	+ } else {
	1379	+ $oldRels = $this->srcRels;
	1380	+ unset( $oldRels['.'] );
	1381	+ $deleteCurrent = true;
	1382	+ }
	1383	+
	1384	+ return array( $oldRels, $deleteCurrent );
	1385	+ }
	1386	+
	1387	+ /protected/ function getHashes() {
	1388	+ $hashes = array();
	1389	+ list( $oldRels, $deleteCurrent ) = $this->getOldRels();
	1390	+
	1391	+ if ( $deleteCurrent ) {
	1392	+ $hashes['.'] = $this->file->getSha1();
	1393	+ }
	1394	+
	1395	+ if ( count( $oldRels ) ) {
	1396	+ $dbw = $this->file->repo->getMasterDB();
	1397	+ $res = $dbw->select(
	1398	+ 'oldimage',
	1399	+ array( 'oi_archive_name', 'oi_sha1' ),
	1400	+ 'oi_archive_name IN (' . $dbw->makeList( array_keys( $oldRels ) ) . ')',
	1401	+ __METHOD__
	1402	+ );
	1403	+
	1404	+ foreach ( $res as $row ) {
	1405	+ if ( rtrim( $row->oi_sha1, "\0" ) === '' ) {
	1406	+ // Get the hash from the file
	1407	+ $oldUrl = $this->file->getArchiveVirtualUrl( $row->oi_archive_name );
	1408	+ $props = $this->file->repo->getFileProps( $oldUrl );
	1409	+
	1410	+ if ( $props['fileExists'] ) {
	1411	+ // Upgrade the oldimage row
	1412	+ $dbw->update( 'oldimage',
	1413	+ array( 'oi_sha1' => $props['sha1'] ),
	1414	+ array( 'oi_name' => $this->file->getName(), 'oi_archive_name' => $row->oi_archive_name ),
	1415	+ __METHOD__ );
	1416	+ $hashes[$row->oi_archive_name] = $props['sha1'];
	1417	+ } else {
	1418	+ $hashes[$row->oi_archive_name] = false;
	1419	+ }
	1420	+ } else {
	1421	+ $hashes[$row->oi_archive_name] = $row->oi_sha1;
	1422	+ }
	1423	+ }
	1424	+ }
	1425	+
	1426	+ $missing = array_diff_key( $this->srcRels, $hashes );
	1427	+
	1428	+ foreach ( $missing as $name => $rel ) {
	1429	+ $this->status->error( 'filedelete-old-unregistered', $name );
	1430	+ }
	1431	+
	1432	+ foreach ( $hashes as $name => $hash ) {
	1433	+ if ( !$hash ) {
	1434	+ $this->status->error( 'filedelete-missing', $this->srcRels[$name] );
	1435	+ unset( $hashes[$name] );
	1436	+ }
	1437	+ }
	1438	+
	1439	+ return $hashes;
	1440	+ }
	1441	+
	1442	+ function doDBInserts() {
	1443	+ global $wgUser;
	1444	+
	1445	+ $dbw = $this->file->repo->getMasterDB();
	1446	+ $encTimestamp = $dbw->addQuotes( $dbw->timestamp() );
	1447	+ $encUserId = $dbw->addQuotes( $wgUser->getId() );
	1448	+ $encReason = $dbw->addQuotes( $this->reason );
	1449	+ $encGroup = $dbw->addQuotes( 'deleted' );
	1450	+ $ext = $this->file->getExtension();
	1451	+ $dotExt = $ext === '' ? '' : ".$ext";
	1452	+ $encExt = $dbw->addQuotes( $dotExt );
	1453	+ list( $oldRels, $deleteCurrent ) = $this->getOldRels();
	1454	+
	1455	+ // Bitfields to further suppress the content
	1456	+ if ( $this->suppress ) {
	1457	+ $bitfield = 0;
	1458	+ // This should be 15...
	1459	+ $bitfield \|= Revision::DELETED_TEXT;
	1460	+ $bitfield \|= Revision::DELETED_COMMENT;
	1461	+ $bitfield \|= Revision::DELETED_USER;
	1462	+ $bitfield \|= Revision::DELETED_RESTRICTED;
	1463	+ } else {
	1464	+ $bitfield = 'oi_deleted';
	1465	+ }
	1466	+
	1467	+ if ( $deleteCurrent ) {
	1468	+ $concat = $dbw->buildConcat( array( "img_sha1", $encExt ) );
	1469	+ $where = array( 'img_name' => $this->file->getName() );
	1470	+ $dbw->insertSelect( 'filearchive', 'image',
	1471	+ array(
	1472	+ 'fa_storage_group' => $encGroup,
	1473	+ 'fa_storage_key' => "CASE WHEN img_sha1='' THEN '' ELSE $concat END",
	1474	+ 'fa_deleted_user' => $encUserId,
	1475	+ 'fa_deleted_timestamp' => $encTimestamp,
	1476	+ 'fa_deleted_reason' => $encReason,
	1477	+ 'fa_deleted' => $this->suppress ? $bitfield : 0,
	1478	+
	1479	+ 'fa_name' => 'img_name',
	1480	+ 'fa_archive_name' => 'NULL',
	1481	+ 'fa_size' => 'img_size',
	1482	+ 'fa_width' => 'img_width',
	1483	+ 'fa_height' => 'img_height',
	1484	+ 'fa_metadata' => 'img_metadata',
	1485	+ 'fa_bits' => 'img_bits',
	1486	+ 'fa_media_type' => 'img_media_type',
	1487	+ 'fa_major_mime' => 'img_major_mime',
	1488	+ 'fa_minor_mime' => 'img_minor_mime',
	1489	+ 'fa_description' => 'img_description',
	1490	+ 'fa_user' => 'img_user',
	1491	+ 'fa_user_text' => 'img_user_text',
	1492	+ 'fa_timestamp' => 'img_timestamp'
	1493	+ ), $where, __METHOD__ );
	1494	+ }
	1495	+
	1496	+ if ( count( $oldRels ) ) {
	1497	+ $concat = $dbw->buildConcat( array( "oi_sha1", $encExt ) );
	1498	+ $where = array(
	1499	+ 'oi_name' => $this->file->getName(),
	1500	+ 'oi_archive_name IN (' . $dbw->makeList( array_keys( $oldRels ) ) . ')' );
	1501	+ $dbw->insertSelect( 'filearchive', 'oldimage',
	1502	+ array(
	1503	+ 'fa_storage_group' => $encGroup,
	1504	+ 'fa_storage_key' => "CASE WHEN oi_sha1='' THEN '' ELSE $concat END",
	1505	+ 'fa_deleted_user' => $encUserId,
	1506	+ 'fa_deleted_timestamp' => $encTimestamp,
	1507	+ 'fa_deleted_reason' => $encReason,
	1508	+ 'fa_deleted' => $this->suppress ? $bitfield : 'oi_deleted',
	1509	+
	1510	+ 'fa_name' => 'oi_name',
	1511	+ 'fa_archive_name' => 'oi_archive_name',
	1512	+ 'fa_size' => 'oi_size',
	1513	+ 'fa_width' => 'oi_width',
	1514	+ 'fa_height' => 'oi_height',
	1515	+ 'fa_metadata' => 'oi_metadata',
	1516	+ 'fa_bits' => 'oi_bits',
	1517	+ 'fa_media_type' => 'oi_media_type',
	1518	+ 'fa_major_mime' => 'oi_major_mime',
	1519	+ 'fa_minor_mime' => 'oi_minor_mime',
	1520	+ 'fa_description' => 'oi_description',
	1521	+ 'fa_user' => 'oi_user',
	1522	+ 'fa_user_text' => 'oi_user_text',
	1523	+ 'fa_timestamp' => 'oi_timestamp',
	1524	+ 'fa_deleted' => $bitfield
	1525	+ ), $where, __METHOD__ );
	1526	+ }
	1527	+ }
	1528	+
	1529	+ function doDBDeletes() {
	1530	+ $dbw = $this->file->repo->getMasterDB();
	1531	+ list( $oldRels, $deleteCurrent ) = $this->getOldRels();
	1532	+
	1533	+ if ( count( $oldRels ) ) {
	1534	+ $dbw->delete( 'oldimage',
	1535	+ array(
	1536	+ 'oi_name' => $this->file->getName(),
	1537	+ 'oi_archive_name' => array_keys( $oldRels )
	1538	+ ), __METHOD__ );
	1539	+ }
	1540	+
	1541	+ if ( $deleteCurrent ) {
	1542	+ $dbw->delete( 'image', array( 'img_name' => $this->file->getName() ), __METHOD__ );
	1543	+ }
	1544	+ }
	1545	+
	1546	+ /**
	1547	+ * Run the transaction
	1548	+ */
	1549	+ function execute() {
	1550	+ global $wgUseSquid;
	1551	+ wfProfileIn( __METHOD__ );
	1552	+
	1553	+ $this->file->lock();
	1554	+ // Leave private files alone
	1555	+ $privateFiles = array();
	1556	+ list( $oldRels, $deleteCurrent ) = $this->getOldRels();
	1557	+ $dbw = $this->file->repo->getMasterDB();
	1558	+
	1559	+ if ( !empty( $oldRels ) ) {
	1560	+ $res = $dbw->select( 'oldimage',
	1561	+ array( 'oi_archive_name' ),
	1562	+ array( 'oi_name' => $this->file->getName(),
	1563	+ 'oi_archive_name IN (' . $dbw->makeList( array_keys( $oldRels ) ) . ')',
	1564	+ $dbw->bitAnd( 'oi_deleted', File::DELETED_FILE ) => File::DELETED_FILE ),
	1565	+ __METHOD__ );
	1566	+
	1567	+ foreach ( $res as $row ) {
	1568	+ $privateFiles[$row->oi_archive_name] = 1;
	1569	+ }
	1570	+ }
	1571	+ // Prepare deletion batch
	1572	+ $hashes = $this->getHashes();
	1573	+ $this->deletionBatch = array();
	1574	+ $ext = $this->file->getExtension();
	1575	+ $dotExt = $ext === '' ? '' : ".$ext";
	1576	+
	1577	+ foreach ( $this->srcRels as $name => $srcRel ) {
	1578	+ // Skip files that have no hash (missing source).
	1579	+ // Keep private files where they are.
	1580	+ if ( isset( $hashes[$name] ) && !array_key_exists( $name, $privateFiles ) ) {
	1581	+ $hash = $hashes[$name];
	1582	+ $key = $hash . $dotExt;
	1583	+ $dstRel = $this->file->repo->getDeletedHashPath( $key ) . $key;
	1584	+ $this->deletionBatch[$name] = array( $srcRel, $dstRel );
	1585	+ }
	1586	+ }
	1587	+
	1588	+ // Lock the filearchive rows so that the files don't get deleted by a cleanup operation
	1589	+ // We acquire this lock by running the inserts now, before the file operations.
	1590	+ //
	1591	+ // This potentially has poor lock contention characteristics -- an alternative
	1592	+ // scheme would be to insert stub filearchive entries with no fa_name and commit
	1593	+ // them in a separate transaction, then run the file ops, then update the fa_name fields.
	1594	+ $this->doDBInserts();
	1595	+
	1596	+ // Removes non-existent file from the batch, so we don't get errors.
	1597	+ $this->deletionBatch = $this->removeNonexistentFiles( $this->deletionBatch );
	1598	+
	1599	+ // Execute the file deletion batch
	1600	+ $status = $this->file->repo->deleteBatch( $this->deletionBatch );
	1601	+
	1602	+ if ( !$status->isGood() ) {
	1603	+ $this->status->merge( $status );
	1604	+ }
	1605	+
	1606	+ if ( !$this->status->ok ) {
	1607	+ // Critical file deletion error
	1608	+ // Roll back inserts, release lock and abort
	1609	+ // TODO: delete the defunct filearchive rows if we are using a non-transactional DB
	1610	+ $this->file->unlockAndRollback();
	1611	+ wfProfileOut( __METHOD__ );
	1612	+ return $this->status;
	1613	+ }
	1614	+
	1615	+ // Purge squid
	1616	+ if ( $wgUseSquid ) {
	1617	+ $urls = array();
	1618	+
	1619	+ foreach ( $this->srcRels as $srcRel ) {
	1620	+ $urlRel = str_replace( '%2F', '/', rawurlencode( $srcRel ) );
	1621	+ $urls[] = $this->file->repo->getZoneUrl( 'public' ) . '/' . $urlRel;
	1622	+ }
	1623	+ SquidUpdate::purge( $urls );
	1624	+ }
	1625	+
	1626	+ // Delete image/oldimage rows
	1627	+ $this->doDBDeletes();
	1628	+
	1629	+ // Commit and return
	1630	+ $this->file->unlock();
	1631	+ wfProfileOut( __METHOD__ );
	1632	+
	1633	+ return $this->status;
	1634	+ }
	1635	+
	1636	+ /**
	1637	+ * Removes non-existent files from a deletion batch.
	1638	+ */
	1639	+ function removeNonexistentFiles( $batch ) {
	1640	+ $files = $newBatch = array();
	1641	+
	1642	+ foreach ( $batch as $batchItem ) {
	1643	+ list( $src, $dest ) = $batchItem;
	1644	+ $files[$src] = $this->file->repo->getVirtualUrl( 'public' ) . '/' . rawurlencode( $src );
	1645	+ }
	1646	+
	1647	+ $result = $this->file->repo->fileExistsBatch( $files, FSRepo::FILES_ONLY );
	1648	+
	1649	+ foreach ( $batch as $batchItem ) {
	1650	+ if ( $result[$batchItem[0]] ) {
	1651	+ $newBatch[] = $batchItem;
	1652	+ }
	1653	+ }
	1654	+
	1655	+ return $newBatch;
	1656	+ }
	1657	+}
	1658	+
	1659	+# ------------------------------------------------------------------------------
	1660	+
	1661	+/**
	1662	+ * Helper class for file undeletion
	1663	+ * @ingroup FileRepo
	1664	+ */
	1665	+class SwiftFileRestoreBatch {
	1666	+ /**
	1667	+ * @var SwiftFile
	1668	+ */
	1669	+ var $file;
	1670	+
	1671	+ var $cleanupBatch, $ids, $all, $unsuppress = false;
	1672	+
	1673	+ function __construct( File $file, $unsuppress = false ) {
	1674	+ $this->file = $file;
	1675	+ $this->cleanupBatch = $this->ids = array();
	1676	+ $this->ids = array();
	1677	+ $this->unsuppress = $unsuppress;
	1678	+ }
	1679	+
	1680	+ /**
	1681	+ * Add a file by ID
	1682	+ */
	1683	+ function addId( $fa_id ) {
	1684	+ $this->ids[] = $fa_id;
	1685	+ }
	1686	+
	1687	+ /**
	1688	+ * Add a whole lot of files by ID
	1689	+ */
	1690	+ function addIds( $ids ) {
	1691	+ $this->ids = array_merge( $this->ids, $ids );
	1692	+ }
	1693	+
	1694	+ /**
	1695	+ * Add all revisions of the file
	1696	+ */
	1697	+ function addAll() {
	1698	+ $this->all = true;
	1699	+ }
	1700	+
	1701	+ /**
	1702	+ * Run the transaction, except the cleanup batch.
	1703	+ * The cleanup batch should be run in a separate transaction, because it locks different
	1704	+ * rows and there's no need to keep the image row locked while it's acquiring those locks
	1705	+ * The caller may have its own transaction open.
	1706	+ * So we save the batch and let the caller call cleanup()
	1707	+ */
	1708	+ function execute() {
	1709	+ global $wgLang;
	1710	+
	1711	+ if ( !$this->all && !$this->ids ) {
	1712	+ // Do nothing
	1713	+ return $this->file->repo->newGood();
	1714	+ }
	1715	+
	1716	+ $exists = $this->file->lock();
	1717	+ $dbw = $this->file->repo->getMasterDB();
	1718	+ $status = $this->file->repo->newGood();
	1719	+
	1720	+ // Fetch all or selected archived revisions for the file,
	1721	+ // sorted from the most recent to the oldest.
	1722	+ $conditions = array( 'fa_name' => $this->file->getName() );
	1723	+
	1724	+ if ( !$this->all ) {
	1725	+ $conditions[] = 'fa_id IN (' . $dbw->makeList( $this->ids ) . ')';
	1726	+ }
	1727	+
	1728	+ $result = $dbw->select( 'filearchive', '*',
	1729	+ $conditions,
	1730	+ __METHOD__,
	1731	+ array( 'ORDER BY' => 'fa_timestamp DESC' )
	1732	+ );
	1733	+
	1734	+ $idsPresent = array();
	1735	+ $storeBatch = array();
	1736	+ $insertBatch = array();
	1737	+ $insertCurrent = false;
	1738	+ $deleteIds = array();
	1739	+ $first = true;
	1740	+ $archiveNames = array();
	1741	+
	1742	+ foreach ( $result as $row ) {
	1743	+ $idsPresent[] = $row->fa_id;
	1744	+
	1745	+ if ( $row->fa_name != $this->file->getName() ) {
	1746	+ $status->error( 'undelete-filename-mismatch', $wgLang->timeanddate( $row->fa_timestamp ) );
	1747	+ $status->failCount++;
	1748	+ continue;
	1749	+ }
	1750	+
	1751	+ if ( $row->fa_storage_key == '' ) {
	1752	+ // Revision was missing pre-deletion
	1753	+ $status->error( 'undelete-bad-store-key', $wgLang->timeanddate( $row->fa_timestamp ) );
	1754	+ $status->failCount++;
	1755	+ continue;
	1756	+ }
	1757	+
	1758	+ $deletedRel = $this->file->repo->getDeletedHashPath( $row->fa_storage_key ) . $row->fa_storage_key;
	1759	+ $deletedUrl = $this->file->repo->getVirtualUrl() . '/deleted/' . $deletedRel;
	1760	+
	1761	+ $sha1 = substr( $row->fa_storage_key, 0, strcspn( $row->fa_storage_key, '.' ) );
	1762	+
	1763	+ # Fix leading zero
	1764	+ if ( strlen( $sha1 ) == 32 && $sha1[0] == '0' ) {
	1765	+ $sha1 = substr( $sha1, 1 );
	1766	+ }
	1767	+
	1768	+ if ( is_null( $row->fa_major_mime ) \|\| $row->fa_major_mime == 'unknown'
	1769	+ \|\| is_null( $row->fa_minor_mime ) \|\| $row->fa_minor_mime == 'unknown'
	1770	+ \|\| is_null( $row->fa_media_type ) \|\| $row->fa_media_type == 'UNKNOWN'
	1771	+ \|\| is_null( $row->fa_metadata ) ) {
	1772	+ // Refresh our metadata
	1773	+ // Required for a new current revision; nice for older ones too. :)
	1774	+ $props = RepoGroup::singleton()->getFileProps( $deletedUrl );
	1775	+ } else {
	1776	+ $props = array(
	1777	+ 'minor_mime' => $row->fa_minor_mime,
	1778	+ 'major_mime' => $row->fa_major_mime,
	1779	+ 'media_type' => $row->fa_media_type,
	1780	+ 'metadata' => $row->fa_metadata
	1781	+ );
	1782	+ }
	1783	+
	1784	+ if ( $first && !$exists ) {
	1785	+ // This revision will be published as the new current version
	1786	+ $destRel = $this->file->getRel();
	1787	+ $insertCurrent = array(
	1788	+ 'img_name' => $row->fa_name,
	1789	+ 'img_size' => $row->fa_size,
	1790	+ 'img_width' => $row->fa_width,
	1791	+ 'img_height' => $row->fa_height,
	1792	+ 'img_metadata' => $props['metadata'],
	1793	+ 'img_bits' => $row->fa_bits,
	1794	+ 'img_media_type' => $props['media_type'],
	1795	+ 'img_major_mime' => $props['major_mime'],
	1796	+ 'img_minor_mime' => $props['minor_mime'],
	1797	+ 'img_description' => $row->fa_description,
	1798	+ 'img_user' => $row->fa_user,
	1799	+ 'img_user_text' => $row->fa_user_text,
	1800	+ 'img_timestamp' => $row->fa_timestamp,
	1801	+ 'img_sha1' => $sha1
	1802	+ );
	1803	+
	1804	+ // The live (current) version cannot be hidden!
	1805	+ if ( !$this->unsuppress && $row->fa_deleted ) {
	1806	+ $storeBatch[] = array( $deletedUrl, 'public', $destRel );
	1807	+ $this->cleanupBatch[] = $row->fa_storage_key;
	1808	+ }
	1809	+ } else {
	1810	+ $archiveName = $row->fa_archive_name;
	1811	+
	1812	+ if ( $archiveName == '' ) {
	1813	+ // This was originally a current version; we
	1814	+ // have to devise a new archive name for it.
	1815	+ // Format is <timestamp of archiving>!<name>
	1816	+ $timestamp = wfTimestamp( TS_UNIX, $row->fa_deleted_timestamp );
	1817	+
	1818	+ do {
	1819	+ $archiveName = wfTimestamp( TS_MW, $timestamp ) . '!' . $row->fa_name;
	1820	+ $timestamp++;
	1821	+ } while ( isset( $archiveNames[$archiveName] ) );
	1822	+ }
	1823	+
	1824	+ $archiveNames[$archiveName] = true;
	1825	+ $destRel = $this->file->getArchiveRel( $archiveName );
	1826	+ $insertBatch[] = array(
	1827	+ 'oi_name' => $row->fa_name,
	1828	+ 'oi_archive_name' => $archiveName,
	1829	+ 'oi_size' => $row->fa_size,
	1830	+ 'oi_width' => $row->fa_width,
	1831	+ 'oi_height' => $row->fa_height,
	1832	+ 'oi_bits' => $row->fa_bits,
	1833	+ 'oi_description' => $row->fa_description,
	1834	+ 'oi_user' => $row->fa_user,
	1835	+ 'oi_user_text' => $row->fa_user_text,
	1836	+ 'oi_timestamp' => $row->fa_timestamp,
	1837	+ 'oi_metadata' => $props['metadata'],
	1838	+ 'oi_media_type' => $props['media_type'],
	1839	+ 'oi_major_mime' => $props['major_mime'],
	1840	+ 'oi_minor_mime' => $props['minor_mime'],
	1841	+ 'oi_deleted' => $this->unsuppress ? 0 : $row->fa_deleted,
	1842	+ 'oi_sha1' => $sha1 );
	1843	+ }
	1844	+
	1845	+ $deleteIds[] = $row->fa_id;
	1846	+
	1847	+ if ( !$this->unsuppress && $row->fa_deleted & File::DELETED_FILE ) {
	1848	+ // private files can stay where they are
	1849	+ $status->successCount++;
	1850	+ } else {
	1851	+ $storeBatch[] = array( $deletedUrl, 'public', $destRel );
	1852	+ $this->cleanupBatch[] = $row->fa_storage_key;
	1853	+ }
	1854	+
	1855	+ $first = false;
	1856	+ }
	1857	+
	1858	+ unset( $result );
	1859	+
	1860	+ // Add a warning to the status object for missing IDs
	1861	+ $missingIds = array_diff( $this->ids, $idsPresent );
	1862	+
	1863	+ foreach ( $missingIds as $id ) {
	1864	+ $status->error( 'undelete-missing-filearchive', $id );
	1865	+ }
	1866	+
	1867	+ // Remove missing files from batch, so we don't get errors when undeleting them
	1868	+ $storeBatch = $this->removeNonexistentFiles( $storeBatch );
	1869	+
	1870	+ // Run the store batch
	1871	+ // Use the OVERWRITE_SAME flag to smooth over a common error
	1872	+ $storeStatus = $this->file->repo->storeBatch( $storeBatch, FileRepo::OVERWRITE_SAME );
	1873	+ $status->merge( $storeStatus );
	1874	+
	1875	+ if ( !$status->isGood() ) {
	1876	+ // Even if some files could be copied, fail entirely as that is the
	1877	+ // easiest thing to do without data loss
	1878	+ $this->cleanupFailedBatch( $storeStatus, $storeBatch );
	1879	+ $status->ok = false;
	1880	+ $this->file->unlock();
	1881	+
	1882	+ return $status;
	1883	+ }
	1884	+
	1885	+ // Run the DB updates
	1886	+ // Because we have locked the image row, key conflicts should be rare.
	1887	+ // If they do occur, we can roll back the transaction at this time with
	1888	+ // no data loss, but leaving unregistered files scattered throughout the
	1889	+ // public zone.
	1890	+ // This is not ideal, which is why it's important to lock the image row.
	1891	+ if ( $insertCurrent ) {
	1892	+ $dbw->insert( 'image', $insertCurrent, __METHOD__ );
	1893	+ }
	1894	+
	1895	+ if ( $insertBatch ) {
	1896	+ $dbw->insert( 'oldimage', $insertBatch, __METHOD__ );
	1897	+ }
	1898	+
	1899	+ if ( $deleteIds ) {
	1900	+ $dbw->delete( 'filearchive',
	1901	+ array( 'fa_id IN (' . $dbw->makeList( $deleteIds ) . ')' ),
	1902	+ __METHOD__ );
	1903	+ }
	1904	+
	1905	+ // If store batch is empty (all files are missing), deletion is to be considered successful
	1906	+ if ( $status->successCount > 0 \|\| !$storeBatch ) {
	1907	+ if ( !$exists ) {
	1908	+ wfDebug( __METHOD__ . " restored {$status->successCount} items, creating a new current\n" );
	1909	+
	1910	+ // Update site_stats
	1911	+ $site_stats = $dbw->tableName( 'site_stats' );
	1912	+ $dbw->query( "UPDATE $site_stats SET ss_images=ss_images+1", __METHOD__ );
	1913	+
	1914	+ $this->file->purgeEverything();
	1915	+ } else {
	1916	+ wfDebug( __METHOD__ . " restored {$status->successCount} as archived versions\n" );
	1917	+ $this->file->purgeDescription();
	1918	+ $this->file->purgeHistory();
	1919	+ }
	1920	+ }
	1921	+
	1922	+ $this->file->unlock();
	1923	+
	1924	+ return $status;
	1925	+ }
	1926	+
	1927	+ /**
	1928	+ * Removes non-existent files from a store batch.
	1929	+ */
	1930	+ function removeNonexistentFiles( $triplets ) {
	1931	+ $files = $filteredTriplets = array();
	1932	+ foreach ( $triplets as $file )
	1933	+ $files[$file[0]] = $file[0];
	1934	+
	1935	+ $result = $this->file->repo->fileExistsBatch( $files, FSRepo::FILES_ONLY );
	1936	+
	1937	+ foreach ( $triplets as $file ) {
	1938	+ if ( $result[$file[0]] ) {
	1939	+ $filteredTriplets[] = $file;
	1940	+ }
	1941	+ }
	1942	+
	1943	+ return $filteredTriplets;
	1944	+ }
	1945	+
	1946	+ /**
	1947	+ * Removes non-existent files from a cleanup batch.
	1948	+ */
	1949	+ function removeNonexistentFromCleanup( $batch ) {
	1950	+ $files = $newBatch = array();
	1951	+ $repo = $this->file->repo;
	1952	+
	1953	+ foreach ( $batch as $file ) {
	1954	+ $files[$file] = $repo->getVirtualUrl( 'deleted' ) . '/' .
	1955	+ rawurlencode( $repo->getDeletedHashPath( $file ) . $file );
	1956	+ }
	1957	+
	1958	+ $result = $repo->fileExistsBatch( $files, FSRepo::FILES_ONLY );
	1959	+
	1960	+ foreach ( $batch as $file ) {
	1961	+ if ( $result[$file] ) {
	1962	+ $newBatch[] = $file;
	1963	+ }
	1964	+ }
	1965	+
	1966	+ return $newBatch;
	1967	+ }
	1968	+
	1969	+ /**
	1970	+ * Delete unused files in the deleted zone.
	1971	+ * This should be called from outside the transaction in which execute() was called.
	1972	+ */
	1973	+ function cleanup() {
	1974	+ if ( !$this->cleanupBatch ) {
	1975	+ return $this->file->repo->newGood();
	1976	+ }
	1977	+
	1978	+ $this->cleanupBatch = $this->removeNonexistentFromCleanup( $this->cleanupBatch );
	1979	+
	1980	+ $status = $this->file->repo->cleanupDeletedBatch( $this->cleanupBatch );
	1981	+
	1982	+ return $status;
	1983	+ }
	1984	+
	1985	+ function cleanupFailedBatch( $storeStatus, $storeBatch ) {
	1986	+ $cleanupBatch = array();
	1987	+
	1988	+ foreach ( $storeStatus->success as $i => $success ) {
	1989	+ if ( $success ) {
	1990	+ $cleanupBatch[] = array( $storeBatch[$i][1], $storeBatch[$i][1] );
	1991	+ }
	1992	+ }
	1993	+ $this->file->repo->cleanupBatch( $cleanupBatch );
	1994	+ }
	1995	+}
	1996	+
	1997	+# ------------------------------------------------------------------------------
	1998	+
	1999	+/**
	2000	+ * Helper class for file movement
	2001	+ * @ingroup FileRepo
	2002	+ */
	2003	+class SwiftFileMoveBatch {
	2004	+ var $file, $cur, $olds, $oldCount, $archive, $target, $db;
	2005	+
	2006	+ function __construct( File $file, Title $target ) {
	2007	+ $this->file = $file;
	2008	+ $this->target = $target;
	2009	+ $this->oldHash = $this->file->repo->getHashPath( $this->file->getName() );
	2010	+ $this->newHash = $this->file->repo->getHashPath( $this->target->getDBkey() );
	2011	+ $this->oldName = $this->file->getName();
	2012	+ $this->newName = $this->file->repo->getNameFromTitle( $this->target );
	2013	+ $this->oldRel = $this->oldHash . $this->oldName;
	2014	+ $this->newRel = $this->newHash . $this->newName;
	2015	+ $this->db = $file->repo->getMasterDb();
	2016	+ }
	2017	+
	2018	+ /**
	2019	+ * Add the current image to the batch
	2020	+ */
	2021	+ function addCurrent() {
	2022	+ $this->cur = array( $this->oldRel, $this->newRel );
	2023	+ }
	2024	+
	2025	+ /**
	2026	+ * Add the old versions of the image to the batch
	2027	+ */
	2028	+ function addOlds() {
	2029	+ $archiveBase = 'archive';
	2030	+ $this->olds = array();
	2031	+ $this->oldCount = 0;
	2032	+
	2033	+ $result = $this->db->select( 'oldimage',
	2034	+ array( 'oi_archive_name', 'oi_deleted' ),
	2035	+ array( 'oi_name' => $this->oldName ),
	2036	+ __METHOD__
	2037	+ );
	2038	+
	2039	+ foreach ( $result as $row ) {
	2040	+ $oldName = $row->oi_archive_name;
	2041	+ $bits = explode( '!', $oldName, 2 );
	2042	+
	2043	+ if ( count( $bits ) != 2 ) {
	2044	+ wfDebug( "Invalid old file name: $oldName \n" );
	2045	+ continue;
	2046	+ }
	2047	+
	2048	+ list( $timestamp, $filename ) = $bits;
	2049	+
	2050	+ if ( $this->oldName != $filename ) {
	2051	+ wfDebug( "Invalid old file name: $oldName \n" );
	2052	+ continue;
	2053	+ }
	2054	+
	2055	+ $this->oldCount++;
	2056	+
	2057	+ // Do we want to add those to oldCount?
	2058	+ if ( $row->oi_deleted & File::DELETED_FILE ) {
	2059	+ continue;
	2060	+ }
	2061	+
	2062	+ $this->olds[] = array(
	2063	+ "{$archiveBase}/{$this->oldHash}{$oldName}",
	2064	+ "{$archiveBase}/{$this->newHash}{$timestamp}!{$this->newName}"
	2065	+ );
	2066	+ }
	2067	+ }
	2068	+
	2069	+ /**
	2070	+ * Perform the move.
	2071	+ */
	2072	+ function execute() {
	2073	+ $repo = $this->file->repo;
	2074	+ $status = $repo->newGood();
	2075	+ $triplets = $this->getMoveTriplets();
	2076	+
	2077	+ $triplets = $this->removeNonexistentFiles( $triplets );
	2078	+
	2079	+ // Copy the files into their new location
	2080	+ $statusMove = $repo->storeBatch( $triplets );
	2081	+ wfDebugLog( 'imagemove', "Moved files for {$this->file->name}: {$statusMove->successCount} successes, {$statusMove->failCount} failures" );
	2082	+ if ( !$statusMove->isGood() ) {
	2083	+ wfDebugLog( 'imagemove', "Error in moving files: " . $statusMove->getWikiText() );
	2084	+ $this->cleanupTarget( $triplets );
	2085	+ $statusMove->ok = false;
	2086	+ return $statusMove;
	2087	+ }
	2088	+
	2089	+ $this->db->begin();
	2090	+ $statusDb = $this->doDBUpdates();
	2091	+ wfDebugLog( 'imagemove', "Renamed {$this->file->name} in database: {$statusDb->successCount} successes, {$statusDb->failCount} failures" );
	2092	+ if ( !$statusDb->isGood() ) {
	2093	+ $this->db->rollback();
	2094	+ // Something went wrong with the DB updates, so remove the target files
	2095	+ $this->cleanupTarget( $triplets );
	2096	+ $statusDb->ok = false;
	2097	+ return $statusDb;
	2098	+ }
	2099	+ $this->db->commit();
	2100	+
	2101	+ // Everything went ok, remove the source files
	2102	+ $this->cleanupSource( $triplets );
	2103	+
	2104	+ $status->merge( $statusDb );
	2105	+ $status->merge( $statusMove );
	2106	+
	2107	+ return $status;
	2108	+ }
	2109	+
	2110	+ /**
	2111	+ * Do the database updates and return a new FileRepoStatus indicating how
	2112	+ * many rows where updated.
	2113	+ *
	2114	+ * @return FileRepoStatus
	2115	+ */
	2116	+ function doDBUpdates() {
	2117	+ $repo = $this->file->repo;
	2118	+ $status = $repo->newGood();
	2119	+ $dbw = $this->db;
	2120	+
	2121	+ // Update current image
	2122	+ $dbw->update(
	2123	+ 'image',
	2124	+ array( 'img_name' => $this->newName ),
	2125	+ array( 'img_name' => $this->oldName ),
	2126	+ __METHOD__
	2127	+ );
	2128	+
	2129	+ if ( $dbw->affectedRows() ) {
	2130	+ $status->successCount++;
	2131	+ } else {
	2132	+ $status->failCount++;
	2133	+ $status->fatal( 'imageinvalidfilename' );
	2134	+ return $status;
	2135	+ }
	2136	+
	2137	+ // Update old images
	2138	+ $dbw->update(
	2139	+ 'oldimage',
	2140	+ array(
	2141	+ 'oi_name' => $this->newName,
	2142	+ 'oi_archive_name = ' . $dbw->strreplace( 'oi_archive_name', $dbw->addQuotes( $this->oldName ), $dbw->addQuotes( $this->newName ) ),
	2143	+ ),
	2144	+ array( 'oi_name' => $this->oldName ),
	2145	+ __METHOD__
	2146	+ );
	2147	+
	2148	+ $affected = $dbw->affectedRows();
	2149	+ $total = $this->oldCount;
	2150	+ $status->successCount += $affected;
	2151	+ $status->failCount += $total - $affected;
	2152	+ if ( $status->failCount ) {
	2153	+ $status->error( 'imageinvalidfilename' );
	2154	+ }
	2155	+
	2156	+ return $status;
	2157	+ }
	2158	+
	2159	+ /**
	2160	+ * Generate triplets for FSRepo::storeBatch().
	2161	+ */
	2162	+ function getMoveTriplets() {
	2163	+ $moves = array_merge( array( $this->cur ), $this->olds );
	2164	+ $triplets = array(); // The format is: (srcUrl, destZone, destUrl)
	2165	+
	2166	+ foreach ( $moves as $move ) {
	2167	+ // $move: (oldRelativePath, newRelativePath)
	2168	+ $srcUrl = $this->file->repo->getVirtualUrl() . '/public/' . rawurlencode( $move[0] );
	2169	+ $triplets[] = array( $srcUrl, 'public', $move[1] );
	2170	+ wfDebugLog( 'imagemove', "Generated move triplet for {$this->file->name}: {$srcUrl} :: public :: {$move[1]}" );
	2171	+ }
	2172	+
	2173	+ return $triplets;
	2174	+ }
	2175	+
	2176	+ /**
	2177	+ * Removes non-existent files from move batch.
	2178	+ */
	2179	+ function removeNonexistentFiles( $triplets ) {
	2180	+ $files = array();
	2181	+
	2182	+ foreach ( $triplets as $file ) {
	2183	+ $files[$file[0]] = $file[0];
	2184	+ }
	2185	+
	2186	+ $result = $this->file->repo->fileExistsBatch( $files, FSRepo::FILES_ONLY );
	2187	+ $filteredTriplets = array();
	2188	+
	2189	+ foreach ( $triplets as $file ) {
	2190	+ if ( $result[$file[0]] ) {
	2191	+ $filteredTriplets[] = $file;
	2192	+ } else {
	2193	+ wfDebugLog( 'imagemove', "File {$file[0]} does not exist" );
	2194	+ }
	2195	+ }
	2196	+
	2197	+ return $filteredTriplets;
	2198	+ }
	2199	+
	2200	+ /**
	2201	+ * Cleanup a partially moved array of triplets by deleting the target
	2202	+ * files. Called if something went wrong half way.
	2203	+ */
	2204	+ function cleanupTarget( $triplets ) {
	2205	+ // Create dest pairs from the triplets
	2206	+ $pairs = array();
	2207	+ foreach ( $triplets as $triplet ) {
	2208	+ $pairs[] = array( $triplet[1], $triplet[2] );
	2209	+ }
	2210	+
	2211	+ $this->file->repo->cleanupBatch( $pairs );
	2212	+ }
	2213	+
	2214	+ /**
	2215	+ * Cleanup a fully moved array of triplets by deleting the source files.
	2216	+ * Called at the end of the move process if everything else went ok.
	2217	+ */
	2218	+ function cleanupSource( $triplets ) {
	2219	+ // Create source file names from the triplets
	2220	+ $files = array();
	2221	+ foreach ( $triplets as $triplet ) {
	2222	+ $files[] = $triplet[0];
	2223	+ }
	2224	+
	2225	+ $this->file->repo->cleanupBatch( $files );
	2226	+ }
	2227	+}
	2228	+
	2229	+/**
	2230	+ * Repository that stores files in Swift and registers them
	2231	+ * in the wiki's own database.
	2232	+ *
	2233	+ * @file
	2234	+ * @ingroup FileRepo
	2235	+ */
	2236	+
	2237	+class SwiftRepo extends LocalRepo {
	2238	+ var $fileFactory = array( 'SwiftFile', 'newFromTitle' );
	2239	+ var $fileFactoryKey = array( 'SwiftFile', 'newFromKey' );
	2240	+ var $oldFileFactory = array( 'OldLocalFile', 'newFromTitle' );
	2241	+ var $oldFileFactoryKey = array( 'OldLocalFile', 'newFromKey' );
	2242	+ var $fileFromRowFactory = array( 'SwiftFile', 'newFromRow' );
	2243	+ var $oldFileFromRowFactory = array( 'OldLocalFile', 'newFromRow' );
	2244	+
	2245	+ function __construct( $info ) {
	2246	+ parent::__construct( $info );
	2247	+
	2248	+ // Required settings
	2249	+ $this->key= $info['key'];
	2250	+ $this->swiftuser= $info['user'];
	2251	+ $this->authurl= $info['authurl'];
	2252	+ $this->container= $info['container'];
	2253	+ }
	2254	+
	2255	+ function storeBatch( $triplets, $flags = 0 ) {
	2256	+ wfDebug( __METHOD__ . " $triplets\n" );
	2257	+ return parent::storeBatch( $triplets, $flags);
	2258	+ }
	2259	+
	2260	+ function storeTemp( $originalName, $srcPath ) {
	2261	+ wfDebug( __METHOD__ . " $originalName, $srcPath\n" );
	2262	+ return parent::storeTemp( $originalName, $srcPath);
	2263	+ }
	2264	+ function append( $srcPath, $toAppendPath, $flags = 0 ){
	2265	+ wfDebug( __METHOD__ . " $srcPath, $toAppendPath, $flags\n" );
	2266	+ return parent::append( $srcPath, $toAppendPath, $flags );
	2267	+ }
	2268	+ function deleteBatch( $sourceDestPairs ) {
	2269	+ wfDebug( __METHOD__ . " $sourceDestPairs\n" );
	2270	+ return parent::deleteBatch( $sourceDestPairs );
	2271	+ }
	2272	+ function fileExistsBatch( $files, $flags = 0 ) {
	2273	+ wfDebug( __METHOD__ . " $files\n" );
	2274	+ return parent::fileExistsBatch( $files, $flags );
	2275	+ }
	2276	+ function getFileProps( $virtualUrl ) {
	2277	+ wfDebug( __METHOD__ . " $virtualUrl\n" );
	2278	+ return parent::getFileProps( $virtualUrl );
	2279	+ }
	2280	+ function newFile( $title, $time = false ) {
	2281	+ if ( empty($title) ) { return null; }
	2282	+ wfDebug( __METHOD__ . " $title, $time " . var_export($this->fileFactory, true) ."\n" );
	2283	+ $f = parent::newFile( $title, $time );
	2284	+ $f->key= $this->key;
	2285	+ $f->swiftuser= $this->swiftuser;
	2286	+ $f->authurl= $this->authurl;
	2287	+ $f->container= $this->container;
	2288	+ return $f;
	2289	+ }
	2290	+ function findFile( $title, $options = array() ) {
	2291	+ global $wgLocalFileRepo;
	2292	+ wfDebug( __METHOD__ . " finding $title\n" );
	2293	+ wfDebug( __METHOD__ . var_export($wgLocalFileRepo, true) . "\n" );
	2294	+ return parent::findFile( $title, $options );
	2295	+ }
	2296	+
	2297	+ function swiftcopy($container, $dstRel, $archiveRel ) {
	2298	+ // Note the assumption that we're not doing cross-container copies.
	2299	+ // The destination must exist already.
	2300	+ $obj = $container->create_object($archiveRel);
	2301	+ $obj->write(".");
	2302	+ $obj->close();
	2303	+ $obj = $container->get_object($dstRel);
	2304	+ $success = $obj->copy($container->name . "/" . $archiveRel);
	2305	+ return $success;
	2306	+ }
	2307	+
	2308	+ /**
	2309	+ * Publish a batch of files
	2310	+ * @param $triplets Array: (source,dest,archive) triplets as per publish()
	2311	+ * @param $flags Integer: bitfield, may be FileRepo::DELETE_SOURCE to indicate
	2312	+ * that the source files should be deleted if possible
	2313	+ */
	2314	+ function publishBatch( $triplets, $flags = 0 ) {
	2315	+ $auth = new CF_Authentication($this->swiftuser, $this->key, NULL, $this->authurl);
	2316	+ wfDebug( __METHOD__ . $this->swiftuser . "," . $this->key . "," . $this->authurl . "\n");
	2317	+ $auth->authenticate();
	2318	+ $conn = new CF_Connection($auth);
	2319	+ $container = $conn->get_container($this->container);
	2320	+ #wfDebug( "Number of Objects: " . $container->object_count . "\n" );
	2321	+ #wfDebug( "Bytes stored in container: " . $container->bytes_used . "\n" );
	2322	+ #wfDebug( "Object: " . var_export($pic, true) . "\n" );
	2323	+
	2324	+ # Delete specific object
	2325	+ #$container->delete_object("disco_dancing.jpg");
	2326	+
	2327	+ # paranoia
	2328	+ $status = $this->newGood( array() );
	2329	+ foreach ( $triplets as $i => $triplet ) {
	2330	+ list( $srcPath, $dstRel, $archiveRel ) = $triplet;
	2331	+
	2332	+ if ( !$this->validateFilename( $dstRel ) ) {
	2333	+ throw new MWException( 'Validation error in $dstRel' );
	2334	+ }
	2335	+ if ( !$this->validateFilename( $archiveRel ) ) {
	2336	+ throw new MWException( 'Validation error in $archiveRel' );
	2337	+ }
	2338	+ if ( !is_file( $srcPath ) ) {
	2339	+ // Make a list of files that don't exist for return to the caller
	2340	+ $status->fatal( 'filenotfound', $srcPath );
	2341	+ }
	2342	+ }
	2343	+
	2344	+ if ( !$status->ok ) {
	2345	+ return $status;
	2346	+ }
	2347	+
	2348	+ foreach ( $triplets as $i => $triplet ) {
	2349	+ list( $srcPath, $dstRel, $archiveRel ) = $triplet;
	2350	+ $dstPath = "{$this->directory}/$dstRel";
	2351	+ $archivePath = "{$this->directory}/$archiveRel";
	2352	+
	2353	+ // Archive destination file if it exists
	2354	+ try {
	2355	+ $pic = $container->get_object($dstRel);
	2356	+ } catch (NoSuchObjectException $e) {
	2357	+ $pic = NULL;
	2358	+ }
	2359	+ if( $pic ) {
	2360	+ // Check if the archive file exists
	2361	+ // This is a sanity check to avoid data loss. In UNIX, the rename primitive
	2362	+ // unlinks the destination file if it exists. DB-based synchronisation in
	2363	+ // publishBatch's caller should prevent races. In Windows there's no
	2364	+ // problem because the rename primitive fails if the destination exists.
	2365	+ try {
	2366	+ $success = $this->swiftcopy($container, $dstRel, $archiveRel );
	2367	+ } catch (NoSuchObjectException $e) {
	2368	+ $success = false;
	2369	+ }
	2370	+
	2371	+ if( !$success ) {
	2372	+ $status->error( 'filerenameerror',$dstPath, $archivePath );
	2373	+ $status->failCount++;
	2374	+ continue;
	2375	+ } else {
	2376	+ wfDebug(__METHOD__.": moved file $dstPath to $archivePath\n");
	2377	+ }
	2378	+ $status->value[$i] = 'archived';
	2379	+ } else {
	2380	+ $status->value[$i] = 'new';
	2381	+ }
	2382	+
	2383	+ $good = true;
	2384	+ // how does this return failure??
	2385	+ $obj = $container->create_object($dstRel);
	2386	+ // we need to do a try here
	2387	+ $obj->load_from_filename( $srcPath, True);
	2388	+ // $status->error( 'filecopyerror', $srcPath, $dstPath );
	2389	+ // $good = false;
	2390	+ #if ( $flags & self::DELETE_SOURCE ) {
	2391	+ #delete ( $srcPath );
	2392	+ #}
	2393	+
	2394	+ if ( $good ) {
	2395	+ $status->successCount++;
	2396	+ wfDebug(__METHOD__.": wrote tempfile $srcPath to $dstPath\n");
	2397	+ // Thread-safe override for umask
	2398	+ $this->chmod( $dstPath );
	2399	+ } else {
	2400	+ $status->failCount++;
	2401	+ }
	2402	+ }
	2403	+ return $status;
	2404	+ }
	2405	+
	2406	+
	2407	+}
	2408	+
	2409	+
	2410	+class Junkyjunk {
	2411	+ var $directory, $deletedDir, $deletedHashLevels, $fileMode;
	2412	+ var $fileFactory = array( 'UnregisteredLocalFile', 'newFromTitle' );
	2413	+ var $oldFileFactory = false;
	2414	+ var $pathDisclosureProtection = 'simple';
	2415	+
	2416	+ function __construct( $info ) {
	2417	+ parent::__construct( $info );
	2418	+
	2419	+ // Required settings
	2420	+ $this->directory = $info['directory'];
	2421	+ $this->url = $info['url'];
	2422	+
	2423	+ // Optional settings
	2424	+ $this->hashLevels = isset( $info['hashLevels'] ) ? $info['hashLevels'] : 2;
	2425	+ $this->deletedHashLevels = isset( $info['deletedHashLevels'] ) ?
	2426	+ $info['deletedHashLevels'] : $this->hashLevels;
	2427	+ $this->deletedDir = isset( $info['deletedDir'] ) ? $info['deletedDir'] : false;
	2428	+ $this->fileMode = isset( $info['fileMode'] ) ? $info['fileMode'] : 0644;
	2429	+ if ( isset( $info['thumbDir'] ) ) {
	2430	+ $this->thumbDir = $info['thumbDir'];
	2431	+ } else {
	2432	+ $this->thumbDir = "{$this->directory}/thumb";
	2433	+ }
	2434	+ if ( isset( $info['thumbUrl'] ) ) {
	2435	+ $this->thumbUrl = $info['thumbUrl'];
	2436	+ } else {
	2437	+ $this->thumbUrl = "{$this->url}/thumb";
	2438	+ }
	2439	+ }
	2440	+
	2441	+ /**
	2442	+ * Get the public root directory of the repository.
	2443	+ */
	2444	+ function getRootDirectory() {
	2445	+ return $this->directory;
	2446	+ }
	2447	+
	2448	+ /**
	2449	+ * Get the public root URL of the repository
	2450	+ */
	2451	+ function getRootUrl() {
	2452	+ return $this->url;
	2453	+ }
	2454	+
	2455	+ /**
	2456	+ * Returns true if the repository uses a multi-level directory structure
	2457	+ */
	2458	+ function isHashed() {
	2459	+ return (bool)$this->hashLevels;
	2460	+ }
	2461	+
	2462	+ /**
	2463	+ * Get the local directory corresponding to one of the three basic zones
	2464	+ */
	2465	+ function getZonePath( $zone ) {
	2466	+ switch ( $zone ) {
	2467	+ case 'public':
	2468	+ return $this->directory;
	2469	+ case 'temp':
	2470	+ return "{$this->directory}/temp";
	2471	+ case 'deleted':
	2472	+ return $this->deletedDir;
	2473	+ case 'thumb':
	2474	+ return $this->thumbDir;
	2475	+ default:
	2476	+ return false;
	2477	+ }
	2478	+ }
	2479	+
	2480	+ /**
	2481	+ * @see FileRepo::getZoneUrl()
	2482	+ */
	2483	+ function getZoneUrl( $zone ) {
	2484	+ switch ( $zone ) {
	2485	+ case 'public':
	2486	+ return $this->url;
	2487	+ case 'temp':
	2488	+ return "{$this->url}/temp";
	2489	+ case 'deleted':
	2490	+ return parent::getZoneUrl( $zone ); // no public URL
	2491	+ case 'thumb':
	2492	+ return $this->thumbUrl;
	2493	+ default:
	2494	+ return parent::getZoneUrl( $zone );
	2495	+ }
	2496	+ }
	2497	+
	2498	+ /**
	2499	+ * Get a URL referring to this repository, with the private mwrepo protocol.
	2500	+ * The suffix, if supplied, is considered to be unencoded, and will be
	2501	+ * URL-encoded before being returned.
	2502	+ */
	2503	+ function getVirtualUrl( $suffix = false ) {
	2504	+ $path = 'mwrepo://' . $this->name;
	2505	+ if ( $suffix !== false ) {
	2506	+ $path .= '/' . rawurlencode( $suffix );
	2507	+ }
	2508	+ return $path;
	2509	+ }
	2510	+
	2511	+ /**
	2512	+ * Get the local path corresponding to a virtual URL
	2513	+ */
	2514	+ function resolveVirtualUrl( $url ) {
	2515	+ if ( substr( $url, 0, 9 ) != 'mwrepo://' ) {
	2516	+ throw new MWException( __METHOD__.': unknown protoocl' );
	2517	+ }
	2518	+
	2519	+ $bits = explode( '/', substr( $url, 9 ), 3 );
	2520	+ if ( count( $bits ) != 3 ) {
	2521	+ throw new MWException( __METHOD__.": invalid mwrepo URL: $url" );
	2522	+ }
	2523	+ list( $repo, $zone, $rel ) = $bits;
	2524	+ if ( $repo !== $this->name ) {
	2525	+ throw new MWException( __METHOD__.": fetching from a foreign repo is not supported" );
	2526	+ }
	2527	+ $base = $this->getZonePath( $zone );
	2528	+ if ( !$base ) {
	2529	+ throw new MWException( __METHOD__.": invalid zone: $zone" );
	2530	+ }
	2531	+ return $base . '/' . rawurldecode( $rel );
	2532	+ }
	2533	+
	2534	+ /**
	2535	+ * Store a batch of files
	2536	+ *
	2537	+ * @param $triplets Array: (src,zone,dest) triplets as per store()
	2538	+ * @param $flags Integer: bitwise combination of the following flags:
	2539	+ * self::DELETE_SOURCE Delete the source file after upload
	2540	+ * self::OVERWRITE Overwrite an existing destination file instead of failing
	2541	+ * self::OVERWRITE_SAME Overwrite the file if the destination exists and has the
	2542	+ * same contents as the source
	2543	+ */
	2544	+ function storeBatch( $triplets, $flags = 0 ) {
	2545	+ wfDebug( __METHOD__ . ': Storing ' . count( $triplets ) .
	2546	+ " triplets; flags: {$flags}\n" );
	2547	+
	2548	+ // Try creating directories
	2549	+ if ( !wfMkdirParents( $this->directory ) ) {
	2550	+ return $this->newFatal( 'upload_directory_missing', $this->directory );
	2551	+ }
	2552	+ if ( !is_writable( $this->directory ) ) {
	2553	+ return $this->newFatal( 'upload_directory_read_only', $this->directory );
	2554	+ }
	2555	+
	2556	+ // Validate each triplet
	2557	+ $status = $this->newGood();
	2558	+ foreach ( $triplets as $i => $triplet ) {
	2559	+ list( $srcPath, $dstZone, $dstRel ) = $triplet;
	2560	+
	2561	+ // Resolve destination path
	2562	+ $root = $this->getZonePath( $dstZone );
	2563	+ if ( !$root ) {
	2564	+ throw new MWException( "Invalid zone: $dstZone" );
	2565	+ }
	2566	+ if ( !$this->validateFilename( $dstRel ) ) {
	2567	+ throw new MWException( 'Validation error in $dstRel' );
	2568	+ }
	2569	+ $dstPath = "$root/$dstRel";
	2570	+ $dstDir = dirname( $dstPath );
	2571	+
	2572	+ // Create destination directories for this triplet
	2573	+ if ( !is_dir( $dstDir ) ) {
	2574	+ if ( !wfMkdirParents( $dstDir ) ) {
	2575	+ return $this->newFatal( 'directorycreateerror', $dstDir );
	2576	+ }
	2577	+ if ( $dstZone == 'deleted' ) {
	2578	+ $this->initDeletedDir( $dstDir );
	2579	+ }
	2580	+ }
	2581	+
	2582	+ // Resolve source
	2583	+ if ( self::isVirtualUrl( $srcPath ) ) {
	2584	+ $srcPath = $triplets[$i][0] = $this->resolveVirtualUrl( $srcPath );
	2585	+ }
	2586	+ if ( !is_file( $srcPath ) ) {
	2587	+ // Make a list of files that don't exist for return to the caller
	2588	+ $status->fatal( 'filenotfound', $srcPath );
	2589	+ continue;
	2590	+ }
	2591	+
	2592	+ // Check overwriting
	2593	+ if ( !( $flags & self::OVERWRITE ) && file_exists( $dstPath ) ) {
	2594	+ if ( $flags & self::OVERWRITE_SAME ) {
	2595	+ $hashSource = sha1_file( $srcPath );
	2596	+ $hashDest = sha1_file( $dstPath );
	2597	+ if ( $hashSource != $hashDest ) {
	2598	+ $status->fatal( 'fileexistserror', $dstPath );
	2599	+ }
	2600	+ } else {
	2601	+ $status->fatal( 'fileexistserror', $dstPath );
	2602	+ }
	2603	+ }
	2604	+ }
	2605	+
	2606	+ // Windows does not support moving over existing files, so explicitly delete them
	2607	+ $deleteDest = wfIsWindows() && ( $flags & self::OVERWRITE );
	2608	+
	2609	+ // Abort now on failure
	2610	+ if ( !$status->ok ) {
	2611	+ return $status;
	2612	+ }
	2613	+
	2614	+ // Execute the store operation for each triplet
	2615	+ foreach ( $triplets as $i => $triplet ) {
	2616	+ list( $srcPath, $dstZone, $dstRel ) = $triplet;
	2617	+ $root = $this->getZonePath( $dstZone );
	2618	+ $dstPath = "$root/$dstRel";
	2619	+ $good = true;
	2620	+
	2621	+ if ( $flags & self::DELETE_SOURCE ) {
	2622	+ if ( $deleteDest ) {
	2623	+ unlink( $dstPath );
	2624	+ }
	2625	+ if ( !rename( $srcPath, $dstPath ) ) {
	2626	+ $status->error( 'filerenameerror', $srcPath, $dstPath );
	2627	+ $good = false;
	2628	+ }
	2629	+ } else {
	2630	+ if ( !copy( $srcPath, $dstPath ) ) {
	2631	+ $status->error( 'filecopyerror', $srcPath, $dstPath );
	2632	+ $good = false;
	2633	+ }
	2634	+ if ( !( $flags & self::SKIP_VALIDATION ) ) {
	2635	+ wfSuppressWarnings();
	2636	+ $hashSource = sha1_file( $srcPath );
	2637	+ $hashDest = sha1_file( $dstPath );
	2638	+ wfRestoreWarnings();
	2639	+
	2640	+ if ( $hashDest === false \|\| $hashSource !== $hashDest ) {
	2641	+ wfDebug( __METHOD__ . ': File copy validation failed: ' .
	2642	+ "$srcPath ($hashSource) to $dstPath ($hashDest)\n" );
	2643	+
	2644	+ $status->error( 'filecopyerror', $srcPath, $dstPath );
	2645	+ $good = false;
	2646	+ }
	2647	+ }
	2648	+ }
	2649	+ if ( $good ) {
	2650	+ $this->chmod( $dstPath );
	2651	+ $status->successCount++;
	2652	+ } else {
	2653	+ $status->failCount++;
	2654	+ }
	2655	+ $status->success[$i] = $good;
	2656	+ }
	2657	+ return $status;
	2658	+ }
	2659	+
	2660	+ /**
	2661	+ * Deletes a batch of files. Each file can be a (zone, rel) pairs, a
	2662	+ * virtual url or a real path. It will try to delete each file, but
	2663	+ * ignores any errors that may occur
	2664	+ *
	2665	+ * @param $pairs array List of files to delete
	2666	+ */
	2667	+ function cleanupBatch( $files ) {
	2668	+ foreach ( $files as $file ) {
	2669	+ if ( is_array( $file ) ) {
	2670	+ // This is a pair, extract it
	2671	+ list( $zone, $rel ) = $file;
	2672	+ $root = $this->getZonePath( $zone );
	2673	+ $path = "$root/$rel";
	2674	+ } else {
	2675	+ if ( self::isVirtualUrl( $file ) ) {
	2676	+ // This is a virtual url, resolve it
	2677	+ $path = $this->resolveVirtualUrl( $file );
	2678	+ } else {
	2679	+ // This is a full file name
	2680	+ $path = $file;
	2681	+ }
	2682	+ }
	2683	+
	2684	+ wfSuppressWarnings();
	2685	+ unlink( $path );
	2686	+ wfRestoreWarnings();
	2687	+ }
	2688	+ }
	2689	+
	2690	+ function append( $srcPath, $toAppendPath, $flags = 0 ) {
	2691	+ $status = $this->newGood();
	2692	+
	2693	+ // Resolve the virtual URL
	2694	+ if ( self::isVirtualUrl( $srcPath ) ) {
	2695	+ $srcPath = $this->resolveVirtualUrl( $srcPath );
	2696	+ }
	2697	+ // Make sure the files are there
	2698	+ if ( !is_file( $srcPath ) )
	2699	+ $status->fatal( 'filenotfound', $srcPath );
	2700	+
	2701	+ if ( !is_file( $toAppendPath ) )
	2702	+ $status->fatal( 'filenotfound', $toAppendPath );
	2703	+
	2704	+ if ( !$status->isOk() ) return $status;
	2705	+
	2706	+ // Do the append
	2707	+ $chunk = file_get_contents( $toAppendPath );
	2708	+ if( $chunk === false ) {
	2709	+ $status->fatal( 'fileappenderrorread', $toAppendPath );
	2710	+ }
	2711	+
	2712	+ if( $status->isOk() ) {
	2713	+ if ( file_put_contents( $srcPath, $chunk, FILE_APPEND ) ) {
	2714	+ $status->value = $srcPath;
	2715	+ } else {
	2716	+ $status->fatal( 'fileappenderror', $toAppendPath, $srcPath);
	2717	+ }
	2718	+ }
	2719	+
	2720	+ if ( $flags & self::DELETE_SOURCE ) {
	2721	+ unlink( $toAppendPath );
	2722	+ }
	2723	+
	2724	+ return $status;
	2725	+ }
	2726	+
	2727	+ /**
	2728	+ * Checks existence of specified array of files.
	2729	+ *
	2730	+ * @param $files Array: URLs of files to check
	2731	+ * @param $flags Integer: bitwise combination of the following flags:
	2732	+ * self::FILES_ONLY Mark file as existing only if it is a file (not directory)
	2733	+ * @return Either array of files and existence flags, or false
	2734	+ */
	2735	+ function fileExistsBatch( $files, $flags = 0 ) {
	2736	+ if ( !file_exists( $this->directory ) \|\| !is_readable( $this->directory ) ) {
	2737	+ return false;
	2738	+ }
	2739	+ $result = array();
	2740	+ foreach ( $files as $key => $file ) {
	2741	+ if ( self::isVirtualUrl( $file ) ) {
	2742	+ $file = $this->resolveVirtualUrl( $file );
	2743	+ }
	2744	+ if( $flags & self::FILES_ONLY ) {
	2745	+ $result[$key] = is_file( $file );
	2746	+ } else {
	2747	+ $result[$key] = file_exists( $file );
	2748	+ }
	2749	+ }
	2750	+
	2751	+ return $result;
	2752	+ }
	2753	+
	2754	+ /**
	2755	+ * Take all available measures to prevent web accessibility of new deleted
	2756	+ * directories, in case the user has not configured offline storage
	2757	+ */
	2758	+ protected function initDeletedDir( $dir ) {
	2759	+ // Add a .htaccess file to the root of the deleted zone
	2760	+ $root = $this->getZonePath( 'deleted' );
	2761	+ if ( !file_exists( "$root/.htaccess" ) ) {
	2762	+ file_put_contents( "$root/.htaccess", "Deny from all\n" );
	2763	+ }
	2764	+ // Seed new directories with a blank index.html, to prevent crawling
	2765	+ file_put_contents( "$dir/index.html", '' );
	2766	+ }
	2767	+
	2768	+ /**
	2769	+ * Pick a random name in the temp zone and store a file to it.
	2770	+ * @param $originalName String: the base name of the file as specified
	2771	+ * by the user. The file extension will be maintained.
	2772	+ * @param $srcPath String: the current location of the file.
	2773	+ * @return FileRepoStatus object with the URL in the value.
	2774	+ */
	2775	+ function storeTemp( $originalName, $srcPath ) {
	2776	+ $date = gmdate( "YmdHis" );
	2777	+ $hashPath = $this->getHashPath( $originalName );
	2778	+ $dstRel = "$hashPath$date!$originalName";
	2779	+ $dstUrlRel = $hashPath . $date . '!' . rawurlencode( $originalName );
	2780	+
	2781	+ $result = $this->store( $srcPath, 'temp', $dstRel );
	2782	+ $result->value = $this->getVirtualUrl( 'temp' ) . '/' . $dstUrlRel;
	2783	+ return $result;
	2784	+ }
	2785	+
	2786	+ /**
	2787	+ * Remove a temporary file or mark it for garbage collection
	2788	+ * @param $virtualUrl String: the virtual URL returned by storeTemp
	2789	+ * @return Boolean: true on success, false on failure
	2790	+ */
	2791	+ function freeTemp( $virtualUrl ) {
	2792	+ $temp = "mwrepo://{$this->name}/temp";
	2793	+ if ( substr( $virtualUrl, 0, strlen( $temp ) ) != $temp ) {
	2794	+ wfDebug( __METHOD__.": Invalid virtual URL\n" );
	2795	+ return false;
	2796	+ }
	2797	+ $path = $this->resolveVirtualUrl( $virtualUrl );
	2798	+ wfSuppressWarnings();
	2799	+ $success = unlink( $path );
	2800	+ wfRestoreWarnings();
	2801	+ return $success;
	2802	+ }
	2803	+
	2804	+ /**
	2805	+ * Move a group of files to the deletion archive.
	2806	+ * If no valid deletion archive is configured, this may either delete the
	2807	+ * file or throw an exception, depending on the preference of the repository.
	2808	+ *
	2809	+ * @param $sourceDestPairs Array of source/destination pairs. Each element
	2810	+ * is a two-element array containing the source file path relative to the
	2811	+ * public root in the first element, and the archive file path relative
	2812	+ * to the deleted zone root in the second element.
	2813	+ * @return FileRepoStatus
	2814	+ */
	2815	+ function deleteBatch( $sourceDestPairs ) {
	2816	+ $status = $this->newGood();
	2817	+ if ( !$this->deletedDir ) {
	2818	+ throw new MWException( __METHOD__.': no valid deletion archive directory' );
	2819	+ }
	2820	+
	2821	+ /**
	2822	+ * Validate filenames and create archive directories
	2823	+ */
	2824	+ foreach ( $sourceDestPairs as $pair ) {
	2825	+ list( $srcRel, $archiveRel ) = $pair;
	2826	+ if ( !$this->validateFilename( $srcRel ) ) {
	2827	+ throw new MWException( __METHOD__.':Validation error in $srcRel' );
	2828	+ }
	2829	+ if ( !$this->validateFilename( $archiveRel ) ) {
	2830	+ throw new MWException( __METHOD__.':Validation error in $archiveRel' );
	2831	+ }
	2832	+ $archivePath = "{$this->deletedDir}/$archiveRel";
	2833	+ $archiveDir = dirname( $archivePath );
	2834	+ if ( !is_dir( $archiveDir ) ) {
	2835	+ if ( !wfMkdirParents( $archiveDir ) ) {
	2836	+ $status->fatal( 'directorycreateerror', $archiveDir );
	2837	+ continue;
	2838	+ }
	2839	+ $this->initDeletedDir( $archiveDir );
	2840	+ }
	2841	+ // Check if the archive directory is writable
	2842	+ // This doesn't appear to work on NTFS
	2843	+ if ( !is_writable( $archiveDir ) ) {
	2844	+ $status->fatal( 'filedelete-archive-read-only', $archiveDir );
	2845	+ }
	2846	+ }
	2847	+ if ( !$status->ok ) {
	2848	+ // Abort early
	2849	+ return $status;
	2850	+ }
	2851	+
	2852	+ /**
	2853	+ * Move the files
	2854	+ * We're now committed to returning an OK result, which will lead to
	2855	+ * the files being moved in the DB also.
	2856	+ */
	2857	+ foreach ( $sourceDestPairs as $pair ) {
	2858	+ list( $srcRel, $archiveRel ) = $pair;
	2859	+ $srcPath = "{$this->directory}/$srcRel";
	2860	+ $archivePath = "{$this->deletedDir}/$archiveRel";
	2861	+ $good = true;
	2862	+ if ( file_exists( $archivePath ) ) {
	2863	+ # A file with this content hash is already archived
	2864	+ if ( !@unlink( $srcPath ) ) {
	2865	+ $status->error( 'filedeleteerror', $srcPath );
	2866	+ $good = false;
	2867	+ }
	2868	+ } else{
	2869	+ if ( !@rename( $srcPath, $archivePath ) ) {
	2870	+ $status->error( 'filerenameerror', $srcPath, $archivePath );
	2871	+ $good = false;
	2872	+ } else {
	2873	+ $this->chmod( $archivePath );
	2874	+ }
	2875	+ }
	2876	+ if ( $good ) {
	2877	+ $status->successCount++;
	2878	+ } else {
	2879	+ $status->failCount++;
	2880	+ }
	2881	+ }
	2882	+ return $status;
	2883	+ }
	2884	+
	2885	+ /**
	2886	+ * Get a relative path for a deletion archive key,
	2887	+ * e.g. s/z/a/ for sza251lrxrc1jad41h5mgilp8nysje52.jpg
	2888	+ */
	2889	+ function getDeletedHashPath( $key ) {
	2890	+ $path = '';
	2891	+ for ( $i = 0; $i < $this->deletedHashLevels; $i++ ) {
	2892	+ $path .= $key[$i] . '/';
	2893	+ }
	2894	+ return $path;
	2895	+ }
	2896	+
	2897	+ /**
	2898	+ * Call a callback function for every file in the repository.
	2899	+ * Uses the filesystem even in child classes.
	2900	+ */
	2901	+ function enumFilesInFS( $callback ) {
	2902	+ $numDirs = 1 << ( $this->hashLevels * 4 );
	2903	+ for ( $flatIndex = 0; $flatIndex < $numDirs; $flatIndex++ ) {
	2904	+ $hexString = sprintf( "%0{$this->hashLevels}x", $flatIndex );
	2905	+ $path = $this->directory;
	2906	+ for ( $hexPos = 0; $hexPos < $this->hashLevels; $hexPos++ ) {
	2907	+ $path .= '/' . substr( $hexString, 0, $hexPos + 1 );
	2908	+ }
	2909	+ if ( !file_exists( $path ) \|\| !is_dir( $path ) ) {
	2910	+ continue;
	2911	+ }
	2912	+ $dir = opendir( $path );
	2913	+ while ( false !== ( $name = readdir( $dir ) ) ) {
	2914	+ call_user_func( $callback, $path . '/' . $name );
	2915	+ }
	2916	+ }
	2917	+ }
	2918	+
	2919	+ /**
	2920	+ * Call a callback function for every file in the repository
	2921	+ * May use either the database or the filesystem
	2922	+ */
	2923	+ function enumFiles( $callback ) {
	2924	+ $this->enumFilesInFS( $callback );
	2925	+ }
	2926	+
	2927	+ /**
	2928	+ * Get properties of a file with a given virtual URL
	2929	+ * The virtual URL must refer to this repo
	2930	+ */
	2931	+ function getFileProps( $virtualUrl ) {
	2932	+ $path = $this->resolveVirtualUrl( $virtualUrl );
	2933	+ return File::getPropsFromPath( $path );
	2934	+ }
	2935	+
	2936	+ /**
	2937	+ * Path disclosure protection functions
	2938	+ *
	2939	+ * Get a callback function to use for cleaning error message parameters
	2940	+ */
	2941	+ function getErrorCleanupFunction() {
	2942	+ switch ( $this->pathDisclosureProtection ) {
	2943	+ case 'simple':
	2944	+ $callback = array( $this, 'simpleClean' );
	2945	+ break;
	2946	+ default:
	2947	+ $callback = parent::getErrorCleanupFunction();
	2948	+ }
	2949	+ return $callback;
	2950	+ }
	2951	+
	2952	+ function simpleClean( $param ) {
	2953	+ if ( !isset( $this->simpleCleanPairs ) ) {
	2954	+ global $IP;
	2955	+ $this->simpleCleanPairs = array(
	2956	+ $this->directory => 'public',
	2957	+ "{$this->directory}/temp" => 'temp',
	2958	+ $IP => '$IP',
	2959	+ dirname( __FILE__ ) => '$IP/extensions/WebStore',
	2960	+ );
	2961	+ if ( $this->deletedDir ) {
	2962	+ $this->simpleCleanPairs[$this->deletedDir] = 'deleted';
	2963	+ }
	2964	+ }
	2965	+ return strtr( $param, $this->simpleCleanPairs );
	2966	+ }
	2967	+
	2968	+ /**
	2969	+ * Chmod a file, supressing the warnings.
	2970	+ * @param $path String: the path to change
	2971	+ */
	2972	+ protected function chmod( $path ) {
	2973	+ wfSuppressWarnings();
	2974	+ chmod( $path, $this->fileMode );
	2975	+ wfRestoreWarnings();
	2976	+ }
	2977	+
	2978	+}

Follow-up revisions

Revision	Commit summary	Author	Date
r87974	Followup r87942, fix svn props	reedy	13:15, 13 May 2011

Comments

#Comment by P858snake (talk | contribs) 01:57, 13 May 2011

localsettings.php?!?!

#Comment by 😂 (talk | contribs) 12:29, 10 August 2011

This never really got an answer. I'm not sure why you're including a full LocalSettings in your extension...most of that is completely irrelevant to setting up a FileRepo.

#Comment by RussNelson (talk | contribs) 03:21, 16 August 2011

Fixed; thanks.

#Comment by Bawolff (talk | contribs) 05:00, 13 May 2011

I don't really know what this extension actually does/how it works, so this comment might be totally wrong, but: I notice you explicitly use the path $IP/extensions/WebStore for something without checking if the WebStore extension is installed. You should probably do a check to see if its installed, and if it isn't throw an exception or something. (Actually it kind of looks like that's used as a dummy parameter to prevent path disclosure (?) If so probably better to use something that doesn't conflict with an actual extension)

#Comment by 😂 (talk | contribs) 17:59, 5 June 2011

To follow up, WebStore is an ancient extension that hasn't gotten very much love in the last 4 years it's been around. I'm not sure what that's for...

#Comment by Bryan (talk | contribs) 19:10, 5 June 2011

It's used for Wikimedia's thumb servers as far as I know.

#Comment by RussNelson (talk | contribs) 19:56, 5 June 2011

As far as I know it's not used by Wikimedia's thumb servers. The thumb scaling cluster (which creates most thumbnails) is simply a set of servers behind a separate load balancer running the same MediaWiki as the main servers. The main server will redirect the request to the scalers when it sees that the thumbnail doesn't exist (404 handler). The scalers have access to the original and thumb storage and will fetch the original, scale it, and write it back out to the thumb store.

If you think otherwise, could you explain how WebStore is being used?

#Comment by Bryan (talk | contribs) 20:15, 5 June 2011

I thought the 404 handler was part of WebStore, but a quick look through the config files shows that this is probably not the case.

#Comment by Catrope (talk | contribs) 09:01, 6 June 2011

No, the 404 handler lives in /trunk/tools/upload-scripts

#Comment by RussNelson (talk | contribs) 12:44, 13 May 2011

Don't worry about that last class -- I just brought it in for reference purposes. I'll delete it in time.

#Comment by RussNelson (talk | contribs) 19:59, 5 June 2011

It's all gone now, except for three nearly identical functions that I'd like to combine once I figure out how to deal with the "nearly" part.

Status & tagging log

23:02, 22 August 2011 Catrope (talk | contribs) changed the status of r87942 [removed: new added: deferred]
12:29, 10 August 2011 😂 (talk | contribs) changed the status of r87942 [removed: deferred added: new]
00:22, 16 June 2011 MarkAHershberger (talk | contribs) changed the status of r87942 [removed: new added: deferred]