r19801 MediaWiki - Code Review archive

Repository:MediaWiki
Revision:< r19800‎ | r19801 | r19802 >
Date:06:59, 6 February 2007
Author:nickj
Status:old
Tags:
Comment:
Add a mechanism to parserTests when run in --compare or --record mode, to give information about
a test's history. This may help to differentiate tests that have never passed, versus new regressions.
Also for a regression the range of dates and SVN version numbers in which it first appeared will be
given, which may help isolate the cause of the regression; equally fixed tests will show a range
to indicate when the test was fixed.

Here is an example of the output (using a parserTests.txt modified specifically to cause all
the conditions to occur) :
---------------------------------------------------------------------------------------
root@bling:/var/www/hosts/mediawiki/wiki# php maintenance/parserTests.php --compare --quick --quiet --color=no
This is MediaWiki version 1.10alpha (r19799).

Reading tests from "maintenance/parserTests.txt"...
This is dvips(k) 5.95a Copyright 2005 Radical Eye Software (www.radicaleye.com)
' TeX output 2007.02.06:0630' ->
<tex.pro><texps.pro>. <cmr12.pfb>[1]
Reading tests from "../extensions/Cite/citeParserTests.txt"...
Reading tests from "../extensions/LabeledSectionTransclusion/lstParserTests.txt"...

1 previously failing test(s) now PASSING! :)
* URL-encoding in URL functions (single parameter) [Fixed between 05-Feb-2007 23:58:17, 1.10alpha (r19799) and 06-Feb-2007 06:30:34, 1.10alpha (r19799)]

1 previously PASSING test(s) removed o_O
* Prevent conversion of text with -{}- tags (language variants) [First recorded appearance: 02-Jan-2007 04:30:32, 1.9alpha (r18762)]

1 new PASSING test(s) :)
* Fake passing test [Has never failed]

1 previously passing test(s) now FAILING! :(
* Plain link to URL [Introduced between 05-Feb-2007 23:58:17, 1.10alpha (r19799) and 06-Feb-2007 06:30:34, 1.10alpha (r19799)]

1 previously FAILING test(s) removed O_o
* TODO: Link containing double-single-quotes '' (bug 4598) [First recorded appearance: 15-Nov-2006 02:53:34, 1.9alpha (r17686)]

1 new FAILING test(s) :(
* Blah blah fake test [Has never passed]

18 still FAILING test(s) :(
* URL-encoding in URL functions (multiple parameters) [Has never passed]
* TODO: Table security: embedded pipes (http://mail.wikipedia.org/pipermail/wikitech-l/2006-April/034637.html) [Has never passed]
* TODO: message transform: <noinclude> in transcluded template (bug 4926) [Has never passed]
* TODO: message transform: <onlyinclude> in transcluded template (bug 4926) [Has never passed]
* TODO: HTML bullet list, unclosed tags (bug 5497) [Has never passed]
* TODO: HTML ordered list, unclosed tags (bug 5497) [Has never passed]
* TODO: HTML nested bullet list, open tags (bug 5497) [Has never passed]
* TODO: HTML nested ordered list, open tags (bug 5497) [Has never passed]
* TODO: Inline HTML vs wiki block nesting [Has never passed]
* TODO: Mixing markup for italics and bold [Has never passed]
* TODO: 5 quotes, code coverage +1 line [Has never passed]
* TODO: dt/dd/dl test [Has never passed]
* TODO: Images with the "|" character in the comment [Has never passed]
* TODO: Parents of subpages, two levels up, without trailing slash or name. [Has never passed]
* TODO: Parents of subpages, two levels up, with lots of extra trailing slashes. [Has never passed]
* Blank ref followed by ref with content [Introduced between 31-Jan-2007 23:52:43, 1.10alpha (r19702) and 05-Feb-2007 23:58:17, 1.10alpha (r19799)]
* Regression: non-blank ref "0" followed by ref with content [Introduced between 31-Jan-2007 23:52:43, 1.10alpha (r19702) and 05-Feb-2007 23:58:17, 1.10alpha (r19799)]
* Regression sanity check: non-blank ref "1" followed by ref with content [Introduced between 31-Jan-2007 23:52:43, 1.10alpha (r19702) and 05-Feb-2007 23:58:17, 1.10alpha (r19799)]

Passed 509 of 529 tests (96.22%)... 20 tests failed!
root@bling:/var/www/hosts/mediawiki/wiki#
---------------------------------------------------------------------------------------

Lastly please note that this is my first play with the MW database API, so if there are better
ways of doing any of the database query stuff than in this patch, then please don't be shy
about telling me how.
Modified paths:
  • /trunk/phase3/maintenance/parserTests.inc (modified) (history)

Diff [purge]

Index: trunk/phase3/maintenance/parserTests.inc
——@@ -1021,8 +1021,8 @@
10221022 if( $differences ) {
10231023 $count = count($differences);
10241024 printf( "\n%4d %s\n", $count, $label );
1025 - foreach ($differences as $differing_test_name) {
1026 - print " * $differing_test_name\n";
 1025+ foreach ($differences as $differing_test_name => $statusInfo) {
 1026+ print " * $differing_test_name [$statusInfo]\n";
10271027 }
10281028 }
10291029 }
——@@ -1034,8 +1034,8 @@
10351035 }
10361036
10371037 /**
1038 - ** @desc: Returns an array of the test names with changed results, based on the specified
1039 - ** before/after criteria.
 1038+ ** Returns an array of the test names with changed results, based on the specified
 1039+ ** before/after criteria.
10401040 */
10411041 private function compareResult( $before, $after ) {
10421042 $testitem = $this->db->tableName( 'testitem' );
——@@ -1065,15 +1065,78 @@
10661066 $result = $this->db->query( $sql, __METHOD__ );
10671067 $retval = array();
10681068 while ($row = $this->db->fetchObject( $result )) {
1069 - $retval[] = $row->t;
 1069+ $testname = $row->t;
 1070+ $retval[$testname] = $this->getTestStatusInfo( $testname, $after, $curRun );
10701071 }
10711072 $this->db->freeResult( $result );
10721073 return $retval;
10731074 }
10741075
10751076 /**
1076 - ** @desc: Helper function for compareResult() database querying.
 1077+ ** Returns a string giving information about when a test last had a status change.
 1078+ ** Could help to track down when regressions were introduced, as distinct from tests
 1079+ ** which have never passed (which are more change requests than regressions).
10771080 */
 1081+ private function getTestStatusInfo($testname, $after, $curRun) {
 1082+
 1083+ // If we're looking at a test that has just been removed, then say when it first appeared.
 1084+ if ( is_null( $after ) ) {
 1085+ $changedRun = $this->db->selectField ( 'testitem',
 1086+ 'MIN(ti_run)',
 1087+ array( 'ti_name' => $testname ),
 1088+ __METHOD__ );
 1089+ $appear = $this->db->selectRow ( 'testrun',
 1090+ array( 'tr_date', 'tr_mw_version' ),
 1091+ array( 'tr_id' => $changedRun ),
 1092+ __METHOD__ );
 1093+ return "First recorded appearance: "
 1094+ . date( "d-M-Y H:i:s", strtotime ( $appear->tr_date ) )
 1095+ . ", " . $appear->tr_mw_version;
 1096+ }
 1097+
 1098+ // Otherwise, this test has previous recorded results.
 1099+ // See when this test last had a different result to what we're seeing now.
 1100+ $changedRun = $this->db->selectField ( 'testitem',
 1101+ 'MAX(ti_run)',
 1102+ array(
 1103+ 'ti_name' => $testname,
 1104+ 'ti_success' => ($after ? "0" : "1"),
 1105+ "ti_run != " . $this->db->addQuotes ( $curRun )
 1106+ ),
 1107+ __METHOD__ );
 1108+
 1109+ // If no record of ever having had a different result.
 1110+ if ( is_null ( $changedRun ) ) {
 1111+ if ($after == "0") {
 1112+ return "Has never passed";
 1113+ } else {
 1114+ return "Has never failed";
 1115+ }
 1116+ }
 1117+
 1118+ // Otherwise, we're looking at a test whose status has changed.
 1119+ // (i.e. it used to work, but now doesn't; or used to fail, but is now fixed.)
 1120+ // In this situation, give as much info as we can as to when it changed status.
 1121+ $pre = $this->db->selectRow ( 'testrun',
 1122+ array( 'tr_date', 'tr_mw_version' ),
 1123+ array( 'tr_id' => $changedRun ),
 1124+ __METHOD__ );
 1125+ $post = $this->db->selectRow ( 'testrun',
 1126+ array( 'tr_date', 'tr_mw_version' ),
 1127+ array( "tr_id > " . $this->db->addQuotes ( $changedRun) ),
 1128+ __METHOD__,
 1129+ array( "LIMIT" => 1, "ORDER BY" => 'tr_id' )
 1130+ );
 1131+
 1132+ return ( $after == "0" ? "Introduced" : "Fixed" ) . " between "
 1133+ . date( "d-M-Y H:i:s", strtotime ( $pre->tr_date ) ) . ", " . $pre->tr_mw_version
 1134+ . " and "
 1135+ . date( "d-M-Y H:i:s", strtotime ( $post->tr_date ) ) . ", " . $post->tr_mw_version ;
 1136+ }
 1137+
 1138+ /**
 1139+ ** Helper function for compareResult() database querying.
 1140+ */
10781141 private function condition( $value ) {
10791142 if( is_null( $value ) ) {
10801143 return 'IS NULL';
——@@ -1094,4 +1157,4 @@
10951158 }
10961159 }
10971160
1098 -?>
 1161+?>
\ No newline at end of file

Follow-up revisions

RevisionCommit summaryAuthorDate
r27906Fix regressions in r27759 -- pages already marked in link cache at link parsi...brion20:45, 27 November 2007

Past revisions this follows-up on

RevisionCommit summaryAuthorDate
r13851Fix bug 5497: Regression in HTML normalization in 1.6 (unclosed <li>)...hashar19:25, 24 April 2006
r13852BACKPORT from trunk:...hashar19:29, 24 April 2006
r14045Parser test cases for bug 5497brion00:24, 3 May 2006
r14046Parser test cases for bug 5497brion00:25, 3 May 2006
r14047adjust notes: * Rolled back the buggy patch for bug 5497brion00:29, 3 May 2006
r14425Add parser tests for bug 4598 ('' in link title)brion00:02, 28 May 2006
r14426Tweak bug 4598 with more sanity checks for casesbrion00:21, 28 May 2006
r14563* Add <ol> to the list of block elements for doBlockLevels; avoids <p>s being...brion02:06, 4 June 2006
r17686icc warning fixesriver23:24, 14 November 2006
r18762Basic search functionality workingmagnusmanske23:09, 1 January 2007
r19702simplify a regex in lsthsanbeg22:20, 31 January 2007
r19799missing includeerik23:38, 5 February 2007