Regression testing in the toolkit
Note: testsuite has been moved to https://github.com/dita-ot/testsuite.
Regression testing with the toolkit
Note: This information is primarily of interest to toolkit developers and those creating plugins for the toolkit. For information on the toolkit itself, see The DITA Open Toolkit.
The core toolkit development team is working on the release of its regression testing package. The package consists of a large number of sample DITA maps and topics representing a wide variety of content, as well as batch files to run the content through a build. The regression test package will be posted to SourceForge, and this page will be updated with documentation on how to run the test.
The test is generally used by running all of the builds to generate output after making a modification to the code. The output of the test is compared to the output with the previous, working set of code to determine if any tests fail or if any output changes in a negative way. This is most useful when making changes to the base code or when testing a plugin that overrides core code (including overrides that should affect only a related specialization).
The toolkit development welcomes additions to the regression test bucket, as well as suggestions for ways to improve the testing process.
The regression suite package is committed to CVS. Developers can access the package by exporting it or checking it out from CVS. Exporting action does not need access authority to CVS. Here is the information about the CVS repository.
Connect protocol: ssh
Host: dita-ot.cvs.sourceforge.net
Port: Default
Repository Path: /cvsroot/dita-ot
Module: RegressionSuite
The structure of regression suite package is listed as followings:
testcase: ant scripts to run corresponding test
testdata: input data that is needed by test
*.bat: windows batch file to run the test
testbuild.xml: the stub ant script to start all testing
After regression suite package is downloaded, the content in the package (testcase, testdata, *.bat and testbuild.xml) should be extracted andcopied to installation directory of DITA OT. Then you need to enter the DITA OT installation directory in console and use ant -f testbuild.xml to start the regression test.
More information will be coming when the test is available at SourceForge.
- Login to post comments
- 8518 reads
Search
Recent Content
- Forum for DITA Users
39 weeks ago - Publishing for DITA
5 years ago - Optimizing the DITA Authoring Experience — Online Course
5 years ago - Linking to another chunked topic
5 years ago - Advanced Reuse Strategies — Online Course
5 years ago - 2019 Content Management Strategies/DITA North America Conference
5 years ago
Subscribe
Test Suite?
I don't see any "tests" in this suite (by "test", I mean something that explicitly passes or fails, like a junit or antunit test)? Currently, this looks like a batch job that you can run before and after changes, and then compare output as you wish.
Why not go a little further and make this a "test suite". E.g. for each of the roughly 75 test cases, you could have a coresponding AntUnit test. for each of the topic files, you could have an assertFilesMatch assertion (that compared the expected output file, with the file that was produced during that specific run of the suite)
As you increment versions of the toolkit, you can modify the suite accordingly and tag/branch it along with the dita-ot