Retest Changes

Automated black box regression testing tool changes

  • Small documentation fixes.
  • Internal improvements.
  • Eliminated dependency on no longer maintained error-chain crate.
  • Updated dependencies.
  • Fixed broken link in (This is the only difference from unpublished 4.0.4.)
  • Added support for using WAIT in .rt files' retest plan test sections. When used this forces retest to wait before running an application to test to prevent it “outrunning” the operating system.
  • Now tries much harder to reset the terminal colors to the defaults at the end (unless nocolor is used), even in the face of Ctrl+C.
  • Added support for setting the wait using the Test::wait() builder method.
  • Updated dependencies.
Changed doc URLs to allow for other crates to be hosted on this site.
Changed the retest application's time format from HhM'S" to the more conventional HhMmSs. Tiny updates and code improvements.
No changes to the executable.
Removed the Plan::app_path_env_var() method that was created by mistake. Instead, when creating new tests use an app name which is in the PATH or which includes the PATH, e.g., Path::new(env!("CARGO_TARGET_DIR")).join("myapp.exe").
Applied cargo fmt to the code.
Tiny change to “info” level messages when generating and tiny internal improvements.
Slightly improved API docs; subtle bug fixes for two rare corner cases; improved error output when using an external diff tool; code improvements.
In .rt files, it is now no longer necessary to specify $OUT_PATH for STDOUT entries. (However, for backwards compatibility, retest will still work correctly whether you use $OUT_PATH in STDOUT entries or not.)
New methods in the retest crate: For retest crate users to upgrade from 2.x to 3.x:
  • Rename PlanBuilder to Plan;
  • Rename Plan's append() method to push();
  • Change calls of the form:
    DiffKind::Custom("diff".to_string()) to
In addition to code changes and improvements, the docs have been improved too.
Improved crate API and tiny output tweak.
For retest crate users to upgrade from 1.x to 2.x:
  • Rename TestBuilder to Test;
  • rename diff_by() to diff();
  • ensure that diff() is passed a DiffKind not a &str.
Errors are shown as such rather than as fails (they were always counted correctly as errors). Improved the summary line.
Tiny doc improvements.
Small API doc improvements.
Tiny internal improvements.
Improved error messages. Improved API: deprecated TestBuilder::diff() in favor of TestBuilder::diff_by(). Improved docs.
Retest is now both an executable and a rust library (see Retest API). Console output is now in color when possible, but this can be turned off using the nocolor option. If you specify numbered tests to generate or retest, the skipped tests are no longer mentioned.
Tiny changes to make it publishable on
Verbosity now defaults to level 1 (show summary, errors, and failures). Use quiet to set it to level 0 (show errors and failures); or use one v to additionally show each generate or retest, or v v to show even more. Added [env] section support for SET entries.
Minor improvements.
When generating, old expecteds are deleted before creating new ones, and when testing, old actuals are deleted before creating new ones: this should avoid comparing against stale files. Minor improvements to some error messages and various internal improvements. (Deprecated: The STDIN file's text can now include $OUT_PATH and which will be correctly replaced with EXPECTED_PATH or ACTUAL_PATH as appropriate. From 1.2.0 the STDIN file's raw bytes will be read as-is and send to the application to test's stdin without any processing.)
Added support for testing interactive console programs using STDIN.
If no .rt is specified, looks for rt-win.rt on Windows or rt-unix.rt on Unix, falling back to rt.rt on all platforms. Minor internal improvements.
Improved error messages.
Slightly changed output texts for very verbose (“skip”, “Pass”, or “FAIL”); and now output test names (if given), for passed or skipped tests when very verbose. Plus minor improvements.
EXPECTED_PATH is now optional and defaults to rt_expected in the current folder; ACTUAL_PATH is now optional and defaults to rt_actual in the current folder; plus various minor internal improvements.
Minor bugfixes (e.g., “env” is no longer case sensitive, just like the “app” and “diff” entry keys); also tests with just STDOUT and APP entries now work correctly; and better error reporting.
Initial release.