TestComplete and eMailing Test Results

The performance of the test log outside of the TestComplete IDE is notoriously hideous. Even it it weren't, most people wouldn't want to wade through it looking for the errors, so here is a collection of techniques to send just the errors in a test run via email without having to parse the log.

The Techniques

Technique 1: Use a closure to filter a log


var TestReport = (function () {
//private variables in the TestReport object
  var _TestSummary;
  var _CurrentTest;
  var _ErrorCount;
  var _TestCount;
  var _thisBuild;
  if (!_CurrentTest) {
    _TestSummary = "";
    _CurrentTest = "";
    _ErrorCount = 0;
    _TestCount = 0;
  }
//private functions in the TestReport object
  function _setBuild(buildDetails) {
    _thisBuild = buildDetails;
  }
  function _addTest(newTest) {
    _CurrentTest = newTest;
    _TestCount++;
  }
  function _addError(errorMessage) {
    _ErrorCount++;
    _TestSummary = _TestSummary + "<b>" + _CurrentTest + "</b> - " + errorMessage + " (Error" + _ErrorCount + ".png)<br />";
    AUTwindow()["Picture"]()["SaveToFile"]("Error" + _ErrorCount + ".png");
  }
  function _addDetail(errorDetail) {
    _TestSummary = _TestSummary + "<i>" + errorDetail + "</i><br />";
  } 
  function _sendMail() {
    var TestSummary = _TestCount + " Interface tests ran on " + _thisBuild + "<br /><br />";
    if (_ErrorCount > 0) {
      TestSummary = TestSummary + _ErrorCount + " error";
      if (_ErrorCount > 1) {
        TestSummary = TestSummary + "s";
      }
      TestSummary = TestSummary + " detected:<br /><br />" + _TestSummary;
    } else {
      TestSummary = TestSummary + "No errors detected.";
    } 
    SendEmail("GUItesting@YourCompany.com", 
      "your.name@YourCompany.com", 
      "GUI test summary", 
      TestSummary, 
      _ErrorCount);
  }
//public interface to the TestReport object
  return {
    setBuild: function (buildDetails) {
      _setBuild(buildDetails);
    },
    addTest: function (newTest) {
      _addTest(newTest);
    },
    addError: function (newError) {
      _addError(newError);
    },
    addDetail: function (newDetail) {
      _addDetail(newDetail);
    },
    sendMail: function () {
      _sendMail();
    }
  } 
}());

Our old friend closure again. It gives us a function that provides "global" variables that are initialised and maintained automatically. See below for examples of higher level scripts that use each of the public methods:

TestReport.setBuild(BuildDetails) - use your own code to get the AUT's build details and feed it into the test report here.
TestReport.addTest("Grid tests") - sets the current test name against which any errors are reported. If there are no errors, this test doesn't appear in the report.
TestReport.addError("Portfolio field should be disabled") - creates a line in the test report consisting of the current test name and this error message.
TestReport.addDetail("Expected: None") - adds any explanatory detail to the error.
TestReport.sendMail() - called at the end of the test run to generate the email if there are any errors.

Note that the closure technique means we don't have to worry about initialising the current test, the email message or the error count variables from our main script as it's all done for us in the first call to the function. This gives us a very useful bit of functionality missing from TestComplete's feature set.

Technique 2: Trap errors


function GeneralEvents_OnLogError(Sender, LogParams) {
  if (LogParams["Str"] != "Script execution was interrupted.") {
    TestReport.addError(LogParams["Str"]);
  }  
}

function GeneralEvents_OnLogWarning(Sender, LogParams) {
  TestReport.addDetail(LogParams["Str"]);  
}

This is set up using TestComplete's Events dialog and merely copies the current Log["Error"]() message to TestReport().
I use Log["Warning"]() to add extra details to error reports.

Technique 3: eMail via CDO


function SendEmail(mFrom, mTo, mSubject, mBody, mErrors) {
  var schema = "http://schemas.microsoft.com/cdo/configuration/";
  var mConfig = Sys.OleObject("CDO.Configuration"); 
  mConfig.Fields.Item(schema + "sendusing") = 2;  // cdoSendUsingPort
  mConfig.Fields.Item(schema + "smtpserver") = "smtpmail.YourCompany.com"; // SMTP server
  mConfig.Fields.Item(schema + "smtpserverport") = 25; // Port number
  mConfig.Fields.Update();

  var mMessage = Sys.OleObject("CDO.Message");
  mMessage.Configuration = mConfig;
  mMessage.From = mFrom;
  mMessage.To = mTo;
  mMessage.Subject = mSubject;
  mMessage.HTMLBody = mBody;

  if (mErrors) { //flag as important
    mMessage.Fields("urn:schemas:httpmail:Importance").Value = 2
    mMessage.Fields("urn:schemas:mailheader:importance").Value = "High"
    mMessage.Fields("urn:schemas:httpmail:priority").Value = 1
    mMessage.Fields("urn:schemas:mailheader:priority").Value = 1
    mMessage.Fields("urn:schemas:mailheader:X-MSMail-Priority").Value = "High"
    mMessage.Fields("urn:schemas:mailheader:X-Priority").Value = 1
    mMessage.Fields("urn:schemas:httpmail:X-MSMail-Priority").Value = "High"
    mMessage.Fields("urn:schemas:httpmail:X-Priority").Value = 1
    for (var i = 1; i <= mErrors; i++) { //and attach the screenshots
      mMessage.AddAttachment(Project["Path"] + "Error" + i + ".png");
    }
  } else { //flag as unimportant
    mMessage.Fields("urn:schemas:httpmail:Importance").Value = 0
    mMessage.Fields("urn:schemas:mailheader:importance").Value = "Low"
    mMessage.Fields("urn:schemas:httpmail:priority").Value = -1
    mMessage.Fields("urn:schemas:mailheader:priority").Value = -1
    mMessage.Fields("urn:schemas:mailheader:X-MSMail-Priority").Value = "Low"
    mMessage.Fields("urn:schemas:mailheader:X-Priority").Value = -1
    mMessage.Fields("urn:schemas:httpmail:X-MSMail-Priority").Value = "Low"
    mMessage.Fields("urn:schemas:httpmail:X-Priority").Value = -1
  }
  mMessage.Fields.update();
  try {
    mMessage.Send();
  } catch(e) {
    Log.Error("E-Mail cannot be sent - " + e.description);
    return false;
  }
  Log.Message("Message to <" + mTo + "> was successfully sent.", mBody);
  return true;
}

Assuming you have a Microsoft SMTP server available, change the parameters above to suit.
Note that mFrom doesn't have to be a genuine account on your domain, so make it obvious that it's to do with your testing.

Technique 4: Use a test runner


function TestRunner(testName) {
    Log["AppendFolder"](testName);
    TestReport.addTest(testName);//sets the current test name for reporting
    Runner["CallMethod"]("InterfaceTests." + testName);
    Log["PopLogFolder"]();
}
//for example, the high level script calls
    TestRunner("Grid_testing");
//rather than
    Grid_testing();

The snazziness of the TestReport() function does come at a price - you need to plan ahead. Using a test runner rather than calling test functions directly gives us the opportunity to pass the current test name to TestReport() so that we know which tests to log any subsequent errors against.

Conclusions

With a bit of planning and minimal extra weight in the script, we can sit back and enjoy summary emails being automatically generated for us at the end of the test run. Any valid HTML can be included in the body of the error message so you can jazz it up as you see fit. The basic TestReport() function above gives us something like:

14 Interface tests ran on Version 1.5 (Build number 1.5.1.42 - Feb 23 2010 19:06:31) 1 error detected: DV_6 - Formula default value wrong (Error1.png) Found:{Offset=5;After=DV_Var_4;Length=3;Size=DV_Var_4;Dimension name=Dim_A} Expected:{Offset=5;After=DV_Var_4;Length=3;Dimension name=Dim_A;Size=DV_Var_4}

TestingMisconfused