Software Testing Tools Question:

Download Questions PDF

Custom Objects

Note: The description of this feature, more than any other in this report, is limited in its scope and coverage. An entire white paper could be dedicated to exploring and describing how each of these tools deal with custom objects. Therefore dedicate several days to evaluating how each of these tools accommodate custom objects in your specific applications. To deal with a custom object [i.e. an object that does not map to standard class] both tools support the use of class mapping [i.e. mapping a custom class to a standard class with like functionality], along with a variety of X:Y pixel coordinate clicking techniques [some screen absolute, some object relative] to deal with bitmap objects, as well as the ability to use external DLL functions to assist in object identification and verification. Beyond these shared capabilities each tool has the following unique custom object capabilities:

• SilkTest has a feature to overlay a logical grid of X rows by Y columns on a graphic that has evenly spaced “hot spots”[this grid definition is then used to define logical GUI declarations for each hot spot]. These X:Y row/column coordinates are resolution independent [i.e. the logical reference says “give me 4th column thing in the 2nd row”, where that grid expands or contracts depending on screen resolution].

• WinRunner has a built-in text recognition engine which works with most standard fonts. This capability can often be used to extract visible text from custom objects, position the cursor over a custom object, etc. This engine can also be taught non-standard font types which is does understand out of the box. Both tools offer support for testing non-graphical controls through the advanced use of custom DLLs [developed by the user], or the Extension Kit [SilkTest, which may have to be purchased at an addition cost] and the Mercury API Function Library [WinRunner]. SilkTest and WinRunner Feature Descriptions Version 1.00 (7/6/00)

2.9 Internationalization (Language Localization)

• SilkTest supports the single-byte IBM extended ASCII character set, and its Technical Support has also indicated “that Segue has no commitment for unicode”. The user guide chapter titled “Supporting Internationalized Applications” shows a straightforward technique for supporting X number of [single-byte IBM extended ASCII character set] languages in a single test frame of GUI declarations.

• WinRunner provides no documentation on how to use the product to test language localized applications. Technical Support has indicated that (1) “WinRunner supports multi-byte character sets for language localized testing…”, (2) “there is currently no commitment for the unicode character set…”, and (3) “it is possible to convert a US English GUI Map to another language using a [user developed] phrase dictionary and various gui_* built-in functions”.

Database Interfaces

Both tools provide a variety of built-in functions to perform Structure Query Language (SQL) queries to control, alter, and extract information from any database which supports the Open Database Connectivity (ODBC) interface.

Database Verification

Both tools provide a variety of built-in functions to make SQL queries to extract information from an ODBC compliant database and save it to a variable [or if you wish, an external file]. Verification at this level is done with hand coding.

WinRunner also provides a visual recorder to create a Database Checkpoint used to validate the selected contents of an ODBC compliant database within a testcase. This recorder creates side files similar to GUI Checkpoints and has a built-in interface to (1) the Microsoft Query application [which can be installed as part of Microsoft Office], and (2) to the Data Junction application [which may have to be purchased at an addition cost], to assist in constructing and saving complex database queries.

Data Driven Testing

Both tools support the notion of data-driven tests, but implement this feature differently:

• SilkTest’s implementation is built around an array of user defined records. A record is a data structure defining an arbitrary number of data items which are populated with values when the array is initialized [statically or at runtime]. Non-programmers can think of an array of records as a memory resident spreadsheet of X number of rows which contain Y number columns where each row/column intersection contains a data item. The test code, as well as the array itself, must be hand coded. It is also possible to populate the array each time the test is run by extracting the array’s data values from an ODBC compliant database, using a series of built-in SQL function calls. The test then iterates through the array such that each iteration of the test uses the data items from the next record in the array to drive the test or provide expected data values.

• WinRunner’s implementation is built around an Excel compatible spreadsheet file of X number of rows which contain Y number of columns where each row/column intersection contains a data item. This spreadsheet is referred to as a Data Table. The test code, as well as the Data Table itself, can be created with hand coding or the use of the DataDriver visual recorder. It is also possible to populate a Data Table file each time the test is run by extracting the table’s data values from an ODBC compliant database using a WinRunner wizard interfaced to the Microsoft Query application.. The test then iterates through the Data Table such that each iteration of the test uses the data items from the next row in the table to drive the test or provide expected data values. Both tools also support the capability to pass data values into a testcase for a more modest approach to data driving a test.

Restoring an Application’s Initial State

• SilkTest provides a built-in recovery system which restores the application to a stable state, referred to as the basestate, when the test or application under test fails ungracefully. The default basestate is defined to be: (1) the application under test is running; (2) the application is not minimized; and (3) only the application’s main window is open and active. There are many built-in functions and features which allow the test engineer to modify, extend, and customize the recovery system to meet the needs of each application under test.

• WinRunner does not provide a built-in recovery system. You need to code routines to return the application under test to its basestate—and dismiss all orphaned dialogs—when a test fails ungracefully.

Scripting Language

Both tools provide proprietary, interpreted, scripting languages. Each language provide the usual flow control constructs, arithmetic and logical operators, and a variety of built-in library functions to perform such activities as string manipulation, [limited] regular expression support, standard input and output, etc. But that is where the similarity ends: • SilkTest provides a strongly typed, object-oriented programming language called 4Test. Variables and constants may be one of 19 built-in data types, along with a user defined record data type. 4Test supports single- and multi-dimensional dynamic arrays and lists, which can be initialized statically or dynamically. Exception handling is built into the language [via the do… except statement].

• WinRunner provides a non-typed, C-like procedural programming language called TSL. Variables and constants are either numbers or strings [conversion between these two types occurs dynamically, depending on usage]. There is no user defined data type such as a record or a data structure. TSL supports sparsely populated associative single- and [pseudo] multidimension arrays, which can be initialized statically or dynamically—element access is always done using string references—foobar[“1”] and foobar[1] access the same element [as the second access is implicitly converted to an associative string index reference]. Exception handling is not built into the language. The only way to truly understand and appreciate the differences between these two programming environments is to use and experiment with both of them.

Exception Handling

Both tools provide the ability to handle unexpected events and errors, referred to as exceptions, but implement this feature differently:

• SilkTest’s exception handling is built into the 4Test language—using the do… except construct you can handle the exception locally, instead of [implicitly] using SilkTest’s default built-in exception handler [which terminates the currently running test and logs an error]. If an exception is raised within the do block of the statement control is then immediately passed to the except block of code. A variety of built-in functions [LogError(), LogWarning, ExceptNum(), ExceptLog(), etc.] and 4Test statements [raise, reraise, etc.] aid in the processing of trapped exceptions within the except block of code.

• WinRunner’s exception handling is built around (1) defining an exception based on the type of exception (Popup, TSL, or object) and relevant characteristics about the exeception (most often its error number); (2) writing an exception hander, and (3) enabling and disabling that exception handler at appropriate point(s) in the code. These tasks can be achieved by hand coding or through the use of the Exception Handling visual recorder.