Test Script Reference
- Overview
- Navigation
- Verification
- Settings
- Output
- Parameterised Scripts
- Setup, Reset and Cleanup
- Random Values
- Request Headers
- Threads and Iterations
- Extensibility
- Links
Overview
All hatatap test scripts start with the root element test:script which must include the hatatap namespace declaration: xmlns:test="urn:hatatap".
<test:script xmlns:test="urn:hatatap">
... script directives ...
</test:script>
There are four main types of script directives:
- Navigation (test:click, test:form, test:send, test:frame, test:window, test:send)
- Verification (test:verify)
- Settings (test:javascript, test:proxy, test:auth, test:host)
- Output (test:log, test:save)
Navigation
Clicking
As when using a web browser, navigation is done by clicking links and buttons. hatatap provides the script declaration test:click which allows for different ways of picking a link out of a page.
A test session generally starts with opening a URL:
Once a page has been opened, its content is available for verification and navigation. Links can be located using their name, id or the text they contain, or if they are image links by the value of their alt tag. Alternatively any XPath (or o:Path) expression that evaluates to a link can be used.
link with name 'search': <test:click name="search"/>
link with id 'search': <test:click ref="search"/>
image link with alt tag 'search': <test:click alt="search"/>
fifth link in page: <test:click select="$dom//a[5]"/>
Forms
To send form data, we need to declare which form (test:form) and what data (test:input).
Forms are selected either by name (using a name attribute) or id (using a ref attribute) , if omitted the first (or only) form of the page is chosen. Input is described by declaring elements with names and values corresponding to input names and values. Alternatively, the test:input element can be used.
<test:form name="user-details">
<firstname>Jane</firstname>
<surname>Doe</surname>
</test:form>
alternative form:
<test:form name="user-details">
<test:input name="firstname" value="Jane"/>
<test:input name="surname" value="Doe"/>
</test:form>
The form data is only sent if a submit declaration is present. If there are several submit buttons in the form, they can be selected by id, name or name and value.
select a submit button by name: <test:submit name="details"/>
select a submit button by name and value: <test:submit name="details" value="send"/>
select a submit button by id: <test:submit ref="send-details"/>
HTML forms can contain not only input fields but also check boxes. These can be selected and unselected using test:check, test:uncheck and test:toggle.
<test:uncheck name="private"/>
<test:toggle name="protected"/>
Drop-down menu and radio button selections are made in the same way as input fields, using the name and value.
Values from drop-down boxes can also be selected by the display name.
<test:input name="occupation" option="Slacker"/>Files can be attached and uploaded by specifying the local file path with a file attribute. The form that is used must have the encoding type multipart/form-data and the input should be of type file.
Frames
When accessing a page that is a frameset, you can access the content of each individual frame.
... get and operate on frameset ...
<test:frame name="left">
... operate on 'left' frame content ...
</test:frame>
<test:frame name="right">
... operate on 'right' frame content ...
</test:frame>
... return to frameset ...
</test:script>
Nested frames are supported by nesting test:frame elements. Outside of a frame element, ie before and after its contents have been processed, the 'current' page remains the same.
Windows
Pop-up windows are accessed in much the same way as frames. To change the window for a part of the script you wrap the processing in a test:window element. You can specify whether or not the window should be closed when the end of the window element is reached. Default is to leave the window open.
... get and operate on a window ...
<test:window name="pop">
... operate on 'pop' window content ...
</test:window>
... return to main window ...
<test:window name="pop" close="yes">
... operate on 'pop' window content again ...
... close 'pop' window when exiting ...
</test:window>
... return to main window ...
</test:script>
Web Services
With hatatap you can create XML or text requests and POST or PUT them to a server.
<test:send url="http://somewhere">
<methodCall>
<methodName>listItems</methodName>
<params/>
</methodCall>
</test:send>
The default content type for simple requests is 'text/xml', and the default method is 'POST'.
Individual request headers can be set inside the test:send element.
<test:send url="http://nowhere" content-type="text/plain" method="PUT">
<test:header name="Pragma" value="no-cache"/>
item 1: empty
</test:send>
SOAP requests are created in the same way as other XML requests, with a SOAP Envelope message body. They can be either literal or RPC, depending on the Envelope data.
<test:send url="http://nowhere">
<se:Envelope>
<se:Body>
<g:doGoogleSearch se:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<key xsi:type="xsd:string">put your key here!</key>
<q xsi:type="xsd:string">what is hatatap</q>
<start xsi:type="xsd:int">0</start>
<maxResults xsi:type="xsd:int">10</maxResults>
<filter xsi:type="xsd:boolean">false</filter>
<restrict xsi:type="xsd:string"/>
<safeSearch xsi:type="xsd:boolean">false</safeSearch>
<lr xsi:type="xsd:string"/>
<ie xsi:type="xsd:string">latin1</ie>
<oe xsi:type="xsd:string">latin1</oe>
</g:doGoogleSearch>
</se:Body>
</se:Envelope>
</test:send>
A request may also include attachments, in which case the HTTP request body will be multipart MIME encoded. The attachment declaration must include a content-type and location attribute. The location attribute specifies the multipart content location or content-id, which may be referenced from within the main message part.
<test:send url="http://nowhere">
<test:header name="SOAPAction" value="placement"/>
<se:Envelope>
<se:Body>
<pl:Placement>
<MsgId>urn:123404321@test</MsgId>
<Location>cid:payload1</Location>
</pl:Placement>
</se:Body>
</se:Envelope>
<test:attachment content-type="text/xml" location="payload1">
<payload>...data...</payload>
</test:attachment>
</test:send>
Binary attachments can be sent by referencing a file that contains the attachment.
<test:attachment location="img1" content-type="image/jpeg" href="images/funnybunny.jpg"/>
XML responses can be verified in the same way as HTML responses, to check content type, length etc. However, things like page title, forms and links obviously don't apply. The most powerful way to process an XML response is with the <test:verify test="xpath expression"/> construct, which gives you full access to the response content.
Verification
Success can be verified in a variety of ways. The simplest form is to check that the web server returns a success code. Regular expressions can be used to match on groups of response codes, eg '20?' for all success codes.
Other aspects of the page can be checked, such as the title, content type, content length, cookie presence and values, and textual content. Content checking can be done using simple string matching (contains), or with full regular expressions (match).
For example, to verify that the total page size is
no less than 1kb: <test:verify content-length="1024+"/>
exactly 2.5Gb: <test:verify content-length="2.5Gb"/>
that the response code is either 200 or 204
that the content type is html
that the title of the page is 'success'
that the page contains the string 'success'
using regular expressions: <test:verify matches="[Ss]uccess"/>
cookie existence and values can be verified:
and that the value is 'krispin': <test:verify cookie="user-id" value="krispin"/>
response headers and header values can be verified:
and that the value is 'Apache': <test:verify header="Server" value="Apache"/>
cookie and header values are tested using regular expression matching.
For more powerful and flexible content verification, the verify command can evaluate XPath expressions against the current page's DOM tree. The page DOM is put in a variable called dom, which is referenced with $dom.
<test:verify test="count($dom//a) = 12"/>
that there are no absolute http links:
<test:verify test="not($dom//a[starts-with(@href, 'http://')])"/>
XPath expressions are evaluated by the o:XML interpretor. Hence not only XPath, but all o:XML core functions can be used. Most useful for test scripts are probably the regular expression functions: split, substitute, matches.
When a verify directive fails, an error message is produced to the output. The exact message can be controlled by adding a msg attribute.
Settings
Javascript
hatatap supports javascript through HttpUnit and the Rhino engine. To enable javascript evaluation, set the top-level javascript attribute to 'yes' or 'true'. When javascript is on, any severe javascript error will cause the script to fail.
Javascript may also be switched on or off at any point in the script using the test:javascript element. This way you may also specify whether or not javascript errors should cause a script error. Default is for an exception to be thrown when javascript is enabled.
... javascript evaluation enabled, errors enabled ...
<test:javascript evaluate="no"/>
... javascript evaluation disabled ...
<test:javascript evaluate="yes" error="no"/>
... javascript evaluation enabled, errors disabled ...
</test:script>
Redirect and Refresh
By default hatatap automatically follows redirect and refresh directives. If required, this may be disabled. Furthermore you may specify a delay period before redirects are followed, to imitate slower user agents.
... delay of 100 milliseconds before redirects are followed through ...
</test:script>
HTTP Errors
When a request results in an http error (eg 404 page not found or 500 internal server error) then by default hatatap will stop and finish the script early.
The default behaviour can be changed with the status-error attribute.
... http errors will be reported but otherwise ignored ...
</test:script>
Proxy
If your client is running behind a http proxy server you might need to specify the proxy settings.
<test:proxy host="192.168.0.88" port="2303"/>
reset to use no proxy:
<test:proxy/>
set http proxy server with authentication:
<test:proxy host="192.168.0.88" username="guest" password="secret"/>
the default proxy port is 80.
Authentication
hatatap supports basic http authentication with the test:auth construct.
Test Servers
To redirect requests to a different server, such as a test server, you can specify a different ip address for a particular server name. This overrides the usual DNS lookups performed on server domain names.
The domain name can be reset at any point by excluding the ip attribute:
Frequently test servers have self-signed or expired SSL certificates which may not even match with the server name. You can tell hatatap to bypass server certificate validation by setting the script attribute trust-ssl to 'yes' or 'true':
...
</test:script>
Note: this can also be enabled with the command line option -trust.
Output
There are two types of output produced by hatatap: log output and an XML record of what pages it navigates, including timings. The XML output can be used to produce test reports in various formats. Both the log and XML output is controlled by the transformation stylesheets.
Additionally, test:log elements for creating output can be included in the scripts. Visited pages can be saved to file using the test:save element. This way scripts can be used to automate web navigation, or create snapshots of entire websites.
Parameterised Scripts
Scripts can be given input parameters that can be used to control their behaviour. To use a parameter it must be declared at the beginning of the script. The declaration can also provide an optional default value.
<test:param name="url" default="http://localhost:8080/"/>
... script ...
</test:script>
The parameter is available as a variable within the script. To use the variable, or any other valid expression, it must simply be enclosed in curly braces. Expressions can be used instead of or as part of almost any string value in the script.
<test:click url="{$url}"/>
send the url as form data:
<test:form>
<web-site>{$url}</web-site>
</test:form>
verify that the page contains at least one link to that url:
<test:verify matches="<a href=.{$url}"/>
or:
<test:verify test="//a[starts-with(@href, $url)]"/>
Other valid expressions include all XPath and o:XML expressions.
All scripts also take two special parameters: test:iterations, which determines how many iterations per thread, and test:threads, how many concurrent threads the script will execute. These parameters must be specified on the command line or web interface with their namespace-qualified names: '{urn:hatatap}iterations' and '{urn:hatatap}threads' respectively.
Setup, Reset and Cleanup
Test scripts may contain a test:setup container element at the start of the script. All instructions it contains will run exactly once for each thread, before any other instructions are executed, regardless of how many iterations the test runs.
After the main instructions the script may include a test:reset element, which contains instructions to be executed after each iteration. Reset instructions are not included in the timing of the iteration itself.
The script may also include a test:cleanup element. If so it should be positioned at the end of the script. The instructions contained in test:cleanup will run once for each thread, after all iterations have either terminated normally or been terminated by an error.
<test:param name="required" default="optional"/> *
<test:setup> ... </test:setup> ?
... instructions ..
<test:reset> ... </test:reset> ?
<test:cleanup> ... </test:cleanup> ?
</test:script>
Random Values
There are several functions available to produce random values. These functions can be used in any string expression, in the same way as parameters. Simply enclose the function call in curly braces.
| random() |
generate a random number between 0 (inclusive) and 1 (exclusive) |
| random(from, to) |
generate a random integer between from (inclusive) and to (exclusive) |
| random-string(length) |
generate a random string of the specified length |
| random-string(length, chars) |
generate a random string using the specified characters |
<test:click url="http://nowhere.com/script?param={random()}"/>
use random values in a form:
<test:form>
<test:input name="query" value="{random-string(20)}"/>
<code>{random(100, 400)}</code>
<chars>example {random-string(10, '_$%!@')}</chars>
</test:form>
Request Headers
Cookies and request headers can be set by the script.
set the cookie 'status' to 'extended': <test:cookie name="status" value="extended"/>
Additionally the User-Agent header field can be set with a convenient attribute on the script element.
...
</test:script>
The default value of User-Agent is 'hatatap (http://hatatap.pingdynasty.com)'.
Threads and Iterations
The default number of threads and iterations may be specified in the script itself.
...
</test:script>
iterations specifies the number of test iterations per thread, so the above script will be executed using 4 threads doing 3 iterations each. The threads and iterations values can be overridden if they are specified as parameters, see 'parameterised scripts'.
Extensibility
hatatap scripts are transformed into o:XML, which is then executed by an o:XML interpretor. Any o:XML code present in the scripts will be included verbatim. The effect of this is that hatatap scripts can be seamlessly extended with o:XML code. o:XML is a fully-featured object-oriented programming language, providing unlimited programming capabilities.