Selenium Web Testing

Demonstration of a Custom Framework

Aaron Shaver

2012-05-17

Application under test: electronic discovery (pre-trial phase in a lawsuit)

High level view of the framework and test suite All test logic and assertions in the test_ directories Categorized, e.g. can run "nosetests tests_smoke"

Subfolders contain individual tests Test dirs are Python packages via the init files, can be imported nose (test runner) discovers tests based on "test" regular expression

setup.cfg: nose, WebDriver, and application-specific settings Change timeouts for WebDriverWait, application task waits Adjust server URLs, usernames, passwords

util.py -- utilities module Product-neutral code: generic functions and classes Could be reused to test other web apps

shift.py (our product is called Shift) Module for all application-specific code Pulls configuration settings from setup.cfg

data.py -- another application-specific module Keeps document count assertions outside of the _test.py files Easier to maintain: doc counts change sometimes as the app improves or degrades, e.g. file formats become supported that weren't earlier

log.txt -- optional test results file Version-controlled via TortoiseHg (Mercurial) + Bitbucket, so I can see past test runs Useful if results exceed cmd.exe buffer size nose needs special syntax to pipe output of console to the log

Example of test run outside of log.txt (outputs to cmd.exe window) Tests slow by usual unit testing standards? App performs many operations on uploads (recursively extracts, indexes metadata properties, deduplicates, generates reports, etc.)

Entire test suite takes ~6 hours, easily fitting into an overnight run, so I haven't bothered to setup Selenium Grid yet

90% of tests in suite use standard setup routine App under test needs to ingest data before doing anything useful All it needs: project name, data source, total docs assertion

Example of a simple test which does not use the "standard intake workflow" Note the product "action" functions: user_login, add_proj clean_up deletes the project and closes the browser, but only if the test was successful Failed tests leave the browser window open, in case it's a new bug

Built my framework before hearing about PageObjects Plan was to make "atomic" app actions ("save a search", etc.) Not as fine-grained as PageObjects Still a robust layer of abstraction for when UI changes Ended up clumping together app actions; even less fine-grained than my own plan

Developers were nice to me: many UI elements generic across app No AJAX yet (fingers crossed) -- timing issues from Webdriver itself

UI is very good / simple / elegant... but not always Example: reuse of list_item_detail class for both <a> and <div> Have to ensure what we're clicking is a link Could do a CSS selector, but would be more fragile

Potential difficulty: can't automate Java or Silverlight uploaders Thankfully, administrator backend allows data intake Uses standard HTML form

Questions about my automation? Suggestions for improvement?