Automation Tool Feasibility Analysis
During this phase focus should be primarily on identification of test automation tool(s) from the technical perspective. The finalization of tool can be worked upon later after due diligence of all factors collected, during this phase.
Prerequisites
- Test environment should be ready and available for feasibility analysis 24*7
- This environment must be dedicated for tool feasibility activity to avoid any schedule slip & test data corruption
Identification of test scenarios & test data
- Number of scenarios should be minimal (to avoid high commercial cost)
- Should cover all screens & all possible unique UI components of the application (e.g.)
- Editing of Grid control
- Picking dates from Calendar
- Filtering and selecting items in ListBox
- Attachments of Files
- Popups
- Trees & Submenus
- Reading value from Graphs (Bar, Chart etc.)
- Dragging and dropping of items
- All scenarios must be accompanied with 3-5 different data sets (positive flow, negative flow, boundary values)
- Every step in a scenario should be accompanied with 2-3 different assertions. Usually assertion is done with text label only (easy way) but it leads to passing of automation, which shouldn’t be the case. If possible, resulting data value should be read and compared against expected.
- If possible, test a scenario where a component is upgraded and check maintenance cost involved in fixing the script
Test suite execution
- Should be able to select and execute any scenario(s)
- A test scenario should be able to execute multiple times in a loop w/o manual intervention
- Should be able to execute multiple scenarios in parallel
- Test automation should support below platforms and browsers
- Windows – IE 6.0 and above
- Windows – Firefox
- Windows – Chrome
- Linux – Firefox
- Linux – Chrome
- Mac – Safari
- Mac – Chrome
- Android - Chrome
- iOS - Safari
Expected output from tool feasibility
- Demo on multiple tools (e.g. Selenium, Ranorex, Test Complete etc.)
- Tool feasibility & comparison report
- Script design productivity information for each tool
- Script maintenance cost for each tool
- License pricing of tool (1st year and onwards)
- Cost of execution in parallel (assume 10 machines)
Factors to consider for tool feasibility
Below is a list of comprehensive factors to determine and finalize an automation tool. Of course weightage of each factor depends on specific needs of teams.
| Factor | Category |
|---|---|
| Identification of all UI components of the application | Scripting |
| Existing Framework (in-built framework) | Scripting |
| Support on mobile platforms like iOS, Android (in future) | Scripting |
| Support of scripting language | Scripting |
| External support, in case of any issues found during scripting or execution | Scripting |
| Support BDD framework | Scripting |
| Support script less automation | Scripting |
| Data driven testing | Scripting |
| Execution support on all browsers (IE, Chrome, Firefox, Safari, Edge) | Execution & Reporting |
| Ease of use in various environments (AUT) | Execution & Reporting |
| Ease of reporting and understanding | Execution & Reporting |
| Integration with Issue Tracker | Extensible |
| Support for Unit Testing | Extensible |
| Execution Flexibility (should be able to execute from anywhere) | Flexibility |
| Script design flexibility (should be able to design script from any machine) | Flexibility |
| Script designing license cost | Cost |
| Runtime license cost | Cost |
| Resource Experience level | Cost |
| Availability of resources in the market | Cost |
| Training (to Jr. Associate) | Cost |
Comments
Post a Comment