Test automation framework is a platform that provides an execution environment for test automation scripts. Each framework offers unique rules, guidelines, protocols and procedures for test creation, association and implementation. In this blog, we are going to discuss some of the widely used test automation frameworks with their pros and cons.
Test automation is a procedure of using an automation tool to execute test cases. The method or process through which such automation is implemented is called a Test Automation Framework.
We can also call it a set of rules or guidelines containing coding standards, test-data handling, etc. for implementing and designing various test cases.
There are several frameworks that have been developed and implemented over the years by different test automation communities, experts as well as numerous commercial vendors. But, many times, it is difficult to decide which test framework to choose and why.
Below is a list of top 6 test automation frameworks that are widely used in the test automation industry. Let’s discuss these test automation frameworks in detail.
This is the simplest test automation framework, which is also known as “Record & Playback Framework”.
Due to its simple architecture, testers can easily record each and every step manually like navigation, user inputs, checkpoints etc. These recorded scripts can be played in a sequential fashion with little or no modifications aligning to test case requirements.
This framework is best suited for test scenarios where there is a need to repeat the same processes several times. Some examples include order management systems, record management etc.
Pros: No coding and much technical expertise required. It is also the fastest and easiest way to generate the script.
Cons: Test scripts can’t be reused. Maintenance is very difficult and synchronization problems may occur frequently.
In this framework, each test case is observed as a function call to which data is inserted from an external source (Excel, Text, CSV, ODBC sources, etc.), so it is possible to run the same test case with different sets of test data.
In Data-driven framework, the application UI can be automated using Selenium test script with C#. Most of the time, the database is in tabular form, stored in Microsoft Excel.
To use Selenium test script, Apache POI library can be used in which you need to download JAR files and add them to the project library. Once the configuration of Apache POI is done, you are ready to write the test in the data-driven framework.
One of the most popular frameworks for Selenium web driver with the data-driven feature is TestNG. This framework is most commonly used in supply chain and financial industry, where data is a very crucial aspect of operations.
Pros: Test data is maintained separately, so it is easy to make changes in the test script. A variety of test scenarios can be executed with multiple external database resources.
Cons: More time is required to plan and prepare both test scripts and test data. Requires a skilled understanding of coding language.
In this framework, keywords and data tables are needed to be developed which are independent of test automation tools for the test case. Each test script is observed as a keyword prompted action and the test data directs the testing here. Thus, when the automated test case is executed, the test data is read which points to the corresponding keyword and then finally, the applicable script is called/executed.
The tools that widely use this framework are HP UFT (formerly known as HP QTP), Selenium, and Robot Framework. You can convert existing desktop, mobile and web keyword tests into scripted tests with languages like JavaScript, JScript, Python, VB.NET etc.
This framework is suitable for scenarios that consider lots of keyword search cases such as web or browser testing, e-commerce testing etc.
Pros: This framework provides codes as well as keywords reusability. Test cases can be designed with or without the application. The testers can decide the execution order of test cases.
Cons: It’s a challenge to update the test scripts according to change in the UI of the applications. High automation and technical expertise are required.
In this framework, testers are required to divide the application under test into separate units, sections or functions, each of which will be tested in isolated modules. A test script is created for each individual module and then combined to build a larger test in a hierarchical fashion. The modules are separated by an abstraction layer so that the changes made in the application don’t affect the entire module.
Tellurium, evolved from Selenium, is built on a UI module concept framework, which makes it easy to create reusable and easy to maintain test cases against the dynamic RIA (Rich Internet Application) based web applications. Another tool called GUIdancer, an Eclipse-based automated GUI testing tool, which works on the same framework allows testers to write modular, flexible test cases without any programming effort.
This framework can be very useful in Healthcare and IT Industry where you need to implement automation on various modules. For healthcare, various modules can be patient monitoring, transporting etc. while for IT, the modules like sales channel, HR, marketing, customer management, etc. are suitable for this framework
Pros: If any changes occur, then only that particular module needs to be fixed, leaving the rest of the application untouched. Creation of test cases takes less time and effort as different modules can be reused.
Cons: Data is embedded into the test scripts since the tests are executed separately, so when testing with a different set of data, changes must be made to the test scripts themselves.
In this framework, the approach is to divide the application into functions that can be used by other parts of the application as well, rather than separating the application into test scripts. A ‘common library’ is created that includes common functions for the application under test, and these libraries can be used whenever needed.
The Selenium WebDriver library is perhaps the most used external test library while Robot Framework can test things other than websites, such as FTP, MongoDB, Android, Appium, and more. The Robot Framework is basically Python-based, but you can also use Jython (Java) or IronPython (.NET).
This framework is most used for inventory management systems, where common activities are repeated frequently.
Pros: Test maintenance and scalability are easier and more cost-effective. It provides a higher degree of reusability.
Cons: More time is needed to plan and prepare the scripts. Test data is hard-coded into the script so changes in the data will require in the test script.
This framework is a combination of more than one framework. It is set up to combine the advantages and moderate the weaknesses of all the associated frameworks. This framework can be easily adapted to get better results.
This framework can be used to get the desired benefits as per the requirements.
Cucumber, SpecFlow, HP UFT (Unified Functional Tester) etc. are open source tools widely used to merge test specification and documentation into a single source.
Considering the usage as well as the pros and cons of each framework, you can select the framework that best suit your test automation needs. You must also make sure that your test automation tool is suitable to the chosen framework to get the desired results of your testing efforts.
eInfochips provides a unified Test Automation Framework that enables early automation, reduction in operating cost and faster time-to-market. It offers end-to-end testing across the web and mobile platforms, guaranteeing high-quality solutions and 40% reduction of effort over traditional automation methods. With a wide range of features and some key capabilities, it provides more robustness and flexibility to the test automation activities.
To know more, read our white paper on Unified Test Automation solutions.
