Test automation frameworks. Introduction to Atata

Automation is an increasingly valued approach to software testing. It not only brings about time and cost savings in projects, but in the long run it allows us to deliver better quality products to a customer. When deciding on automation, it is worth carefully considering what tools to use, including test frameworks. In this article you will find an overview of the most important types of test automation frameworks. I will also take a closer look at the not yet widely known tool Atata, based on Selenium WebDriver, and compare its capabilities with those offered by pure Selenium.

When should we automate software tests and why?

What is Automation?

Before we focus on the topic of the Atata test automation framework, I would like to briefly remind you what software test automation is in general and why it is worth taking an interest in it. There are a number of myths around this topic.

Testing is a very important part of the software development process. One of the approaches to this aspect is automation, i.e., the use of special tools and frameworks that enable the programming of tests and automatic execution without further human intervention.

Benefits of Automation

Automation of these processes can bring countless benefits, including some of the major ones – time and cost savings. In addition, automatic tests are more repeatable and more accurate than manual tests, which translates to reliability and effectiveness.

Elevate Your Application Development

Our tailored Application Development services meet your unique business needs. Consult with Marek Czachorowski, Head of Data and AI Solutions, for expert guidance.

Schedule a meeting

Scenarios and test cases and automation

It is worth considering test automation in your project due to the option of conducting tests with complex scenarios and in many environments at the same time, which could be a challenge when manually implementing a test scenario on a daily basis. It is possible to perform tests on different platforms, in different hardware and network configurations, which would be difficult or even impossible to achieve in the case of manual tests.

Creating test cases in an automated form ensures their repeatability, and thus increases their effectiveness when it comes to detecting defects. This is why automation is particularly useful in testing the critical functionalities of the app. However, it is worth remembering that the test reality is not black and white , and like everything, it has its pros and cons. What are the downsides of automation?

The Dark Side of Automation

Test automation can be a fantastic solution in many projects, but it is not a universal solution. In some cases, manual testing can be a much better, more effective option, or even the only one available. Therefore, before deciding to automate tests, it is worth carefully analyzing the specifics of the project, the needs, and any available tools and frameworks.

You also need to bear in mind the time that must be invested in the creation of automated test cases. This requires many hours of work, configuration, and troubleshooting that you do not have to deal with during manual software testing. This is a process that can develop very slowly at first, and credible results are often visible only after putting a lot of work into the project. There is also a need to hire an automation tester who will be able to design the entire system so that the tests are stable and in line with good programming practices.

Types of test automation frameworks

Automation frameworks are tools that make this task easier by providing solutions to many common problems. Depending on the needs and specifics of the project, we can choose from different types of frameworks. Below is a brief description of some of them.

Library Architecture Framework

The Library Architecture Framework (LAF) is a popular software test automation technique. As part of this methodology, libraries with ready-made test functions that can be called in the test code are used. LAF is often used in cases where tests are complicated and require multiple stages, as well as in situations where tests must work on different operating systems and web browsers.
This type of framework is particularly useful for low-level tests, such as unit tests.

Module Based Testing Framework (MBTF)

This is a type of software test automation framework that is based on a modular approach to test design. In this model, tests are designed in a modular way, and each module covers one functionality or test cases related to one application module. In MBTF, test modules are designed in such a way as to be universal and can be used repeatedly for various applications and systems. Once you’ve created a set of modules, you can use them in different projects without having to rewrite the code.

Linear Automation Framework

Linear Automation Framework is a type of test automation framework that is based on a linear approach. At LAF, tests are designed to reflect business processes or application usage scenarios. This means that they are performed in a sequential manner. In this model, tests are designed as sets of steps, and each step of the test is associated with a user action or result that is expected in the application. The advantage of the Linear Automation Framework is that the tests are closer to the actual use cases, which can help us to find more realistic defects in the application, such as those that the user would come across.

Hybrid Test Automation Framework

This is a kind of framework that combines the features of all the above-mentioned approaches. This enables the design and execution of tests in a flexible way, tailored to the needs of the project. In HTAF, tests are created in a modular way, similar to the Module Based Testing Framework, but with more flexibility and access to a wide range of test tools. This framework allows the integration of various tools and technologies, such as Selenium WebDriver, TestNG, Appium, JMeter, etc., which allows us to automate tests in various environments, such as web, mobile, desktop applications and many others.

Read also: End to end testing with Cypress

Atata automation framework

Atata is a full-featured test automation framework that is based on Selenium WebDriver and designed for web applications. It was written in C# and offers a wide range of functionalities that make it easier to automate web application tests.

Atata – Advantages

• Availability – The Atata framework is fully open source and available on the GitHub platform under the Apache License 2.0.
• Transparency and ease of use – one of the key elements of the Atata framework is the use of the Fluent Page Object Pattern. This pattern allows you to write code that is easy to read and understand. The built-in login system makes it easy to track and troubleshoot problems during testing.
• Triggers – what is more, the framework has a unique trigger functionality that allows you to run custom code when specific events occur.
• Low Entrance Threshold – Atata is designed to be easy to use and maintain. The framework provides a simple and intuitive syntax for defining and using the objects on the page. This requires a minimum amount of code, which positively affects readability and facilitates the introduction of further testers to the project.
• Ready-made components – the framework also provides a set of ready-to-use components to facilitate work. These components include controls, pages, and layouts. Atata provides a wide range of built-in controls that can be used to interact with web elements on the site. Controls include buttons, text boxes, check boxes, radio buttons, and many, many more.
• Data-driven testing – Atata also supports data-driven testing, allowing testers to create tests using the Data-Driven Testing function. With this feature, developers can run the same test scenario with different datasets.
• Parallel testing – the framework also supports parallel testing. With parallel testing, testers can run multiple tests at the same time, which can significantly reduce the time taken for testing. This is especially useful for large web applications that require extensive quality control.

Atata – disadvantages

Despite its many advantages, the Atata framework also has some drawbacks.

• Limited use – one of the main drawbacks is that the framework usage is limited to websites only. This is an important limitation, for example, for companies developing software that also design other types of applications in addition to websites. In this case, you may need to use another test automation tool.
• Available only for C# – this framework is only created for the C# language, which makes it unavailable to many programmers who specialize in other popular programming languages.
• Small community – another disadvantage, compared to other test automation tools, may be the small community surrounding it. This framework is not as well-known as other test tools which have been available on the market for years. We need to take into account the fact that there are fewer online resources and documentation, as well as support materials available to users. Fortunately, the user community is highly engaged and ready to help. It is also worth noting that contact with the creators of the framework is extremely simple, and communication is highly efficient.

However, despite these disadvantages, the Atata framework is still one of the best tools for automating C# web tests. Its numerous advantages make it popular among testers who are looking for a simple and intuitive way to automate.

Atata vs Selenium

Let me start by saying that the question “Atata or Selenium?” is not quite appropriate in the context of the issue we are discussing. In particular, comparing Selenium to an overlay, which in fact Atata is, may seem baseless. After all, Atata is nothing more than a framework covering the database with its functionalities, which provides us with the necessary basic tools.

Thanks to this we get a “nicer” version of Selenium, which is more user-friendly and faster to implement. Nevertheless, I decided to lay out the important differences in the table below. It is worth being aware of the benefits of using such an overlay for the project. After all, when planning work, choosing a framework is not easy; the more options there are, the harder it is to make a decision even within frameworks that rely on other tools which are already available.

Atata

One of the biggest advantages of the Atata framework is its useful login system which allows you to track and diagnose problems during testing. Atata also offers unique features such as a trigger mechanism that allows you to execute custom code in response to specific events during testing.

Atata also offers an intuitive syntax for defining and using page objects, making it easy to program and maintain tests. It supports the use of the Page Object Pattern model, which is a huge advantage for fans of this pattern. This makes the structure clear and easier to understand.

The framework has a built-in set of ready-made components such as controls, pages and layouts, which speeds up the test automation process. Below I have provided an example of what Atata offers – it not only contains basic elements of pages such as text fields, but also differentiates fields for e-mail addresses, passwords, numbers or files. Furthermore, it supports choice lists, clickable items, and many, many more. A full list of specifications and capabilities can always be found in the very clear documentation at https://atata.io/components/.

using _ = TextBoxPage; //alias for better readability
    [Url(TextBoxUrl)]      //Url extension to base url placed in SpecFlowHooks.cs
    [VerifyTitle("ToolsQA")]
    public class TextBoxPage : BasePage<_> 
    {
        private const string TextBoxUrl = "/text-box";


        [FindById("userName")] //Selector to find element
        public TextInput<_> UserNameTextBoxInput { get; set; } //An element with its type and name


        [FindById("userEmail")]
        public EmailInput<_> UserEmailTextBoxInput { get; set; }


        [FindById("currentAddress")]
        public TextArea<_> CurrentAddressTextBoxTextArea { get; set; }


        [FindById("permanentAddress")]
        public TextArea<_> PermanentAddressTextBoxTextArea { get; set; }


        [FindById("submit")]
        public Button<_> SubmitTextBoxButton { get; set; }
       
        [FindById("output")]
        public Text<_> TextBoxOutputBox { get; set; }


        [FindById("name")]
        public Text<_> TextBoxOutputName { get; set; }


        [FindById("email")]
        public Text<_> TextBoxOutputEmail { get; set; }


        [FindById("currentAddress")]
        public Text<_> TextBoxOutputCurrentAddress { get; set; }


        [FindByXPath("output")]
        public Text<_> OutputBox { get; set; }


        [FindById("permanentAddress")]
        public Text<_> TextBoxOutputPermanentAddress { get; set; }
    }

Use of ready-made components

public class ProjectDetailsPage : BasePage<ProjectDetailsPage>
    {
        public int findGarageParkingPlaces()
        {
            var element = AtataContext.Current.Driver.FindElement(By.XPath("//div[contains(text(),'Garage Parking Places')]"));
            return int.Parse(element.FindElement(By.ClassName("value")).Text);
        }
    }

Calling AtataContext.Current.Driver

Despite its enormous capabilities, however, it sometimes turns out that we lack a component or it is impossible to use a ready-made solution offered by Atata. Luckily, the developers haven’t removed the option to access a pure version of Selenium, and we can search for our elements or apply specific actions to them with no issues. Below is an example of a WebDriver call.

Fans of reports and tables will also find a useful function here. Atata offers a built-in tool to take screenshots according to the selected pattern, e.g., after performing each step. If we connect this function with a library for creating reports, such as ExtendedReports, we will get a very pleasant environment for generating reports in the form of a .html file. Once the tests are run, we have an immediate view of the results along with statistics, marked errors, tags and graphs. At first glance, we can say that IDE itself can also offer such data but the form of compiling them in this library is much more readable and easy on the eye. After small changes to adapt to the needs of the project, we get a report that can be presented almost effortlessly during a business meeting.

[Screenshot(AfterAnyAction)]
    public abstract class BasePage<TOwner> : Page<TOwner>
        where TOwner : BasePage<TOwner>
    {
        //Some general helping methods that can be used in all pages
        public int[] GetElementSize(Text<TOwner> element)
        {
            return new int[2] { element.ComponentSize.Width.Value, element.ComponentSize.Height.Value };
        }
        
        public void AcceptAlert()
        {
            AtataContext.Current.Driver.SwitchTo().Alert().Accept();
        }


        public string GetAlertMessage()
        {
            return AtataContext.Current.Driver.SwitchTo().Alert().Text;
        }
    }

An example of adding screenshots to a report (1 line), I’ve thrown in the context of the whole class for clarity but I’ll leave it to you to edit as it suits you.

[Binding]
    public sealed class TextBoxSteps : BaseSteps
    {
        private readonly ScenarioContext _scenarioContext;


        [Obsolete("Visual Studio IntelliSense Work Around", true)]
        public TextBoxSteps(ScenarioContext scenarioContext)
        {
           _scenarioContext = scenarioContext;   //Initializing scenario context, you can save test session data in it
           SetRandomUser(_scenarioContext);      //Creating new fake user data and saving it in context so it can be used in multiple tests without generating new set 
        }


        [When(@"User input all personal data on address form page")] //SpecFlow binding to feature files, methods are named the same as scenario steps 
        public void WhenUserInputAllPersonalDataOnAddressFormPage()
        {
            Go.To<TextBoxPage>()                 //Open page   
                .UserNameTextBoxInput.Set(User.FirstName + " " + User.LastName) //Make actions on web elements from page you just opened eg. set -> input text 
                .UserEmailTextBoxInput.Set(User.Email) //User.Email is a reference to our generated fake data
                .CurrentAddressTextBoxTextArea.Click()
                .CurrentAddressTextBoxTextArea.Set(User.Country)
                .PermanentAddressTextBoxTextArea.Set(User.Country);
        }


        [When(@"confirm the data by clicking button")]
        public void WhenConfirmTheDataByClickingButton()
        {
            On<TextBoxPage>()                     //On already opened page
                .SubmitTextBoxButton.Click();
        }


        [Then(@"user should see provided data below")]
        public void ThenUserShouldSeeProvidedDataBelow()
        {
            On<TextBoxPage>()
                .TextBoxOutputBox.Should.BeVisible()  //Should is assertion verification if something is as you wanted
                .TextBoxOutputName.Content.Should.Contain($"Name:{User.FirstName} {User.LastName}")
                .TextBoxOutputEmail.Content.Should.Contain($"Email:{User.Email}")
                .TextBoxOutputCurrentAddress.IsVisible.Should.BeTrue()
                .TextBoxOutputCurrentAddress.GetContent(ContentSource.Value).Should.Contain(User.Country)
                .TextBoxOutputPermanentAddress.GetContent(ContentSource.Value).Should.Contain(User.Country);
        }
    }

Example of page navigation (Go.To and On), I paste again the full context of the class

The issue of navigating between application pages during one session was also intuitively solved. In the example below, you can see that using the Go function, It (go to) or He (stay on the page) is sufficient. Just upload the appropriate page address according to the method – there is no need to open and close the session properly (as with Selenium) without an overlay. This contributes to significant time savings when configuring your environment.

Unfortunately, this framework was created only for the C# programming language, which is a significant limitation when compared to Selenium, which also works with Java and Python.

Which test automation framework is best for my project?

The choice of test automation framework depends on many factors, such as project type, business requirements, developer preferences, and resources available. Here are some steps that can help you choose the best framework for your project:

1. Define your requirements

Before deciding to use a given framework to automate tests, you need to know what your requirements are. For example, if your project involves the development of a web application, then a framework that supports user interface testing might be the best choice. There is also no point in choosing a technology that supports other types of applications as well at the expense of the one that interests us.

2. Find the right tools

Consult developers and testers who have experience in test automation and ask them for recommendations on tools. Collect information about different frameworks, their functionalities, limitations and social resources. 

3. Test the frameworks

Test selected frameworks in practice. Prepare a set of tests representative of your project and implement them in each of the frameworks. Analyze which frameworks are the most intuitive and easy to use.  

4. Check the integration

If the developed application uses external tools, make sure that the framework you choose integrates with them. For example, if you use a Continuous Integration platform, make sure the framework you choose has a plugin for it.  

5. Pay attention to the availability of documentation and community support

Choose a framework that has good documentation and a wide community of users. This will help to solve the technical problems you may encounter when working with the framework.  

Summary

To sum up, the choice of a test automation framework depends on many factors and there is no clear answer to the question of which one will work best in your project. Currently, there are many types of frameworks available on the market, so you need to explore different options and choose the one that best suits your requirements and project specifics. I hope that with this article I have also managed to detail the capabilities of the not as yet well-known tool Atata.

Although Selenium is one of the more popular and widely appreciated tools, it is always worth knowing about other options. Atata is constantly being developed by the developer community, which means that anyone can contribute to advancements by reporting bugs, making improvements or writing documentation. This way, Atata becomes a better tool.