What are the challenges of Quality Assurance in 2019?

From 15th to 17th October, TestCon Europe took place in Vilnius (Lithuania) dedicated to software testing. Sharing knowledge and methodologies via workshops and conferences is the main target of TestCon. Among topics which have been discussed, we could recognize a list which indicates the main issues in Testing:

•  Lack of practical working definition of what ‘quality’ means for software.
• Lack of quality.
• Lack of understanding by project management that quality is on target.
• Inadequate use of reviews and inspections.
• Inadequate defect prevention.
• Insufficient or careless testing.
• Excessive scheduling pressure leading to unwise attempts to short-cut quality control.
• Unstable and ambiguous user requirements.

Speed to market delivery requires improvement of key capabilities of software development and deployment performance (delivery frequency, lead time to change, meantime to repair). Industries found solutions to answer to these concerns.

DevOps, CI/CD and Agility

In 2018 many projects migrated to agile methodologies (Scrum, Kanban, SAFe) with a DevOps approach. One of the impacts of these changes in methodologies is the place and role of the tester. First, the tester is more and more integrated on the project team. With other actors such as developers or business stakeholders, testers take part in solution design. In this way, organizations try to apply Shift left, starting their testing activities earlier, to decrease time to market and increase quality.
Moreover, it means that new skills are required to answer these transformations. The Tester should be able to understand technical purposes on solution design or to implement automated tests as well. Also new tasks appear:

• define and clarify acceptance criteria in term of testability
• estimate the test effort
• reduce risks including working closely with developers
• coach other members of his team to promote quality
• create and update often his test strategy
• Finally, test strategies need to change: small iterations to deliver parts of feature involve the use of drivers/stubs (to mock missing implementation) and test automation to track any regression.

TDD / BDD / Contract testing

Test Driven Development (TDD) is a goal-oriented method, based where tests are being specified before the actual code is written. Definition of tests is the first step of a development task – the following implementation will try to reach success on many iterations. In other words, write a test helps in the design of the solution. Although the methodology has to be structured, it offers quiet a transition on the way of working.
Behavior Driven Development (BDD) is applicable on a higher-level, on an acceptance criteria definition. BDD is also known as Specification by examples. This strategy is mostly being used on agile projects and could be a base of test automation (thanks to frameworks such as Cucumber / Specflow or FitNesse). Providing a common language for business stakeholders, developers and testers, is also an important benefit to share requirements and works time-saving. Test referentials could even become the documentation of a project with an advantage to be still up-to-date. This is called ‘living documentation’.
Micro-services or componentization are an architectural response for reducing the delivery time. But how do we check a micro-service (when component itself uses other components) or feature of a very integrated-application, dealing with dependencies? Contract testing can tackle this issue (as well as related false failures and unstable tests). The principle is to define a contract between the provider and the consumer. Both will agree on the format of data that will be exchanged and that will be the contract. The provider has to ensure that his servicealways delivers the same response, and consumers could mock his input in order to stabilize his tests. Some tools, like Spring Cloud Contract, manage the design of a contract and generation of stuck.

Security & Performance

Nowadays, security testing becomes more and more important in software development. However this is not a functional requirement, it’s importance is growing from business perspective. Improving security of applications is a continuous and team effort and is part of a long process. Build a strategy means to define a scope of security testing, integrated on SDLC: review of requirements, threat modeling on design, code analysis and penetration testing. Some easy actions could be deployed to raise awareness, creating some checklists (based on existing standard) or appointing a security ambassador on the team.
Another non-functional requirement is system performance, which is linked to subjective feedback. How to ensure that users have good experiences on this application? 3 Seconds is the maximum (on average) before a user gives up the webpage. On agile process, performance should be present in the definition of done. We should evaluate the impact for each user story and define metrics, dashboard and alters to monitor it. Last but not least, testing continues also on production via A/B testing, canary test or chaos testing.

Tools for automation

Finally, this conference was also the opportunity to explore new tools, especially related to test automation. Some of them are more and more popular in organizations.
– Robot Framework: It allows to get a BDD approach and is an open-source project. One interesting feature is its ability to perform end-to-end testing across different platforms (different browsers, desktop applications, SOAP messages, etc.). Robot framework has a native integration in Python, provides a complete library for testing and could be integrated on a continuous delivery pipeline.
– Cypress.IO: it’s a chrome-based tool which runs in the browser. There is the main difference with selenium framework – Cypress.IO executes actions inside the browser and limits the network lag. There are currently still some limitations such as upload / download files, manage iframe or access to items of page via XPath commands, but there is as well an active community who found solutions and improves this open-source project.
– Selenoid: It’s an alternative lightning fast open-source Selenium protocol implementation running browsers and Android emulators inside Docker containers. It is distributed with a set of ready-to-use Docker images corresponding to the majority of popular browser versions, has a one-command installation utility and works more efficiently than traditional Selenium Grid. As solution is open source, has ongoing development and support it could be worth considering. Especially, when project requires support of many platforms/browsers/browser versions and can be executed on virtual machines.

Nicolas and Lukasz
Consultants