Quality Assurance with OutSystems
Ensuring high quality at a rapid pace.
Agile and DevOps practices has for long been focusing on enabling rapid development in a safe and secure manner, partly due to the frequent execution of automated tests. In my own experience, testing is very often a central culprit, when organizations are struggling to meet project deadlines or maintain a quality standard.
OutSystems, as a low code platform, includes pre-built deployment pipelines out of the box, allowing developers to rapidly deploy changes to the development environment in order to test out their creations. With such a rapid speed, the potential for introducing errors is certainly there. Given the ease and speed of deployments, one can of course easily fix bugs found through manual testing and redeploy the fixes in a hurry. However, this is not scalable. If you a building an application of some scale, or if you are interested in really utilizing the amazing speed-to-market, that OutSystems has to offer, test automation is a must.
Lessons from automated testing in traditional programming teach us, that in order to be fast and efficient, most of our tests should be at the unit level, as this yields both faster feedback and a better economy (as it is more expensive to fix bugs later in the cycle). Traditionally, unit tests are something that you would write together with the code implementation — which brings us to the question: Can we even do this with low code? And is it needed?
The short answer to both is yes!
In this post, I will take you through, how OutSystems enables you to automate tests on various levels similar to what you would do with traditional programming.
Why do you need to automate your tests to ensure quality?
When developing applications with low code, several things are different compared to normal. First of all, although the visual low code implementations might be easier to grasp by most people, it by no means eliminates the possibility to create logical errors in your functions. Second, you no longer have pull requests as a way to gate changes to your code repository.
So what do you do then? By automating tests, you can gate the deployment to various environments by running the tests prior to deployment to that environment, and then only go forward if everything works as expected.
At the same time, I would advise you to up your efforts within exploratory testing, as this could give you insights what has been implemented to your application , as well as insights into missing tests for future automation.
Unit testing — low code style!
You are by no means the first one, who would like to unit test your low code methods (called “actions” in OutSystems). As OutSystems do not provide any unit testing framework out of the box, others have created one available in the OutSystems Forge. Look up the BDDFramework. As the name indicates, this extension provides you with a behavior driven development framework for OutSystems. Use the known “Given, When, Then” wording together with test implementation in the same low code features as you use for application development.
In order to execute the tests as a suite, use the extension named TestFramework. This extension allows you to also report on your projects code coverage, analyze test results and schedule test executions.
Finally, make use of OutSystems building applications as sets of components. Develop your automated unit tests in a separate component referencing your application. This allows you to exclude the tests from production deployments.
Create higher level automated tests with your favorite tools
OutSystems produces absolutely normal applications. This means, that once build and deployed, your application is possible to test in the same ways as traditional applications. This allows you to perform integration and UI testing with external tools, such as Postman or Selenium, respectively — or whatever tool you prefer.
Utilize your own orchestration and pipeline for continuous testing
Out-of-the-box OutSystems provides you with pipelines for deploying your application to new environments. However, these pipelines are not configurable.
In order to incorporate your automated tests of any kind, you will have to use your own orchestration. OutSystems provide easy-to-use guidelines for setting up these pipelines with Azure DevOps and Jenkins.
As part of these pipelines, you can run all of your automated tests along with other tools, such as static code analysis or vulnerability scans (requiring you to have access to the generated source code).
