Here we’ll learn how to create a basic test, execute it, and find the bugs before your users do! You’ll learn about the tools available to write and execute tests and also see testing terms like automated, manual, unit and integration tests as well.
Automated vs. Manual Testing
| Automated | Manual Testing |
| Test automation software along with testing tools executes the test cases. | Explicitly humans involved in executing the tests step by step, and they may be testing without test scripts. |
| Test cases are written by QA engineers but execution is fully automatic and it is quite faster and will run for n number of different scenarios. | Analysts and QA engineers need to get involved in end-to-end testing and time consuming. |
| In Python, we need to install a package named “unittest”. | In Python, via “White Box Testing”,” Unit Testing”,” Integration Testing” etc., methodologies, Manual testing can be carried out. |
Unit Tests vs. Integration Tests
| Unit Tests | Integration Tests |
| Unit testing works on component by component basis and hence if any small component to be tested, we need to go for unit testing. | Integration testing works on the whole component, we can conclude as many unit tests make integration testing. And also a software is complete if whole integration testing is complete |
| Testing of addition calculation alone in a calculator program for specific set of arguments | Testing of all arithmetic operations like addition, subtraction etc., (whatever present in calculator program) with different set of arguments |
Example for Unit testing:
Python3
# define a functiondef test_sum_numbers(): assert sum([100, 200, 300,400]) == 1000, "Result should be 1000"# define a functiondef test_sum_tuple_values(): assert sum((100, 200, 200,400)) == 1000, "Result should be 1000"# Driver codeif __name__ == "__main__": test_sum_numbers() test_sum_tuple_values() print("Checking whether all tests are passed or not") |
Output:
Traceback (most recent call last): File "....../test_example.py", line 9, in <module> test_sum_tuple_values() File "...../test_example.py", line 5, in test_sum_tuple_values assert sum((100, 200, 200,400)) == 1000, "Result should be 1000" AssertionError: Result should be 1000
In the above example, We have two methods here and when code is executed, second method throws error as the given values do not produce 1000.
Choosing a Test Runner:
Test Runner is a library or a testing tool which reads the source code that contains unit tests and a bunch of settings which can be executed and produces its output to the console or log files.
There are different Test Runners available in Python. Popular one are
- unittest
- nose or nose2
- pytest
unittest : It is built into the standard python library. import unittest should be the starting line of code for using it. Depends upon the python version, it should differ as later versions of Python supports unittest and earlier versions supported unittest2.
Sample snippet of python code using unittest is given below:
Python3
# import libraryimport unittest# create a classclass TestXXXXX(unittest.TestCase): # define a function def test_xxxxxxx(self): data = [100, 200, 300] result = sum(data) self.assertEqual(result, 6000)# driver codeif __name__ == '__main__': unittest.main() |
Output:
====================================================================== .F FAIL: test_xxxxxxx (__main__.TestXXXXX) ---------------------------------------------------------------------- Traceback (most recent call last): File "......py", line 8, in test_xxxxxxx self.assertEqual(result, 6000) AssertionError: 600 != 6000 ---------------------------------------------------------------------- Ran 1 test in 0.001s FAILED (failures=1)
nose or nose2: This is an open source application and similar to unittest only.It is compatible with numerous kinds of tests that are written using unittest framework. nose2 is the recent version one, and they are installed by using.
pip install nose2
pytest: It supports unittest test cases execution. It has benefits like supporting built in assert statement, filtering of test cases, returning from last failing test etc
Python3
def test_sum_numbers_using_pytest(): assert sum([700, 900]) == 1600, "Resultant should be 1600"def test_sum_tuple_using_pytest(): assert sum((700,1900)) == 1600, "Resultant should be 1600" |
No need to write the class, command line entry point etc.,
How to Structure a Simple Test:
- For writing a test, need to know what is going to get tested.
- Does it cover unit testing and/or integration testing?
- All kind of necessary inputs (it can range between integer, float, type data types etc., ) and the code for their execution and get the output and compare the output with expected result.
How to Write Assertions:
Assertion is nothing but validating the output against a known response. i.e. in above code, we have passed the list containing 3 values namely 10, 20 and 30, and we know that its multiplication result is 6000. So as a last step in code, we will be writing assertion code and above code comes up with assertEqual and surely it will give the output as 6000 and hence the testcase passes.
unittest has lots of methods to assert on the values, types, and existence of variables.
Let us see few :
.assertEqual(one,two) means one == two (Our above example) .assertTrue(expr) means boolean(expr) is True .assertFalse(expr) means boolean(expr) is False .assertIs(one,two) means one is two
We have opposite methods too like:
.assertIsNot(one,two) means one is not two. .assertIsNone(x) vs .assertIsNone(x) .assertIn(x, y) vs .assertNotIn(x, y) .assertIsInstance(m, n) vs .assertNotIsInstance(m,n)
Side Effects:
Continuous execution of piece of code has the possibility to alter other things to change like attribute of a class or even a value in the database. So these are need to be decided before doing testing. Refactoring of code needs to be considered to break the side effects and write repeatable and simple unit tests.
Running Tests From PyCharm:
We can run unittest or pytest by following the steps and this is applicable for running all tests in a directory:
- First from project tool window, select the tests directory
- Then, on the context menu, choose “UnitTests in” or “PyTests in “
For individual test,- Via main toolbar by using Run or Debug command
- Via context menu by using Run or Debug by clicking the specific file
Testing for Web Frameworks Like Django and Flask:
Based on unittest, Django and Flask makes things easier, and they have their testing framework:
Django Test Runner: The Django startapp template will have created a tests.py file inside your application directory. If you don’t have that already, you can create it with the following contents:
Python3
from django.test import TestCaseclass RequiredTestCases(TestCase): # Write all Test methods |
For executing the test suit, we need to give as:
python manage.py test
Use unittest and Flask : Flask requires that the app be imported in file and then set in test mode. You can instantiate a test client and use the test client to make requests to any routes in your application.
All the test client instantiation is done in the setUp() method of your test case.
Python3
# import our application fileimport my_app# import libraryimport unittestclass FlaskTestCase(unittest.TestCase): def setUp(self): my_app.app.testing = True self.app = my_app.app.test_client() def test_home(self): result = self.app.get('/') |
Test cases can be executed by using below command (via command line) :
python -m unittest discover
More Advanced Testing Scenarios:
- Fixture is used which is nothing but the data that is created as an input and reusing them.
- Parameterization can be followed which is nothing but running the same test several times by passing different values and expecting the same result.
- Need to cover different scenarios like handling expected failures, writing integration tests, testing in multiple environments etc.
Below example shows how to write test for bad data type:
Python3
import unittestclass TestSumDifferentPatterns(unittest.TestCase): def test_list_tuple_values(self): # Summation of numbers present # in a tuple value data = (10*2, 200*2) result = sum(data) self.assertEqual(result, 600) def test_bad_data_type(self): data = "alpha value passed instead of numeric" # Because of the below condition, TypeError # occurs and hence it will not proceed # to next step with self.assertRaises(TypeError): result = sum(data)if __name__ == '__main__': unittest.main() |
Output
.F ====================================================================== FAIL: test_list_tuple_values (__main__.TestSumDifferentPatterns) ---------------------------------------------------------------------- Traceback (most recent call last): File "......py", line 10, in test_list_tuple_values self.assertEqual(result, 600) AssertionError: 420 != 600 ---------------------------------------------------------------------- Ran 2 tests in 0.001s FAILED (failures=1)
Automating the Execution of Tests :
Continuous Integration/Continuous Deployment tools are available. They help to run tests, compile, publish and also deploy into production.
https://travis-ci.com/ is one among them which works well with Python and it is an open source project. Login into the site and create.travis.yml with the below contents:
language: python
python:
<Include the versions as 2.7 or 3.7 whichever required>install:
– pip install -r <your requirements file>script:
– python -m unittest discover
Above file instructs “Travis CI” to look into this file and for given Python versions, the test cases are getting tested by installing the necessary packages as given in the requirements file and finally run below command to run the test.
python -m unittest discover
Results are on the website against your credential
