🏆 Can explain testing

Quality Assurance → Testing →

What

Testing: Testing is operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component. –- source: IEEE

When testing, we execute a set of test cases. A test case specifies how to perform a test. At a minimum, it specifies the input to the software under test (SUT) and the expected behavior.

📦 Example: A minimal test case for testing a browser:

  • Input – Start the browser using a blank page (vertical scrollbar disabled). Then, load longfile.html located in the test data folder.
  • Expected behavior – The scrollbar should be automatically enabled upon loading longfile.html.

Test cases can be determined based on the specification, reviewing similar existing systems, or comparing to the past behavior of the SUT.

A more elaborate test case can have other details such as those given below.

  • A unique identifier : e.g. TC0034-a
  • A descriptive name: e.g. vertical scrollbar activation for long web pages
  • Objectives: e.g. to check whether the vertical scrollbar is correctly activated when a long web page is loaded to the browser
  • Classification information: e.g. priority - medium, category - UI features
  • Cleanup, if any: e.g. empty the browser cache.

For each test case we do the following:

  1. Feed the input to the SUT
  2. Observe the actual output
  3. Compare actual output with the expected output

A test case failure is a mismatch between the expected behavior and the actual behavior. A failure is caused by a defect (or a bug).

📦 Example: In the browser example above, a test case failure is implied if the scrollbar remains disabled after loading ‘longfile.html’. The defect/bug causing that failure could be an uninitialized variable.

Here is another definition of testing:

Software testing consists of the dynamic verification that a program provides expected behaviors on a finite set of test cases, suitably selected from the usually infinite execution domain. -– source: Software Engineering Book of Knowledge V3

Some things to note (indicated by keywords in the above definition):

  • Dynamic: Testing involves executing the software. It is not by examining the code statically.
  • Finite: In most non-trivial cases there are potentially infinite test scenarios but resource constraints dictate that we can test only a finite number of scenarios.
  • Selected: In most cases it is not possible to test all scenarios. That means we need to select what scenarios to test.
  • Expected: Testing requires some knowledge of how the software is expected to behave.

Explain how the concepts of testing, test case, test failure, and defect are related to each other.