Smock Testing VS. Sanity Testing

SMOKE TESTING:

  • Smoke testing originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch fire and smoke. In software industry, smoke testing is a shallow and wide approach whereby all areas of the application without getting into too deep, is tested.
  • A smoke test is scripted, either using a written set of tests or an automated test
  • A Smoke test is designed to touch every part of the application in a cursory way. It’s shallow and wide.
  • Smoke testing is conducted to ensure whether the most crucial functions of a program are working, but not bothering with finer details. (Such as build verification).
  • Smoke testing is normal health check up to a build of an application before taking it to testing in depth.

So, smoke testing is designed to test new feathers of a system, it is wide but not deep. By doing this, we can decide whether we need to do further testing.

For example, if a tester is asked to find out all the impurities of a river. Smoke testing is just for finding out all the impurities at the surface of the river, widely but not deeply.

SANITY TESTING:

  • A sanity test is a narrow regression test that focuses on one or a few areas of functionality. Sanity testing is usually narrow and deep.
  • A sanity test is usually unscripted.
  • A Sanity test is used to determine a small section of the application is still working after a minor change.
  • Sanity testing is a cursory testing, it is performed whenever a cursory testing is sufficient to prove the application is functioning according to specifications. This level of testing is a subset of regression testing.
  • Sanity testing is to verify whether requirements are met or not, checking all features breadth-first.

Sanity is narrow but quite deep. It focuses on a particular or some particular feathers.

For example, find out all the impurities of the river, but only for a particular area, say like within 30 mile radius.

(Source: http://www.softwaretestinghelp.com/smoke-testing-and-sanity-testing-difference)

Black Box Testing

What is Black Box Testing?

Unlike White Box Testing, which is focusing on the code or syntax stuff, Black Box Testing is focusing on functional requirements, and it compliments White Box Testing. it only concerns with testing the specification, and cannot guarantee that all parts of the implementation have been tested.

Thus black box testing is testing against the specification and will discover faults of omission, indicating that part of the specification has not been fulfilled. White box testing is testing against the implementation and will discover faults of commission, indicating that part of the implementation is faulty. In order to fully test a software product, both black and white box testing are required.

Black box testing attempts to find out: (1) incorrect or missing functions, (2) interface errors, (3) errors in data structures or external database access, (4) performance errors, and (5) initialisation and termination errors.

 

Advantages of black box testing?

  • The test will be unbiased because the developer and tester are independent of each other;
  • The tester does not need to know deeply the specific programming languages;
  • The test can be done from the perspective of end user, instead of developer.

 

Disadvantages of black box testing?

  • The test can be redundant if the developer has already run a test case;
  • The test cases are sometimes difficult to design;
  • Testing every possible input stream is unrealistic because it would take an inordinate amount of time. Therefore, many program paths will go untested.
  • Only test the performance or functions, sometimes the errors exist in the code cannot be found. Maybe will cause further problems which cannot be disclosed now.

 

Black Box Testing methods:

  1. Equivalence Partitioning
  2. Boundary Value Analysis.
  3. Cause Effect Graphing Techniques.
  4. Comparison Testing

 

Equivalence Partitioning:

is a typical black box testing technique which divides the input domain of a program into classes of data from which test cases can be derived. Equivalence partitioning method strives to define a test case that uncovers classes of errors, thereby reducing the total number of test cases that must be developed.

  • equivalence partitions are <10,000, 10,000-99,999 and >100, 000
  • Test cases would be selected from each equivalence class i.e. 00000, 09999, 10000, 99999, 100001

Choose the most representative one for each set.

Boundary Value Analysis:

is a black box testing method that where test cases are designed to test the boundary of an input domain. Studies have shown that more errors happen on the “boundary” of an input domain rather than on the “center area”.

So here, we can select the minimum and maximum values of an input and output range to test.

Example?

  • 10,000, 99,999,100,000

Cause Effect graphing Techniques

The basic idea of this method is that, a “cause” if an input condition, and an “effect” is a specific sequence of computations to be performed. A cause-effect graph is basically a directed graph that describes the logical combinations of causes and their relationship to the effects to be produced. It can help testers to select test cases to chekc if the program will produce right effect for every possible combination of causes.

Cause Effect Graph

(source: https://www.tutorialspoint.com/software_testing_dictionary/cause_effect_graph.htm)

Why we are using cause-effect graph/When should we use/What circumstances?

  • To identify the possible root causes, the rease for a specific effect, problem, or outcomes;
  • To relate the interaction of the system among the factors affecting a particular process of effect;
  • To analyse the existing problems so that corrective action can be taken at the earliest.

Benefit:

  • It Helps us to determine the root causes of a problem or quality using a structured approach.
  • It Uses an orderly, easy-to-read format to diagram cause-and-effect relationships.
  • It Indicates possible causes of variation in a process.
  • It Identifies areas, where data should be collected for further study.
  • It Encourages team participation and utilizes the team knowledge of the process.
  • It Increases knowledge of the process by helping everyone to learn more about the factors at work and how they relate.

Comparison Testing:

which comprises of comparing the content of multiple objects, e.g. files, databases, against actual results.

  • One can easily detect the underlying weakness and strengths of the software
  • It helps to realize the design structure of competing products as well as allows you to use them as a benchmark for future enhancements and improvements

 

(Some good sources and articles about black box testing and test cases writing:

http://www.cnblogs.com/Jackc/archive/2009/02/24/1397433.html

http://agile.csc.ncsu.edu/SEMaterials/BlackBox.pdf)

Some common challenges faced by software testers in general

 

Can you list few common challenges faced by software testers in general?

Well, it is a good question.

First of all, one of the challenges a tester will be faced is that how to build a good relationship with developers. For example, how you are going to tell developers that their job has not been completed properly? Or full of bugs? It requires skilled testers to handle this relation positively and even by completing the work in testers way. It always to see some disagreement on some points between the developer and testers. So how to express my self, how to describe a particular defect appropriately, how to discuss with developers, all of these are quite challengeable.

Second, understanding the requirements. When a tester cannot understand clearly or in the right way about what the system is proposed to do, what kinds of functions it should provide, etc., it can lead to a really bad and serious situation. So I think testers require a good listening and understanding capabilities.

Also, when facing a time constraint, we also need to decide how to decide which test case should be executed firstly and which should not be executed until the very last period? So a tester should define properly the priority and severity of each test case and defect, to ensure that the quality of a software system is maximised or optimised.

What’s more, it is not possible to test the whole application/software system. There are millions of test combinations. So if a tester is trying to test all these combinations he/she will never ship the product. Remember that there is no perfect quality.

(A good source here: http://www.softwaretestinghelp.com/manual-and-automation-testing-challenges/)

(Also if want to read more about the challenges of agile testing: http://blog.smartbear.com/sqc/top-5-common-challenges-for-agile-testing-teams/)

Root Cause Analysis (RCA)

How do you analyse the root cause for a defect?

As a software tester, we can use Root Cause Analysis.

Root Cause Analysis (RCA) is an approach used in software testing to analyse and identify the root causes of defects and address them, instead of treating the symptoms. If some problems happen, instead of just fixing it, we can also investigate and try to fix the underlying cause so that this kind of problems will be prevented from the next or further phases.

How to do the RCA?

Five whys technique: whose goal is to determine the root cause of a defect or problem by repeating the question “Why”. Each “why” forms the basis of the next question. For example:

6How to do RCA?
Why we've received an issue?
Why Save button is not working on users page?
Why does specific symbol affect...

(Source: https://www.slideshare.net/DavidGevorgyan1/root-cause-analysis-of-defects)

The answer to these questions will give us the exact phase, where the defect actually exists. Then, we can come up with “What” question:

What will you do to avoid this in future?

So now we can prevent the same defect to arise again by implementing the answer of this question.

Why doing this?

Doing the RCA accurately helps to prevent defects in the later releases or phases.

Examples?

Problem Statement: You are on your way home from work and your car stops in the middle of the road.

1. Why did your car stop?
– Because it ran out of gas.

2. Why did it run out of gas?
– Because I didn’t buy any gas on my way to work.

3. Why didn’t you buy any gas this morning?
– Because I didn’t have any money.

4. Why didn’t you have any money?
– Because I lost it all last night in a poker game.

5. Why did you lose your money in last night’s poker game?
– Because I’m not very good at “bluffing” when I don’t have a good hand.

(Source: https://www.isixsigma.com/tools-templates/cause-effect/determine-root-cause-5-whys/)

When do you do impact analysis and how does it help improve the quality of software developed

When do you do impact analysis and how does it help improve the quality of software developed

Impact Analysis:

Impact analysis is basically analysing the impact of the changes in the deployed software product or system. It tells us about the parts of the system that may be unintentionally affected because of the changes in the application and therefore need careful regression testing.

The decision is taken together with the stakeholders.

Test Case Optimisation

How do you optimize test cases?

Having minimum ongoing test case will lead to increased efficiency and amplified savings.

Choose the most representative ones, ensure that this particular test case uncovers a class of errors that might otherwise require many cases to be executed before the general error is observed.

Can use techniques of black box testing to select and optimise test cases.