4 Tips to write test cases

It is important for tester to improve how to write test cases. So, I summarize the key things that I normally consider when I am writing test cases. From this article, I will start with the meaning and the required information for test cases. Then, I will explain the tips to write test cases.

What is the test case?

The test case is a sequence of steps to test the correct behavior of software by comparing with the expected result or expected outcome.

Required information in the test case

- Test case ID
- Component
- Test case description
- Steps to be executed
- Test data
- Expected result
- Pass/Fail

4 Tips to write test cases

1. Focus on requirement analysis
First of all, we should analyze what we are going to test. This activity occurs during development phase so we should focus in requirement analysis. We should keep in mind that we cannot write test cases well by using the poor requirement.

2. Make the test case for unfamiliar tester
We should write test cases which is easy to understand. Then, the tester who is unfamiliar with the application can execute the test case easily. Someone may think why he have to spend more time to write test cases for this purpose. Let's think what will happen if we have to assign tester from another team to help us testing but he cannot help because of our test cases.

3. Apply black box testing technique
We should always use black box testing technique to derive the effective test case.

4. Always cover negative flow
We should cover negative flow in test case because developer often make mistake with negative flow. From my experience, the problem from negative flow often cause big problem with software.

Read More

Guideline for ISTQB Certification Exam and Recommended ISTQB Book

If you are preparing for the ISTQB Certification Exam, here is guideline from me.

1. Reading document provided by ISTQB website.
You should start with the syllabus, glossary and sample ISTQB Certification Exam which are available in ISTQB website. The information in these documents will provide you the scope of software testing knowledge that will be used for ISTQB Certification Exam.

2. Learning from internet.
You can find more information from many forums and articles available in internet. However, you should be aware of the error of information if you need to prepare for ISTQB Certification Exam.

3. Reading ISTQB book.
I recommend you to use an ISTQB book and documents provided by ISTQB from above together. Using documents from ISTQB as a checklist and then, using the ISTQB book for more detail.

Here is the list of recommended ISTQB book for ISTQB Certification Exam. You can see more detail and review by follows the link.

	Foundations of Software Testing: ISTQB Certification A Guide To Passing The ISTQB Foundation Level Exam Foundations of Software Testing. Whether you are a students or tester, this book is an essential purchase if you want to benefit from the knowledge and experience of those involved in the writing of the ISTQB Syllabus.
	Advanced Software Testing - Vol. 1: Guide to the ISTQB Advanced Certification as an Advanced Test Analyst (Rockynook Computing)A Guide To Passing The ISTQB Advanced Level Test Analyst. This book is written for the test analyst who wants to achieve advanced skills in test analysis, design, and execution. With a hands-on, exercise-rich approach, this book teaches you how to define and carry out the tasks required to put a test strategy into action.
	Advanced Software Testing - Vol. 2: Guide to the Istqb Advanced Certification as an Advanced Test ManagerA Guide To Passing The ISTQB Advanced Level Test Manager. This book teaches test managers what they need to know to achieve advanced skills in test estimation, test planning, test monitoring, and test control. Readers will learn how to define the overall testing goals and strategies for the systems being tested. Target Audience: Software Testers, Programmers, System Designers, IT Managers.
	Advanced Software Testing - Vol. 3: Guide to the ISTQB Advanced Certification as an Advanced Technical Test AnalystA Guide To Passing The ISTQB Advanced Level Technical Test Analyst. This book is written for the technical test analyst who wants to achieve advanced skills in test analysis, design, and execution. With a hands-on, exercise-rich approach, this book teaches you how to define and carry out the tasks required to put a test strategy into action.

With my guideline for ISTQB Certification Exam, I hope that you will success in ISTQB Certification Exam. Good luck with your exam.

Read More

Software Testing Certification For Testing Tools

After you have visited Software Testing Certifications in previous article, you may would like to know the Software Testing Certification For Testing Tools as well. There are many Software Testing Certifications which are developed by tool vendors. To get the Software Testing Certification will help you to enhance the software testing knowledge in specific software testing tool.

Here is the list of available Software Testing Certification for testing tools:

1. HP
Certification: There are many software testing certifications available for testing tools from HP (e.g., Quick Test Professional (QTP), Quality Center (QC), Load Runner).
Related Document: HP Certifications

2. IBM
Certification: There are many software testing certifications available for testing tools from HP (e.g., IBM Solution Designer - Rational Performance Tester, IBM Certified Solution Designer - Rational Manual Tester).
Related Document: IBM Certifications

If you know more Software Testing Certification available, please let me know. Then, we can consolidate and have single point for all of available software testing certifications.

Read More

Software Testing Certification

There are many Software Testing Certifications available for QA and Software Tester which are developed by many organizations. To get the Software Testing Certification will help you to enhance the software testing knowledge in this field. I will use this article to maintain the list of Software Testing Certification.

Here is the list of available Software Testing Certification:

1. The International Software Testing Qualifications Board (ISTQB)
Certification: There are two levels of software testing certification: The Foundation Level certificate and the Advanced Level certificate.
Related Document: http://istqb.org/display/ISTQB/Downloads?atl_token=F5FhF5FHY6
Guideline: Guideline for ISTQB Certification Exam and Recommended ISTQB Book

2. International Software Certification Board (ISCB)
Certification: There are three levels of software testing certification for both Software Testers and Quality Assurance:
Software Testers - CAST -> CSTE -> CMST 
Software Quality Assurance - CASQ -> CSQA -> CMSQ

3. International Institute for Software Testing (IIST)
Certification: Certified Software Test Professional (CSTP) 

See more Software Testing Certifications in Software Testing Certifications For Testing Tools.

Read More

Localization and Globalization Testing

Globalization Testing

Globalization Testing is the software testing process for checking if the software can work properly in various culture/locale settings using every type of international input.

Localization Testing

Localization Testing in the software testing process for checking the localized version of a product (e.g., Japanese product for Japanese user). Localization Testing checks the quality of a localized version for a particular that culture/locale setting. Localization Testing is based on the results of Globalization Testing which verifies the functional support for that particular culture/locale setting. We should invite the local staff to help our localization testing by checking the quality of translation as well.

You can test on localized OS build (e.g.,Windows XP Japanese version) or MUI on English OS Build. I suggest you to start with MUI version for Globalization Testing because MUI can help you switch to other language settings easier and faster. From my software testing experience, most bugs occur with German and Japanese culture/locale setting.

Here are the common bugs found from Testing
Cannot display the correct format
The software cannot display the format as defined by the regional setting of OS (e.g., numbers, dates, time, currency, and calendars). I have found that developers often display the static format as English culture/locale setting although it is international software.

Functionality is broken
The functionality which works properly on English setting may not work on other settings. I have found that one root cause of this problem is the communication problem between the API's owner and consumer. They forgot to make an agreement for the data format; one party uses the local format but another uses the English format.

Now we understand the importance of Localization Testing and Globalization Testing and the risk if we don't have this type of software testing. So, it is good idea to implement this testing type for your international software.

Read More

Static Testing and Dynamic Testing

The difference between Static Testing and Dynamic Testing is as below.

Static Testing

Static Testing is a software testing method that software is not really executed. It normally checks defects for the deviations from standards, missing requirements, design defects, test plan, non-maintainable code and inconsistent interface specifications.

Dynamic Testing

Dynamic Testing is a software testing method that software is really executed. Software is executed using a set of input values and its output is compared to the expectation.

Read More

The Risk From Roaming User Profile. One Of Tester's Checklist For Software Testing.

From my software testing experience, I have got many issues related to the Roaming User Profile from customers. So, I think this issue is important enough to consider when we start doing the software testing.

Roaming User Profile

First of all, I would like to introduce the term of Roaming User Profile. Roaming User Profile is a system process that allows a user can access his documents and has a consistent desktop experience on any computer. To enable this feature these computers have to join the Active Directory. The key concept is that a group of files for a user will be transferred to the machine that user is logging in.

What is its risk?

As I have said, user profile's files will bind to user but not bind to machine. So, if developers store any software configuration in a machine, we should ensure that there is no any Configuration File which tries to refer any item in a machine cause the problem. From my software testing experience, the possible problem can be:

1. Configuration File in user profile refers to an installation path of software. It can cause the problem because the installation path in any machine can be customized by user - C:\Program Files in the first machine, D:\Program Files in the second machine, for example.

2. Configuration File in user profile refers to version of software. It can cause the problem because there is no guarantee that all of the machines will be upgraded to the same version at the same time.

From this example, I think testers should consider User Roaming Profile as one checklist when execute software testing. The suggested solution for this issue is to set up Roaming User Profile as one test environment when do any software testing.

Read More

Verification and Validation

Verification

Verification is the process used for evaluating the system if it meets the requirements set. It focuses on the question "Are you building it right?"

Validation

Validation is the process used for evaluating the system if it meets the user needs. It focuses on the question "Are you building the right thing?"

Read More

Process When Testing A Software

Here are basic activities of testing process that I normally use when I need to execute testing a software. The process will start from planning and end with evaluation the process.

1. Test Planning and Control

First of all, we set the goal and scope of testing when we plan to do testing a software. This goal is based on the objective of the project. We define approach and plan for software testing. Then, we define the test approach which includes coverage, test resources, schedule of all the tasks, exit criteria and so on. We will control and measure project's progress against the plan continuously.

2. Test Analysis and Design

After we have a plan from the first step, we start building test design and test procedure. Evaluate the requirement and system. Identify test condition, required test environment and tools for software testing.

3. Execution

We build the test cases by using Black Box Testing Techniques and set up the test environment and tools for software testing. The test cases will be reviewed if it is required. Then, we execute software testing and recording the test result. If there are difference between expected result and actual result, we will analyze them to see the root cause of defect and record them.

4. Evaluating and Reporting

We use the exit criteria to check if software testing can be done. If it is done, we can make a summary report for stakeholders.

5. Project Evaluation

Evaluate and analyze the lesson from this project. Implement action plan if it is need for continuous improvement. Please keep in mind that we always need to evaluate when we do testing a software.

Read More

Black Box Testing

Black Box Testing is a software testing method which testers use when they have no idea about software's internal algorithm. Testers only know the input and its expected output from the software which is based on specification.

Test Design Techniques

Test Design Techniques are used for test case design. They include:
- Equivalence Partitioning
- Boundary Value Analysis
- Pairwise Testing (All-pairs testing)
- Decision Table
- Error Guessing

Advantages of Black Box Testing

- Test Cases can be designed as soon as specifications are ready.
- Programming skill is not required. Business user can involve easily.
- Help to identify ambiguity and contradiction in specifications.

Disadvantages of Black Box Testing

- It is hard to identify all possible cases.
- There is possibility of duplicated test which have been done by developer.

Tools for Black Box Testing

Test Design Tools
- CTE XL

Automation Tools for Regression Test
- QTP
-----------------------------------------------------------------

Related topic: White Box vs Black Box Testing

Read More

Is it necessary to have Integration Testing Phase?

Since the Integration testing can be done in several levels, I will choose one level as example for this article. The Integration testing that I choose is the software testing for interaction between 2 systems which occurs after System Testing Phase. However, the idea from this article can be applied to other level of Integration Testing Phase as well.


"Is Integration Testing Phase necessary?" This question may be in some testers' mind after they need to do the same testing as System Testing Phase. My answer of this always is "Yes, Of Course". Let see what I have experienced.

I have assigned to do website testing and this website consumes data from various data sources. There are plans to release new version of many data sources on the same date as my website.

I have done my website testing in System Testing Phase and the result looks good. No pending bug from my website testing. Then, I wait until start date of the Integration Testing Phase which every data sources are ready. Shock!!! My website is broken when I do website testing on Integration Testing Phase. What's wrong with my test result from prior website testing?

The issue has been raised to developers for investigation and they take one week to investigate the issue. The problem comes from the new version of one data source. Its version in website testing's report from system test environment is older than integration test.
--------------------------------------------------------------------------

From this lesson, I always think about the dependency of my system. Then, manage the risk from them. Finally, do the software testing as soon as possible if the Integration Testing Phase is ready.

Read More

Software Testing in Requirement Analysis

I have found that many bugs are introduced during Requirement Phase but testers can detect them during do software testing. If they can detect bugs in Requirement Phase, the cost for fixing them will be cheaper. So, it is worth to remove them during this phase. Here is some activity that tester can involve.

Keep in mind that the requirement must be clear

The good requirement must be clear, measurable, testable and defined enough for system design. Moreover, the requirement must be fulfilled by using client's information. Don't try to answer the query ourselves with any assumption. If they are not clear, we should not let it go to the next development phase.

Using software testing skill to explore the missing cases in requirement

Let using testing design skill for requirement analysis by thinking about the possible cases for the requirement. Then, try to summarize the list of question for client. Again!!! Don't try to answer the query ourselves.

Don't limit the first idea by technical constraint

The first idea for detail of requirement often has been limited by technical constraint. From my view, we should drive the requirement by business need first. The requirement adjustment because of technical constraint should come later.

Don't shy to ask any question

Tester is one of the project member. So, we should not shy to ask any question or raise any concern. Our question or concern may be the significant issue for the requirement analysis.
----------------------------------------------------------------------------

Tester is one of the project member. We can improve the quality of project by contributing our value since the Requirement Analysis activity.

Read More

Development Model : V Model For Testing

V Model is a development model which demonstrates the relationship between each phase of development life cycle and related phase of software testing. The left side of the V Model consists of development phase and the right side of the V Model consists of software testing phase. The flow of V Model will be start at Requirement phase and end at Acceptance Testing phase. 

Testing document in a software testing phase will be derived from the document in development phase that stays at the same level (e.g., Acceptance Testing and Requirement Phase, System Testing and Specification Phase). So, tester can start working on software testing activity as soon as the development activity starts.


The benefit from early software testing activity is that we can detect some faults early. As a result of early fault detection, the cost for fixing will be less.

Read More

Black Box Testing Technique : Error Guessing

Black Box Testing Technique

If I would like to do software testing, I start with thinking about how to select the test cases. If I have no idea about software's internal algorithm and cannot cover all combinations of input values and preconditions for a module of software because of resource constraint (Time or manpower), I will use Black Box Testing Technique to design the test cases.

Error Guessing

Error Guessing is a software testing technique that generates test cases which can find the error by using past experience of tester. The key concept is using the prior software testing experience to supplement other software testing techniques. 

The known error in any software is varied so using input from experienced tester for that software is a good idea. From my software testing experience, the typical errors are related to "divide by zero" and "various date and time format".

Read More

Black Box Testing Technique : Decision Table Testing

Black Box Testing Technique

If I would like to execute testing a software (testing web or windows application), I start with thinking about how to select the test cases. If I have no idea about software's internal algorithm and cannot cover all combinations of input values and preconditions for a module of software because of resource constraint (Time or manpower), I will use Black Box Testing Technique to design the input values.

Decision Table Testing

Decision Table Testing is a software testing technique that tries to create the complete test cases by using decision table. This decision table will contain all possible values of each input and output. Then, we mark each possible scenario to that decision table.

Read More

How To Design Test Cases for testing the correctness of Calculation

I have found that many testers feel afraid when they are assigned to do software testing with the calculation module. Here is some guideline that will help you do software testing better.

Learn how it calculates

It will be good if you understand what you will test. With this information, you can design the test strategy for software testing properly.

Apply Black Box testing technique

Apply Black Box testing technique; Equivalence partitioning, Boundary value analysis and Pairwise testing (All-pairs testing); for test cases design will improve the coverage of software testing.

We can use the equivalence partitioning testing technique to partition the possible cases. Don't forget to put the exception case from error guessing to the software testing scope as well (e.g., Divide by Zero case). From my experience, many bugs can be detected by testing the exception case.

The Boundary value analysis will help you to select the test case for checking also. Pairwise testing can be used for testing the formula that contains many input variables (e.g.,"=Floor(NUMBER,SIGNIFICANCE)" in MS Excel ).

Don't try to calculate manually for comparing the result

You should not calculate manually for comparison because it often contains the human error. Using the calculation from another program, e.g., MS Excel, is better because it does not contain human error. If the result is different, one of them should be wrong. So, the next step is to investigate this issue.

---------------------------------------------------------------
With this guideline, I hope that you will feel more confident when do software testing for any calculation.

Read More

No resource for testing everything !!!

Most issue that QA or tester faces is that the limitation of resource (time and manpower) for software testing. With this constraint, we need to decide what is the proper software testing scope and find the way to execute testing a software faster.

Prioritize software testing scope

The first thing that we need to do is to prioritize the software testing scope. We can use the software feature's  importance and likelihood / impact of failure to prioritize. Then, make a commitment about the software testing scope with stakeholder

Reduce the test cases

After we have the list of software testing scope, we can reduce the number of test cases by designing the test cases for software testing. This can be done by the Black Box testing technique; Equivalence partitioning, Boundary value analysis and Pairwise testing (All-pairs testing).

Use the software testing tools

There are many software testing tools available for using in many activities in software testing. Using them to make the software testing activities done faster and easier. For example, CTE XL tool for generating test cases and Quick Test Pro for Test Automation.

This is the basic guideline that can help you to do software testing activities when you would like to do this tasks faster.

Read More

Black Box Testing Technique : Pairwise Testing (All-pairs testing) - Part3

Pairwise Testing (All-pairs testing) 

After we have installed CTE XL which is one of software testing tools for Pairwise Testing (All-pairs testing), we will start using it with following scenario.


CTE XL: How to use

I will use this following scenario for test case design in this example.

Scenario

System: Fund Transfer feature in ATM system of A Bank with 3 dependent factors:

1st factor: How to transfer

- A Bank to A Bank

- A Bank to B Bank
2nd factor: Amount of money
- Less than $1.00
- Between $1.00 and $5,000
- More than $5,000
3rd factor: SMS alert?
- Yes
- No

Step

1. Start CTE XL.

Read More

Black Box Testing Technique : Pairwise Testing (All-pairs testing) - Part2

Pairwise Testing (All-pairs testing) 

After you have a clear view in Pairwise Testing (All-pairs testing) technique, I will introduce one of the software testing tools that can help you to design test case by this software testing technique. The software testing tool's name is CTE XL by Berner & Mattner. You can find more information of this software testing tool in link.

CTE XL Set Up Process

1. Click here to open Download Page.

Read More

Black Box Testing Technique : Pairwise Testing (All-pairs testing) - Part1

Black Box Testing Technique

If I would like to execute software testing (web testing or windows application testing), I start with thinking about how to select the test cases. If I have no idea about software's internal algorithm and cannot cover all combinations of input values and preconditions for a module of software because of resource constraint (Time or manpower), I will use Black Box Testing Technique to design the input values.

Pairwise Testing (All-pairs testing) 

Pairwise Testing (All-pairs testing) is a software testing technique that designs test cases for testing all possible combinations of each pair of input parameters. The number of this test cases will be less than testing all possible combinations. You can see more detail in the Part3 of articles. To get the list of test cases for software testing we need to use some algorithm to extract them.

The key idea of this technique for software testing is "the most bugs in a software are introduced by a single input parameter. The next one are introduced by interaction between pairs of input parameters.".

If you have found that many bugs in your software are introduced by two factors and would like to design test cases for software testing by using this condition, Pairwise Testing (All-pairs testing) will be the answer.

Sample Bug: Bug in Fund Transfer feature in ATM system is introduced by 2 conditions:

1st condition: From A Bank to B Bank

2nd condition: More than $5,000

---------------------------------------------------------------------------------

Now you understand the concept of testing a software (web testing or windows application testing) by using Pairwise Testing (All-pairs testing) testing technique.

Since this software testing technique is hard to do manually, I will share you how to create test cases by tool in the next articles.

---------------------------------------------------------------

Related topic of Pairwise Testing (All-pairs testing) 

Basic concept: Black Box Testing Technique : Pairwise Testing (All-pairs testing) - Part1
How to install test tool: Black Box Testing Technique : Pairwise Testing (All-pairs testing) - Part2
How to use test tool: Black Box Testing Technique : Pairwise Testing (All-pairs testing) - Part3

Read More

Black Box Testing Technique : Boundary value analysis

Black Box Testing Technique

If I would like to execute testing a software (testing web or windows application), I start with thinking about how to select the test cases. If I have no idea about software's internal algorithm and cannot cover all combinations of input values and preconditions for a module of software because of resource constraint (Time or manpower), I will use Black Box Testing Technique to design the input values.

Boundary value analysis

Boundary value analysis is a software testing technique that selects the input data which represents boundary values. These data includes both boundary and each side of boundary which should be in the smallest increment. 

Example
I would like to test withdrawal on ATM system. The maximum amount is $500 and minimum amount is $1.

Find boundary value:

1. Maximum value is $500
2. Minimum value is $1

Find each side of boundary value

1. $501 and $499 for "Maximum value is $500" case
2. $2 and $0 for "Maximum value is $1" case

The final list of input values are 

1. $501, $499 and $500
2. $2, $0 and $1

----------------------------------------------------------

Now you can design and execute testing a software (testing web or windows application) by using Boundary value analysis testing technique. 

Let's find the pairwise testing (All-pairs testing) technique in the next article.

Read More

Black Box Testing Technique : Equivalence partitioning

Black Box Testing Technique

If I would like to execute testing a software (web testing or windows application testing), I start with thinking about how to select the test cases. Then, if I have no idea about software's internal algorithm and cannot cover all combinations of input values and preconditions for a module of software because of resource constraint (Time or manpower), I will use Black Box Testing Technique to design the input values for this software testing.

Equivalence partitioning

Equivalence partitioning is a software testing technique that groups the input data for a module of a software into partitions of data that can represent each possible case. Then, select an input data from each partition.

Example
I would like to execute software testing with withdrawal module on ATM system. The maximum amount is $500 and minimum amount is $1.

Partitioning:

1. More than $500
2. Between $500 and $1
3. Less than $1

Select input data:

1. $800 for "More than $500" case
2. $200 for "Between $500 and $1" case
3. $0 for "Less than $1" case

----------------------------------------------------------

Now you can design and execute testing a software (web testing or windows application testing) by using Equivalence partitioning testing technique. 

Let's find the Boundary value analysis testing technique for software testing in the next article.

Read More

Methods of software testing: White Box vs Black Box Testing

White Box vs Black Box Testing

When someone is assigned to execute software testing (Both web testing and windows application testing), they sometimes have question for me about the type of methods of software testing, my answer usually is "2 types the White Box and Black Box Testing". Then, the next question will be "What is different?". Here is the answer...

	White Box	Black Box
Require knowledge of software's internal algorithm?	Yes	No
Sample of Test design technique	Path testing Branch testing	Equivalence partitioning Boundary value analysis Pairwise testing (All-pairs testing
Software Testing Phase	Unit Testing	Integration/System/Acceptance Testing

Each methods of software testing has different purpose and are suitable for different testing phase. I will focus only on the Black Box method for this blog.

The Black Box method is suitable for tester who would like to design testing but has no idea about software's internal algorithm. However, they know only the input and output of this software. If they have to select input for execute software testing, the test design technique will help them. There are some software testing tools for design which make our life easier also.

I will explain the detail of Test design technique of Black Box method in the next articles.

Read More

The earlier a bug is found, the cheaper it costs

This is the key message showing why we need to do software testing activity as early as we can.

Let's find the reason by starting with following SDLC model (V Model). It shows us that there are many phases during implementation. It starts from Requirement Phase and ends with Acceptance Testing Phase.

If a problem from Requirements phase is found during System Testing phase, then it would cost more to fix than if it had been found during the Requirements phase. The reason is that we need to rework everything again instead of just changing the detail of requirement.

From my experience, most bugs which are found during software testing can be removed at the requirement phase. So, we should spend more effort at the beginning of project. Don't wait until software testing phase which is too late.

Read More

What is Software Testing? Why do we need it?

What is Software Testing?

Software Testing is the process of checking if the software can meet the user's expectation. This process can be used for testing a software for both web testing and windows application testing.

Why Software Testing?

Without Software Testing Process,

1) The software has poor quality.

2) The software may contain the critical error or bug.

3) The cost of error or bug may be too high. 

Why do I say that "The cost of error or bug may be too high."?  Let's see the sample of high cost bug: A hospital uses the computer-assisted surgery technology and we don't have the proper software testing activity in SDLC during implementing this tool. Unfortunately. this tool contains a bug that can cause the problem with patient's safety. Let's think what will happen next.

Read More