Functional testing
Application functional testing is the most widely accepted testing practice in software development organizations today, and hundreds of millions of dollars have been spent on automated tools to support the practice. Yet, organizations still struggle to achieve software quality and suffer inconsistent, inefficient, and often inaccurate application of the practice. Over 70 percent of software testing today is still done manually despite the investment in automation. Understanding what has or hasn't been tested is still most organizations' major challenge, and the time commitment required for QA functional testing is increasingly one of organizations' greatest concerns for meeting application delivery and time-to-market demands.
Without a well-defined and implemented "best" practice, functional testing will continue to be practiced incorrectly and suffer inaccurate or, at best, incomplete results. Automated tools alone cannot ensure that functional testing is complete and accurate. Achieving this goal requires that a tool be supported by a well-defined practice for its use and be integrated into an organization's software development process. The necessary level of commitment and practice maintenance is often underestimated and is typically the reason for poor results and the abandonment of automated tools in favor of manual testing. An effective and consistently-applied functional testing practice is the only way to ensure that software satisfies expected functionality and reliability objectives.
Software development organizations with an effective functional testing practice have a fast and objective way to determine whether each functional requirement is actually implemented in the code. With functional testing, the team translates functional requirements into executable test cases that confirm how well the code satisfies the requirements at any given time. It provides unprecedented objective insight into requirement status and prevents the missing or incorrect functionality implementations that can lead to countless rewrites (and then budget overruns and missed deadlines), user dissatisfaction, and project failure.
An effective functional testing practice should be a natural extension of a requirements management policy. Ideally, a requirements management policy describes how to build effective use cases for the features expected in the current release, and how to map those use cases to verifiable test cases. A test plan is then developed and implemented to build a suite of executable tests that frame and verify the functionality requirements, providing a fast and objective way to assess the status of expected functionality. These tests can then be executed regularly to ensure that code modifications do not unintentionally change previously verified functionality.
An effective functional testing practice involves the definition of guidelines for using functional testing technologies effectively, and then the implementation and integration of those guidelines (along with supporting technologies and configurations) into your software development lifecycle to ensure that your teams apply the policy consistently and regularly. It also requires a means to monitor and measure the policy's application, as well as report the data it is tracking.
UAT
The user acceptance testing is usually a black box type of testing. In other words, the focus is on the functionality and the usability of the application rather than the technical aspects. It is generally assumed that the application would have already undergone Unit, Integration and System Level Testing. UAT is useful if it is carried out in an environment that closely resembles the real world or production environment.
The steps taken for User Acceptance
1) User Acceptance Test (UAT) Planning
2) Designing UA Test Cases
3) Selecting a Team that would execute the (UAT) Test Cases
4) Executing Test Cases
5) Documenting the Defects found during UAT
6) Resolving the issues/Bug Fixing
7) Sign Off
User Acceptance Test (UAT) Planning:
As always the Planning Process is the most important of all the steps. This affects the effectiveness of the Testing Process. The Planning process outlines the User Acceptance Testing Strategy. It also describes the key focus areas, entry and exit criteria.
Designing UA Test Cases:
The User Acceptance Test Cases help the Test Execution Team to test the application thoroughly. This also helps ensure that the UA Testing provides sufficient coverage of all the scenarios. The Use Cases created during the Requirements definition phase may be used as inputs for creating Test Cases. The inputs from Business Analysts and Subject Matter Experts are also used for creating. Each User Acceptance Test Case describes in a simple language the precise steps to be taken to test something.
Selecting a Team that would execute the (UAT) Test Cases:
- Selecting a Team that would execute the UAT Test Cases is an important step.
- The UAT Team is generally a good representation of the real world end users.
- The Team thus comprises of the actual end users who will be using the application.
Executing Test Cases:
The Testing Team executes the Test Cases and may additional perform random Tests relevant to them
Documenting the Defects found during UAT:
- The Team logs their comments and any defects or issues found during testing.
- Resolving the issues/Bug Fixing
- The issues/defects found during Testing are discussed with the Project Team, Subject Matter Experts and Business Analysts. The issues are resolved as per the mutual consensus and to the satisfaction of the end users.
Sign Off
Upon successful completion of the User Acceptance Testing and resolution of the issues the team generally indicates the acceptance of the application. This step is important in commercial software sales. Once the User “Accept” the Software delivered they indicate that the software meets their requirements.
Usability testing
Usability testing evaluates how easy a system is to learn and use and it offers many benefits to both users and development companies. Benefits to users include increased pace of learning, decreased user task time and errors and increased jobsatisfaction. Development companies benefit through providing less technical support and training to users, improved management of risks associated with usability rework as well as improved user acceptance. Companies also develop software faster and more cheaply since only relevant functionality is produced and they can identify which designs are effective and which to avoid Benefits of web usability include increased sales, increased traffic and improved user productivity and features usage.
Cost-Benefit Value of Usability Testing
The basic assumption of a cost-benefit analysis of usability strategies is that a usable system will result in tangible, measurable benefits. There are many different areas where usability strategies can have cost-benefitvalue. Four of these are discussed in the forthcoming sections:
- Development – how usability results in optimization of the development process
- Technical Support – how usability results in technical support savings
- Sales – how usability increases product sales
- User – how usability positively impacts user performance and satisfaction
Development Benefits
Reduced development time : A user-centered design approach ensures that the needs of users are built into the product development cycle. This, in turn, ensures that only relevant functionality is designed into a product – no more, no less. The user-centered design approach combats the common industry misconception that “the more features a product has, the better”. An explosion of features in a system can be highly detrimental because extra features can clutter an interface and make it more difficult to find useful features. Features are also expensive to develop and, if they are not used, developing them becomes futile.
Reduced rework: If usability testing is done early in the design life cycle, changes may be made to eliminate problems and achieve benefits much more cheaply because code has not been written. Thus, changing and fixing usability problems early in the development life cycle can substantially reduce rework costs later on.
Technical Support Benefits
- Decreased user training: Many organizations spend large amounts of money training users on new software systems. When usable systems are designed, they become more intuitive and users require less training on the system before can they use it.
- Decreased help-desk calls and on-site technical support: Usability decreases the burden on a company’s help-desk and on-site technical support staff because users do not require assistance addressing usability problems.
- Decreased unofficial technical support: Users often choose to disturb non-technical colleagues to help them solve usability problems. When a system is more usable, these disturbances are less likely to occur.
Sales Benefits
- Competitive edge of marketing usable products: Usable software can accrue increased revenues from external customers due to increased marketability. An area where this is particularly critical is that of ecommerce. On the Web, instead of buying software first and then experiencing its level of usability later, customers encounter usability first and then decide whether to pay or not. In addition, in e-commerce, the company’s website becomes the crucial primary interface to the customer.
- More satisfied customers: When users suffer frustration caused by unusable software, the company’s credibility is tarnished. In contrast, satisfied customers have brand loyalty and are much more likely to buy the same brand in the future with less researching of the particular product.
- Higher ratings for usability in the press: Usability is one of the main criteria used to critically rate software packages in commercial publications. A wide variety of trade magazines evaluate new software releases that are sent to market. According to Nielsen (1993), approximately 15% of reviews by trade journals is devoted to analyzing the “user-friendliness” of new software products.
User Benefits
- Reduced user task time: Optimizing user efficiency relates to cutting down unnecessary work-hours resulting in potentially large savings to companies. The benefits due to improved user efficiency are particularly evident for high-volume, repetitive tasks, such as data capturing.
- Reduced user errors: Users will not make as many errors on a usable system because they have a better understanding of the system. Typical error-prone situations can be “designed-out”. This, of course, relates closely to the previous benefit of improved user task time.
- Improved user satisfaction: This relates to higher levels of staff morale, which have been shown to reduce staff turnover and increase job motivation and company loyalty.
Performance testing
A performance test is a technical investigation done to determine or validate the speed, scalability, and/or stability characteristics of the product under test. Performance testing is the superset containing all other subcategories of performance testing as Stress test, Load Test, Endurance Test and etc.
Stress Test:
A stress test is a type of performance test designed to evaluate an application's behavior when it is pushed beyond normal or peak load conditions. The goal
of stress testing is to reveal application bugs that surface only under high load conditions. These bugs can include such things as synchronization issues,
race conditions, and memory leaks. Stress testing enables you to identify your application's weak points, and shows how the application behaves under extreme load conditions.
Endurance Test:
An endurance test is a type of performance test focused on determining or validating performance characteristics of the product under test when subjected to workload models and load volumes anticipated during production operations over an extended period of time. Endurance testing is a subset of load testing.
Load Test:
Load testing is conducted to verify that your application can meet your desired performance objectives; these performance objectives are often specified in a
service level agreement (SLA). A load test enables you to measure response times, throughput rates, and resource-utilization levels, and to identify your
application's breaking point, assuming that the breaking point occurs below the peak load condition.
Standard and Complaince Testing:
Complaince are basically standards that is globally accepted and need to adhere and validating the application with resspect to standards is called Complaince testing. for eg. 508 Complaince, SOX complaince and etc.
Configuration Testing:
Configuration testing is the system testing of different variations of an integrated, blackbox application against its configurability requirements.The objective of configuration testing are:
- Cause the application to fail to meet its configurability requirements so that the underlying defects can be identified, analyzed, fixed, and prevented in
the future.
Compatibility testing
Compatibility testing, part of software non-functional tests, is testing conducted on the application to evaluate the application's compatibility with the computing environment. Computing environment may contain some or all of the below mentioned elements:
- Computing capacity of Hardware Platform (IBM 360, HP 9000, etc.)
- Bandwidth handling capacity of networking hardware
- Compatibility of peripherals (Printer, DVD drive, etc.)
- Operating systems (MVS, UNIX, Windows, etc.)
- Database (Oracle, Sybase, DB2, etc.)
- Other System Software (Web server, networking/ messaging tool, etc.)
- Browser compatibility (Firefox, Netscape, Internet Explorer, Safari, etc.)
- Carrier compatibility (Verizon, Sprint, Orange, O2, AirTel, etc.)
- Backwards compatibility.
- Hardware (different phones)
- Different Compilers (compile the code correctly)
- Runs on multiple host/guest Emulators
Globalization Testing:
The goal of globalization testing is to detect potential problems in application design that could inhibit globalization. It makes sure that the code can handle all international support without breaking functionality that would cause either data loss or display problems. Globalization testing checks proper functionality of the product with any of the culture/locale settings using every type of international input possible.
Proper functionality of the product assumes both a stable component that works according to design specification, regardless of international environment settings or cultures/locales, and the correct representation of data.
The following must be part of your globalization-testing plan:
- Decide the priority of each component
- To make globalization testing more effective, assign a testing priority to all tested components. Components that should receive top priority:
- Support text data in the ANSI (American National Standards Institute) format
- Extensively handle strings (for example, components with many edit controls)
- Use files for data storage or data exchange (e.g., Windows metafiles, security configuration tools, and Web-based tools)
Localization Testiing:
Localization translates the product UI and occasionally changes some initial settings to make it suitable for another region. Localization testing checks the
quality of a product's localization for a particular target culture/locale. This test is based on the results of globalization testing, which verifies the functional support for that particular culture/locale. Localization testing can be executed only on the localized version of a product. Localizability testing does not test for localization quality.
The test effort during localization testing focuses on:
- Areas affected by localization, such as UI and content
- Culture/locale-specific, language-specific, and region-specific areas
- Improve SEO Rankings
- In addition, localization testing should include:
- Basic functionality tests Setup and upgrade tests run in the localized environment
- Plan application and hardware compatibility tests according to the product's target region.
