Domain Testing--Perspective (Part 2/3) (BBST)

Risk-Based Software Testing

ISTQB certification

Content of the exams

The exam for the Foundation Level has more theoretical nature and requests for knowledge about the software development area - especially in the field of testing.

The different Advanced Level exams are more practical and they deepen the gained knowledge in special areas. Test Manager deal with the planning and control of the test process. Functional Tester concern themselves among other things with reviews and Black box testing methods. Technical Tester look into the subject component tests (also called unit test), where they use inter alia White box testing methods and non-functional testing methods and also includes test tools. The Expert Level is still in preparation.

Premises

The exam Foundation Level has no premises. To take the Advanced Level exam, candidates need to pass the Foundation Level exam first and must prove having at least 60 months of professional experience (in USA). In India 24 months and Germany it is 18 months.

Exam

The exam consists of a multiple-choice test, which until 2005 about 80% of the candidates passed. A reason for this can be, that people can learn by self-study without visiting preparation seminars, because the syllabus is available for free in the internet.

In USA pure exam fees for Foundation Level are $250.00. For Advanced Level it is for each component exam US $200.00 and there is a one-time qualification fee of US $100.00 (date: september 2006).

Certificates are valid for life time (Foundation Level and Advanced Level) and do not need recertification.

So called testing boards are responsible for the quality and the auditing. Worldwide there are in 33 countries testing boards (date: april 2007). In USA the corresponding organ is the ASTQB (American Software Testing Qualifications Board).

Register for Exams

Learn more about ISTQB certification and exams in your country by clicking here [1].

Figures about Certified Testers worldwide

At the moment there are over 60000 ISTQB Certified Testers worldwide. One reason for this is, that organizations which offer also certifications for software testers affiliate with ISTQB (e.g. ISEB (Information Systems Examination Board)). At the moment Germany, India and USA are leading in the real Certified Tester programs.
The QAI (Quality Assurance Institute) is another organization, which offers in more than 40 countries certificates.

Six Sigma Black Belt Certification - SSBB

The Certified Six Sigma Black Belt is a professional who can explain Six Sigma philosophies and principles, including supporting systems and tools. A Black Belt should demonstrate team leadership, understand team dynamics and assign team member roles and responsibilities. Black Belts have a thorough understanding of all aspects of the DMAIC model in accordance with Six Sigma principles. They have basic knowledge of Lean enterprise concepts, are able to identify non-value-added elements and activities and are able to use specific tools.

1. Is this the right certification for you?
Understand the minimum expectations, requirements, experience, and the exam specifics for a Six Sigma Black Belt. Note: Six Sigma Black Belt certification requires that you complete at least one Six Sigma project and submit a project affidavit. Review frequently asked questions regarding projects and the affidavit/verfication.

2. Prepare for the exam.
Review the Body of Knowledge and references. Take advantage of the study guide and sample exams. See what training and books ASQ has to offer.

3. Choose an exam date.
Find the best date and location that works for you. (Note the application deadlines for each exam date.)

4. Apply for certification.
ASQ offers several ways to apply. Have your documentation and credit card ready to apply online.

5. Recertify.
ASQ requires that you recertify as a Six Sigma Black Belt every three years—either by documenting RU credits or by testing.

Download the Six Sigma Black Belt Certification Brochure (PDF, 176 KB)

*$50 of your fee is an application fee, and is not refundable.

Quality Improvement Associate Certification - CQIA

The Certified Quality Improvement Associate has a basic knowledge of quality tools and their uses and is involved in quality improvement projects, but doesn't necessarily come from a traditional quality area.

1. Is this the right certification for you?
Understand the requirements and exam specifics for a Quality Improvement Associate.

2. Prepare for the exam.
Review the Body of Knowledge and references. Take advantage of the study guide and sample exams. See what training and books ASQ has to offer.

3. Choose an exam date.
Find the best date and location for you. (Note the application deadlines for each exam date.)

4. Apply for certification.
ASQ offers several ways to apply. Have your documentation and credit card ready to apply online.

NOTE: The Quality Improvement Associate Certification has no recertification requirements. It is a lifetime certification.

Download the Quality Improvement Associate Certification Brochure (PDF, 234 KB)

*$50 of your fee is an application fee, and is not refundable.

Software Quality Engineer Certification - CSQE

The Certified Software Quality Engineer understands software quality development and implementation, software inspection, testing, verification and validation; and implements software development and maintenance processes and methods.

1. Is this the right certification for you?
Understand the minimum expectations, requirements, experience, and the exam specifics for a Software Quality Engineer.

2. Prepare for the exam.
Review the Body of Knowledge and references. Take advantage of the study guide and sample exams. See what training and books ASQ has to offer.

3. Choose an exam date.
Find the best date and location that works for you. (Note the application deadlines for each exam date.)

4. Apply for certification.
ASQ offers several ways to apply. Have your documentation and credit card ready to apply online.

5. Recertify.
ASQ requires that you recertify as a Software Quality Engineer every three years—either by documenting RU credits or by testing.

Download the Software Quality Engineer Certification Brochure (PDF, 192 KB)

*$50 of your fee is an application fee, and is not refundable.

Usability test for Cyworld Tw: Task 4

Usability test for Cyworld Tw: Task 3

SPL 2007 Low Fidelity Usability Test

Usability test for Cyworld Tw: Task 2

Usability test for Cyworld Tw: Task 1

Web Testing with Sahi - Part 2

Web Testing with Sahi

How to test a web app

Testing WebApps with OttoMate

Unit Testing with Team System

freeSpeech test

Visual Web Service Client

file management

JTR 4.0 Distributed Testing Demo

Testing Swing Apps w/ Frankenstein

Muvee Framework for Autonomous..

Ruby Tools for Building Web Testing

Specification based Testing

Bill Schongar - Making Automated Testing

performance testing

Jmeter

Presenting the apple

Stress test

How to write test case

Software Testing Notes

Software Testing Types:

Black box testing - Internal system design is not considered in this type of testing. Tests are based on requirements and functionality.

White box testing - This testing is based on knowledge of the internal logic of an application’s code. Also known as Glass box Testing. Internal software and code working should be known for this type of testing. Tests are based on coverage of code statements, branches, paths, conditions.

Unit testing - Testing of individual software components or modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. may require developing test driver modules or test harnesses.

Incremental integration testing - Bottom up approach for testing i.e continuous testing of an application as new functionality is added; Application functionality and modules should be independent enough to test separately. done by programmers or by testers.

Integration testing - Testing of integrated modules to verify combined functionality after integration. Modules are typically code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to client/server and distributed systems.

Functional testing - This type of testing ignores the internal parts and focus on the output is as per requirement or not. Black-box type testing geared to functional requirements of an application.

System testing - Entire system is tested as per the requirements. Black-box type testing that is based on overall requirements specifications, covers all combined parts of a system.

End-to-end testing - Similar to system testing, involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.

Sanity testing - Testing to determine if a new software version is performing well enough to accept it for a major testing effort. If application is crashing for initial use then system is not stable enough for further testing and build or application is assigned to fix.

Regression testing - Testing the application as a whole for the modification in any module or functionality. Difficult to cover all the system in regression testing so typically automation tools are used for these testing types.

Acceptance testing -Normally this type of testing is done to verify if system meets the customer specified requirements. User or customer do this testing to determine whether to accept application.

Load testing - Its a performance testing to check system behavior under load. Testing an application under heavy loads, such as testing of a web site under a range of loads to determine at what point the system’s response time degrades or fails.

Stress testing - System is stressed beyond its specifications to check how and when it fails. Performed under heavy load like putting large number beyond storage capacity, complex database queries, continuous input to system or database load.

Performance testing - Term often used interchangeably with ’stress’ and ‘load’ testing. To check whether system meets performance requirements. Used different performance and load tools to do this.

Usability testing - User-friendliness check. Application flow is tested, Can new user understand the application easily, Proper help documented whenever user stuck at any point. Basically system navigation is checked in this testing.

Install/uninstall testing - Tested for full, partial, or upgrade install/uninstall processes on different operating systems under different hardware, software environment.

Recovery testing - Testing how well a system recovers from crashes, hardware failures, or other catastrophic problems.

Security testing - Can system be penetrated by any hacking way. Testing how well the system protects against unauthorized internal or external access. Checked if system, database is safe from external attacks.

Compatibility testing - Testing how well software performs in a particular hardware/software/operating system/network environment and different combination s of above.

Comparison testing - Comparison of product strengths and weaknesses with previous versions or other similar products.

Alpha testing - In house virtual user environment can be created for this type of testing. Testing is done at the end of development. Still minor design changes may be made as a result of such testing.

Beta testing - Testing typically done by end-users or others. Final testing before releasing application for commercial purpose.

how to become an software testing expert

AJAX Testing

With the increasing popularity of Web 2.0, a large number of web applications have become more complex using advanced techniques such as Asynchronous JavaScript and XML (AJAX) which gives tremendous usability benefits. AJAX makes asynchronous calls to the web server to create more responsive web applications. Testing AJAX-based web applications is even more challenging compared to traditional web applications.
AJAX Testing Needs

* Functional Testing
* Load Testing
* Regression Testing

In traditional web applications, the HTML elements are already loaded in the web pages which has been fairly easy to test in an automated fashion. Unfortunately, AJAX-based web applications are not as easy or consistent to test as the traditional web applications. The problem with AJAX testing is that the web page under test will be modified asynchronously such as an element will be dynamically added based on a check box click or an element will be removed based on a list element selection, etc. To handle this dynamic element creation or removal using AJAX technique, QEngine provides some specific AJAX functions which are as follows.

*

waitForDynamicElement - During playback, the test waits until the HTML Element with the given tag name and matching property name and value is identified.
*

waitForDynamicElementRemoval - During playback, the test waits until the specified HTML Element with the given tag name and matching property name and value, is removed from the HTML page.
*

waitForElement - During playback, the test waits until the specified HTML Element with the given Element ID is identified.
*

waitForElementRemoval - During playback, the test waits until the HTML Element with the given Element ID is removed.
*

waitForText - During playback, the test waits until the specified search text (with the given prefix and suffix text) appears in the HTML page that is identified in the last user action.
*

waitForTitle - During playback, the test waits until the specified window title appears in the last identified HTML page.

You can insert any of the above AJAX functions in non-recording mode to handle the user actions that uses AJAX calls. These functions allow you to specify a condition such as an element to identify or text to appear and the maximum time to wait until the given condition is satisfied. Returns true and the test proceeds to the next line in the script, if the given condition is satisfied within the specified wait time. Returns false and the test proceeds to the next line in the script, if the given condition is not attainable within the specified wait time.

Tools varieties for testing

Functional Testing
Functional Testing Features Easy to Learn
Functional Testing Features Unicode Enabled
Functional Testing Features Powerful Checkpoints
Functional Testing Features Data-driven Testing
Functional Testing Features Error Recovery
Functional Testing Features Built-in Functions
Functional Testing Features Batch Mode Testing


Performance Testing
Load Testing Load Test Recorder
Performance Testing Bandwidth Simulation
Performance Testing DB & Server Monitors
Performance Testing Session Data Handling
Performance Testing Distributed Load Testing
Performance Testing Realistic Load
Performance Testing Detailed Reports

Web Services Testing
Web Services Testing Auto-generated Scripts
Web Services Testing Complex Data/Arrays
Web Services Testing Multiple Binding
Web Services Testing Attachment Support
Web Services Testing Data-driven Testing
Web Services Testing Powerful Validation
Web Services Testing Realistic Load Testing

software testing tools

Load and Performance Test Tools
Java Test Tools
Link Checkers
HTML Validators
Free On-the-Web HTML Validators and Link Checkers
PERL and C Programs for Validating and Checking
Web Functional/Regression Test Tools
Web Site Security Test Tools
External Site Monitoring Services
Web Site Management Tools
Log Analysis Tools
Other Web Test Tools

QA Tester Certifications

QA Tester Certifications

* Certification Comparisons
* American Society for Quality Certified Software Quality Engineer (CSQE)
* American Socity for Quality Quality Improvement Associate (CQIA)
* American Socity for Quality Six Sigma Black Belt Certification (SSBB)
* British Computer Society Information Systems Examinations Board (ISEB) qualification in Software Testing
* International Software Quality Institute ISTQB Certified Tester. The ISTQB is the umbrella organization for the national testing boards, which have already been established in many countries across Europe and around the world.
* Mercury Tools certification
* Quality Assurance Institute Certified Software Quality Analyst (CSQA)
* Rational Function and Performance tester certification
* Segue Tools certification

Mercury TestDirector

• Can you estimate efforts required to fully test requirements and make informed decisions regarding effort and risk tradeoffs?
• Does your organization lack the software testing tools to deploy high-quality applications quickly and effectively?
• Does your organization have the appropriate level of communication, organization, documentation, and structure in place for every testing project?
• Do you have a consistent, repeatable testing process?
• Are you able to track application release progress and project tasks?

Mercury TestDirector™ for Quality Center allows you to deploy high-quality applications quickly and effectively by providing a consistent, repeatable process for gathering requirements, planning and scheduling tests, analyzing results, and managing defects and issues. TestDirector is a single, web-based application for all essential aspects of test management — Requirements Management, Test Plan, Test Lab, and Defects Management. You can leverage these core modules either as a standalone solution or integrated within a global Quality Center of Excellence environment.

With Mercury TestDirector for Quality Center, you can:

• Proactively track the progress of releases and cycles so your organization can make more informed budgetary and release decisions and have the ability to analyze, evaluate, and improve your current processes over the lifetime of the application.
• Perform risk-based quality management enabling you to create appropriate test strategies by calculating test requirements against business risk levels and available resources.
• Facilitate information access across geographical and organization boundaries by supporting high levels of communication and collaboration among IT teams.
• Get real-time visibility for inter-related quality assets allowing you to test only what’s been impacted, giving you the time to run a full regression test after each change.

Service-Oriented Architecture (SOA) Test Management

Mercury Service Test Management™ automatically imports SOA services from the Mercury Systinet Registry into Mercury Quality Center to create test requirements and plans. Once the services are imported, Mercury Service Test Management identifies any changes to the services and automatically generates the necessary test cases that need to be run.

Using TestDirector for Quality Center, multiple groups throughout your organization can contribute to the quality process:

• Business analysts define application requirements and testing objectives.
• Test managers and project leads design test plans and develop test cases.
• Test managers automatically create QA testing requirements and test assets for SOA services and environments.
• Test automation engineers create automated scripts and store them in the repository.
• QA testers run manual and automated tests, report execution results, and enter defects.
• Developers review and fix defects logged into the database.
• Project managers can export test and resource data in various reports, or in native Microsoft Excel for analysis.
• Product managers decide whether an application is ready to be released.
• QA analysts can auto-generate test asset documentation in Microsoft Word format.

Types of Testing

As you get involved in the development of a new system a vast number of software tests appear to be required to prove the system. While they are consistent in all having the word "test" in them, their relative importance to each other is not clear. This advice article gives an outline of the various types of software testing.

The main software testing types are:

• Component.
• Interface.
• System.
• Acceptance.
• Release.

To put these types of software testing in context requires an outline of the development process.

Development Process

The development process for a system is traditionally as a Waterfall Model where each step follows the next, as if in a waterfall. This shows how various products produced at each step are used in the process following it. It does not imply that any of the steps in a process have to be completed, before the next step starts, or that prior steps will not have to be revisited later in development. It is just a useful model for seeing how each step works with each of the others.

Business Case

The first step in development is a business investigation followed by a "Business Case" produced by the customer for a system. This outlines a new system, or change to an existing system, which it is anticipated will deliver business benefits, and outlines the costs expected when developing and running the system.

Requirements

The next broad step is to define a set of "Requirements", which is a statement by the customer of what the system shall achieve in order to meet the need. These involve both functional and non-functional requirements. Further details are in the requirements article.

System Specification

"Requirements" are then passed to developers, who produce a "System Specification". This changes the focus from what the system shall achieve to how it will achieve it by defining it in computer terms, taking into account both functional and non-functional requirements.

System Design

Other developers produce a "System Design" from the "System Specification". This takes the features required and maps them to various components, and defines the relationships between these components. The whole design should result in a detailed system design that will achieve what is required by the "System Specification".

Component Design

Each component then has a "Component Design", which describes in detail exactly how it will perform its piece of processing.

Component Construction

Finally each component is built, and then is ready for the test process.

Types of Testing

The level of test is the primary focus of a system and derives from the way a software system is designed and built up. Conventionally this is known as the "V" model, which maps the types of test to each stage of development.

Component Testing

Starting from the bottom the first test level is "Component Testing", sometimes called Unit Testing. It involves checking that each feature specified in the "Component Design" has been implemented in the component.

In theory an independent tester should do this, but in practise the developer usually does it, as they are the only people who understand how a component works. The problem with a component is that it performs only a small part of the functionality of a system, and it relies on co-operating with other parts of the system, which may not have been built yet. To overcome this, the developer either builds, or uses special software to trick the component into believing it is working in a fully functional system.

Interface Testing

As the components are constructed and tested they are then linked together to check if they work with each other. It is a fact that two components that have passed all their tests, when connected to each other produce one new component full of faults. These tests can be done by specialists, or by the developers.

Interface Testing is not focussed on what the components are doing but on how they communicate with each other, as specified in the "System Design". The "System Design" defines relationships between components, and this involves stating:

• What a component can expect from another component in terms of services.
• How these services will be asked for.
• How they will be given.
• How to handle non-standard conditions, i.e. errors.

Tests are constructed to deal with each of these.

The tests are organised to check all the interfaces, until all the components have been built and interfaced to each other producing the whole system.

System Testing

Once the entire system has been built then it has to be tested against the "System Specification" to check if it delivers the features required. It is still developer focussed, although specialist developers known as systems testers are normally employed to do it.

In essence System Testing is not about checking the individual parts of the design, but about checking the system as a whole. In effect it is one giant component.

System testing can involve a number of specialist types of test to see if all the functional and non-functional requirements have been met. In addition to functional requirements these may include the following types of testing for the non-functional requirements:

• Performance - Are the performance criteria met?
• Volume - Can large volumes of information be handled?
• Stress - Can peak volumes of information be handled?
• Documentation - Is the documentation usable for the system?
• Robustness - Does the system remain stable under adverse circumstances?

There are many others, the needs for which are dictated by how the system is supposed to perform.

Acceptance Testing

Acceptance Testing checks the system against the "Requirements". It is similar to systems testing in that the whole system is checked but the important difference is the change in focus:

• Systems Testing checks that the system that was specified has been delivered.
• Acceptance Testing checks that the system delivers what was requested.

The customer, and not the developer should always do acceptance testing. The customer knows what is required from the system to achieve value in the business and is the only person qualified to make that judgement. To help them courses and training are available.

The forms of the tests may follow those in system testing, but at all times they are informed by the business needs.

Release Testing

Even if a system meets all its requirements, there is still a case to be answered that it will benefit the business. The linking of "Business Case" to Release Testing is looser than the others, but is still important.

Release Testing is about seeing if the new or changed system will work in the existing business environment. Mainly this means the technical environment, and checks concerns such as:

• Does it affect any other systems running on the hardware?
• Is it compatible with other systems?
• Does it have acceptable performance under load?

These tests are usually run the by the computer operations team in a business. The answers to their questions could have significant a financial impact if new computer hardware should be required, and adversely affect the "Business Case".

It would appear obvious that the operations team should be involved right from the start of a project to give their opinion of the impact a new system may have. They could then make sure the "Business Case" is relatively sound, at least from the capital expenditure, and ongoing running costs aspects. However in practise many operations teams only find out about a project just weeks before it is supposed to go live, which can result in major problems.

Regression Tests

With modern systems one person's system, becomes somebody else's component. It follows that all the above types of testing could be repeated at many levels in order to deliver the final value to the business. In fact every time a system is altered.

Testing Types

• COMPATIBILITY TESTING. Testing to ensure compatibility of an application or Web site with different browsers, OSs, and hardware platforms. Compatibility testing can be performed manually or can be driven by an automated functional or regression test suite.
• CONFORMANCE TESTING. Verifying implementation conformance to industry standards. Producing tests for the behavior of an implementation to be sure it provides the portability, interoperability, and/or compatibility a standard defines.
• FUNCTIONAL TESTING. Validating an application or Web site conforms to its specifications and correctly performs all its required functions. This entails a series of tests which perform a feature by feature validation of behavior, using a wide range of normal and erroneous input data. This can involve testing of the product's user interface, APIs, database management, security, installation, networking, etcF testing can be performed on an automated or manual basis using black box or white box methodologies.
• LOAD TESTING. Load testing is a generic term covering Performance Testing and Stress Testing.
• PERFORMANCE TESTING. Performance testing can be applied to understand your application or WWW site's scalability, or to benchmark the performance in an environment of third party products such as servers and middleware for potential purchase. This sort of testing is particularly useful to identify performance bottlenecks in high use applications. Performance testing generally involves an automated test suite as this allows easy simulation of a variety of normal, peak, and exceptional load conditions.
• REGRESSION TESTING. Similar in scope to a functional test, a regression test allows a consistent, repeatable validation of each new release of a product or Web site. Such testing ensures reported product defects have been corrected for each new release and that no new quality problems were introduced in the maintenance process. Though regression testing can be performed manually an automated test suite is often used to reduce the time and resources needed to perform the required testing.
• SMOKE TESTING. A quick-and-dirty test that the major functions of a piece of software work without bothering with finer details. Originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch on fire.
• STRESS TESTING. Testing conducted to evaluate a system or component at or beyond the limits of its specified requirements to determine the load under which it fails and how. A graceful degradation under load leading to non-catastrophic failure is the desired result. Often Stress Testing is performed using the same process as Performance Testing but employing a very high level of simulated load.
• UNIT TESTING. Functional and reliability testing in an Engineering environment. Producing tests for the behavior of components of a product to ensure their correct behavior prior to system integration.