Testing Methodology

When you begin your stress testing, you will want to increase your capacity testing to make sure you are able to handle the increased load of data such as ASP pages and graphics. When you test the ASP pages, you may want to create a page similar to the original page that will simulate the same items on the ASP page and have it send the information to a test bed with a process that completes just a small data output. By doing this, you will have your processor still stressing the system but not taking up the bandwidth by sending the HTML code along the full path. This will not stress the entire code but will give you a basis from which to work. Dividing the requests per second by the total number of user or threads will determine the number of transactions per second. It will tell you at what point the server will start becoming less efficient at handling the load. Let's look at an example. Let's say your test with 50 users shows your server can handle 5 requests per seconf, with 100 users it is 10 requests per second, with 200 users it is 15 requests per second, and eventually with 300 users it is 20 requests per second. Your requests per second are continually climbing, so it seems that you are obtaining steadily improving performance. Let's look at the ratios:

05/50 = 0.1

10/100 = 0.1

15/200 = 0.075

20/300 = 0.073

From this example you can see that the performance of the server is becoming less and less efficient as the load grows. This in itself is not necessarily bad (as long as your pages are still returning within your target time frame). However, it can be a useful indicator during your optimization process and does give you some indication of how much leeway you have to handle expected peaks.

Did a new build break an existing function? Repeat testing after changes for managing risk relate to product enhancement.

A regression test is performded when the tester wishes to see the progress of the testing processs by performing identical tests before and after a bug has been fixed. A regression test allows the tester to compare expeted test results with the actual results.

Regression testing's primary objective is to ensure that all bugfree features stay that way. In addition, bugs which have been fixed once should not turn up again in subsequent program versions.

Regression testing: After every software modification or before next release, we repeat all test cases to check if fixed bugs are not show up again and new and existing functions are all working correctly.

Regression testing is used to confirm that fixed bugs have, in fact, been fixed and that new bugs have not been introduced in the process, and that festures that were proven correctly functional are intact. Depending on the size of a project, cycles of regression testing may be perform once per milestone or once per build. Some bug regression testing may also be performed during each accceptance test cycle, forcusing on only the most important bugs. Regression tests can be automated.

CONDITIONS DURING WHICH REGRESSION TESTS MAY BE RUN Issu fixing cycle. Once the development team has fixed issues, a regression test can be run t ovalidate the fixes. Tests are based on the step-by-step test casess that were originally reported:

* If an issue is confirmeded as fixed, then the issue report status should be changed to Closed.

* If an issue is confirmed as fixed, but with side effects, then the issue report status should be changed to Closed. However, a new issue should be filed to report the side effect.

* If an issue is only partially fixed, then the issue report resolution should be changed back to Unfixed, along with comments outlining the oustanding problems

Open-status regression cycle. Periodic regression tests may be run on all open issue in the issue-tracking database. During this cycle, issue status is confirmed either the report is reproducible as is with no modification, the report is reproducible with additional comments or modifications, or the report is no longer reproducible

Closed-fixed regression cycle. In the final phase of testing, a full-regression test cycle should be run to confirm the status of all fixed-closed issues.

Feature regression cycle. Each time a new build is cut or is in the final phase of testing depending on the organizational procedure, a full-regression test cycle should be run to confirm that the proven correctly functional features are still working as expected.

The original intent for client/server applications was to separete presentation logic from business logic. In an ideal system design, the client was reponsible for presenting the user interface, command elements (drop-down menus, buttons, controls), displayed results information in a set of windows, charts, and dials. The client connected to a server to process functions and the server responded with data.

In a client/server environment the protocols are cleanly defined so that all the clients use the same protocols to communicate with the server.

The client-side frameworks to provide the same functionality of desktop application frameworks plus most of the needed communication code to issue commands to the server and the code needed to automatically update a client with new functions received from the server.The server-side frameworks provide code needed to received and handle requests from multiple clients, and code to connect to database for data persistence and remote information providers. Additionally, these framworks need to handle stateful transations and intermittent network connections. Stateful transactions require multiple steps to accomplish a task.

Client/server applications are normally transactional in nature and usually several interactions with the user are needed to finish a single request. For example, in a stock trading application the user begins a transaction by identifying themselves to the server, looking up an order code, and then submitting a request to the server, and receives and presents the results to the user. The client-side application normally knows something about the transaction - for example, the client-side application will normally store the user identification and a session code such as cookie value across the user's interaction with the server-based application. Users like it better when the client-side application knows about the transaction because each step in a request can be optimized in the client application. For example. in the stock trading example the client application could calculate a stock trade commission locally without having to communication with server.

Client/server application test automation provides the functionality of desktop application test automation plus these:

* Client/server applications operate in a network environment. The tests need to not only check for the function of an application, they need to test how the application handles slow or intermittent network performance.

* Automated test are ideal to determine the number of client applications a server is able to efficiently handle at any given time.

* The server is usually a middle tier between the client application and several data sources. Automated tests need to check the server for correct functionality while it communicates with the data source.

Web-Enabled application go further in these areas:

* Web-enabled application are meant to be stateless. HTTP was designed to be stateless. Each request from a Web-enabled application is meant to be atomic and not rely on any previouse requests. This has huge advantages for system architecture and datacenter provisioning. When requests are stateless, then any sserver can respond to the request and any request handler on any server may service the request.

* Web-enabled application are platform independent. The client application may be written for Windows, Macintosh, Linux, and any other platform that is capable of implementing the command protocol and network connection to the server.

* Web-enabled application expect the client application to provide presentation rendering and simple scripting capabilities. The client application is usually a browser, however, it may also be a dedication client application such as a retail cash register, a Windows-based data analysis tool, ot an electronic address book in your mobile phone.

The missing context in a Web-enabled application test automation means that software developers and QA technicians must manually script tests for each Web-enalbled application. Plus, they need to maintain the test scriots as the application changes. Web-enabled application test automation tools focus on making the scriot writing and maintenance tasks easier. The test automation tool offer these features:

* A friendly, graphical user interface to integrate the record, edit, and run-time script functions.

* A recorder that watches how an application is used and writes a test script for you.

* A playback utility that drives a Web-enalbed application by processing the test script and logging. The playback utility also provides the facility to play back several concurrently running copies of the same script to check the system for scalability and load testing.

* A report utility to show how the playback differed from the original recording. The differences may be slower or faster performance times, errors, and incomplete transactions.

Usage " To ensure that system performs correctly

" To ensure that correctness can be sustained for a considerable period of time.

" System can be tested for correctness through all phases of SDLC but incase of reliability the programs should be in place to make system operational.

Objectives " Successfully implementation of user requirements

" Correctness maintained over considerable period of time

" Processing of the application complies with the organization's policies and procedures.

" Secondary users needs are fulfilled -

Security officer

DBA

Internal auditors

Record retention

Comptroller

How to Use " Test conditions created

These test conditions are generalized ones, which becomes test cases as the SDLC progresses until system is fully operational.

Test conditions are more effective when created from user's requirements.

Test conditions if created from documents then if there are any error in the documents those will get incorporated in Test conditions and testing would not be able to find those errors.

Test conditions if created from other sources (other than documents) error trapping is effective.

" Functional Checklist created.

When to use " Every application should be Requirement tested

" Should start at Requirements phase and should progress till operations and maintenance phase.

" The method used to carry requirement testing and the extent of it is important.

Examples " Creating test matrix to prove that system requirements as documented are the requirements desired by the user.

" Creating checklist to verify that application complies to the organizational policies and procedures.

The forced-error test (FET) consists of negative test cases that are designed to force a program into error conditions. A list of all error messages thatthe program issues should be generated. The list is used as a baseline for developing test cases. An attempt is made to generate each error message in the list. Obviously, test to validate error-handling schemes cannot be performed until all the handling and error message have been coded. However, FETs should be thought through as early as possible. Sometimes, the error messages are not available. The error cases can still be considered by walking through the program and deciding how the program might fail in a given user interface such as a dialog or in the course of executing a given task or printing a given report. Test cases should be created for each condition to determine what error message is generated.

Web system often require both client-side and server-side installs. Testing of the installer checks that installed features function properly--including icons, support documentation , the README file, and registry keys. The test verifies that the correct directories are created and that the correct system files are copied to the appropriate directories. The test also confirms that various error conditions are detected and handled gracefully.

Testing of the uninstaller checks that the installed directories and files are appropriately removed, that configuration and system-related filea are also appropriately removed or modified, and that the operating environment is recovered in its original state.

1. White Box

Also called ‘Structural Testing / Glass Box Testing’ is used for testing the code keeping the system specs in mind. Inner working is considered and thus Developers Test..

* Mutation Testing

Number of mutants of the same program created with minor changes and none of their result should coincide with that of the result of the original program given same test case.

* Basic Path Testing

Testing is done based on Flow graph notation, uses Cyclometric complexity & Graph matrices.

* Control Structure Testing

The Flow of control execution path is considered for testing. It does also checks :-

Conditional Testing : Branch Testing, Domain Testing.

Data Flow Testing.

Loop testing :Simple, Nested, Conditional, Unstructured Loops.

2. Gray Box

3. Black Box

Also called ‘Functional Testing’ as it concentrates on testing of the functionality rather than the internal details of code.

Test cases are designed based on the task descriptions

* Comparison Testing

Test cases results are compared with the results of the test Oracle.

* Graph Based Testing

Cause and effect graphs are generated and cyclometric complexity considered in using the test cases.

* Boundary Value Testing

Boundary values of the Equivalence classes are considered and tested as they generally fail in Equivalence class testing.

* Equivalence class Testing

Test inputs are classified into Equivalence classes such that one input check validates all the input values in that class.

Gray Box Testing : Similar to Black box but the test cases, risk assessments, and test methods involved in gray box testing are developed based on the knowledge of the internal data and flow structures

System tests check that the software functions properly from end-to-end. The components of the system include: A database, Web-enable application software modules, Web servers, Web-enabled application frameworks deploy Web browser software, TCP/IP networking routers, media servers to stream audio and video, and messaging services for email.

A common mistake of test professionals is to believe that they are conducting system tests while they are actually testing a single component of the system. For example, checking that the Web server returns a page is not a system test if the page contains only a static HTML page.

System testing is the process of testing an integrated hardware and software system to verify that the system meets its specified requirements. It verifies proper execution of the entire set of application components including interfaces to other applications. Project teams of developers and test analysts are responsible for ensuring that this level of testing is performed.

System testing checklist include question about:

* Functional completeness of the system or the add-on module

* Runtime behavior on various operating system or different hardware configurantions.

* Installability and configurability on various systems

* Capacity limitation (maximum file size, number of records, maximum number of concurrent users, etc.)

* Behavior in response to problems in the programming environment (system crash, unavailable network, full hard-disk, printer not ready)

* Protection against unauthorized access to data and programs.