Testing Methodology

Data stored in the database should include such items as the catalog, pricing, shipping tables, tax tables, order database, and customer information. Testng must verify the integrity of the stored data. Testing should be done on a regular basis because data changes over time.

Data integrity tests

Data integrity can be tested as follows to ensure that the data is valid and not corrupt:

* Test the creation, modification, and deletion of data in tables as specified in the business requirement.

* Test to make sure that sets of radio buttons represent a fixed set of values. You should also check for NULL or EMPTY values.

* Test to make sure that data is save to the database and that each values gets saved fully. You should watch for the truncation of strings and that numeric values are not rounded off.

* Test to make sure that default values are stored and saved.

* Test the compatibility with old data. You should ensure that all updates do not affect the data you have on file in your database.

Usage " Security is a protection system that is needed for both securing the confidential information and for competitive purposes to assure third parties that their data will be protected.

" Amount of security provided depends upon risks associated with compromise or loss of information.

" Protecting the confidentiality of the information is designed to protect the resources of the organization.

" Used to check the adequacy of protective procedures and countermeasures..

Objectives " To identify the defects which are very difficult to identify.

" The failures in security system operation may not be detected , resulting in a loss or compromise of information without the knowledge of that loss.

" To determine that adequate attention is paid to identify security risks.

" Determine realistic definition and enforcement of access to the system has been implemented.

" To determine that sufficient expertise exists to perform adequate security testing.

" Conducting reasonable tests to ensure that the implemented security measures function properly.

How to Use " Involves a wide spectrum of conditions.

" Testing divided into physical and logical security.

" Physical security - deals with penetration by people in order to physically gather information.

" Logical Security - deals with use of computer operations / communications capabilities to improperly access information.

When to use " Security testing should be used when the information and/or assets protected by the application system are of significant value to the organization.

" Should be conducted before system goes to operational status.

" Extent of testing should depend upon the security risk.

Examples " Access denied " Procedures in place

The functional acceptance simple test(FAST) is run on each development release to check that key features of the program are appropriately accessible and functioning properly on the at least one test configuration (preferable the minimum or common configuration).This test suite consists of simple test cases that check the lowest level of functionality for each command- to ensure that task-oriented functional tests(TOFTs) cna be performed on the program. The objective is to decompose the functionality of a program down to the command level and then apply test cases to check that each command works as intended. No attention is paid to the combination of these basic commands, the context of the feature that is formed by these combined commands, or the end result of the overall feature. For example, FAST for a File/Save As menu command checks that the Save As dialog box displays. However, it does not validate that the overall file-saving feature works nor does it validate the integrity of save files.

The software was written to provide a friendly interface for information workers: Spreadsheet jockeys, business people needing written reports, and game players. The full spectrum of desktop software could pretty well be categorized into spreadsheet, wordprocessor, database, and entertainment categories since desktop computers were rarely networked to other information resources. Desktop applications used the keyboard, and then later a mouse, to navigate through windows and a drop-down menu. Inside a desktop application software package one would find an event-driven framework surrounding individual procedural functions. The automation focused on improving the time it took to test a desktop application for functionality. The test utilities link into desktop applications and try each command as though a user were accessing the menu and window commands. Most QA technicians testing a desktop application compare the function of all the menus and windows to a written functional specification document. The variation from the document to the performance shows the relative health of a desktop application.

Editable Fields Checking and Validation:

* Valid/invalid characters/strings data in all editable fields

* Valid minimum/maximum/mid range values in fields

* Null strings (or no data ) in required fields

* Record length (character limit)in text/memo fields

* Cut/copy/paste into/from fields when possible

Not Editable Fields Checking:

* Check for all test/spelling in warnings and error messages/dialogs

* Invoke/check all menu items and their options

Application Usability:

* Appearance an outlook (Placement an alignment of objects on screen)

* User Interface Test (open all menus, check all items)

* Basic functionality checking (File+Open+Save, etc..)

* Right mouse clicking sensitivity

* Resize/min/max/restore app, windows (check min app size)

* Scrollability when applicable (scrollbars, keyboard, autoscrolling)

* Keyboard and mouse navigation, highlighting, dragging, drag/drop

* Print in landscape an portrait modes

* Check F1, What's This , Help menu

* Short-cut and Accelerator keys

* Tab Key order and Navigation in all dialog boxes and menus

Background " One segment of system is developed and thoroughly tested.

" Another segment is changed which has disastrous effect on the tested segment.

" The implemented change, new data or parameters have created change in the already tested segment.

Usage " All aspects of system remain functional after testing.

" Change in one segment does not change the functionality of other segment.

Objectives " Determine -

system documents remain current

System test data and test conditions remain current.

Previously tested system functions properly without getting effected though changes are made in some other segment of application system.

How to Use " Test cases, which were used previously for the already tested segment is, re-run to ensure that the results of the segment tested currently and the results of same segment tested earlier are same.

" Test automation is needed to carry out the test transactions (test condition execution) else the process is very time consuming and tedious.

" In this case of testing cost/benefit should be carefully evaluated else the efforts spend on testing would be more and payback would be minimum.

When to use " When there is high risk that the new changes may effect the unchanged areas of application system.

" In development process: Regression testing should be carried out after the pre-determined changes are incorporated in the application system.

" In Maintenance phase : regression testing should be carried out if there is a high risk that loss may occur when the changes are made to the system

Examples " Re-running of previously conducted tests to ensure that the unchanged portion of system functions properly.

" Reviewing previously prepared system documents (manuals) to ensure that they do not get effected after changes are made to the application system.

" Obtaining printout of data dictionary to ensure that the data elements reflect the changes being incorporated.

Disadvantage " Time consuming and tedious if test automation not done.

1. Browser functionality:

* Is the browser compatible with the application design?

* There are many different types of browsers available.

# GUI design components

* Are the scroll bars, buttons, and frames compatible with the browser and functional?

* To check the functionality of the scroll bars on the interface of the Web page to make sure the the user can scroll through items and make the correct selection from a list of items.

* The button on the interface need to be functional and the correct hyperlink should go to the correct page.

* If frames are used on the interface, they should be checked for the correct size and whether all of the components fit within the viewing screen of the monitor.

2. User Interface

One of the reasons the web browser is being used as the front end to applications is the ease of use. Users who have been on the web before will probably know how to navigate a well-built web site. While you are concentrating on this portion of testing it is important to verify that the application is easy to use. Many will believe that this is the least important area to test, but if you want to be successful, the site better be easy to use.

3.Instructions

You want to make sure there are instructions. Even if you think the web site is simple, there will always be someone who needs some clarification. Additionally, you need to test the documentation to verify that the instructions are correct. If you follow each instruction does the expected result occur?

4. Site map or navigational bar

Does the site have a map? Sometimes power users know exactly where they want to go and don't want to wade through lengthy introductions. Or new users get lost easily. Either way a site map and/or an ever-present navigational bar can help guide the user. You need to verify that the site map is correct. Does each link on the map actually exist? Are there links on the site that are not represented on the map? Is the navigational bar present on every screen? Is it consistent? Does each link work on each page? Is it organized in an intuitive manner?

5. Content

To a developer, functionality comes before wording. Anyone can slap together some fancy mission statement later, but while they are developing, they just need some filler to verify alignment and layout. Unfortunately, text produced like this may sneak through the cracks. It is important to check with the public relations department on the exact wording of the content.

You also want to make sure the site looks professional. Overuse of bold text, big fonts and blinking (ugh) can turn away a customer quickly. It might be a good idea to consult a graphic designer to look over the site during User Acceptance Testing. You wouldn't slap together a brochure with bold text everywhere, so you want to handle the web site with the same level of professionalism.

Finally, you want to make sure that any time a web reference is given that it is hyperlinked. Plenty of sites ask you to email them at a specific address or to download a browser from an address. But if the user can't click on it, they are going to be annoyed.

6. Colors/backgrounds

Ever since the web became popular, everyone thinks they are graphic designers. Unfortunately, some developers are more interested in their new backgrounds, than ease of use. Sites will have yellow text on a purple picture of a fractal pattern. (If you've never seen this, try most sites at GeoCities or AOL.) This may seem "pretty neat", but it's not easy to use.

Usually, the best idea is to use little or no background. If you have a background, it might be a single color on the left side of the page, containing the navigational bar. But, patterns and pictures distract the user.

7. Images

Whether it's a screen grab or a little icon that points the way, a picture is worth a thousand words. Sometimes, the best way to tell the user something is to simply show them. However, bandwidth is precious to the client and the server, so you need to conserve memory usage. Do all the images add value to each page, or do they simply waste bandwidth? Can a different file type (.GIF, .JPG) be used for 30k less?

In general, you don't want large pictures on the front page, since most users who abandon a page due to a large load will do it on the front page. If you can get them to see the front page quickly, it will increase the chance they will stay.

8. Tables

You also want to verify that tables are setup properly. Does the user constantly have to scroll right to see the price of the item? Would it be more effective to put the price closer to the left and put miniscule details to the right? Are the columns wide enough or does every row have to wrap around? Are certain columns considerably longer than others?

9. Wrap-around

Finally, you will want to verify that wrap-around occurs properly. If the text refers to "a picture on the right", make sure the picture is on the right. Make sure that widowed and orphaned sentences and paragraphs don't layout in an awkward manner because of pictures.

10. Functionality

The functionality of the web site is why your company hired a developer and not just an artist. This is the part that interfaces with the server and actually "does stuff".

11. Links

A link is the vehicle that gets the user from page to page. You will need to verify two things for each link: that the link brings you to the page it said it would and that the pages you are linking to actually exists. It may sound a little silly but I have seen plenty of web sites with internal broken links.

12. Forms

When a user submits information through a form it needs to work properly. The submit button needs to work. If the form is for an online registration, the user should be given login information (that works) after successful completion. If the form gathers shipping information, it should be handled properly and the customer should receive their package. In order to test this, you need to verify that the server stores the information properly and that systems down the line can interpret and use that information.

13. Data verification

If the system verifies user input according to business rules, then that needs to work properly. For example, a State field may be checked against a list of valid values. If this is the case, you need to verify that the list is complete and that the program actually calls the list properly (add a bogus value to the list and make sure the system accepts it).

14. Cookies

Most users only like the kind with sugar, but developers love web cookies. If the system uses them, you need to check them. If they store login information, make sure the cookies work. If the cookie is used for statistics, verify that totals are being counted properly. And you'll probably want to make sure those cookies are encrypted too, otherwise people can edit their cookies and skew your statistics.

Application specific functional requirements Most importantly, you want to verify the application specific functional requirements. Try to perform all functions a user would: place an order, change an order, cancel an order, check the status of the order, change shipping information before an order is shipped, pay online, ad naseum.

This is why your users will show up on your doorstep, so you need to make sure you can do what you advertise.

16. Interface Testing

Many times, a web site is not an island. The site will call external servers for additional data, verification of data or fulfillment of orders.

16. Server interface

The first interface you should test is the interface between the browser and the server. You should attempt transactions, then view the server logs and verify that what you're seeing in the browser is actually happening on the server. It's also a good idea to run queries on the database to make sure the transaction data is being stored properly.

17. External interfaces

Some web systems have external interfaces. For example, a merchant might verify credit card transactions real-time in order to reduce fraud. You will need to send several test transactions using the web interface. Try credit cards that are valid, invalid, and stolen. If the merchant only takes Visa and MasterCard, try using a Discover card. (A script can check the first digit of the credit card number: 3 for American Express, 4 for Visa, 5 for MasterCard, or 6 for Discover, before the transaction is sent.) Basically, you want to make sure that the software can handle every possible message returned by the external server.

18. Error handling

One of the areas left untested most often is interface error handling. Usually we try to make sure our system can handle all of our errors, but we never plan for the other systems' errors or for the unexpected. Try leaving the site mid-transaction - what happens? Does the order complete anyway? Try losing the internet connection from the user to the server. Try losing the connection from the server to the credit card verification server. Is there proper error handling for all these situations? Are charges still made to credit cards? Is the interruption is not user initiated, does the order get stored so customer service reps can call back if the user doesn't come back to the site?

19. Compatibility

You will also want to verify that the application can work on the machines your customers will be using. If the product is going to the web for the world to use, you will need to try different combinations of operating system, browser, video setting and modem speed.

20. Operating systems

Does the site work for both MAC and IBM-Compatibles? Some fonts are not available on both systems, so make sure that secondary fonts are selected. Make sure that the site doesn't use plug-ins only available for one OS, if your users will use both.

21. Browsers

Does your site work with Netscape? Internet Explorer? Lynx? Some HTML commands or scripts only work for certain browsers. Make sure there are alternate tags for images, in case someone is using a text browser. If you're using SSL security, you only need to check browsers 3.0 and higher, but verify that there is a message for those using older browsers.

22. Video settings

Does the layout still look good on 640x400 or 600x800? Are fonts too small to read? Are they too big? Does all the text and graphic alignment still work?

23. Modem/connection speeds

Does it take 10 minutes to load a page with a 28.8 modem, but you tested hooked up to a T1? Users will expect long download times when they are grabbing documents or demos, but not on the front page. Make sure that the images aren't too large. Make sure that marketing didn't put 50k of font size -6 keywords for search engines.

Usage " To determine if the system can function when subject to large volumes.

" It includes testing of

input transactions

Internal tables

Disk Space

Out put

Communication

Computer capacity

Interaction with people.

Objectives " To simulate production environment

" Normal or above normal volumes of transactions can be processed through the transaction within expected time frame.

" Application system would be able to process larger volume of data.

" System capacity should have sufficient resources to meet expected turnaround time.

How to Use " It should simulate as closely as possible to production environment.

" Online system should be stress tested with users entering test data with normal or above normal pace.

" Batch system should be tested with huge volumes/ numbers of batches

" The test conditions should have error conditions.

" Transactions used in stress testing are obtained from following 3 sources :

Test data generators

Test transactions created by test group

Transactions which were previously used in production.

" In stress testing the system should run as it would in the production environment.

When to use " When there is uncertainty that system will work with huge volumes of data and without generating any faults.

" Attempt is made to break system with huge amount of data.

" Most commonly used technique to test for online transaction systems as other

techniques are not effective.

Examples " Sufficient disk space allocated

" Communication lines are adequate

Disadvantage " Amount of time taken to prepare for testing

" Amount of resources utilized during test execution.

1. Unit Testing.

* Unit Testing is primarily carried out by the developers themselves.

* Deals functional correctness and the completeness of individual program units.

* White box testing methods are employed

2. Integration Testing.

* Integration Testing: Deals with testing when several program units are integrated.

* Regression testing : Change of behavior due to modification or addition is called ‘Regression’. Used to bring changes from worst to least.

* Incremental Integration Testing : Checks out for bugs which encounter when a module has been integrated to the existing.

* Smoke Testing : It is the battery of test which checks the basic functionality of program. If fails then the program is not sent for further testing.

3. System Testing.

* System Testing - Deals with testing the whole program system for its intended purpose.

* Recovery testing : System is forced to fail and is checked out how well the system recovers the failure.

* Security Testing : Checks the capability of system to defend itself from hostile attack on programs and data.

* Load & Stress Testing : The system is tested for max load and extreme stress points are figured out.

* Performance Testing : Used to determine the processing speed.

* Installation Testing : Installation & uninstallation is checked out in the target platform.

4. Acceptance Testing.

* UAT ensures that the project satisfies the customer requirements.

* Alpha Testing : It is the test done by the client at the developer’s site.

* Beta Testing : This is the test done by the end-users at the client’s site.

* Long Term Testing : Checks out for faults occurrence in a long term usage of the product.

* Compatibility Testing : Determines how well the product is substantial to product transition.

The release acceptance test (RAT), also referred to as a build acceptance or smoke test, is run on each development release to check that each build is stable enough for further testing. Typically, this test suite consists of entrance and exit test cases plus test cases that check mainstream functions of the program with mainstream data. Copies of the RAT can be distributed to developers so that they can run the tests before submitting builds to the testing group. If a build does not pass a RAT test, it is reasonable to do the following:

* Suspend testing on the new build and resume testing on the prior build until another build is received.

* Report the failing criteria to the development team.

* Request a new build.