How to do REAL TESTING

real time testing 

Evaluation of a system (or its components) at its normal operating frequency, speed, or timing.

Contents:

• Introduction to Real-time systems
• Two main types of system
• Testing real-time software
• Difficulties with testing such software

Introduction:

• What is a Real-time System?

ú  A real time system is one that “is subject to real time constraints”  meaning that operations must  produce a response within a given time.

ú  Examples include:

           Air traffic control systems

           Medical monitoring

           Weapons delivery systems

Space navigation and guidance

Types of Real-time System:

• There are two main types of real-time systems:
• Hard real-time system – Requires that operations must be completed      within their deadline e.g. systems in space shuttles, fight control.
• Soft real-time system – Although it is emphasised that deadlines are important they are not as strict as in hard real-time systems.

Introduction to testing…

• How do we test a real-time software?

ú  There isn’t one single method for testing real-time software.

ú  In order to test such software need to use several methods and strategies such as:

           White box testing

           Black box testing

           Sequence testing

                                        Integration testing

How do we test software in real-time systems?

• A strategy for testing real-time software should include:

ú  Low-level tests – single lines of code

ú  High-level tests – methods and functions

• In order to test such software one technique might be to generate and test sequences by applying time constraints in which they must be completed.

The 3-stage approach to testing real time software:

• 1. Identification  of tasks

ú  Output Testing

ú  Performance Testing

• 2. Incorporation of tasks

ú  Output Testing

ú  Performance Testing in a real environment

• 3. Integration Testing

Difficulties with testing Real-time software?

• It is difficult to test a system that is running in real time due to factors such as:

ú  Time constraints

ú  Conventional methods are not applicable for real-time software

• This can lead to…

ú  Unpredictable outcomes

Summary:

• A real-time system is subject to time constraints
• Two main types of real-time system
• No single way of testing this software
• Difficulties with testing such software

• http://msdn.microsoft.com/en-us/library/aa459157.aspx
• http://en.wikipedia.org/wiki/Real-time_computing
• G53SRP module page
• http://www.ndt.net/article/wcndt00/papers/idn235/idn235.htm
• http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=139423

 Real Time:

Software testing is performed to detect and help correct bugs in the software. Testing also involves ensuring that the software gives the desired functionality to the user. Static and conventional ways of testing predetermine bugs, but such techniques can't ensure same result in real time software systems.
Real-time software systems have strict timing constraints and have a deterministic behavior.This is so because real time software systems have to schedule their tasks such that the timing constraints imposed on them are met. Conventional static way of analysis is not adequate to deal with such timing constraints, hence real time testing is important

l'� 7 a � �t olor:#0B0080'>^ . http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=139423.

2.      ^ Software Engineering: A Practitioner's Approach by Roger S Pressman

3.      ^ . http://www.sdl-forum.org/issre04-witul/papers/EbnerTTCN3.pdf.

4.      ^ . http://www.aws.cit.ie/personnel/dpesch/notes/msc_sw/sdl.pdf.

5.      ^ . http://www.ttcn-3.org.

Strategy

Test case design for real time testing can be proposed in four steps 

Task testing

In the very first step, each task is tested individually with conventional ways of static testing.This testing is performed only to discover the errors in logic or syntax of the program. It has nothing to do with software's behaviour. Order of the events doesn't matter as task testing doesn't deal with timing constraints and time properties of events.

Behavioral testing

Using the system models designed with the help of automated testing tools, it is possible to simulate behavior of real time system and impact of concurrent external events on it's behavior.

Intertask testing

Once the testing with the individual task is done, then task is supposed to be error free in coding and behavioral area. Time-related constraints are tested with intertask testing. To reveal the errors in communication, asynchronous tasks are tested with variable data rates and different payloads.

System testing

In this testing, software and hardware are integrated and full range of system tests are conducted to discover errors, if any, during software and hardware interfacing.

Tools for real time testing

As testing of real time systems is becoming more important, there are some tools designed

MSC

Message Sequence Charts is an internationally accepted standard for capturing requirements.MSC Provides a graphical 2-D language often required for collecting requirements through some interaction scenarios.

SDL

System Design Language is a standard used for design and analysis. SDL  supports the specification of complex software systems and has been extensively applied across a broad array of domains from telecommunications, automation, through to general software development

TTCN

Testing and Test Control Notation is the only internationally standard testing language. TTCN3  provides a broader applicability, as compared to earlier versions of TTCN, which were primarily focussed on OSI protocols only.

These three standards together are used for testing of real time applications. It is necessary that requirements be satisfied with these models and test cases generated must capture the functional and real time information needed to test systems. Also,the changes in the requirements of design and new information about the real time properties of systems should be fed into models so that its impact can be found out.

To accurately capture the real time properties of a given test system and to ensure that requirements and models are used to generate realistic and enforceable timing information, it is essential that the language itself (TTCN-3) has a well understood and semantically sound model of time.

TTCN3

Background

TTCN3 is the only currently available,internationally standardised testing language. Prior to TTCN3, its earlier versions were having limited functionality and limited scope over OSI protocol.But,TTCN3 is an advanced version and has broader applicability.
Characteristics of TTCN3 are:

ability to specify dynamic concurrent testing

operations for message-based and procedure based communication

the ability to specify data and signature templates with powerful matching mechanisms

type and value parameterization

the assignment and handling of test verdicts

test suite parameterization and test case selection mechanisms

The reason for using TTCN3 for real time testing is because of its timers. These timers are defined in function test-suites. There are no any global kind timers used in TTCN3. These timers can be started ,stopped and checked using simple functions like timer.start, timer.stop, and timer.read.

Snapshot Semantics is a technique in TTCN3 (also in TTCN2), which deals with the message passed during communication by system to system or implementation under test. When a series of responses are received by system under test,then snapshot is taken and they are evaluated in order of their arrival.So,each time around a set of attributes,a snapshot is taken and only those events are evaluated which are present in snapshot.

But this technique is not efficient as some events and their attribute information might get lost while the snapshot is taken. Some events might get recorded on processing queue,but not on snapshot. Such events can never get processed.Also,if the test executer equipment is not fast enough, then it can not communicate properly with the system under test. So, faults might get generated during such test evaluation.

References

^ . http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=139423.

^ Software Engineering: A Practitioner's Approach by Roger S Pressman

^ . http://www.sdl-forum.org/issre04-witul/papers/EbnerTTCN3.pdf.

^ . http://www.aws.cit.ie/personnel/dpesch/notes/msc_sw/sdl.pdf.

^ . http://www.ttcn-3.org.

mer.rea� P : > �� � pan>

Snapshot Semantics is a technique in TTCN3 (also in TTCN2), which deals with the message passed during communication by system to system or implementation under test. When a series of responses are received by system under test,then snapshot is taken and they are evaluated in order of their arrival.So,each time around a set of attributes,a snapshot is taken and only those events are evaluated which are present in snapshot.

But this technique is not efficient as some events and their attribute information might get lost while the snapshot is taken. Some events might get recorded on processing queue,but not on snapshot. Such events can never get processed.Also,if the test executer equipment is not fast enough, then it can not communicate properly with the system under test. So, faults might get generated during such test evaluation.

References

1.      ^ . http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=139423.

2.      ^ Software Engineering: A Practitioner's Approach by Roger S Pressman

3.      ^ . http://www.sdl-forum.org/issre04-witul/papers/EbnerTTCN3.pdf.

4.      ^ . http://www.aws.cit.ie/personnel/dpesch/notes/msc_sw/sdl.pdf.

5.      ^ . http://www.ttcn-3.org.

 Web Testing, Example Test cases

WEB TESTING
While testing a web application you need to consider following Cases: 

• Functionality Testing
• Performance Testing
• Usability Testing
• Server Side Interface
• Client Side Compatibility
• Security

Functionality:
In testing the functionality of the web sites the following should be tested:
• Links
i. Internal Links
ii. External Links
iii. Mail Links
iv. Broken Links

• Forms
i. Field validation
ii. Error message for wrong input
iii. Optional and Mandatory fields

• Database
* Testing will be done on the database integrity.

• Cookies
* Testing will be done on the client system side, on the temporary Internet files.

Performance :
Performance testing can be applied to understand the web site’s scalability, or to benchmark the performance in the environment of third party products such as servers and middleware for potential purchase.

• Connection Speed:
Tested over various networks like Dial Up, ISDN etc
• Load:
i. What is the no. of users per time?
ii. Check for peak loads and how system behaves
iii. Large amount of data accessed by user
• Stress:
i. Continuous Load
ii. Performance of memory, CPU, file handling etc..

Usability:
Usability testing is the process by which the human-computer interaction characteristics of a system are measured, and weaknesses are identified for correction.
• Ease of learning
• Navigation
• Subjective user satisfaction
• General appearance

Server Side Interface:
In web testing the server side interface should be tested. This is done by verify that communication is done properly. Compatibility of server with software, hardware, network and database should be tested.

Client Side Compatibility:
The client side compatibility is also tested in various platforms, using various browsers etc.

Security: 
The primary reason for testing the security of a web is to identify potential vulnerabilities and subsequently repair them.
• Network Scanning
• Vulnerability Scanning
• Password Cracking
• Log Review
• Integrity Checkers
• Virus Detection

How can a Web site be tested?

Points to be considered while testing a Web site:

Web sites are essentiallyclient/server applications -
with web servers and ‘browser’ clients.

Consideration should be given to the interactions betweenhtml pages, TCP/IP communications, Internet connections, firewalls, applications that run in web pages (such as applets, javascript, plug-in applications), and applications that run on the server side (such as cgi scripts, database interfaces, logging applications, dynamic page generators, asp, etc.).

Additionally, there are a wide variety of servers and browsers, various versions of each, small but sometimes significant differences between them, variations in connection speeds, rapidly changing technologies, and multiple standards and protocols. The end result is that testing for web sites can become a major ongoing effort.

Other considerations might include:

What are the expected loads on the server (e.g., number of hits per unit time?), and what kind of performance is required under such loads (such as web server response time, database query response times). What kinds of tools will be needed for performance testing (such as web load testing tools, other tools already in house that can be adapted, web robot downloading tools, etc.)?

Who is the target audience? What kind of browsers will they be using? What kind of connection speeds will they by using? Are they intra- organization (thus with likely high connection speeds and similar browsers) or Internet-wide (thus with a wide variety of connection speeds and browser types)?

What kind of performance is expected on the client side (e.g., how fast should pages appear, how fast should animations, applets, etc. load and run)?

Will down time for server and content maintenance/upgrades be allowed? how much?

What kinds of security (firewalls, encryptions, passwords, etc.) will be required and what is it expected to do? How can it be tested?

How reliable are the site’s Internet connections required to be? And how does that affect backup system or redundant connection requirements and testing?

What processes will be required to manage updates to the web site’s content, and

what are the requirements for maintaining, tracking, and controlling page content, graphics, links, etc.?

Which HTML specification will be adhered to? How strictly? What variations will be allowed for targeted browsers?

Will there be any standards or requirements for page appearance and/or graphics throughout a site or parts of a site??

How will internal and external links be validated and updated? how often?
Can testing be done on the production system, or will a separate test system be required? How are browser caching, variations in browser option settings, dial-up connection variabilities, and real-world internet ‘traffic congestion’ problems to be accounted for in testing?

How extensive or customized are the server logging and reporting requirements; are they considered an integral part of the system and do they require testing?

How are cgi programs, applets, javascripts, ActiveX components, etc. to be maintained, tracked, controlled, and tested?
Pages should be 3-5 screens max unless content is tightly focused on a single topic. If larger, provide internal links within the page.

The page layouts and design elements should be consistent throughout a site, so that it’s clear to the user that they’re still within a site.

Pages should be as browser-independent as possible, or pages should be provided or generated based on the browser-type.

All pages should have links external to the page; there should be no dead-end pages.
The page owner, revision date, and a link to a contact person or organization should be included on each page.

I am working on the search engine website. Testing a search engine site is a little bit different than a regular website. In my next posts I will explain how to test WWW in detail.