UNLIMITED FREE ACCESS TO THE WORLD'S BEST IDEAS

close
Already an Engineering360 user? Log in.

This is embarrasing...

An error occurred while processing the form. Please try again in a few minutes.

Customize Your Engineering360 Experience

close
Privacy Policy

This is embarrasing...

An error occurred while processing the form. Please try again in a few minutes.

SNZ AS/NZS ISO/IEC 25045

Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Evaluation module for recoverability

active, Most Current
Buy Now
Organization: SNZ
Publication Date: 24 May 2013
Status: active
Page Count: 48
scope:

This International Standard is one of the SQuaRE series of International Standards, which contains general requirements for specification and evaluation of systems and software quality and clarifies the associated general concepts. SQuaRE provides a framework for evaluating the quality of software products and states the requirements for methods of software product measurement and evaluation.

This International Standard uses a methodology involving two types of evaluation for recoverability. One part of the method makes use of the disturbance injection methodology and a list of disturbances based on common categories of operational faults and events to evaluate the quality measure of resiliency. The second quality measure is based on a set of questions that is defined for each disturbance to evaluate the quality measure of autonomic recovery index by assessing how well the system detects, analyses, and resolves the disturbance without human intervention.

This International Standard is applicable to information systems executing transactions in a system supporting single or multiple concurrent users, where speedy recovery and ease of managing recovery is important to the acquirer, owner/operator, and the developer.

Characteristics

This evaluation module measures the quality measures defined under the following characteristic and sub-characteristics of the quality model as defined in ISO/IEC 9126-1:2001.

NOTE The reference to ISO/IEC 9126-1 will be replaced by a reference to ISO/IEC 25010 when published.

Characteristic - Reliability

Sub-characteristic - Recoverability

Quality measure - Resiliency

Quality measure - Autonomic recovery index 

Level of evaluation

Level D as defined in ISO/IEC 14598-5. This evaluation is intended for a system with executable products.

NOTE The reference to ISO/IEC 14598-5 will be replaced by a reference to ISO/IEC 25040 when published.

Technique

A disturbance injection methodology is a test methodology where disturbances are injected against the application and other components of the system while it is running a workload of interest to the acquirer. A disturbance injection methodology and a list of disturbances based on common categories of operational faults and events are used to evaluate the quality measure of Resiliency. For each disturbance, the Resiliency of the system is calculated based on the ratio between the number of transactions that complete successfully while the system is under disturbance and the number of transactions that complete successfully in a system that does not encounter the disturbance. A set of disturbances is defined under the following categories:

• Unexpected shutdown - e.g. abrupt operating system (OS) shutdown, process shutdown, network shutdown;

• Resource contention - e.g. CPU/memory/IO hogs, memory leak, database management system (DBMS) runaway query, DBMS deadlock, DBMS and queuing server storage exhaustion;

• Loss of data - e.g. DBMS loss of data, DBMS loss of file, DBMS and queuing server loss of disk;

• Load resolution - e.g. a moderate or significant increase of users or workload;

• Restart failures - e.g. restart failure on OS and middleware server process.

Other disturbance categories may be identified if appropriate.

A set of questions to assess how well the system detects, analyses, and resolves the disturbance is defined for each disturbance to evaluate the quality measure of autonomic recovery index. A score is calculated for each disturbance based on the answers to those questions.

The overall Resiliency and autonomic recovery index are calculated respectively as an average of those individual scores.

The detailed evaluation methodology involved is given in 5.1.

Applicability

This evaluation module is applicable to an information system that involves a software product and other software components. The information system must have a workload that has a consistently reproducible performance result to properly assess the impact of disturbance and recovery.

The evaluation module can be used in the following situations:

a) evaluation as part of the system verification testing;

b) evaluation against the test environment of a production system to gauge recoverability and identify weakness;

c) evaluation of the recoverability of different solutions proposed by vendors using a common workload.

The evaluation result is only applicable to the specific release and configuration of the software and hardware components on which they were evaluated. Two results are comparable if they use the same workload and workload parameter set defined in 5.2.2.2 and fault load and fault load parameter set defined in 5.2.3.2 for the evaluation.

Document History

SNZ AS/NZS ISO/IEC 25045
May 24, 2013
Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Evaluation module for recoverability
This International Standard is one of the SQuaRE series of International Standards, which contains general requirements for specification and evaluation of systems and software quality and clarifies...

References

Advertisement