scope:

In this Recommendation, a subjective test procedure is described which is able to assess the quality of speech or audio-visual telephone calls, the so-called "call quality". In contrast to standard listening-only tests as described in [P.800], call quality refers to the experience of a user during a typically 1-2 min long call. Such experiences are commonly addressed in conversation tests as described in [P.800] and [P.805], however real conversation tests commonly lack analytical power, because naïve test participants have to concentrate mainly on the flow of the conversation, and are unable to rate instances of good or bad quality. In turn, methods for the continuous assessment of quality as described in [P.880] do not put the listener in a situation which is comparable to a conversation.

In order to circumvent this problem, ETSI has developed a method for so-called "simulated conversation tests" in its TR 102 506 (2011), which has been adopted here and extended to audiovisual telephony. Simulated conversations consist of a number of short (8...12 s) stretches of speech, typically five, which are presented to the listening or listening-and-viewing test participants, and which follow a logical storyline. After each stretch of speech, the test participant has to answer a short content-related question, e.g. in terms of an answer to a multiple-choice question. Following the answer, the next stretch of speech or audio-visual material is presented, followed by the next question, and so forth until the end of the logical sequence is reached. Upon the end of the sequence, the test participant has to judge the perceived quality of the entire call on a standard (typically 5-point ACR according to [P.800] and [P.911]) quality scale. Those multiple-choice questions may refer to information transmitted via the audio, video or audio-visual modality.

Quality of short segments can either be predicted with subjective methods, e.g. [P.911] for audiovisual segments or [P.800] for speech segments, or can be assessed with instrumental models like [P.863] or [J.341], in order to analyse the relationship between the quality of individual samples and the quality of the entire call as assessed with P.ACQ.

The method is not intended to replace listening-only tests according to [P.800] or listening-andviewing tests according to [P.911], or speech conversation tests according to [P.800] or audio-visual conversation tests according to [P.920]. Rather, it is recommended to make use of the appropriate method as follows:

• In case that the listening-only or listening-and-viewing quality is of interest and that the quality is considered to be constant throughout the duration of a speech-only or audio-visualcall, the method according to [P.800] or [P.911] should be used. [P.800] is, however, not recommended for longer durations.1 
• In case that the listening-and-viewing quality is of interest and that the quality is considered not to be constant throughout the duration of a speech-only or audio-visual call, the present method should be used; in contrast to the task of rating overall quality in [P.880] it addresses also audio-visual material and aims at establishing a more valid situation by introducing active participation of the participants 
• In case that conversational quality which may be impacted by conversation-specific degradations like echo or delay is of interest, the method according to [P.800], [P.805] or [P.920] should be used