scope:

This Recommendation concerns subjective quality assessment of telemeeting systems that provide communication between multiple parties at remote locations. As multiparty telemeetings can differ in a large number of aspects, the assessment of such systems requires a proper selection and control of the considered aspects and a precise description when reporting results.

The main aspects defining the scope of this Recommendation are:

• Number of participants and number of locations

The term multiparty refers to more than two meeting participants who can be located at two, or more than two, locations. Hence, several multiparty situations are possible on which methodological aspects could depend: two sites with more than one person at at least one site (multiparty point-to-point), more than two sites with one person at each site (multiparty one-per-site), and more than two sites with more than one person at at least one site (multiparty multi-point).

• Communication mode and rendering conditions

The telemeeting systems considered in this Recommendation can provide audio-only, video-only (i.e., sign language or lip reading), or audiovisual communication. Telemeeting systems can render communication modes using different techniques such as mono channel vs. spatial sound reproduction or 2D video vs. 3D video display. Furthermore, web conferencing applications are considered as telemeeting systems that can provide additional text-based (chat, e-mail, etc.), and graphical information means (presentation slides, etc.).

• Evaluation mode and type of quality

Evaluation of multiparty telemeeting systems can focus on audio-only, video-only, or audiovisual quality aspects and it can assess non-interactive or conversational/interactive quality. Hence, the assessment of multiparty telemeetings quality can be organized along five communication modes (audio, video, audiovisual, text-based, graphical), three test modes (audio, video, audiovisual), and two types of qualities (non-interactive, conversational/interactive).

• Controlled and non-controlled environments

Assessment tests can be conducted in a laboratory or in a real-life environment where the system is supposed to be used, for example, in a telepresence room. This Recommendation concerns testing in both controlled and not controlled environments. Accordingly, the test environments should be properly described and specified when reporting the test results.

• Symmetric and asymmetric set-ups

All telemeeting participants can be connected with the same type of equipment (symmetric) or different types of equipment (asymmetric). This Recommendation concerns testing of both symmetric and asymmetric telemeeting set-ups.

At the moment of writing this Recommendation, a number of ITU-T and ITU-R Recommendations are in force describing subjective quality evaluation methods; each of those methods focusing on individual communication modes, test modes, or types of quality. Table 1 provides a corresponding overview. Note that in order to evaluate several quality aspects of a system intended for multiparty telemeetings, several assessment tests might be needed to execute.