Quality, Bandwidth, and Connectivity

The Amazon Chime SDK for JavaScript, in conjunction with WebRTC, allows you to configure quality and bandwidth options for meeting sessions. It provides both automatic and manual controls to adapt to changing network conditions. Finding the right balance of quality and performance can be challenging when either the client's network or CPU is constrained. As network reliability and available bandwidth decrease, quality and functionality trade-offs must be made in real time to preserve the viability of the meeting. This guide gives an overview of challenges associated with devices and networks, degradation detection mechanisms, and mitigations for various levels of degradation.

Quality, Bandwidth, and Connectivity

Goals

When designing an application that shares audio and video you should determine which functionality is most important to your end users. Just as responsive web design allows a single web application to adapt to changing screen sizes, the audio-video component of your application can also be responsive to changing device and network conditions. For example, in a collaborative meeting it may be acceptable to gracefully degrade to an audio-only experience. However, in a presentation where the screen is shared, the presenter audio and screen share video may be the most important.

Challenges

To join an Amazon Chime SDK meeting, each client traverses the public internet to connect to the media services for that meeting which are hosted by the AWS Global Infrastructure in one of the available SDK media regions. The following key factors influence the client experience:

Application performance: if your client application is consuming a lot of CPU, then it may hinder the device from simultaneously processing media and transmitting and receiving network packets.
Client device performance: transcoding audio and video in real-time is CPU intensive and may both decrease performance and further constrain the device's network adapter. Browsers may or may not support hardware acceleration depending on the codec profile and device hardware. Furthermore, browsers do not allow you to directly monitor CPU usage, so it can be difficult to tell whether CPU consumption is a factor in performance issues for your end-user devices.
Differences in uplink and downlink: In order for an attendee to send audio or video to a remote attendee, both the sender's uplink and the receiver's downlink must have enough bandwidth. However, most end users will be on a network that has more downlink (receiving) bandwidth than uplink (sending) bandwidth. It can be challenging to determine whether a drop in quality is due to constraints of the sender, the receiver, or both.
Last mile network connectivity: issues with WiFi, local area network, or the internet service provider may limit the effective bandwidth due to packet loss, bandwidth caps, or other hardware limitations.
Complexity and length of the network path: meeting attendees that are geographically distant from the SDK media region may experience higher packet loss, jitter, and round-trip times due to the number of intermediate networks, which may vary in quality.

Detection Mechanisms

The Amazon Chime SDK for JavaScript produces several kinds of events on the AudioVideoObserver to monitor connectivity and quality. Use the following events and key health metrics to monitor the performance of the meeting session in real time. For code snippets showing how to subscribe to these events, see Monitoring and Alerts.

Metrics derived from WebRTC stats are not guaranteed to be present in all browsers. In such cases the value may be missing.

For the browser support columns below, "All" refers to the browsers officially supported by the Chime SDK.

Events for monitoring local attendee downlink

Event	Notes	Browsers
connectionDidSuggestStopVideo	Indicates that the audio connection is experiencing packet loss. Stopping local video and pausing remote video tiles may help the connection recover by reducing CPU usage and network consumption.	All
connectionDidBecomeGood	Indicates that the audio connection has improved.	All
connectionDidBecomePoor	Similar to the previous metric, but is fired when local video is already turned off.	All

Events for monitoring remote attendee uplink

Event	Notes	Browsers
realtimeSubscribeToVolumeIndicator	The `signalStrength` field indicates whether the server is receiving the remote attendee's audio. A value of 1 indicates a good connection, a value of 0.5 or 0 indicates some or total packet loss. Since each attendee receives the signal strength for all attendees, this event can be used to monitor the ability of attendees to share their audio in real-time.	All

Metrics exposed directly from the WebRTC peer connection

Event	Notes	Browsers
metricsDidReceive	Exposes the WebRTC getStats metrics. There may be differences among browsers as to which metrics are reported.	All

Note: We have built a video WebRTC statistics widget in the demo browser meeting app. This widget is shown as an overlay on the video, when the video is enabled.

You get the following WebRTC stats in the video stats widget:

Upstream video information: Frame height, Frame width, Bitrate (bps), Packets Sent, Frame Rate (fps), Audio Jitter.

Downstream video information: Frame height, Frame width, Bitrate (bps), Packet Loss (%), Frame Rate (fps).

You can also use this event to get the raw RTCStatsReport through clientMetricReport.getRTCStatsReport()

You can check the implementation in the demo app to build your own custom widget (look for getObservableVideoMetrics method in the demo). This video stats widget is built using the getObservableVideoMetrics and the metricsDidReceive APIs. Through the information provided in these stats, the application can monitor key attributes and take action. For instance if the bitrate or resolution falls below a certain threshold, the user could be notified in some manner, or diagnostic reporting could take place.

Events for monitoring currently active simulast layers

Event	Notes	Browsers
encodingSimulcastLayersDidChange	Called when simulcast is enabled and simulcast uplink encoding layers are changed. Check the simulcast guide for more information on simulcast.	Chromium-based

Mitigations

Automatic mitigations

WebRTC will automatically reduce video frame rate, resolution, and bandwidth if it detects that it is unable to send video at the specified rate due to bandwidth or CPU.

Mitigations to conserve CPU

Application profiling

Use the browser's built-in developer tools to profile your application and determine whether there are any hotspots. When handling real-time events (prefixed with realtime) ensure that you are doing as little processing as possible. See the API Overview section on building a roster for more information. In particular, look out for expensive DOM updates (such as when manipulating the roster or video tile layout).

When possible, profile on devices that have similar performance characteristics to the ones you expect to be used by your end users.

Choose a lower local video quality

Sometimes it is better to sacrifice video quality in order to prioritize audio. You can call chooseVideoInputQuality(width, height, frameRate) and setVideoMaxBandwidthKbps(maxBandwidthKbps) to lower the maximum bandwidth in real-time. You can also adjust the resolution and frame rate if you call the method before starting local video (or stop and then restart the local video). See the section below on values you can use for chooseVideoInputQuality.

Pause remote videos

Calling pauseVideoTile on remote video tiles will reduce the amount of CPU consumed due to decoding remote attendee video.

Mitigations to conserve bandwidth

In the absence of packet loss, keep in mind that the sender uplink and receiver downlink consume the same bandwidth for each video tile. Mitigations affecting one sender's uplink can benefit all receiver's downlinks. This means that in order to help receiver's, sometimes the best course of action is to lower the bandwidth consumed by the sender.

Adjust local video quality

You can choose a video quality of up to 1280x720 (720p) at 30 fps using chooseVideoInputQuality(width, height, frameRate) and setVideoMaxBandwidthKbps(maxBandwidthKbps) APIs before the meeting session begins. However, in some cases it is not necessary to send the highest quality and you can use a lower values. For example, if the size of the video tile is small then the highest quality may not be worth the additional bandwidth and CPU overhead.

Example:

// 360p
meetingSession.audioVideo.chooseVideoInputQuality(640, 360, 15);
meetingSession.audioVideo.setVideoMaxBandwidthKbps(600);

// 540p
meetingSession.audioVideo.chooseVideoInputQuality(960, 540, 15);
meetingSession.audioVideo.setVideoMaxBandwidthKbps(1400);

// 720p
meetingSession.audioVideo.chooseVideoInputQuality(1280, 720, 15);
meetingSession.audioVideo.setVideoMaxBandwidthKbps(1400);

The default resolution in the SDK is 540p at 15 fps and 1400 Kbps. Lower resolutions can be set if you anticipate a low bandwidth situation. Browser and codec support for very low resolutions may vary.

The value maxBandwidthKbps is a recommendation you make to WebRTC to use as the upper limit for upstream sending bandwidth. The Chime SDK default is 1400 Kbps for this value. The following table provides recommendations of minimum bandwidth value per resolution for typical video-conferencing scenarios. Note that when low values are selected the video can be appear pixelated. Using 15 fps instead of 30 fps will substantially decrease the required bit rate and may be acceptable for low-motion content.

Resolution	Frame Rate Per Sec	Min Kbps
180p	30	100
360p	30	250
480p	30	400
540p	30	500
720p	30	1400

Setting a frame rate below 15 is not recommend and will cause the video to appear jerky and will not significantly improve the bandwidth consumed since key frames will still be sent occasionally. It would be better to adjust the resolution than set a very low frame rate.

Turning off local video

In some situations it may be best to turn off the local video tile to improve audio uplink, CPU consumption, or remote attendee downlink.

You can also observe the connectionDidSuggestStopVideo event to monitor audio packet loss and use that as a cue to turn off local video using stopLocalVideoTile.

Configure a video uplink policy

The SDK by default uses the NScaleVideoUplinkBandwidthPolicy which monitors number of participants in the meeting and automatically scales down the maxBandwidthKbps as the number of remote video tiles increases. This can be customized by implementing a VideoUplinkBandwidth Policy and setting it in the MeetingSessionConfiguration class.

Pause remote attendee video

When more video is being received than available estimated downlink bandwidth can support, the event videoNotReceivingEnoughData can is triggered with a list of attendee IDs and the bandwidth being consumed due to them. You can use this information to selectively pause attendees that are sending the highest bitrate video streams using pauseVideoTile. When a video tile is paused, the action only affects your client. It does not pause the video for other attendees.

Use active speaker detection

You can use the active speaker detector to show only the video of the active speakers and pause other videos.

Configure a video downlink policy

By default the SDK uses the AllHighestVideoBandwidthPolicy, but VideoAdaptiveProbePolicy is used if simulcast is enabled. AllHighestVideoBandwidthPolicy subscribes to the highest quality video of all participants and VideoAdaptiveProbePolicy tries to subscribe to videos based on the browser downlink estimate, dropping videos in case of not enough bandwidth.

You can customize the default downlink policy by setting the VideoDownlinkBandwidthPolicy in MeetingSessionConfiguration class.

Browser clients currently only send one stream resolution. You would only need to use this function if you were also using the Amazon Chime SDK for iOS or the Amazon Chime SDK for Android.

Use video simulcast

Simulcast is a standardized technique in WebRTC to enhance video quality in a multi-party call. Amazon Chime SDK for JavaScript provides a video simulcast solution for Chromium-based browsers. To use simulcast, please refer to the documentation here. Give feedback on this guide