When designing an application that shares audio and video you should determine which functionality is most important to your end users. Just as responsive web design allows a single web application to adapt to changing screen sizes, the audio-video component of your application can also be responsive to changing device and network conditions. For example, in a collaborative meeting it may be acceptable to gracefully degrade to an audio-only experience. However, in a presentation where the screen is shared, the presenter audio and screen share video may be the most important.
To join an Amazon Chime SDK meeting, each client traverses the public internet to connect to the media services for that meeting which are hosted by the AWS Global Infrastructure in one of the available SDK media regions. The following key factors influence the client experience:
Metrics derived from WebRTC stats are not guaranteed to be present in all browsers. In such cases the value may be missing.
For the browser support columns below, "All" refers to the browsers officially supported by the Chime SDK.
|Indicates that the audio connection is experiencing packet loss. Stopping local video and pausing remote video tiles may help the connection recover by reducing CPU usage and network consumption.
|Indicates that the audio connection has improved.
|Similar to the previous metric, but is fired when local video is already turned off.
signalStrength field indicates whether the server is receiving the remote attendee's audio. A value of 1 indicates a good connection, a value of 0.5 or 0 indicates some or total packet loss. Since each attendee receives the signal strength for all attendees, this event can be used to monitor the ability of attendees to share their audio in real-time.
|Exposes the WebRTC getStats metrics. There may be differences among browsers as to which metrics are reported.
Note: We have built a video WebRTC statistics widget in the demo browser meeting app. This widget is shown as an overlay on the video, when the video is enabled.
You get the following WebRTC stats in the video stats widget:
Upstream video information: Frame height, Frame width, Bitrate (bps), Packets Sent, Frame Rate (fps), Audio Jitter.
Downstream video information: Frame height, Frame width, Bitrate (bps), Packet Loss (%), Frame Rate (fps).
You can also use this event to get the raw
You can check the implementation in the demo app to build your own custom widget (look for
getObservableVideoMetricsmethod in the demo). This video stats widget is built using the getObservableVideoMetrics and the metricsDidReceive APIs. Through the information provided in these stats, the application can monitor key attributes and take action. For instance if the bitrate or resolution falls below a certain threshold, the user could be notified in some manner, or diagnostic reporting could take place.
|Called when simulcast is enabled and simulcast uplink encoding layers are changed. Check the simulcast guide for more information on simulcast.
WebRTC will automatically reduce video frame rate, resolution, and bandwidth if it detects that it is unable to send video at the specified rate due to bandwidth or CPU.
Use the browser's built-in developer tools to profile your application and determine whether there are any hotspots. When handling real-time events (prefixed with
realtime) ensure that you are doing as little processing as possible. See the API Overview section on building a roster for more information. In particular, look out for expensive DOM updates (such as when manipulating the roster or video tile layout).
When possible, profile on devices that have similar performance characteristics to the ones you expect to be used by your end users.
Sometimes it is better to sacrifice video quality in order to prioritize audio. You can call chooseVideoInputQuality(width, height, frameRate) and setVideoMaxBandwidthKbps(maxBandwidthKbps) to lower the maximum bandwidth in real-time. You can also adjust the resolution and frame rate if you call the method before starting local video (or stop and then restart the local video). See the section below on values you can use for
Calling pauseVideoTile on remote video tiles will reduce the amount of CPU consumed due to decoding remote attendee video.
In the absence of packet loss, keep in mind that the sender uplink and receiver downlink consume the same bandwidth for each video tile. Mitigations affecting one sender's uplink can benefit all receiver's downlinks. This means that in order to help receiver's, sometimes the best course of action is to lower the bandwidth consumed by the sender.
You can choose a video quality of up to 1280x720 (720p) at 30 fps using chooseVideoInputQuality(width, height, frameRate) and setVideoMaxBandwidthKbps(maxBandwidthKbps) APIs before the meeting session begins. However, in some cases it is not necessary to send the highest quality and you can use a lower values. For example, if the size of the video tile is small then the highest quality may not be worth the additional bandwidth and CPU overhead.
meetingSession.audioVideo.chooseVideoInputQuality(640, 360, 15);
meetingSession.audioVideo.chooseVideoInputQuality(960, 540, 15);
meetingSession.audioVideo.chooseVideoInputQuality(1280, 720, 15);
The default resolution in the SDK is 540p at 15 fps and 1400 Kbps. Lower resolutions can be set if you anticipate a low bandwidth situation. Browser and codec support for very low resolutions may vary.
maxBandwidthKbps is a recommendation you make to WebRTC to use as the upper limit for upstream sending bandwidth. The Chime SDK default is 1400 Kbps for this value. The following table provides recommendations of minimum bandwidth value per resolution for typical video-conferencing scenarios. Note that when low values are selected the video can be appear pixelated. Using 15 fps instead of 30 fps will substantially decrease the required bit rate and may be acceptable for low-motion content.
|Frame Rate Per Sec
Setting a frame rate below 15 is not recommend and will cause the video to appear jerky and will not significantly improve the bandwidth consumed since key frames will still be sent occasionally. It would be better to adjust the resolution than set a very low frame rate.
In some situations it may be best to turn off the local video tile to improve audio uplink, CPU consumption, or remote attendee downlink.
The SDK by default uses the NScaleVideoUplinkBandwidthPolicy which monitors number of participants in the meeting and automatically scales down the maxBandwidthKbps as the number of remote video tiles increases. This can be customized by implementing a VideoUplinkBandwidth Policy and setting it in the MeetingSessionConfiguration class.
When more video is being received than available estimated downlink bandwidth can support, the event videoNotReceivingEnoughData can is triggered with a list of attendee IDs and the bandwidth being consumed due to them. You can use this information to selectively pause attendees that are sending the highest bitrate video streams using pauseVideoTile. When a video tile is paused, the action only affects your client. It does not pause the video for other attendees.
You can use the active speaker detector to show only the video of the active speakers and pause other videos.
By default the SDK uses the AllHighestVideoBandwidthPolicy, but VideoAdaptiveProbePolicy is used if simulcast is enabled. AllHighestVideoBandwidthPolicy subscribes to the highest quality video of all participants and VideoAdaptiveProbePolicy tries to subscribe to videos based on the browser downlink estimate, dropping videos in case of not enough bandwidth.
Browser clients currently only send one stream resolution. You would only need to use this function if you were also using the Amazon Chime SDK for iOS or the Amazon Chime SDK for Android.
Generated using TypeDoc