Retry Utilities

An abstraction of utilities for retrying with exponential back off and jitter.

Overview

The retry utilities are a set of APIs that aid in retrying with exponential backoff and jitter. Exponential backoff with jitter is strongly recommended for retrying failed actions over the network with servers. Please see https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ for more information about the benefits with AWS.

Exponential backoff with jitter is typically used when retrying a failed connection to the server. In an environment with poor connectivity, a client can get disconnected at any time. A backoff strategy helps the client to conserve battery by not repeatedly attempting reconnections when they are unlikely to succeed.

Before retrying the failed communication to the server, there is a delay period. In this delay period, the task that is retrying must sleep for some random amount of seconds between 0 and the lesser of the backoff window (related to the retry attempt) and a predefined maximum delay value. The backoff window is doubled with each retry attempt until the maximum delay value is reached.

sleep_seconds = random_between( 0, min( 2^{attempts_count} * base_seconds, maximum_seconds ) )

Implementing Retry Utils

The functions that must be implemented are:

• RetryUtils_ParamsReset
• RetryUtils_BackoffAndSleep

The functions are used as shown in the diagram below. This is the exponential backoff with jitter loop:

The following steps give guidance on implementing the Retry Utils. An example implementation of the Retry Utils for a POSIX platform can be found in file retry_utils_posix.c.

1. Implementing RetryUtils_ParamsReset

   void RetryUtils_ParamsReset( RetryUtilsParams_t * pRetryParams );

   
   This function initializes RetryUtilsParams_t. It is expected to set RetryUtilsParams_t::attemptsDone to zero. It is also expected to set RetryUtilsParams_t::nextJitterMax to INITIAL_RETRY_BACKOFF_SECONDS plus some random amount of seconds, jitter. This jitter is a random number between 0 and MAX_JITTER_VALUE_SECONDS. This function must be called before entering the exponential backoff with jitter loop using RetryUtils_BackoffAndSleep.
   
   Please follow the example below to implement your own RetryUtils_ParamsReset. The lines with FIXME comments should be updated.

   void RetryUtils_ParamsReset( RetryUtilsParams_t * pRetryParams )
   {
       uint32_t jitter = 0;
    
       // Reset attempts done to zero so that the next retry cycle can start.
       pRetryParams->attemptsDone = 0;
    
       // Seed pseudo random number generator with the current time. FIXME: Your
       // system may have another method to retrieve the current time to seed the
       // pseudo random number generator.
       srand( time( NULL ) );
    
       // Calculate jitter value using picking a random number.
       jitter = ( rand() % MAX_JITTER_VALUE_SECONDS );
    
       // Reset the backoff value to the initial time out value plus jitter.
       pRetryParams->nextJitterMax = INITIAL_RETRY_BACKOFF_SECONDS + jitter;
   }

   
   
2. Implementing RetryUtils_BackoffAndSleep

   RetryUtilsStatus_t RetryUtils_BackoffAndSleep( RetryUtilsParams_t * pRetryParams );

   
   When this function is invoked, the calling task is expected to sleep a random number of seconds between 0 and RetryUtilsParams_t::nextJitterMax. After sleeping this function must double RetryUtilsParams_t::nextJitterMax, but not exceeding MAX_RETRY_BACKOFF_SECONDS. When MAX_RETRY_ATTEMPTS are reached, this function should return RetryUtilsRetriesExhausted, unless MAX_RETRY_ATTEMPTS is set to zero. When RetryUtilsRetriesExhausted is returned, the calling application can stop trying with a failure, or it can call RetryUtils_ParamsReset again, and restart the exponential back off with jitter loop.
   
   Please follow the example below to implement your own RetryUtils_BackoffAndSleep. The lines with FIXME comments should be updated.

   RetryUtilsStatus_t RetryUtils_BackoffAndSleep( RetryUtilsParams_t * pRetryParams )
   {
       RetryUtilsStatus_t status = RetryUtilsRetriesExhausted;
       // The quiet period delay in seconds.
       int backOffDelay = 0;
    
       // If MAX_RETRY_ATTEMPTS is set to 0, try forever.
       if( ( pRetryParams->attemptsDone < MAX_RETRY_ATTEMPTS ) ||
           ( 0 == MAX_RETRY_ATTEMPTS ) )
       {
           // Choose a random value for back-off time between 0 and the max jitter value.
           backOffDelay = rand() % pRetryParams->nextJitterMax;
    
           //  Wait for backoff time to expire for the next retry.
           ( void ) myThreadSleepFunction( backOffDelay ); // FIXME: Replace with your system's thread sleep function.
    
           // Increment backoff counts.
           pRetryParams->attemptsDone++;
    
           // Double the max jitter value for the next retry attempt, only
           // if the new value will be less than the max backoff time value.
           if( pRetryParams->nextJitterMax < ( MAX_RETRY_BACKOFF_SECONDS / 2U ) )
           {
               pRetryParams->nextJitterMax += pRetryParams->nextJitterMax;
           }
           else
           {
               pRetryParams->nextJitterMax = MAX_RETRY_BACKOFF_SECONDS;
           }
    
           status = RetryUtilsSuccess;
       }
       else
       {
           // When max retry attempts are exhausted, let application know by
           // returning RetryUtilsRetriesExhausted. Application may choose to
           // restart the retry process after calling RetryUtils_ParamsReset().
           status = RetryUtilsRetriesExhausted;
           RetryUtils_ParamsReset( pRetryParams );
       }
    
       return status;
   }