Accessing AVSession

In addition to the implementation of audio and video playback, media applications may need to access AVSession provided by AVSession Kit for display and control purposes. This topic describes typical display and control scenarios of accessing AVSession.

In different access scenarios, different UIs are displayed in the controller of the system, and different specifications are posed for application access processing.

Scenarios That Require AVSession Access

AVSession restricts background audio playback and VoIP calls. As such, applications that provide long-duration audio or video playback, audiobook applications, and VoIP applications need to access AVSession. If such an application does not access AVSession, the system stops its audio playback or mutes the ongoing call when detecting that the application is running in the background. In this way, the application behavior is restricted. You can verify the restriction locally before the application is released.

For applications that may use audio playback, such as gaming and live broadcast applications, accessing AVSession is optional. However, if they want to continue audio playback after switching to the background, they must access AVSession.

To implement background playback, the application must also use Background Tasks Kit to request a continuous task to avoid being suspended.

Access Process

The process for implementing AVSession access is as follows:

1. Determine the type of AVSession to be created for the application, and then create one. The AVSession type determines the style of the control template displayed in the controller.
2. Create a background task.
3. Set necessary metadata, which is the response information displayed in the controller. The metadata includes the IDs of the current media asset (assetId), previous media asset (previousAssetId), and next media asset (nextAssetId), title, author, album, writer, and duration.
4. Set playback state information. The information includes the playback state, position, speed, buffered time, loop mode, media item being played (activeItemId), custom media data (extras), and whether the media asset is favorited (isFavorite).
5. Register control commands. The control commands include play, pause, previous, next, fastForward, rewind, toggleFavorite, setLoopMode, and seek.
6. Destroy AVSession when the application exits or stops providing service.

Creating AVSession

AVSessionType in the constructor determines the type of AVSession to create. Different AVSession types represent the control capabilities in various scenarios and display different control templates in the controller.

• For audio AVSession, the controller provides the following control buttons: favorite, previous, play/pause, next, and loop mode.

• For video AVSession, the controller provides the following control buttons: rewind, previous, play/pause, next, and fast-forward.

• For voice_call AVSession, the application is not displayed in the controller.

Refer to the code snippet below:

import AVSessionManager from '@ohos.multimedia.avsession';

// Start to create and activate an AVSession object.
// Create an AVSession object.
let context: Context = getContext(this);
async function createSession() {
let type: AVSessionManager.AVSessionType = 'audio';
let session = await AVSessionManager.createAVSession(context,'SESSION_NAME', type);

// Call activate() after the metadata and control commands are registered.
await session.activate();
  console.info(`session create done : sessionId : ${session.sessionId}`);
}

Creating a Background Task

To implement background playback, the application must also use Background Tasks Kit to request a continuous task to avoid being suspended.

Media playback applications must request a continuous task of the AUDIO_PLAYBACK background mode.

Setting Metadata

Setting Common Metadata

The application can call setAVMetadata() to set AVSession metadata to the system so that the metadata can be displayed in the controller. The metadata includes the IDs of the current media asset (assetId), previous media asset (previousAssetId), and next media asset (nextAssetId), title, author, album, writer, and duration.

import AVSessionManager from '@ohos.multimedia.avsession';
import { BusinessError } from '@ohos.base';

let context: Context = getContext(this);
async function setSessionInfo() {
  // It is assumed that an AVSession object has been created. For details about how to create an AVSession object, see the node snippet above.
  let session = await AVSessionManager.createAVSession(context, 'SESSION_NAME', 'audio');
  // Set necessary AVSession metadata.
  let metadata: AVSessionManager.AVMetadata = {
    assetId: '0', // Specified by the application, used to identify the media asset in the application media library.
    title: 'TITLE',
    artist: 'ARTIST'
  };
  session.setAVMetadata(metadata).then(() => {
    console.info(`SetAVMetadata successfully`);
  }).catch((err: BusinessError) => {
    console.error(`Failed to set AVMetadata. Code: ${err.code}, message: ${err.message}`);
  });
 }

Setting Lyrics

The controller provides the UI to show lyrics. The application only needs to set the lyrics content. The controller parses the lyrics content and displays it based on the playback progress.

import AVSessionManager from '@ohos.multimedia.avsession';
import { BusinessError } from '@ohos.base';

let context: Context = getContext(this);
async function setListener() {
  // It is assumed that an AVSession object has been created. For details about how to create an AVSession object, see the node snippet above.
  let type: AVSessionManager.AVSessionType = 'audio';
  let session = await AVSessionManager.createAVSession(context, 'SESSION_NAME', type);

  // Set the lyric to AVSession.
  let metadata: AVSessionManager.AVMetadata = {
    assetId: '0',
    lyric: 'http://www.test.lyric',
  };
  session.setAVMetadata(metadata).then(() => {
    console.info(`SetAVMetadata successfully`);
  }).catch((err: BusinessError) => {
    console.error(`Failed to set AVMetadata. Code: ${err.code}, message: ${err.message}`);
  });

}

Display Tags of Media Assets

The controller displays a special type identifier for long-duration media assets. Currently, only the AudioVivid identifier is displayed.

The application notifies the system of the display tag of the media asset through the AVMetadata during the access, and the controller displays the tag when the media asset is being played.

import AVSessionManager from '@ohos.multimedia.avsession';
import { BusinessError } from '@ohos.base';

let context: Context = getContext(this);
async function setListener() {
  // It is assumed that an AVSession object has been created. For details about how to create an AVSession object, see the node snippet above.
  let type: AVSessionManager.AVSessionType = 'audio';
  let session = await AVSessionManager.createAVSession(context, 'SESSION_NAME', type);

  // Set the media audio source information to AVSession.
  let metadata: AVSessionManager.AVMetadata = {
    assetId: '0',
    // The display tag of the audio source is AudioVivid.
    displayTags: AVSessionManager.DisplayTag.TAG_AUDIO_VIVID,
  };
  session.setAVMetadata(metadata).then(() => {
    console.info(`SetAVMetadata successfully`);
  }).catch((err: BusinessError) => {
    console.error(`Failed to set AVMetadata. Code: ${err.code}, message: ${err.message}`);
  });
}

Setting Playback State Information

Setting General State Information

The application can call setAVPlaybackState() to set the playback state information to the system so that the information can be displayed in the controller.

Generally, the playback state information includes the playback state, position, speed, buffered time, loop mode, media item being played (activeItemId), custom media data (extras), and whether the media asset is favorited (isFavorite). It changes during the playback.

import AVSessionManager from '@ohos.multimedia.avsession';
import { BusinessError } from '@ohos.base';

let context: Context = getContext(this);
async function setSessionInfo() {
  // It is assumed that an AVSession object has been created. For details about how to create an AVSession object, see the node snippet above.
  let session = await AVSessionManager.createAVSession(context, 'SESSION_NAME', 'audio');

  // The player logic that triggers changes in the AVSession metadata and playback state information is omitted here.
  // Set the playback state to paused and set isFavorite to false.
  let playbackState: AVSessionManager.AVPlaybackState = {
    state:AVSessionManager.PlaybackState.PLAYBACK_STATE_PAUSE,
    isFavorite:false
  };
  session.setAVPlaybackState(playbackState, (err: BusinessError) => {
   if (err) {
      console.error(`Failed to set AVPlaybackState. Code: ${err.code}, message: ${err.message}`);
    } else {
      console.info(`SetAVPlaybackState successfully`);
    }
  });
}

Setting the Progress Bar

To display a progress bar in the controller, the application must set the duration, playback state (pause or play), playback position, and playback speed. The controller displays the progress bar based on the information.

import AVSessionManager from '@ohos.multimedia.avsession';

let context: Context = getContext(this);
async function setListener() {
  // It is assumed that an AVSession object has been created. For details about how to create an AVSession object, see the node snippet above.
  let type: AVSessionManager.AVSessionType = 'audio';
  let session = await AVSessionManager.createAVSession(context, 'SESSION_NAME', type);

  // Set the playback state information, including the playback state, position, speed, buffered time, and duration.
  let playbackState: AVSessionManager.AVPlaybackState = {
    state: AVSessionManager.PlaybackState.PLAYBACK_STATE_PLAY, // Playing state.
    position: {
      elapsedTime: 1000, // Playback position, in milliseconds.
      updateTime: 30000, // Timestamp when the application updates the current position, in milliseconds.
    },
    speed: 1.0, // Optional. The default value is 1.0. The playback speed is set based on the speed supported by the application. The system does not verify the speed.
    bufferedTime: 14000, // Optional. Buffered time, in milliseconds.
    duration: 23000, // Duration of the media asset, in milliseconds.
  };
  session.setAVPlaybackState(playbackState, (err) => {
    if (err) {
      console.error(`Failed to set AVPlaybackState. Code: ${err.code}, message: ${err.message}`);
    } else {
      console.info(`SetAVPlaybackState successfully`);
    }
  });
}

The controller calculates the playback progress based on the information set by the application. The application does not need to update the playback progress in real time. However, it needs to update the playback state when the following information changes to avid calculation errors:

• state
• position
• speed

Certain special processing is required when setting the progress bar.

1. Songs that can be previewed

   If a VIP song can be previewed, the application should set the preview duration of the song, rather than the total duration.

   If only the preview duration is set, when the user triggers progress control in the controller, the application receives the relative timestamp within the preview duration, rather than that within the total duration. The application needs to calculate the absolute timestamp from the very beginning of the song.

2. Songs that do not support preview

   If a song cannot be previewed, it cannot be displayed by the application. In this case, the application should set the duration to -1, so the system does not display the actual duration.

3. Special contents such as ads

   For media assets with pre-roll or post-roll ads, you are advised to:

   • Set the ad duration separately.
   • Set a new duration for the actual content, to distinguish it from the ad.

Registering Control Commands

The application can register different control commands through on() to implement control operations in the controller. For details, see the API reference.

NOTE

After an AVSession object is created, register control commands supported by the application before activating the object.

The table below lists the control commands supported by media assets.

The table below lists the control commands for calling applications.

Handling Unsupported Commands

If the application does not support a control command supported by the system, for example, the playPrevious command, it can use off() to deregister the control command. Then the controller grays out the control page accordingly, so that users know that the control command is not supported.

import AVSessionManager from '@ohos.multimedia.avsession';

let context: Context = getContext(this);
async function unregisterSessionListener() {
  // It is assumed that an AVSession object has been created. For details about how to create an AVSession object, see the node snippet above.
  let type: AVSessionManager.AVSessionType = 'audio';
  let session = await AVSessionManager.createAVSession(context, 'SESSION_NAME', type);

  // Cancel the listener of the AVSession object.
  session.off('play');
  session.off('pause');
  session.off('stop');
  session.off('playNext');
  session.off('playPrevious');
}

Setting Fast-Forward or Rewind

The application can call APIs to set the fast-forward or rewind intervals in three different ways. It also registers the fast-forward or rewind control command to respond to user operations.

import AVSessionManager from '@ohos.multimedia.avsession';
import { BusinessError } from '@ohos.base';

let context: Context = getContext(this);
async function unregisterSessionListener() {
  // It is assumed that an AVSession object has been created. For details about how to create an AVSession object, see the node snippet above.
  let type: AVSessionManager.AVSessionType = 'audio';
  let session = await AVSessionManager.createAVSession(context, 'SESSION_NAME', type);

  // Set the supported fast-forward or rewind duration for AVSession.
  let metadata: AVSessionManager.AVMetadata = {
    assetId: '0', // Specified by the application, used to identify the media asset in the application media library.
    skipIntervals: SkipIntervals.SECONDS_10,
  };
  session.setAVMetadata(metadata).then(() => {
    console.info(`SetAVMetadata successfully`);
  }).catch((err: BusinessError) => {
    console.error(`Failed to set AVMetadata. Code: ${err.code}, message: ${err.message}`);
  });

  session.on('fastForward', (time ?: number) => {
    console.info(`on fastForward , do fastForward task`);
    // do some tasks ···
  });
  session.on('rewind', (time ?: number) => {
    console.info(`on rewind , do rewind task`);
    // do some tasks ···
  });
}

Favoriting Media Assets

To implement favoriting, a music application must call on('toggleFavorite') to register the toggleFavorite control command.

import AVSessionManager from '@ohos.multimedia.avsession';
import { BusinessError } from '@ohos.base';

let context: Context = getContext(this);
async function setListener() {
 // It is assumed that an AVSession object has been created. For details about how to create an AVSession object, see the node snippet above.
 let type: AVSessionManager.AVSessionType = 'audio';
 let session = await AVSessionManager.createAVSession(context, 'SESSION_NAME', type);
 session.on('toggleFavorite', (assetId) => {
   console.info(`on toggleFavorite `);
   // The application receives the toggleFavorite command and favorites or unfavorites the media asset.

   // Set the new state to AVSession after the application finishes favoriting or unfavoriting.
   let playbackState: AVSessionManager.AVPlaybackState = {
     isFavorite:true,
   };
   session.setAVPlaybackState(playbackState).then(() => {
     console.info(`SetAVPlaybackState successfully`);
   }).catch((err: BusinessError) => {
     console.info(`SetAVPlaybackState BusinessError: code: ${err.code}, message: ${err.message}`);
   });

 });
}

Setting the Loop Mode

For music applications, the controller displays control operations in loop mode by default. Currently, the system supports four fixed loop modes, namely, shuffle, sequential playback, single loop, and playlist loop. The controller notifies the application of the loop mode changes, and the application responds accordingly.

Refer to the code snippet below:

import AVSessionManager from '@ohos.multimedia.avsession';
import { BusinessError } from '@ohos.base';

let context: Context = getContext(this);
async function setListener() {
 // It is assumed that an AVSession object has been created. For details about how to create an AVSession object, see the node snippet above.
 let type: AVSessionManager.AVSessionType = 'audio';
 let session = await AVSessionManager.createAVSession(context, 'SESSION_NAME', type);

 // When the application starts, it sets the loop mode to AVSession.
 let playBackState: AVSessionManager.AVPlayBackState = {
   loopMode: AVSessionManager.LoopMode.LOOP_MODE_SINGLE,
 };
 session.setAVPlayBackState(playBackState).then(() => {
   console.info(`set AVPlayBackState successfully`);
 }).catch((err: BusinessError) => {
   console.error(`Failed to set AVPlayBackState. Code: ${err.code}, message: ${err.message}`);
 });

 // The application listens for loop mode changes.
 session.on('setLoopMode', (mode) => {
   console.info(`on setLoopMode ${mode}`);
   // After receiving the setLoopMode command, the application switches to the corresponding loop mode.
 });

}

Performing Progress Control

An application that supports progress display can further supports progress control. To support progress control, the application must respond to the seek control command. When users drag the progress bar in the controller, the application receives a callback. Refer to the code snippet below:

import AVSessionManager from '@ohos.multimedia.avsession';

let context: Context = getContext(this);
async function setListener() {
 // It is assumed that an AVSession object has been created. For details about how to create an AVSession object, see the node snippet above.
 let type: AVSessionManager.AVSessionType = 'audio';
 let session = await AVSessionManager.createAVSession(context, 'SESSION_NAME', type);

 session.on('seek', (time: number) => {
   console.info(`on seek , the time is ${JSON.stringify(time)}`);

   // The seek operation may trigger a long buffering time. Generally, set the playback state to PLAYBACK_STATE_BUFFERING.
   let playbackState: AVSessionManager.AVPlaybackState = {
     state: AVSessionManager.PlaybackState.PLAYBACK_STATE_BUFFERING, // Buffering state.
   };
   session.setAVPlaybackState(playbackState, (err) => {
     if (err) {
       console.error(`Failed to set AVPlaybackState. Code: ${err.code}, message: ${err.message}`);
     } else {
       console.info(`SetAVPlaybackState successfully`);
     }
   });

   // The application responds to the seek command and seeks to the specified position.

   // After seeking to the specified position, the application synchronizes the new position to the system.
   playbackState.state = AVSessionManager.PlaybackState.PLAYBACK_STATE_PLAY; // Playing state.
   playbackState.position = {
     elapsedTime: 4000, // Playback position, in milliseconds.
     updateTime: 34000, // Timestamp when the application updates the current position, in milliseconds.
   }
   session.setAVPlaybackState(playbackState, (err) => {
     if (err) {
       console.error(`Failed to set AVPlaybackState. Code: ${err.code}, message: ${err.message}`);
     } else {
       console.info(`SetAVPlaybackState successfully`);
     }
   });

 });
}

Adapting to Media Notification

Currently, the system does not provide APIs for proactively sending control notifications to applications. When an application enters the playing state, the system automatically sends a notification and displays the notification in the notification center and on the lock screen.

NOTE

Currently, notifications are displayed for audio AVSession, but not video AVSession.

The system sends playback control widgets in the notification center and on the lock screen and controls their lifecycle.