skip to Main Content

I am building a video conferencing app using LiveKit in Flutter and want to implement Picture-in-Picture (PiP) mode on iOS. My goal is to display a view showing the speaker’s initials or avatar during PiP mode. I successfully implemented this functionality on Android but am struggling to achieve it on iOS.

I am using a MethodChannel to communicate with the native iOS code. Here’s the Flutter-side code:

import 'package:flutter/foundation.dart';
import 'package:flutter/services.dart';

class PipController {
  static const _channel = MethodChannel('pip_channel');

  static Future<void> startPiP() async {
    try {
      await _channel.invokeMethod('enterPiP');
    } catch (e) {
      if (kDebugMode) {
        print("Error starting PiP: $e");
      }
    }
  }

  static Future<void> stopPiP() async {
    try {
      await _channel.invokeMethod('exitPiP');
    } catch (e) {
      if (kDebugMode) {
        print("Error stopping PiP: $e");
      }
    }
  }
}

On the iOS side, I am using AVPictureInPictureController. Since it requires an AVPlayerLayer, I had to include a dummy video URL to initialize the AVPlayer. However, this results in the dummy video’s audio playing in the background, but no view is displayed in PiP mode.

Here’s my iOS code:

import Flutter
import UIKit
import AVKit

@main
@objc class AppDelegate: FlutterAppDelegate {
    
    var pipController: AVPictureInPictureController?
    var playerLayer: AVPlayerLayer?
    
    override func application(
        _ application: UIApplication,
        didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
    ) -> Bool {
        
        let controller: FlutterViewController = window?.rootViewController as! FlutterViewController
        let pipChannel = FlutterMethodChannel(name: "pip_channel", binaryMessenger: controller.binaryMessenger)

        pipChannel.setMethodCallHandler { [weak self] (call: FlutterMethodCall, result: @escaping FlutterResult) in
            if call.method == "enterPiP" {
                self?.startPictureInPicture(result: result)
            } else if call.method == "exitPiP" {
                self?.stopPictureInPicture(result: result)
            } else {
                result(FlutterMethodNotImplemented)
            }
        }

        GeneratedPluginRegistrant.register(with: self)
        return super.application(application, didFinishLaunchingWithOptions: launchOptions)
    }
    
    private func startPictureInPicture(result: @escaping FlutterResult) {
        guard AVPictureInPictureController.isPictureInPictureSupported() else {
            result(FlutterError(code: "UNSUPPORTED", message: "PiP is not supported on this device.", details: nil))
            return
        }

        // Set up the AVPlayer
        let player = AVPlayer(url: URL(string: "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4")!)
        let playerLayer = AVPlayerLayer(player: player)
        self.playerLayer = playerLayer

        // Create a dummy view
        let dummyView = UIView(frame: CGRect(x: 0, y: 0, width: 1, height: 1))
        dummyView.isHidden = true
        window?.rootViewController?.view.addSubview(dummyView)
        dummyView.layer.addSublayer(playerLayer)
        playerLayer.frame = dummyView.bounds

        // Initialize PiP Controller
        pipController = AVPictureInPictureController(playerLayer: playerLayer)
        pipController?.delegate = self

        // Start playback and PiP
        player.play()
        pipController?.startPictureInPicture()
        print("Picture-in-Picture started")
        result(nil)
    }
    
    private func stopPictureInPicture(result: @escaping FlutterResult) {
        guard let pipController = pipController, pipController.isPictureInPictureActive else {
            result(FlutterError(code: "NOT_ACTIVE", message: "PiP is not currently active.", details: nil))
            return
        }

        pipController.stopPictureInPicture()
        playerLayer = nil
        self.pipController = nil
        result(nil)
    }
}

extension AppDelegate: AVPictureInPictureControllerDelegate {
    func pictureInPictureControllerDidStartPictureInPicture(_ pictureInPictureController: AVPictureInPictureController) {
        print("PiP started")
    }

    func pictureInPictureControllerDidStopPictureInPicture(_ pictureInPictureController: AVPictureInPictureController) {
        print("PiP stopped")
    }
}

Questions:

  1. How can I implement PiP mode on iOS without using a video URL (or AVPlayerLayer)?
  2. Is there a way to display a custom UIView (like a speaker’s initials or an avatar) in PiP mode instead of requiring a video?
  3. Why does PiP not display any view, even though the dummy video URL is playing in the background?

I am new to iOS development and would greatly appreciate any guidance or alternative approaches to achieve this functionality. Thank you!

2

Answers


  1. 1. Implementing PiP without using a video URL or AVPlayerLayer

    The native AVPictureInPictureController on iOS requires an AVPlayerLayer or an AVSampleBufferDisplayLayer as its content source. Unfortunately, iOS doesn’t natively support custom views (e.g., showing initials or an avatar) for PiP mode.

    Workaround:

    If you don’t want to use a video, you could create a silent video (or an empty video file with only a single frame and no audio). This would allow you to use AVPlayerLayer for PiP while maintaining the ability to display custom content. Here’s how:
    1. Create a dummy MP4 file containing a single frame with a custom background (like a speaker’s initials) and no audio.
    2. Use this file as the source for the AVPlayer.

    let player = AVPlayer(url: Bundle.main.url(forResource: "dummy", withExtension: "mp4")!)
    let playerLayer = AVPlayerLayer(player: player)
    

    2. Displaying a Custom UIView in PiP Mode

    Custom PiP Solutions: Implement your own floating view mechanism that mimics PiP behavior. You can use a draggable UIView with custom animations, allowing users to minimize and resize it.

    Using CallKit/SceneKit: If you’re creating a video conferencing app, integrate with CallKit to provide system-native handling for active calls. While CallKit doesn’t directly support PiP, it might work well with a minimized UI for ongoing calls.

    Updated Solution-

    private func startPictureInPicture(result: @escaping FlutterResult) {
        guard AVPictureInPictureController.isPictureInPictureSupported() else {
            result(FlutterError(code: "UNSUPPORTED", message: "PiP is not supported on this device.", details: nil))
            return
        }
    
        // Use a local or remote dummy video URL
        guard let videoURL = Bundle.main.url(forResource: "dummy", withExtension: "mp4") else {
            result(FlutterError(code: "INVALID_URL", message: "Could not find dummy video.", details: nil))
            return
        }
    
        let player = AVPlayer(url: videoURL)
        player.isMuted = true
        let playerLayer = AVPlayerLayer(player: player)
        self.playerLayer = playerLayer
    
        // Set up dummy view
        let dummyView = UIView(frame: CGRect(x: 0, y: 0, width: 1, height: 1))
        dummyView.isHidden = true
        window?.rootViewController?.view.addSubview(dummyView)
        dummyView.layer.addSublayer(playerLayer)
        playerLayer.frame = dummyView.bounds
    
        // Initialize PiP Controller
        pipController = AVPictureInPictureController(playerLayer: playerLayer)
        pipController?.delegate = self
    
        // Start playback and PiP
        player.play()
        pipController?.startPictureInPicture()
        result(nil)
    }
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search