Question 263972 in Flutter Question posted in
The official Flutter documentation can be found here.

In flutter I am not able to play uint8List type as audio when streaming via websocket with just_audio package

Here is my code :

try {
      _channel = IOWebSocketChannel.connect(
          'wss://api.elevenlabs.io/v1/text-to-speech/$voiceId/stream-input?model_id=$model');

      _channel!.stream.listen(
        (data) async {
          final response = jsonDecode(data);
          if (response['audio'] != null) {
            try{
              final audioChunk = response['audio'] as String;
              final uint8List = Uint8List.fromList(base64.decode(audioChunk));

              await _audioPlayer.setAudioSource(
                ConcatenatingAudioSource(
                  children: [
                    AudioSource.uri(


                      Uri.dataFromBytes(
                        uint8List,
                        mimeType: 'audio/mpeg',
                      ),
                    ),
                  ],
                ),
              );
              await _audioPlayer.play();
            }
            catch (e){
              print("Error setting audio source and playing: $e");

            }

I am able to get the response and am converting it to uint8List and trying to play it but its not playing anything and I am getting error :

flutter: Error setting audio source and playing: (-11828) Cannot Open
flutter: Error setting audio source and playing:
MissingPluginException(No implementation found for method load on
channel
com.ryanheise.just_audio.methods.abaaa9b2-f8ed-4c88-9d89-4623ab523beb)
flutter: Error setting audio source and playing: (-11828) Cannot Open

How can I play it? Is there any other package that can do it? I am successfully getting the response but am just not able to play it

Tags:

2

Answers


  1. 0

    This example comes from the just_audio README:

    // Feed your own stream of bytes into the player
class MyCustomSource extends StreamAudioSource {
  final List<int> bytes;
  MyCustomSource(this.bytes);
  
  @override
  Future<StreamAudioResponse> request([int? start, int? end]) async {
    start ??= 0;
    end ??= bytes.length;
    return StreamAudioResponse(
      sourceLength: bytes.length,
      contentLength: end - start,
      offset: start,
      stream: Stream.value(bytes.sublist(start, end)),
      contentType: 'audio/mpeg',
    );
  }
}

await player.setAudioSource(MyCustomSource(...));
player.play();

    You can define your own custom subclass of StreamAudioSource that feeds audio data into just_audio. Since part of the returned StreamAudioResponse is a stream, you can just transform and redirect your websocket stream into this StreamAudioResponse. So instead of:

    Stream.value(bytes.sublist(start, end)),

    you could use something like:

    channel!.stream
        .map(json.decode)
        .map((data) => data['audio'])
        .map(base64.decode),

    Since this audio source would continuously stream the audio data to just_audio, you would just call setAudioSource once and not repeatedly.

    Login or Signup to reply.
  2. 0

    I just made my account so I unfortunately don’t have enough reputation to leave a comment however, I did also try the answer from Ryan Heise. What I found is that all the bytes need to be loaded into the StreamAudioSource before the player starts to buffer. I made a project to mimc my stream of bytes by using a random .mp3 file.

    I have pasted my code and the logs below:

        class MyCustomSource extends StreamAudioSource {
      StreamController<List<int>> bytes;
      MyCustomSource(this.bytes);
    
      @override
      Future<StreamAudioResponse> request([int? start, int? end]) async {
        start = 0;
        end = 0;
        return StreamAudioResponse(
          sourceLength: end,
          contentLength: end - start,
          offset: start,
          stream: bytes.stream,
          contentType: 'audio/mpeg',
        );
      }
    }
    
    void mimicStream(AudioPlayer player) async {
      StreamController<Uint8List> streamController = StreamController<Uint8List>();
      ByteData data = await rootBundle.load('assets/temp_response.mp3');
      Uint8List bytesU8 = data.buffer.asUint8List();
      Uint8List accumulatedBytes = Uint8List(0);
      int start = 0;
      int end = 8191;
      print('Starting stream of chunks');
      Duration timeTaken = Duration.zero;
      Timer.periodic(Duration(milliseconds: 100), (Timer t) {
        print('start $start');
    
        // Emit the bytes
        streamController.add(bytesU8.sublist(start, end));
        // Extend the accumulatedBytes with the newBytes
        accumulatedBytes =
            Uint8List.fromList(accumulatedBytes + bytesU8.sublist(start, end));
    
        start = end + 1;
        end += 8192;
        timeTaken += Duration(milliseconds: 500);
    
        // Stop the timer if end exceeds the length of uint8List
        if (end >= bytesU8.length) {
          print('stream of chunks ended ended -- time taken ${timeTaken.inSeconds} seconds');
          t.cancel();
          streamController.close();
        }
      });
    
      MyCustomSource audioInBytes = MyCustomSource(streamController);
    
      player.playerStateStream.listen((event) {
        print('playerStateStream -- ${event}');
      });
    
      await player.setAudioSource(audioInBytes);
      player.play();
    }

    From the logs, you can see that the stream had to complete before the audio player actually played. I’m not too sure how we can get the bytes to play as each chunk is received. Any help would also be appreciated.
    Logs:

    Starting stream of chunks
playerStateStream -- playing=false,processingState=ProcessingState.idle
playerStateStream -- playing=false,processingState=ProcessingState.loading
start 0
start 8192
start 16384
start 24576
start 32768
start 40960
start 49152
start 57344
start 65536
start 73728
start 81920
start 90112
start 98304
start 106496
start 114688
start 122880
start 131072
start 139264
start 147456
stream of chunks ended ended -- time taken 9 seconds
playerStateStream -- playing=false,processingState=ProcessingState.buffering
playerStateStream -- playing=false,processingState=ProcessingState.ready
playerStateStream -- playing=true,processingState=ProcessingState.ready
playerStateStream -- playing=true,processingState=ProcessingState.buffering
playerStateStream -- playing=true,processingState=ProcessingState.ready
playerStateStream -- playing=true,processingState=ProcessingState.completed
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search