Question posted in
Some documentation to get you started can be found here.

Stream audio from microphone to pocketsphinx and browser at once – Apache

I want to stream audio from my Raspberry Pi’s microphone to a custom pocketsphinx application and a browser at one time. How would I do this? I want to get it working on my local network at first, but the technique should be usable over WAN as well.

What I already have:

server:

gst-launch-1.0 alsasrc ! audio/x-raw, endianness=1234, signed=true, width=16, depth=16, rate=44100, channels=1, format=S16LE ! audioconvert ! audioresample ! tcpserversink host=127.0.0.1 port=3000

client:

gst-launch-1.0 tcpclientsrc host=127.0.0.1 port=3000 ! audio/x-raw, endianness=1234, signed=true, width=16, depth=16, rate=44100, channels=1, format=S16LE ! audioconvert ! audioresample ! pocketsphinx ! fakesink

I would replace 127.0.0.1 with my streaming server’s local ip.

This seems to work fine with pocketsphinx. But how do I need to adapt the command in order to stream to a browser as well? I have Apache2 html + PHP server installed. When a client visits my website, I want him to hear the sound from the mic, while another application on the computer needs pocketsphinx at the same time.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I finally figured out how to do it. Thanks to Nikolay Shmyrev for pointing me in the right direction. I wrote complete instructions how to do it and I post them here if anyone wants to do the same.

    This will describe how to setup a live audio streaming server + pocketsphinx speech recognition service on raspberry pi. Should work on other Linux distros as well.

    1) Install packages

    sudo apt-get install gstreamer-1.0 gstreamer1.0-tools apache2 libapache2-mod-php libopus-dev libmicrohttpd-dev libjansson-dev libssl-dev libsrtp-dev libsofia-sip-ua-dev libglib2.0-dev libopus-dev libogg-dev libcurl4-openssl-dev liblua5.3-dev libconfig-dev pkg-config gengetopt libtool automake git bison python-dev swig make gedit firefox-esr

    2) Compile janus

    git clone https://github.com/meetecho/janus-gateway
cd janus-gateway
sh autogen.sh
./configure --prefix=/opt/janus
make -j4
sudo make install
sudo make configs
cd ..

    3) Compile cmusphinx

    git clone https://github.com/cmusphinx/sphinxbase
git clone https://github.com/cmusphinx/pocketsphinx
git clone https://github.com/cmusphinx/sphinxtrain
cd sphinxbase
./autogen.sh
make -j4
sudo make install
cd ..
cd pocketsphinx
./autogen.sh
make -j4
sudo make install
cd ..
cd sphinxtrain
./autogen.sh
make -j4
sudo make install
cd ..
pip install pocketsphinx

    4) Update janus configuration file

    Just replace the content of /opt/janus/etc/janus/janus.plugin.streaming.jcfg with the following:

    ; You should ensure that one of the streams configured for Janus is the
; following. You need to edit 
;
;       /opt/janus/etc/janus/janus.plugin.streaming.cfg
;
; and add/modify it to have the following section. Make sure all other
; sections in that file are commented out/deleted.

[gstreamer-sample]
type = rtp
id = 1
description = Opus/VP8 live stream coming from gstreamer
audio = yes
audioport = 5002
audiopt = 111
audiortpmap = opus/48000/2
secret = adminpwd

    5) Set environment vars

    You need to run this in every terminal where you use pocketsphinx, instead you could add this to your .bashrc file:

    export LD_LIBRARY_PATH=/usr/local/lib 
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

    6) Copy the content of janus-gateway/html to your apache2 directory (probably /var/www/html)

    sudo cp -a janus-gateway/html/. /var/www/html

    7) Set default audio to usb sound card and reboot

    gedit /usr/share/alsa/alsa.conf

    will show:

    defaults.ctl.card 0
defaults.pcm.card 0

    change it to:

    defaults.ctl.card 1
defaults.pcm.card 1

    reboot:

    sudo reboot

    8) restart apache2

    sudo service apache2 restart

    9) start janus

    janus -F /opt/janus/etc/janus

    10) Run gstreamer command 1:

    gst-launch-1.0 alsasrc ! audio/x-raw, endianness=1234, signed=true, width=16, depth=16, rate=44100, channels=1, format=S16LE ! tee name=t 
    t. ! queue ! audioconvert ! audioresample ! tcpserversink host=127.0.0.1 port=3000 
    t. ! queue ! audioresample ! audio/x-raw, channels=1, rate=16000 ! opusenc bitrate=20000 ! rtpopuspay ! udpsink host=127.0.0.1 port=5002

    You should see something like "New Audio Stream!" in the terminal where you started janus.

    11) Run gstreamer command 2:

    gst-launch-1.0 tcpclientsrc host=127.0.0.1 port=3000 ! audio/x-raw, endianness=1234, signed=true, width=16, depth=16, rate=44100, channels=1, format=S16LE ! audioconvert ! audioresample ! pocketsphinx ! fakesink

    **) Finished!

    You can go to http://127.0.0.1/streamingtest.html, click "Start" on Plugin Demo: Streaming tab, select "Opus/VP8 live stream coming from gstreamer (live)" and click "Watch or Listen". You will hear the soud coming from your microphone. You can visit this site from any computer in the local network by replacing 127.0.0.1 with the raspberry pi's ip address. I have experienced problems with chrome, use firefox. We installed it in 1). You can start it from the menu or by typing "firefox-esr" in a terminal.

    You can check pocketsphinx/src/gst-plugin/livedemo.c or livedemo.py to see how to use pocketsphinx with gstreamer.


  2. 0

    Like in Implement multi-stream in Gstreamer you can split gstreamer streams on two with tee plugin and stream them to separate endpoints:

    gst-launch-1.0 alsasrc ! audio/x-raw, endianness=1234, signed=true, width=16, depth=16, rate=44100, channels=1, format=S16LE ! audioconvert ! audioresample ! tee name=t ! tcpserversink host=127.0.0.1 port=3000 t. ! rtpL16pay name=pay0

    See also Gsteamer Tee plugin documentation.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search