Since I'm engaged in VoIP-related job, I often do things other than SIP (Session Initiation Protocol) devices. In some cases, Android or IOS-based SIP terminal apps are made, and in some cases, hardware phones sold in the market are tested. Sometimes I have inquiries regarding the production of special purpose SIP devices. One of the recent proposals was to develop a telephone for the elderly living alone or in the home of a woman living alone. This phone is not a phone that usually makes a call by pressing a phone button, but requires a function that detects an emergency situation and automatically makes a call when an emergency situation occurred. When the embedded voice recognition device recognizes an urgent sentence, it transmits a signal to the phone, and the phone automatically calls the rescue center to connect with the rescuer.

The requested phone has the following features.

There is no need to connect with a typical telco carrier.
It can be connected to a specific PBX of the rescue center through SIP protocol.
There is no need for a DTMF button because the outbound number is always the same.
No inbound required.
I need a device that can receive events delivered by voice recognition devices.

I mostly use Raspberry Pi or Odroid for small device development. These devices support Android and Linux. So, I had to decide which of the two OSs to choose. Since I have experienced making Android soft phones many times using SDK, I first considered the Android OS. However, the event delivery required by the voice recognition device was the H/W method, not the S/W method.

In other words, it was a method of turning on/off the 3.3V voltage to the GPIO (General Purpose IO) pin of the Raspberry Pi, not serial communication or tcp/ip communication. Linux is much more advantageous to control the GPIO pins, so I decided to use the Linux OS. And it decided to use the Raspberry Pi, which has rich sound card support for interworking with peripheral devices such as microphones and speakers.

Materials

The preparations for making VoIP phones are as follows.

Raspberry Pi

Most Raspberry Pi models are available. Selecting one of the following models makes it easy to use.

Raspberry Pi 3B, 3B+, 4B+, Zero, Zero W

I'm going to use Rpi4 B.

Sound Card(Hat) for Raspberry Pi

The Raspberry Pi supports built-in sound over a 3.5mm audio jack. However, you need a sound card to use the microphone function, which is essential for phone functions.

Both the USB sound card and the Hat type used to connect to the GPIO pin are available.

In this article, I will use a HAT type sound card from SeeedStudio. This product is convenient to use because it has a built-in microphone and can drive low-power speakers without an amplifier.

Speaker and Microphone

You need a speaker and microphone that can be used by connecting to a sound card. If you are using a speaker with a large output, you may need a separate amplifier. Also, some sound cards have built-in microphones. In this case, you do not need to prepare a separate microphone.

Assembling the Sound HAT

A detailed description of the ReSpeaker 2-Mics Pi HAT product can be found at https://wiki.seeedstudio.com/ReSpeaker_2_Mics_Pi_HAT/.

SeeedStudio ReSpeaker 2-Mics Pi HAT overview

<https://wiki.seeedstudio.com/ReSpeaker_2_Mics_Pi_HAT/>

BUTTON: a User Button, connected to GPIO17
MIC_Land MIC_R: 2 Microphones on both sides of the board
RGB LED: 3 APA102 RGB LEDs, connected to SPI interface
WM8960: a low power stereo codec
Raspberry Pi 40-Pin Headers: support Raspberry Pi Zero, Raspberry Pi 1 B+, Raspberry Pi 2 B , Raspberry Pi 3 B, B+ and Raspberry Pi 4 B
POWER: Micro USB port for powering the ReSpeaker 2-Mics Pi HAT, please power the board for providing enough current when using the speaker.
I2C: Grove I2C port, connected to I2C-1
GPIO12: Grove digital port, connected to GPIO12 & GPIO13
JST 2.0 SPEAKER OUT: for connecting speaker with JST 2.0 connector
3.5mm AUDIO JACK: for connecting headphone or speaker with 3.5mm Audio Plug

Raspberry Pi OS

First, download the OS image and install it.

Raspberry Pi installs "Raspberry Pi OS with desktop". You can download the image at https://www.raspberrypi.org/software/operating-systems/#raspberry-pi-os-32-bit .

Assembling

Connect ReSpeaker 2-Mics Pi HAT to Raspberry Pi

Mount ReSpeaker 2-Mics Pi HAT on your Raspberry Pi, make sure that the pins are properly aligned when stacking the ReSpeaker 2-Mics Pi HAT.

I used Raspberry Pi 4B. And since there was no cable that fits the JST 2.0 terminal, I soldered and connected the 4 ohm speaker.

Setup the driver on Raspberry Pi

Remove the default sound driver(bcm2835). Modify the /boot/config.txt file as follows

dtparam=audio=off

Then reboot the Raspberry Pi.

While the upstream wm8960 codec is not currently supported by current Pi kernel builds, upstream wm8960 has some bugs, SeeedStudio had fixed it. We must build it manually. Make sure that you are running the lastest Raspbian Operating System(debian 9) on your Pi.

Installation of portaudio and pyaudio is not required. However, to use pyaudio, you need to install portaudio together.

sudo apt-get update
sudo apt-get upgrade

apt-get install libasound2 libasound2-dev

wget http://www.portaudio.com/archives/pa_stable_v190600_20161030.tgz

tar -xvzf pa_stable_v190600_20161030.tgz

cd portaudio

./configure

make

sudo make install

pip3 install pyaudio
cd ..

Now install the seeed device drivers.

git clone https://github.com/respeaker/seeed-voicecard.git
cd seeed-voicecard
sudo ./install.sh
reboot

Check that the sound card name matches the source code seeed-voicecard by command aplay -l and arecord -l.

root@raspberrypi:~# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: seeed2micvoicec [seeed-2mic-voicecard], device 0: bcm2835-i2s-wm8960-hifi wm8960-hifi-0 [bcm2835-i2s-wm8960-hifi wm8960-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
root@raspberrypi:~# arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: seeed2micvoicec [seeed-2mic-voicecard], device 0: bcm2835-i2s-wm8960-hifi wm8960-hifi-0 [bcm2835-i2s-wm8960-hifi wm8960-hifi-0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

Test, you will hear what you say to the microphones(don't forget to plug in an earphone or a speaker):

This command transfers the data recorded using arecord to aplay and outputs it. In the following command, 0 in Dhw:0 is the card number of the alsa -l command output.

root@raspberrypi:~# arecord -f cd -Dhw:0 | aplay -Dhw:0
Recording WAVE 'stdin' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo
Playing WAVE 'stdin' : Signed 16 bit Little Endian, Rate 44100 Hz, Stereo

Nore: Don't be disappointed if you don't hear what you want and only a little noise. It is necessary to adjust the input, output of the microphone and speaker.

Input/output level adjustment of microphone and speaker

alsamixer is a graphical mixer program for the Advanced Linux Sound Architecture (ALSA) that is used to configure sound settings and adjust the volume. run alsamixer.

root@raspberrypi:~# alsamixer

For SeedStudio products, you can get a good microphone level by setting it near -20dB. Press the F4 Key to select Capture setting. Then adjust the L,R CAPTURE levels.

Press the F3 Key to select Playback setting. Then adjust the Speaker and Headphone levels. And adjust the Playback level. This value also affects the sound output.

You can change the device selection at the bottom of the screen by using the right,left arrow keys.

Once you have adjusted the microphone and speaker levels appropriately, test the microphone and speakers with the command "arecord -f cd -Dhw:0 | aplay -Dhw:0" again. When clear voice is input and output, level control is successful.

Development of VoIP softphone Vs. Use existing package

From the conclusion, I decided to use the existing package. The reason is that the SDK I have is for Android, IOS, and Windows. Therefore, in order to develop for Linux, there is a risk that a lot of trial and error occurs because a new SDK needs to be found, and there is a risk of exceeding the working deadline.

Twinkle

I looked for a VoIP softphone that works on Linux that fits my needs and came to the conclusion that twinkle was a good fit. Almost all softphones are GUI-based. However, GUI-based software is not suitable for automatic operation. CLI command-based software is easy to operate or control automatically without user assistance. twinkle supports both GUI and CLI. Therefore, when an event is received from a voice recognition device, it is easy to implement a function that automatically makes a call without user's manipulation.

apt-get update
apt-get install twinkle

When twinkle is installed, you can see that Twinkle is newly created along with the existing Chromium Web Browser in "internet" of the Raspberry Pi desktop menu.

Twinkle configuration

In the "Edit/System settings..." menu, set the audio as follows. The newly installed SeeedStudio's audio device will appear.

In the "Edit/User profile..." menu, set the SIP information as follows. Modify the extension value, password, and SIP Server address to suit your environment.

These settings are saved in the current user's "home directory/.twinkle" directory.

root@raspberrypi:/home/pi/.twinkle# pwd
/home/pi/.twinkle
root@raspberrypi:/home/pi/.twinkle# ls -al
total 136
drwx------  3 pi pi  4096 Jan 21 22:50 .
drwxr-xr-x 16 pi pi  4096 Jan 21 22:58 ..
-rw-r--r--  1 pi pi  2477 Jan 15 21:53 1234.cfg
srwxr-xr-x  1 pi pi     0 Jan 21 22:50 .cmdsock
-rw-r--r--  1 pi pi   497 Jan 16 19:19 gui_state.ini
drwx------  2 pi pi  4096 Jan 15 21:00 tmp
-rw-r--r--  1 pi pi  1187 Jan 16 19:39 twinkle.ch
-rw-------  1 pi pi    10 Jan 16 20:51 twinkle.history
-rw-------  1 pi pi     0 Jan 15 21:00 twinkle.lck
-rw-r--r--  1 pi pi 87777 Jan 21 23:04 twinkle.log
-rw-r--r--  1 pi pi  6572 Jan 16 20:52 twinkle.log.old
-rw-r--r--  1 pi pi  1240 Jan 21 22:50 twinkle.sys
-rw-r--r--  1 pi pi  1237 Jan 16 20:51 twinkle.sys~

Note : Running twinkle in daemon mode uses the root account. Therefore, copy the configuration file in the /home/pi/.twinkle directory to the /root/.twinkle directory.

Running twinkle from the command window

root@raspberrypi:~# twinkle -c

Firewall/NAT discovery in progress.
Please wait.

Twinkle 1.10.1, October 7, 2016
Copyright (C) 2005-2015  Michel de Boer and contributors

Users:
* twinkle
    1234 <sip:1234@192.168.11.50>

Local IP:       255.255.255.255


twinkle: registering phone...

Twinkle>
twinkle: registration succeeded (expires = 3600 seconds)
Registrar Date: Thu, 21 Jan 2021 14:16:59 GMT

If registration succeeded is displayed, you are connected to the exchange normally. You can now make and receive calls using commands in the command window.

twinkle commands

You can use a command like this:

Twinkle> help

call            Call someone
answer          Answer an incoming call
answerbye       Answer an incoming call or end a call
reject          Reject an incoming call
redirect        Redirect an incoming call
transfer        Transfer a standing call
bye             End a call
hold            Put a call on-hold
retrieve        Retrieve a held call
conference      Join 2 calls in a 3-way conference
mute            Mute a line
dtmf            Send DTMF
redial          Repeat last call
register        Register your phone at a registrar
deregister      De-register your phone at a registrar
fetch_reg       Fetch registrations from registrar
options         Get capabilities of another SIP endpoint
line            Toggle between phone lines
dnd             Do not disturb
auto_answer     Auto answer
user            Show users / set active user
message         Send an instant message
presence        Publish your presence state
quit            Quit
help            Get help on a command


Users:
* twinkle
    1234 <sip:1234@192.168.11.50>

Local IP:       255.255.255.255


twinkle: registering phone...

Twinkle>
twinkle: registration succeeded (expires = 3600 seconds)
Registrar Date: Thu, 21 Jan 2021 14:16:59 GMT

inbound call auto answer

Twinkle> auto_answer
Auto answer enabled.

Twinkle>
Line 1: incoming call
From:           Extension 1235 <sip:1235@192.168.11.50>
To:             sip:1234_192_168_11_50@192.168.11.53

warning: Unknown speex_preprocess_ctl request:  2
warning: Unknown speex_preprocess_ctl request:  6
warning: Unknown speex_preprocess_ctl request:  30
warning: The VAD has been replaced by a hack pending a complete rewrite

Line 1: far end supports DTMF telephone event.


Line 1: call established.


Line 1: far end ended call.

outbound call

Twinkle> call 1235
Twinkle>
Line 1: received 100 Trying


Line 1: received 180 Ringing

warning: Unknown speex_preprocess_ctl request:  2
warning: Unknown speex_preprocess_ctl request:  6
warning: Unknown speex_preprocess_ctl request:  30
warning: The VAD has been replaced by a hack pending a complete rewrite

Line 1: far end supports DTMF telephone event.


Line 1: far end answered call.
200 OK
To: sip:1235@192.168.11.50


Line 1: far end ended call.

Controlling twinkle with Python

Our final goal is the ability to make a call automatically in background mode by detecting the signal on the GPIO pin without a UI. For this, a program that monitors GPIO pins and controls twinkle is required. Let's make it using Python.

Note : The voice recognition device must transmit a 3.3V voltage signal to GPIO 12. Do not use 5V signal.

To control twinkle in Python you can use the subprocess module. But we need to control the twinkle and monitor the state of the GPIO pins at the same time. If GPIO pin 12 is ON, you must call the rescue center. To do the two things effectively, we use asyncio.subprocess instead of synchronous subprocess. The advantage of asyncio.subprocess is that you can set a timeout when reading the stdout value of the twinkle process. If there is no stdout value for a certain period of time, the GPIO value can be read without blocking. The following Python code runs the twinkle program at startup and then continues to monitor twinkle's stdout output. If there is no stdout value for 3 seconds of timeout time, it reads the value of GPIO12 to determine whether to outbound.

import argparse
import logging
import logging.handlers
from datetime import datetime
import os, sys, time
#import threading
import asyncio
from asyncio.subprocess import PIPE, STDOUT
import RPi.GPIO as GPIO

logger_file = '/var/log/bluebay_voip_' + datetime.now().strftime("%Y%m%d") + '.log'
logger = logging.getLogger('mylogger')
fomatter = logging.Formatter('[%(levelname)s|%(filename)s:%(lineno)s] %(asctime)s > %(message)s')
fileHandler = logging.FileHandler(logger_file)
streamHandler = logging.StreamHandler()
fileHandler.setFormatter(fomatter)
streamHandler.setFormatter(fomatter)
logger.addHandler(fileHandler)
logger.addHandler(streamHandler)
logger.setLevel(logging.DEBUG)

phone_registered = False
state = ['idle','calling', 'talk']
call_state = state[0]
g_proc = None
dial_number = 'rescue_center'
gpio_pin = 12

  
async def twinkle_proc(*args, timeout=None):
    global g_proc, call_state, phone_registered
    g_proc = await asyncio.create_subprocess_exec(*args, stdin=PIPE, stdout=PIPE, stderr=STDOUT)    
    cnt = 0
    while True:
        try:
            line = await asyncio.wait_for(g_proc.stdout.readline(), timeout)
        except asyncio.TimeoutError:
            logger.debug('time out')
            event = GPIO.input(gpio_pin)
            if(event == True):
                logger.info('GPIO PIN[%d] STATE:%d  call_state[%s] registered[%d]'%(gpio_pin, event, call_state, phone_registered))
                if(call_state == state[0] and phone_registered == True):
                    dial_str = 'call ' + dial_number + '\n'
                    g_proc.stdin.write(dial_str.encode())
                    call_state = state[1]
            pass
        else:
            if not line: # EOF(Process end)
                logger.info('no data')
                break
            else:
                l = str(line.rstrip())
                if (l.find("registration succeeded") > 0):
                    phone_registered = True
                    logger.info("registration success")

                if (l.find("call failed") > 0): #전화걸기 실패
                    logger.error('call to[%s] failed'%(dial_number))
                    call_state = state[0]

                if (l.find("far end answered call") > 0): call_state = state[2]
                if (l.find("far end ended call") > 0): call_state = state[0]
                if (l.find("call established") > 0): call_state = state[2]  #인바운드 통화시작
                if (l.find("incoming call") > 0): #인바운드 전화.무조건 받음
                    g_proc.stdin.write('answer'.encode())

                if len(l) > 3: logger.info(l)
                continue # While some criterium is satisfied
    #g_proc.kill()        
    return await g_proc.wait()            


if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('--dial', type=str, default='rescue_center', help='Dialing Number')
    FLAGS, unparsed = parser.parse_known_args()
    dial_number = FLAGS.dial

    GPIO.setmode(GPIO.BCM)  
    GPIO.setup(gpio_pin, GPIO.IN)
    loop = asyncio.get_event_loop()
    returncode = loop.run_until_complete(twinkle_proc("twinkle", "-c", timeout=3))
    loop.close()
    logger.info('twinkle end!')

<twinkle.py>

Wrapping up

While using Seeed Studio's Respeaker, I found a problem. There is no problem with the speaker part, but there was often severe noise from the input microphone. Due to the noise that makes it difficult to understand what the other person is saying. I have researched several times and found that this happens when I connect a 4ohm speaker directly to the Respeaker. Therefore, instead of connecting the speaker directly to the Respaker, we used a method of outputting an audio signal using a headset terminal and then connecting the speaker using an amplifier. According to the document, a separate 5V power supply is provided through the Seeed Studio Respeaker's USB port to connect the speaker. But when I tested it, it was more stable with a headphone jack and an external amplifier.

The following figure shows the audio output after amplifying the audio through a 12V amplifier using a USB sound card. The microphone is connected to the microphone terminal of the USB sound card. I have confirmed that this method works well too.

You can easily implement a SIP soft phone by installing twinkle and then modifying the example above to suit your purpose and controlling it in the background.

In this configuration, it was confirmed that calls were made cleanly without noise. When a finished product is released soon, I will update the blog content again.

이 블로그 검색

Linux Tips

Making VoIP Phone Using Raspberry Pi