Feature Extraction for Distinguishing a Limited Set of Words
Abstract Generated abstract
This paper addresses feature extraction for recognizing a limited vocabulary of spoken words intended for oral computer control. It argues that the required distinctions can be obtained from clipped envelopes of the speech signal, the whole word, and the fundamental tone, and analyzes why direct envelope extraction by a half-wave detector is inadequate for broadband speech unless the signal is first converted to a narrowband form by single-sideband modulation. The proposed device combines modulation, amplitude detection, thresholding, fundamental-tone extraction, and logic operations to identify voiced stops, voiceless stops, voiced consonants and vowels, and voiceless consonants, and is described as implementable largely with integrated circuits.
Full Text
Feature Extraction for Distinguishing a Limited Set of Words
N. F. KUCHER, A. N. OSOKIN
(Presented by the scientific seminar of the Department of Computer Engineering)
In [1] it is shown that, for distinguishing a limited set of words (about 50), sufficient for oral control of an electronic digital computer, it is enough to extract the following features: voiced stops, voiceless stops, voiced consonants and vowels, voiceless consonants. To form the indicated features it is necessary to form, from the speech signal, the clipped envelope of the speech signal and the clipped envelope of the entire word.
To form the clipped envelope of a speech signal it is necessary to extract the envelope of the speech signal, represented in the form
\[ S(t)=A(t)\cos\varphi(t). \tag{1} \]
The envelope \(A(t)\) can be expressed through the signal conjugate to the speech signal, \(\hat S(t)\), as follows:
\[ A(t)=\sqrt{S^2(t)+\hat S^2(t)}, \tag{2} \]
where
\[ \hat S(t)=H[S(t)]=\frac{1}{\pi}\int_{-\infty}^{\infty}\frac{S(\tau)}{t-\tau}\,d\tau \tag{3} \]
is the Hilbert transform of the signal \(S(t)\). However, determining \(A(t)\) according to (2) is difficult because of the ambiguity that arises when performing the square-root extraction operation. In addition, the hardware implementation of operations (2) and (3) proves to be very complicated, or even altogether impossible, for broadband signals.
A speech signal represented in the form (1) is a process with amplitude and phase modulation; therefore the envelope \(A(t)\) could be extracted by means of amplitude detection, and the detection operation is readily implemented by hardware methods.
Let us show the applicability of detection to speech signals. In the theory and practice of radio reception and measuring technology, linear single- and two-half-period detectors and square-law detectors are used to extract signal envelopes [2]. In most cases, high-quality extraction of the envelope is provided by a linear single-half-period detector. Let us consider the use of such a detector to extract the envelope \(A(t)\) of the speech signal \(S(t)\).
Let us determine the response \(S_1(t)\) of a linear detector to the speech signal \(S(t)\). A linear half-period detector is a nonlinear element with the characteristic shown in Fig. 1.
To solve the problem of finding the response, we shall use the method of transformations [3]. Introduce the following notation:
\[ \begin{aligned} S(t)&=x,\\ S_1(t)&=y,\\ A(t)&=A,\\ \varphi(t)&=\varphi. \end{aligned} \tag{4} \]
Fig. 1
Then the characteristic of the nonlinear detector element will take the form
\[ y=g(x), \tag{5} \]
where, in accordance with Fig. 1,
\[ \begin{aligned} g_{+}(x)&=y=x \quad \text{for } x>0,\\ g_{-}(x)&=y=0 \quad \text{for } x\leq 0. \end{aligned} \tag{6} \]
In accordance with the method of transformations, we find the image \(f(p)\) of the characteristic of the linear detector
\[ f(p)=f_{+}(p)+f_{-}(p)=\int_{0}^{\infty} g_{+}(x)e^{-px}\,dx+\int_{-\infty}^{0} g_{-}(x)e^{-px}\,dx= \]
\[ =\int_{0}^{\infty} xe^{-px}\,dx+\int_{-\infty}^{0}0\cdot e^{-px}\,dx=\int_{0}^{\infty} xe^{-px}\,dx. \]
Denote:
\[ px=t;\qquad dx=\frac{dt}{p}. \]
Then
\[ f(p)=\int_{0}^{\infty}\frac{t}{p}e^{-pt}\frac{1}{p}\,dt =\frac{1}{p^2}\int_{0}^{\infty}te^{-t}\,dt =\frac{1}{p^2}\Gamma(2)=\frac{1}{p^2}\cdot 1!, \tag{7} \]
where \(\Gamma(2)\) is the gamma function.
The response of the half-period linear detector is
\[ y=g(x)=\frac{1}{2\pi j}\int_{c} f(p)e^{px}\,dp =\frac{1}{2\pi j}\left[\int_{c_{+}} f_{+}(p)e^{px}\,dp+\int_{c_{-}} f_{-}(p)e^{px}\,dp\right]. \]
Since \(f_{-}(p)=0\), then
\[ y=g(x)=\frac{1}{2\pi j}\int_{c_{+}} f_{+}(p)e^{px}\,dp, \]
where \(c_{+}\) is the contour of integration.
Taking into account that from (4) \(x=A\cos\varphi\), we obtain
\[ y=g(x)=\frac{1}{2\pi j}\int_{c_{+}}\frac{1}{p^2}e^{pA\cos\varphi}\,dp. \tag{8} \]
An integral of the form (8) is evaluated approximately. For this purpose the exponential is expanded in a series by the Jacobi–Anger formula
\[ e^{(pA)\cdot\cos\varphi} =\sum_{m=0}^{\infty}\varepsilon_m I_m(pA)\cos m\varphi, \tag{9} \]
where \(I_m(pA)\) is the modified Bessel function of the first kind of order \(m\);
\(\varepsilon_m\) is the Neumann factor;
\[ \varepsilon_m = \begin{cases} 1, & \text{for } m=0,\\ 2, & \text{for } m=1,2,3\ldots \end{cases} \tag{10} \]
Taking (9) into account,
\[ y=\frac{1}{2\pi j}\int_{c_+}\frac{1}{p^2}\sum_{m=0}^{\infty} I_m(pA)\cos m\varphi\, dp. \]
Let us denote:
\[ pA=\omega,\qquad p=\frac{\omega}{A},\qquad dp=\frac{1}{A}\,d\omega, \]
then
\[ \begin{aligned} y &=\sum_{m=0}^{\infty}\cos m\varphi\, \frac{\varepsilon_m}{2\pi j} \int_{c_+}\frac{A^2}{\omega^2} I_m(\omega)\,\frac{d\omega}{A} =\sum_{m=0}^{\infty} A\cos m\varphi\, \frac{\varepsilon_m}{2\pi j} \int_{c_+}\frac{I_m(\omega)}{\omega^2}\,d\omega \\ &=\sum_{m=0}^{\infty} c_m A\cos m\varphi, \end{aligned} \tag{11} \]
where
\[ c_m=\frac{\varepsilon_m}{2\pi j}\int_{c_+}\frac{I_m(\omega)}{\omega^2}\,d\omega \quad\text{is a constant.} \tag{12} \]
Thus, the response \(S_1(t)\) of a linear half-wave detector to the speech signal \(S(t)\) has the following form:
\[ S_1(t)=y=\sum_{m=0}^{\infty} c_m A(t)\cos m\varphi(t) =c_0A(t)+\sum_{m=1}^{\infty} c_m A(t)\cos m\varphi(t). \tag{13} \]
In [3], the general solution of integral (12) is given in the form
\[ c_m=\frac{\varepsilon_m}{\pi}\cdot \frac{\cos m\pi/2}{1-m^2}. \tag{14} \]
In particular, for \(m=0\), \(\varepsilon_m=1\),
\[ c_0=\frac{1}{\pi}\frac{\cos 0}{1}=\frac{1}{\pi}. \]
It follows from (13) that, in order to obtain the desired envelope \(A(t)\), the spectra of \(A(t)\) and \(S(t)\) must not overlap; that is, the signal \(S(t)\) must be narrowband. Only in this case will it be possible to filter out the response component \(C_0A(t)\) by means of a low-pass filter. However, a speech signal is broadband, since for it the narrowband condition is not satisfied,
\[ \frac{\Delta F}{f_0}\ll 1, \]
where \(\Delta F\) is the width of the signal spectrum;
\(f_0\) is the central frequency of the signal spectrum.
Consequently, the successful use of a linear half-wave detector for extracting the amplitude envelope of a speech signal requires a preliminary transformation of the original broadband speech signal into a narrowband one. Such a transformation is carried out by means of single-sideband modulation of the speech signal.
The structural diagram of the device for extracting the clipped envelope is shown in Fig. 2 [4]. The speech signal is picked up by a microphone. By means of a single-sideband modulation block, the original speech signal arriving from the microphone is transformed into a single-sideband
Fig. 2
(narrow-band) signal, whose envelope disappears at the moment when a closure appears in the speech sequence being pronounced in front of the microphone. This envelope is extracted by an amplitude detector, whose time constant is determined not by the duration of the closure and the segment in which sound is present, but chiefly only by the carrier frequency of the single-sideband signal. The presence or absence of the envelope is recorded by a threshold device, at whose output we obtain rectangular pulses with amplitude equal to zero at the moment when a closure is present in the speech signal; i.e., we obtain the clipped envelope of the speech signal.
The time diagram (Fig. 3) shows
Fig. 3,
\(U_1(t)\) — the speech signal;
\(U_2(t)\) — the speech signal after single-sideband modulation;
\(U_3(t)\) — the envelope \(A(t)\) of the speech signal;
\(U_4(t)\) — the clipped envelope of the speech signal.
The envelope of the entire word is extracted analogously to the envelope of the speech signal, except that the time constant of the amplitude detector is chosen to be larger.
Top annotations in the diagram: Vowel; voiced closure; voiced consonant; voiceless closure; voiceless consonant.
Waveform labels along the left: \(U_1, U_2, U_3, U_4, U_5, U_6, U_7, U_8, U_9, U_{10}, U_{11}, U_{12}, U_{13}, U_{14}, U_{15}, U_{16}\).
Horizontal-axis label for each waveform: \(t\).
Fig. 4
For extracting the fundamental-tone signals we shall use the method described in [5].
Extraction of the clipped envelope of the fundamental tone is similar to the extraction, described above, of the clipped envelope of the speech signal.
The structural diagram of the feature-extraction device is shown in Fig. 4. It can be seen that the circuit consists of generators of the three signals indicated earlier and a logic part, which operates in accordance with the expressions given in [1].
It is known from a number of sources [6] that voiced stops before the sounds б, д, г are characterized by disappearance of the speech signal. However, in contrast to a voiceless stop before the sounds п, т, к, ц, ч, when a voiced stop is pronounced the fundamental tone is present (the vocal cords vibrate), which is the distinguishing feature of a voiced stop. Voiced consonants and vowels are characterized by the presence of the fundamental tone and the envelope of the speech signal during the utterance of the word. Voiceless consonants are characterized by the presence of the envelope, but by the absence of the fundamental tone during pronunciation.
Fig. 5
The timing diagram of the device operation is shown in Fig. 4, where the following notation is used:
$U_1$ — speech signal;
$U_2$ — speech signal after single-sideband modulation;
$U_3$ — envelope of the speech signal;
$U_4$ — clipped envelope of the speech signal;
$U_5$ — envelope of the whole word;
$U_6$ — clipped envelope of the whole word;
$U_7$ — fundamental-tone signal;
$U_8$ — fundamental-tone signals after single-sideband modulation;
$U_9$ — envelope of the fundamental tone;
$U_{10}$ — clipped envelope of the fundamental tone;
$U_{11}$ — inverted clipped envelope of the fundamental tone;
$U_{12}$ — inverted clipped envelope of the speech signal;
$U_{13}$ — feature of a voiceless stop;
$U_{14}$ — feature of a voiced stop;
$U_{15}$ — feature of a voiceless consonant;
$U_{16}$ — feature of a voiced consonant or vowel.
The device is implemented using integrated microcircuits, except for the single-sideband modulation block.
References
-
N. F. Kucher, A. N. Osokin. Selection of a feature system for distinguishing a limited set of words. In this collection.
-
I. S. Gonorovskii. Radio Engineering Circuits and Signals. “Soviet Radio,” 1971.
-
V. Davenport, V. Root. Introduction to the Theory of Random Signals and Noise. Moscow, IIL, 1962.
-
N. F. Kucher, A. N. Osokin. Inventor’s certificate No. 534782. — Bulletin “Inventions, Discoveries, Industrial Designs, Trademarks,” 1976, No. 41.
-
B. N. Epifantsev, N. F. Kucher, A. N. Osokin, V. M. Razin. Automatic distinguishing of a limited set of commands for machine-tool control. In: Proceedings of the All-Union Sixth Annual School-Seminar (ARSO-6). Tallinn, Publishing House of the Academy of Sciences of the Estonian SSR, 1972.
-
M. A. Sapozhkov. The Speech Signal in Cybernetics and Communications. Svyazizdat, 1963.