On the Evaluation of the Effectiveness of Speech Signal Parameters

B. N. Epifantsev, V. P. Evmenov

(Presented by the scientific seminar of the Department of Computer Engineering)

It is known that, in the transition from an absolute description of speech signals to a system of features, the latter are selected by the method of “trial and error” [1]. But in order to assess how successfully a given feature has been chosen, a criterion is needed that gives a quantitative evaluation of the feature from the point of view of its subsequent application. Until recently, in most cases the percentage of classification errors was used as such a criterion. For this purpose, in the space of the selected feature (features), the intrinsic regions of patterns were constructed; their form (convexity, concavity), the degree of overlap of regions belonging to different classes, etc., were evaluated. After this a classification procedure was proposed, an experiment was set up, and it was checked what error should be expected when recognizing the given patterns. The effectiveness of the feature (features) was characterized by the indicated error value. Whether because of the enormous number of features and their correlation, or as a result of using unsuitable classification techniques, acceptable results have not yet been obtained when oriented toward the method under consideration. It is not difficult to understand that this procedure for evaluating the effectiveness of features does not provide information on the extent to which the feature under investigation is important for a person recognizing speech. Therefore we cannot say whether the percentage of classification errors reflects information necessary either for understanding what has been said or for characterizing the particular features of pronunciation. This, perhaps, explains the interest now being shown in the semi-objective criterion, the intelligibility coefficient, and in the methods of evaluating the effectiveness of features that follow from it.

One of the methods based on the intelligibility coefficient is the method of “analysis through synthesis.” Its idea is that, by selecting the parameters of an electrical model of the speech tract, sounds are synthesized, and, if a signal acceptable from the point of view of intelligibility is obtained, a mathematical description of the model is produced.

Another method may be called the “distortion method.” The effectiveness of features by this method is evaluated by the intelligibility coefficient of speech subjected to deliberate, targeted distortions. The fact that the amplitude ratio in a signal has a vanishingly small influence on intelligibility (clipped speech) was obtained as a result of applying the “distortion method.”

Despite its apparent simplicity, the methodology for setting up the methods considered (except for the first) has not been developed, which, in fact, explains the episodic nature of work based on them. On the other hand, the intelligibility coefficient can be used to evaluate individual features and a number of their combinations.

Below, another approach to evaluating the effectiveness of features is proposed. In essence, it is a product of combining the methods considered and makes it possible to obtain information about a feature that the existing approach cannot provide.

Speech analysis has yielded many fundamental facts. We shall be interested in such facts as, for example, the increase in intelligibility of clipped speech that has first been passed through a four-terminal network whose amplitude–frequency characteristic rises toward the high frequencies, i.e., transformations that undoubtedly affect the informational structure of speech signals. Then the influence of one or another transformation can be described by a vector $\bar{\delta}$, each component of which is $\delta_i = \pm (A_i/A)$, where $A_i$ is the intelligibility coefficient of speech subjected to the $i$-th transformation, $A$ is the intelligibility coefficient of undistorted speech; the sign “$+$” is used when $A < A_i$, and “$-$” when $A > A_i$.

But it is not difficult to obtain an analogous vector $\bar{\sigma}$. In this case $\sigma_i = \pm (d_i/d_0)$, where $d_i$ is the criterion of discrimination by the selected features between the speech signals being classified and subjected to the $i$-th transformation, and $d_0$ is the same criterion for undistorted speech; the sign “$+$” is used under the condition $d_0 < d_i$, and “$-$” when $d_0 > d_i$. Since the dimensionality of the vectors $\bar{\sigma}$ and $\bar{\delta}$ is the same, the effectiveness of a feature may be evaluated by the quantity

$$ d(\bar{\sigma}, \bar{\delta}) = \frac{\displaystyle \sum_{i=1}^{n}(\sigma_i - \delta_i)^2} {\displaystyle \sum_{i=1}^{n}(|\sigma_i| + |\delta_i|)^2}. $$

Obviously, $0 \leq d(\bar{\sigma}, \bar{\delta}) \leq 1$; moreover, when $d(\bar{\sigma}, \bar{\delta}) = 0$, the effectiveness of the features under consideration is maximal (100%); when $d(\bar{\sigma}, \bar{\delta}) = 1$, the effectiveness $(\mathcal{E})$ is equal to 0. The dependence $\mathcal{E} = f[d(\bar{\sigma}, \bar{\delta})]$ is a subject for special investigations. The quantity $d(\bar{\sigma}, \bar{\delta})$ contains information about how important the feature (features) being investigated is in recognition for a human being and how close the recognition procedure used is to that employed by a human.

A development of this method may be the incorporation of information on the influence of factors (loudness, duration of sounds, etc.) on the value of the selected parameter (parameters), as well as heuristic considerations concerning the functional dependence of the intelligibility coefficient on the factors under consideration.

Let us illustrate the foregoing with examples.

1. It is required to evaluate the effectiveness of the mean-high frequency of sounds (a quantity proportional to the density of zero crossings of a function $\rho_{T_3}$, found over the interval of existence of the sound $T_3$).

To solve this problem, the sounds were successively entered through an analog-to-code converter into the main memory of the BESM-2M computer. By software means, for each sound the number of zero crossings of the function $\rho_{T_3}$ was counted; this was then divided by the time $T_3$, i.e., the quantity found was

$$ F_{T_3} = \frac{\rho_{T_3}}{T_3}, $$

Checking the correspondence of the signal entered into the computer to the original was carried out by listening to the information recorded in the machine, by outputting it through a synthesizer to a loudspeaker.

Each of the vowel sounds \(j\) is pronounced by one speaker at the same loudness 40 times, and for each realization \(i\) \((1, 2 \ldots, \ldots 40)\) the quantity \(F_{T_3 i}^{j}\) was determined. Then, from the data of the statistical series \(F_{T_3 i}^{j}\), the frequencies of the distribution polygon \(P_{\delta F_{T_3 i}^{j}}\), falling into the corresponding series \(\delta F_{T_3}\), and the Euclidean distances were calculated

\[ d\left(P_{\delta F_{T_3 i}^{j}},\, P_{\delta F_{T_3 i}^{k}}\right) = \sqrt{\sum_{\delta=1}^{N} \left(P_{\delta F_{T_3 i}^{j}}-P_{\delta F_{T_3 i}^{k}}\right)^2}. \]

From the series of values \(d(\ldots)\), the overall distance was found as

\[ d_{\mathrm{av}}=\frac{1}{M}\sum_{jk} d\left(P_{\delta F_{T_3 i}^{j}},\, P_{\delta F_{T_3 i}^{k}}\right) \]

by averaging all possible distances \(jk\) \((j \ne k)\).

The next stage was to obtain the quantities \(d_{\mathrm{av}}^{(\mathrm{fc})}\), \(d_{\mathrm{av}}^{(\mathrm{bf})}\), which differ from \(d_{\mathrm{av}}\) in that, before being entered into the machine, the speech was passed either through a frequency corrector having a rise in the frequency characteristic toward high frequencies of 6 dB/oct, or through a filter (250–5000 Hz). Other speech distortions are also possible [2], which can be characterized by quantities \(d_{\mathrm{av}}^{(\ldots)}\). In our example we shall restrict ourselves to two \(\left(d_{\mathrm{av}}^{(\mathrm{fc})},\, d_{\mathrm{av}}^{(\mathrm{bf})}\right)\). The quantities \(d_{\mathrm{av}}\), \(d_{\mathrm{av}}^{(\mathrm{fc})}\), \(d_{\mathrm{av}}^{(\mathrm{bf})}\) are given in Table 1.

It is known from the literature [2, 3] that the introduction of a frequency corrector and of a band-pass filter increases the intelligibility of speech. This means that the indicated transformations separate the proper regions of the images, constructed in the coordinates of the features by which a person recognizes speech signals. The objective data, however \(\left(d_{\mathrm{av}}^{(\ldots)}\right.\), Table 1), give a different picture. The only conclusion is that, in recognizing speech sounds, a person does not use such a parameter as \(F_{T_3}\).

The value of the introduced criterion \(d(\bar{\sigma}, \bar{\delta})=1\) says the same thing.

Table 1

\[ d(\bar{\sigma},\bar{\delta})=1,\qquad \mathcal{E}=0 \]

Incidentally, a segment of a sinusoid of frequency \(F_{T_3}\), specified on the interval \(T_3\), never resembles any of the sounds; and the examples cited in the literature of more successful solutions of certain problems than those produced by nature are hardly acceptable in the case of speech recognition. It should finally be noted that if the factors of loudness and speaker have a substantial influence on the value of \(F_{T_3}\), then for under-

understanding of what was said, it is immaterial to the person which of the announcers spoke and, with minor allowances, how loudly the sounds were pronounced.

1. As a second example, let us evaluate the effectiveness of the probability-density distributions of the durations of intervals between zeros. Proceeding analogously to item 1, we determine the overall Euclidean distances between the distributions, averaged over realizations, of the \(j\)-th and \(k\)-th sounds for the cases with a frequency corrector, a band-pass filter, and without them. The results of this work are given in Table 2. The same table also presents the results of studying the influence of the loudness and announcer factors on the parameters of the distributions under consideration, borrowed from [4].

Table 2

\[ d(e,b) = 1,\qquad E = 0 \]

It is not difficult to conclude that the form of the probability-density distributions of the durations of intervals between zeros is immaterial for a person recognizing speech signals.

References

1. A. A. Kharkevich. On the selection of features in machine recognition. Proceedings of the Academy of Sciences of the USSR, Department of Technical Sciences, Technical Cybernetics, No. 2, 1963.

2. Yu. G. Rostovtsev. On the possibility of applying limiting amplitude restriction of speech signals in communication systems. Elektrosvyaz, No. 6, 1958.

3. Yu. S. Bykov. The Theory of Speech Intelligibility and Increasing the Efficiency of Radiotelephone Communication. Gosenergoizdat, 1959.

4. V. P. Evmenov, B. N. Epifantsev. On the distribution of time intervals between zeros of speech signals. This collection.