ON THE TECHNICAL IMPLEMENTATION OF IDEAS IN SPEECH ANALYSIS

B. N. EPIFANTSEV

(Presented by the scientific seminar of the Department of Computer Engineering)

A review of works on automatic recognition of speech signals [1] gives grounds to assert that, despite the considerable efforts recently undertaken in this direction, the results obtained do not make it possible to satisfy the demands of numerous technical applications. There are many reasons for this, and undoubtedly one of them consists in the use, by the majority of researchers in their experiments, of narrowly specialized analog apparatus with its limited capabilities.

In 1964, the lack of prospects for using specialized devices for speech analysis was substantiated [2]. Instead of analog apparatus at all stages of the work, it was proposed to use general-purpose computers [2]. This point of view has not undergone any substantial change to this day, although the authors of a number of works [3, 4] consider it expedient to use general-purpose computers at the final stage of analysis—the processing of features that are, in turn, extracted with the aid of specialized installations.

The indicated proposals on the technical embodiment of ideas in speech analysis have a certain commonality: they are not substantiated in any way. As will be seen from what follows, the necessity of the latter is quite obvious.

When entering into a computer and processing special information on a computer, one has to deal with a number of fairly general conditions:

1. \(V_c \leqslant V_{\text{ram}}\), i.e., the volume of the signal must be less than the volume of the random-access memory of the computer.

2. The throughput capacity of the machine’s information channels must be matched to the density of the information entering the computer.

3. The construction of the computer must not be subjected to any kind of modifications that would make it difficult to solve other problems.

4. Provision must be made for monitoring the correspondence of the entered signal to the original one.

5. In choosing a scheme for the technical implementation of ideas in speech analysis, the factor of the cost of machine time must be taken into account.

Indeed, the classical scheme for entering special information into an electronic computer (sensor—analog-to-code converter—computer) proposes the measurement and input of each count of the original function, i.e., the volume of information entered into the computer is thus

the so-called absolute description of the input signal [5], representing the signal under study with all the completeness and accuracy determined by the methods of observation and the resolving power of the means of observation. At the sampling frequency adopted for speech signals, \(f_{\mathrm{qv}} = 22\ \text{kHz}\), and a dynamic range of \(45—48\ \text{dB}\) [6], the amount of information entered into a computer to describe a word of duration, say, \(1\ \text{sec}\), is about \(200 \cdot 10^3\ \text{bits}\). This quantity exceeds the capacity of the random-access memory of most general-purpose computers. Therefore, when the classical scheme for inputting special information is used, the duration of the function under study has to be reduced to unjustifiably small values. For example, for the BESM-2M machine, which has a random-access memory capacity \(V_{\text{RAM}} \approx 80 \cdot 10^3\ \text{bits}\), the time interval over which the function exists is, in the best case, limited to \(0.4\ \text{sec}\), which is clearly insufficient if one takes into account the presence in speech of correlations over sections larger than \(0.4\ \text{sec}\) [7].

Furthermore, the enormous density of information (\(200\,000\ \text{bits/sec}\)) arriving from the analog-to-code converter rules out all possible methods of introducing special signals into a computer that are based on sequential recording principles, since the sequential recording principles of modern machines allow an information input density not exceeding \(50 \cdot 10^3\ \text{bits/sec}\). The use of parallel recording principles, however, first, limits the class of machines suitable for speech analysis and, second, significantly complicates the structure of analog-to-code converters [8], bringing it to an unjustified complexity in attempts to introduce a check on the correspondence of the entered signal to the original one [8].

Finally, processing an absolute description of a signal is extremely uneconomical from the standpoint of machine-time expenditure. Thus, if we take the signal considered above, whose absolute description is \(200 \cdot 10^3\ \text{bits}\), and assume that the machine operates with 9-bit binary numbers, then in order to compute, say, the \(k\)-th point of the autocorrelation function of the given signal,

\[ B(\tau_k) = \frac{1}{m} \sum_{i=1}^{m} f_i \cdot f_{i+k} \]

it will be necessary to perform \(22 \cdot 10^3\) multiplications, \(22 \cdot 10^3\) additions, and one division. To compute all possible points, the figures given increase to \(484 \cdot 10^6\) multiplications, \(484 \cdot 10^6\) additions, and \(22 \cdot 10^3\) divisions.

If we now take into account that the quality of solving problems corresponding to the third and fourth points is directly dependent on the volume of the signal, and that realizations of one and the same sound, pronounced by one and the same speaker, differ significantly from one another, attempts to represent the signal with high accuracy [6] make no sense.

Let us consider another idea for the application of electronic computers to the study of speech signals. Its meaning is that, with the aid of analog apparatus, some feature is extracted from the speech signal, which is then fed into the computer through an analog-to-code converter (see, for example, [3, 8]). Such a method makes it possible, generally speaking, to extract the features of interest “painlessly,” using general-purpose machines for their subsequent processing.

But what does it give?

To date there are no data whatsoever on a method for coding semantic information in speech. The experience of previous studies convincingly indicates that hopes of solving the problem of recognizing speech sig-

signals, using a number of known integral transformations, have no foundation. Moreover, the existing premise of the fundamental possibility of passing from an absolute description of signals to a system of features is not obvious. As is known [8], according to this premise the organ of hearing performs some transformation, as a result of which the information density of the signal entering the nervous system does not exceed 50 bits/sec and, consequently, the compression coefficient effected by the ear reaches a value of 400–1000. But the latter figure will correspond to the truth only when, in determining the volume of the signal at the input, we rely on the same concepts on which psychologists relied in determining the volume of the signal at the output—which, precisely, is not actually observed.

Returning now to the question of the expediency of studying parameters on a digital computer, it should be said that until the limiting value of the compression coefficient has been strictly established, it is impossible to regard as possible the solution of the problem of recognizing speech signals at the level of individual preselected features. Therefore the use of specialized analog equipment limits the subsequent possibilities of the scheme under consideration for the application of a digital computer. Another shortcoming inherent in this scheme is that checking the correspondence of the input signal to the original is carried out by means of the laborious process of recording, on a loop oscillograph, the signals that are fed into the digital computer.

In search of a research scheme that would satisfy points 1–5 and at the same time would make it possible to operate with signals for which there was confidence that the amount of information contained in them is sufficient for understanding what has been said, let us recall the fact that the intelligibility of “vocoder” speech is sufficient for understanding words, and the intelligibility of a clipped signal differs little from that of the original. At the same time, the volume of a “vocoder” or clipped

Fig. 1

the signal is more than an order of magnitude smaller than that of the undistorted one. Naturally, the idea suggests itself of making use of these facts. Then the scheme for studying speech signals will look as follows (Fig. 1).

The speech signal is first subjected to compression, with the signal coefficient chosen so that it is possible to restore the original signal from the point of view of its intelligibility. Through an analog-to-code converter the compressed signal is entered into the electronic digital computer. With a compression coefficient \(K_{\mathrm{n}} = 10 \div 20\), conditions 1, 2, and 5 will not be violated. In this case, checking the correspondence of the entered signal to the original can easily be carried out by outputting the entered signal from the RAM to a synthesizing device and subsequently listening to it. The experimenter can readily determine by ear whether the recording of the signal in the computer was satisfactory, or whether it should be erased and a new one recorded.

The scheme considered has one further positive feature. By acting on the recorded signal by program methods, we can evaluate by ear various transformations of the signal which are practically impossible to perform with analog equipment.

In conclusion, let us note that the schemes considered for implementing ideas in speech analysis have been tested. The computer used was the BESM-2M. The most successful application proved to be for the analysis of clipped speech. The segment of the signal that could be recorded in the RAM exceeded 7 sec.

REFERENCES

1. N. G. Zagoruiko, G. Ya. Voloshin, V. N. Elkina. Automatic recognition of sound patterns (literature review). Computing Systems, issue 14, 1964.

2. N. G. Zagoruiko. On the exchange of oral information between a human being and computing systems. Computing Systems, issue 10, 1964.

3. O. A. Petrov. Statistical processing of primary features of speech signals using an electronic computing machine (ECM). “Pattern Recognition, Information Transmission Theory,” 1965.

4. V. P. Trunin-Donskoi, A. S. Firer. Input into the BESM-2 computer of certain speech features. “Input and Processing of Special Information in an ECM,” issue 4, Moscow, Computing Center of the USSR Academy of Sciences, 1964.

5. A. A. Kharkevich. On the selection of features in machine recognition. Publishing House of the USSR Academy of Sciences, Department of Technical Sciences, Technical Cybernetics, No. 2, 1963.

6. G. Ya. Voloshin. Analog-to-digital converter for inputting speech signals into an ECM. Computing Systems, issue 10, 1964.

7. A. N. Velichkin. Statistical study of the speech process. Elektrosvyaz’, VIII, No. 8, 1961.

8. Moles. Information Theory and Esthetic Perception. “Mir,” 1966.