SOME ISSUES CONCERNING THE ACCURACY OF INITIAL SOCIOLOGICAL INFORMATION

E. N. Kamyshev

(Presented by the scientific seminar of the Department of Scientific Communism)

Any act of cognition of reality, in its development, presupposes the passage through at least two stages. These are: the abstract, theoretical-level resolution of a question, which, as a rule, concludes with the advancement of various kinds of hypotheses; and the collection of empirical material, which makes it possible to settle upon hypotheses corresponding to reality, to fix them in theory as already proven, and thereby to open the way to new propositions.

The creation of a system for the education of engineering personnel is no exception to this rule. The need to verify many theoretical conjectures naturally leads to the sociological method of resolving the question, as the one that most fully corresponds to the stated aim. However, despite the value of the final results that this method promises to provide, it contains a mass of difficulties, both of a general sociological nature and arising from the specificity of the subject of the investigation. One such difficulty is the assessment of the accuracy of the primary information with which the researcher will operate.

In the natural sciences it has long been customary, together with the measured data, to give the carefully calculated error of observation, so that these data may be used only where this is beneficial. “When an error is not mentioned, this either means that it is known to everyone and universally accepted, or it is done because the data being presented have already entered into the theory of the exact sciences, which determine the permissible degree of error, and their boundedness is the same as that of the corresponding theory—for example, in measuring the speed of light, since this indicator is inseparably connected with the theory of relativity. The same is true with respect to other constants.”¹ In the social sciences such practice, unfortunately, has not become widespread. The main reason for this is, undoubtedly, the extraordinary difficulty of quantitatively determining the degree of error. But even a qualitative description of errors is, to begin with, a necessary step so that in the future researchers may be able to arrive at a more exact quantitative assessment of them. Until the theory of errors in the social sciences has been developed to a sufficient degree, in practice one will have to be guided in assessing them exclusively by common sense

meaning, logic, and responsibility of the researcher. Therefore, at the present stage in the development of empirical sociology, it is important for us to know not only the results of one study or another but, perhaps first of all, how these results were obtained—that is, what methodological and technical procedures underlay the work, and whether the conclusions obtained on their basis can be considered reliable.

In this article we shall not examine all possible cases in which errors arise in a sociological study. Consideration of this question would require a deeper analysis than is envisaged in the present work. We shall attempt to show certain methodological and technical problems in the study of graduates of the Tomsk Polytechnic Institute, conducted by the sociological group of the Department of Scientific Communism of that same institute in 1969–1971, which are connected with the possibility that errors may arise in the primary information.

In the ideal case, the following requirements may be imposed on the initial data in any sociological study: first, they must be reliable; second, it is desirable that the initial data be quantitative, that is, that each phenomenon be expressed unambiguously by some number or system of numbers; third, the initial data must be sufficient in quantity so that the conclusions obtained on their basis may be extended to other objects as well.^2

Hence there are three large sources of error affecting the quality of the initial empirical material: errors connected with the method of collecting the material; errors connected with the quantitative measurement of social characteristics; and the so-called sampling errors.

As regards the first of these, there is still debate about the effectiveness of particular techniques for gathering information about the object of interest to the researcher. Thus, the English sociologist F. Yates believes that “direct observation is preferable to the survey...”^3 In contrast, the well-known Soviet sociologist B. A. Grushin, on the contrary, considers the questionnaire survey method to be “the leading, basic method of studying public opinion.”^4

Despite such seemingly fundamental disagreements, at present, relying on the decade-long experience of the development of concrete sociological research in our country, as well as on foreign experience, it may be stated that the underestimation of methods of collecting primary information in sociology is being overcome. A number of works have already appeared in Soviet literature that analyze in detail the methods of conducting concrete sociological studies.^5

Here one should note one point that is unanimously emphasized by almost all authors and that consists in the simple idea that the researcher will achieve the greatest effect only by applying combinations of different methods and not by singling out any one of them.^6 Usually this proceeds from the assumption that the shortcomings of one method are covered by the advantages of another. However, no less just is the reverse—

^2 A. A. Frenkel, Mathematical Analysis of Labor Productivity. Moscow, 1968, p. 42.

^3 F. Yates, The Sampling Method in Censuses and Surveys. Moscow, 1965, p. 89.

^4 B. A. Grushin, Opinion about the World and the World of Opinions. Moscow, 1967, p. 332.

^5 For example: V. A. Yadov, Methodology and Procedures of Sociological Research. Tartu, TSU, 1968; A. P. Svenitsky, “The Interview as a Method of Concrete Sociological Research,” Filosofskie nauki, 1965, No. 4; T. I. Pavlova, “The Survey as a Method of Studying Public Opinion,” Filosofskie nauki, 1968, No. 3, etc.

^6 Social Research. Moscow, 1965, p. 34.

...a contrary thought. Indeed, why could a situation not arise in which errors would be layered one upon another? Therefore every assertion about the neutralization of errors must be proved, and finding such proof is difficult. It is not easy to decide whether it is fair, true, and sufficiently correct. Here one has to rely exclusively on the logical coherence of the proof.

In our study, the primary information about graduates of the institute was obtained mainly by three methods fairly well known in sociology. These were a direct written survey of the young engineer, a standardized interview with his immediate supervisor, and an analysis of factory and higher-education documentation. Accordingly, three documents were developed, characterizing each graduate of the institute and intended to be filled out: the young engineer’s questionnaire, the interview with the young engineer’s supervisor, and the graduate’s individual card.

We tried to formalize all the characteristics under study by known methods so that in the future it would be possible to use the effective apparatus of quantitative methods for analyzing the collected material. At the same time, taking into account the need for further work on the problem of training engineering personnel, we gave both the engineers themselves and their supervisors the opportunity to express their views on problems important, in their opinion, in light of increasing the effectiveness of this training. Therefore the questions were mainly of a semi-closed type, which makes it possible to solve the basic tasks of the study and to outline paths for further investigation of this problem.

From the standpoint of the content of the information obtained, each document played a different role in the study. Thus, the interview and the data from factory documentation were compiled in such a way as to obtain a characterization of the engineer’s activity at the enterprise in light of two functions: directly productive, and as an organizer and educator of the political activity of broad masses of working people. The questionnaire and the graduate’s individual card made it possible to collect information about the past activity of the institute graduate within the walls of the higher educational institution.

Thus, each of the three documents listed could be filled out independently of the others, and therefore there was a real possibility of assigning responsibility for filling them out to different executors. This fact, in our view, testifies to the reduction to a minimum of errors arising from the observer’s bias. In fact, each of them acted according to the developed instructions, which included only organizational requirements and said nothing about the main hypotheses of the study. However, the danger of this kind of bias exists in any study, especially if the collection of material is conducted directly by a person interested in proving one hypothesis or another. And here the question is not so much the researcher’s scientific integrity as his involuntary striving to seek material that confirms his statements.

The greatest danger of obtaining nonobjective data about the young engineer exists when filling out the questionnaire and the interview. Until recently, an anonymous survey was considered the best remedy for distortions of this kind. We could not follow this path because, first, we needed the surname as a connecting link among the three above-named documents, and second, the idea that an anonymous survey impedes the appearance, to some degree, of falsified information is fairly widespread...

so common, so little proven. Of course, guided by common sense, one may admit their effectiveness in matters concerning the intimate aspects of a person’s life, where it would be unpleasant for the respondent to have his name invoked, as well as in cases where there is a danger that the information given will be used against the respondent. Neither the one nor the other is present in our study; therefore the question of anonymity of the study documents falls away by itself.

There is another danger here. As has been noted, it is characteristic of a person to think of himself as somewhat better than he actually is, although each person’s ability to differentiate his own actions varies. Therefore we have the right to expect that the data obtained from the questionnaire about an institute graduate will be somewhat embellished in comparison with the real data. A different picture, in our view, is observed during the interview with the young engineer’s supervisor. We formed the impression that, in striving to point out shortcomings in the specialist’s higher-education training, many supervisors gave somewhat distorted characteristics of their charges in a negative direction. Thus, when analyzing the material, one may expect something like a gap between the quality of the graduate’s work and his training, which undoubtedly makes it more difficult to carry out the main task of the study. It is difficult to say specifically to what extent this will manifest itself, but we hope that this kind of error will nevertheless make it possible to identify the most striking shortcomings in the higher-education system of training and educating engineering personnel.

Another major source of errors in primary sociological information is the procedure for measuring the social concepts used in the study. The problem of measurement itself follows from the proposition that the more or less effective conduct of any concrete sociological study is unthinkable without the application of all the modern tools and methods of scientific cognition, which presuppose that a measure must be defined—the unity of the quantitative and qualitative determinacy of the phenomena and processes being studied.^7 Before the researcher has the possibility of analyzing phenomena not only at the qualitative but also at the quantitative level, he must formalize the phenomena under study to such a degree that they receive numerical expression, i.e., solve the problem of measurement. If in many technical sciences such a problem at first glance does not exist (since the phenomena under study already possess a measure understandable to all), then in the field of the social sciences, where it is incomparably more difficult to single out a measure, it acquires enormous significance.

In Soviet sociological literature a number of authors have already addressed this area,^8 but, as E. P. Andreev and G. V. Osipov rightly note, there is still great diversity in the understanding of sociological measurement.^9 In the view of many authors, measurement is the assignment, according to definite rules, of numbers to the studied properties of an object. These rules consist in establishing the degree of isomorphism between certain properties of numbers and certain formal

^7 Quantitative Methods in Sociology. Moscow, 1966.

^8 E. V. Belyaev, “Problems of Sociological Measurement,” Voprosy filosofii, 1967, No. 7; Yu. A. Patrugin, “On the Measurement of Qualitative Attributes,” in: Modeling of Social Processes, Moscow, 1970; V. N. Shubkin, “Quantitative Methods in Sociology,” Voprosy filosofii, 1967, No. 3; V. A. Yadov, Methodology and Procedures of Sociological Research. Tartu, 1968.

^9 E. P. Andreev, G. V. Osipov, “Mathematics and Sociology,” Voprosy filosofii, 1968, No. 11.

properties of the characteristics being studied.^10 The further course of the investigation depends on the rules by which the researcher is able to designate by numbers particular properties of the object—that is, on the scale by which he manages to measure the attribute of interest to him. Every researcher’s dream is to measure attributes on a ratio scale,^11 which means establishing the most complete isomorphism between the properties of the object being studied and their numerical expression, and therefore makes it possible to replace a direct natural experiment on the object by an experiment on numbers. This is especially important for the social sciences, where the object of study is usually large masses of people and where experiment on them is often difficult for familiar reasons.

At present, all sociological measurements are still being carried out on nominal and ordinal scales, and only tentative steps are being taken toward moving to equal-interval scales and ratio scales. Probably in the future we shall learn to measure social characteristics also within interval and ratio scales, since, in Homans’s opinion, “ideally the nature of explanatory theory in the social sciences is no different from what obtains in the physical sciences...”^12 At the same time, however, one must not forget that “mathematics for the sociologist is more abstract than for the natural scientist.”^13

It is important to note that measurement is not only a quantitative procedure. To a greater degree it depends on qualitative analysis. Before measuring anything directly, it is necessary to establish the measure of the objects of measurement beyond which they lose their former quality. In practically any concrete sociological study, two problems arise in this respect: correlating the general concepts used by the researcher in theoretical analysis with less general ones that are ultimately determined by direct observation of real events; and constructing the scale proper for measuring general concepts. The two problems condition each other; without solving one there can be no solution to the other.

The first, the empirical interpretation of theoretical concepts, is nothing other than an ascent from the abstract, given in concepts and definitions, to the concrete, given in direct observation. It must be noted that such a dialectical-materialist approach to the empirical interpretation of theoretical concepts differs fundamentally from the positivist point of view, where, as is known, scientific abstractions are considered permissible insofar as they can be verified, i.e., checked empirically, by direct comparison with the data of sensory experience.

The principle of verification was first defined by Schlick and Wittgenstein as the principle of checking the truth or falsity of scientific concepts, and found many followers in bourgeois empirical sociology.^14 For a positivist, theoretical concepts are true only when their content can be entirely decomposed into single elements of the sensuously given.

^10 Psychological Measurements. Moscow, 1967.
^11 S. S. Stevens. Experimental Psychology. Vol. 1, Moscow, 1960.
^12 E. P. Andreev, G. V. Osipov. Mathematics and Sociology. — Voprosy filosofii, 1968, No. 11, p. 105.
^13 P. P. Maslov. Sociology and Statistics. Moscow, 1967, p. 69.
^14 G. M. Andreeva. Contemporary Bourgeois Empirical Sociology. Moscow, 1965, p. 200.

But the striving to reduce abstract concepts entirely to operational ones, to a mechanical sum of empirical referents, is hardly an attractive undertaking. Just as the phenomenon does not fully reflect the essence, so the content of an abstract concept cannot be fully reduced to the data of sensory experience. However, such irreducibility of the theoretical content of concepts to their empirical equivalents, as A. G. Zdravomyslov and V. A. Yadov note, does not mean that their essence is unknowable; it means only the relative independence of theoretical knowledge itself². The comparison of theoretical and operational concepts in a concrete sociological study is “absolutely necessary, for otherwise we cannot find an outlet into the sphere of practice and will be forced to remain confined to the sphere of abstract definitions”³.

The point, therefore, is to keep constantly in mind those limitations possessed by operational concepts, so that, at the stage of ascent from empirically interpreted concepts to scientific abstractions, we do not lose sight of the essence of the phenomena being analyzed, which are only partially expressed in the operational definition.

Here it is necessary to pause over one more important point. Any object subjected to investigation is connected with the surrounding reality by a multitude of connections and relations. Strictly speaking, it is precisely its various connections and relations with other objects that make it possible to give it various definitions through concepts. It is enough to recall Lenin’s proposition on the dialectical definition of an object, as distinct from the eclectic one: “If I need a glass now as an instrument for drinking, then it is completely unimportant to me to know whether its form is fully cylindrical and whether it is really made of glass; what is important is that there be no crack in the bottom, that one cannot cut one’s lips when using this glass, and so on.”⁴ And further: “…all human practice must enter into the complete definition of the object … as the practical determinant of the object’s connection with what man needs.”⁵ Consequently, the empirical interpretation of a given concept means the selection of such empirical referents as would reflect the connection of the object under study with reality in some respect essential for the researcher. And this, in turn, will follow from the tasks of the study.⁶

The direct-production function of the young engineer is precisely such a concept, the need for effective interpretation and measurement of which is dictated by the principal task of the study of graduates of the Tomsk Polytechnic Institute. It is part of a more general concept, namely, the activity of the engineer in general.

The singling out, from the structure of the engineer’s activity, of a single direct-production function is, naturally, purely conditional and does not aim to oppose it to the others; this singling out is possible, of course, only in abstraction and is difficult to isolate in real life. In general, this shortcoming is inherent in any classification in the social sciences, which carry, alongside the information needed by the researcher, auxiliary information, which in the ideal case is taken into account—

should not be classified. Morris Reschlen shows this as follows: “To classify subjects according to their profession means to distinguish them, on average, by level of culture, standard of living, the duration and nature of leisure, etc.”^20

Thus, in its pure form, however much we might want it, there is no possibility of measuring the direct productive activity of a young engineer by modern methods; nevertheless, the positive aspects of such a classification allow us to accept the fact that the content of the information does not fully reflect the content of the phenomenon being measured.

In a concrete study, this direct productive activity can be grasped only in its individual manifestation—namely, in its manifestation at a particular workplace. At every enterprise there is something of its own, something individual, which to some degree leaves its imprint on the measurement being made. Therefore, for us, an empirical interpretation of the direct productive activity of an institute graduate means, first, identifying those aspects of activity subject to measurement that could serve as a kind of criterion of the higher-education training of the specialist (this follows from the objectives of the study), and, second, these aspects should, as far as possible, not depend on the individual conditions of the activity of a particular specialist.

In our view, the following characteristics meet these conditions: promotion in service, growth of wages, incentives and reprimands, and management’s opinions about the productive activity of the young engineer.

The attentive reader may immediately notice that these characteristics are closely connected with one another. Thus, for example, promotion in service leads, as a rule, to an increase in wages, and vice versa. There is nothing surprising in this, since they are linked to the interpreted concept as the general and the particular.

In the study, the selected empirical referents of direct productive activity were measured as follows. Wage growth was measured as the difference between the young engineer’s wage at the time of the survey and the wage he received upon entering the enterprise, using the following scale of 9 points:

Promotion in service was measured as the difference between the position he held at the time of the survey and the lowest possible position, on a scale of 4 points:

1 — engineer (foreman),
2 — senior engineer (senior foreman),
3 — lead engineer (deputy shop chief),
4 — head of a laboratory, bureau, department (shop chief).

The presence of incentives and penalties—the very fact of their presence—was recorded on a three-member scale:

^20 Experimental Psychology. Moscow, 1966, p. 203.

1 — has incentives,
2 — does not have them,
3 — has penalties.

The opinion of the immediate supervisor as to whether the young engineer copes with the work required by his position was measured according to the following three points:

1 — copes well,
2 — copes,
3 — does not quite cope.

From the standpoint of measurement, these are rather heterogeneous elements. For all the elements it is still necessary to find an experimental operation that would make it possible to operate with the numerical values of these elements within some scale.

At first glance, the salary of a young engineer already possesses a ready-made, well-measured quantitative measure expressed in rubles. However, upon closer examination, when an increase in salary is considered as an indicator, a criterion of the specialist’s university training, it turns out that the proportionality of the scale is not so obvious after all. Indeed, if one engineer’s salary was raised by 10 rubles and another’s by 20 rubles, then to conclude that the second engineer works exactly twice as well as the first would be unjustified. The growth of salary is subject to a whole series of limitations, which make it impossible (proceeding from the tasks of our study) to use such a unit of measurement of direct production activity as the ruble within an equal-interval scale and a ratio scale.

Generally speaking, the ruble as a unit of measurement is suitable for measuring income, the cost of living, but it reflects such a characteristic as the direct production activity of a young engineer in a disproportional way. In such a case it is necessary either to take into account the degree of this disproportionality, or to find another unit of measurement that would uniformly reflect the measured attribute. It should also not be forgotten that the empirical referents that have been identified must be measured in a unified way, so that in the future the researcher will have the possibility of moving from them to a general operational definition of the direct production activity of the young engineer.

In sociology there exist various ways of constructing measurement scales. However, as scholars from the GDR rightly note, the overwhelming majority of them were developed by bourgeois sociologists and, as a rule, must be treated with great caution. Thus, if knowledge of the technique of constructing scales can play a positive role in the more rapid development of concrete social research, then the initial points for the choice of indicators usually require serious rethinking⁷.

Today the best-known methods, which have become widespread especially in bourgeois empirical sociology, are those of L. Thurstone, R. Likert, L. Guttman, and P. Lazarsfeld. The characteristics of the various techniques of scaling, and their development in the history of sociology, are ably presented by B. Green⁸. Here we shall only note that,

that the methods analyzed by B. Green—judgments, summated ratings, scalogram analysis, the unfolding technique, analysis of latent structures, and rating—aim to obtain an ideal scalogram (as, for example, in the physical sciences), but none of these methods has yet succeeded in realizing it.

This desire to reduce measurement in the social sciences to measurement of the kind found in the physical sciences is characteristic of many authors. It is often identified with some absolute measurement, whereas in physics itself all measurements are relative. It is enough to cite the example of measuring the weight of an object. As is known, in physics weight is measured on a ratio scale or on a scale of equal intervals with a fixed zero point. The scale is based on the unit of measurement “kg.” One could have chosen any other unit as well, as is shown by the English system of weights, where the pound is taken as the basis. This is not the point; the point is that all measurements are correlated with the force of the earth’s gravity, which manifests itself more or less identically at any point on the globe. The force of the earth’s gravity is, as it were, the criterion, the evaluator, the judge in the matter of measuring the weight of various objects.

As applied to social characteristics, in particular to the immediate production activity of a young engineer, only the usefulness of the results of each engineer’s activity for social development can serve as such a criterion. Such an approach, although in the ideal case it is the only correct one, has not been developed at all, since it encounters serious difficulties. Therefore, in our study we have to judge the usefulness of the results of an engineer’s activity not directly, but indirectly, through the empirical referents identified above. Of course, the evaluation that is taken as the basis is in a definite dependence on the level of development of the enterprise and the branch, on the degree of their contribution to social progress, and so on. The only thing that can be done here is to conduct the study at enterprises that stand in the first rank, that determine the production policy of their branch, that is, at advanced and more or less homogeneous enterprises. Only on condition of the homogeneity of enterprises in the sense indicated above can measurements of the selected empirical referents be comparable.

The single indicator we need—how much each engineer stands out from the general mass in a positive or negative sense in terms of the quality of his work—also determined the method for constructing the scale for the immediate production activity of a young engineer. The idea used was one expressed by R. Angell in the USA and by A. Zdravomyslov and V. Yadov in the USSR.^23 Its meaning consists in determining the “weight,” or significance, of each point of the attribute at which measurement is made, on the basis of the probability of occurrence of the event. The “weight” of such a point is inversely proportional to the frequency of occurrences and to the number of points in the given characteristic. It is assumed that such a determination of the degree of inequality of the intervals between measurement points on the scale of the attribute makes it possible to perform operations on numbers that are characteristic of an equal-interval scale. But such an assertion requires its own rigorous mathematical justification. At the same time, in our view, such a technique for constructing a scale must take into account the form of the distribution of the attribute in the sample population.

^23 Man and His Work. Moscow, 1967.

Thus, the growth of wages and advancement in position among young engineers will have a form of distribution characteristic of monotonic attributes, that is, those in which, as the magnitude of the attribute changes, the probability of its occurrence changes in one direction or remains constant, but never changes in the opposite direction.

The “weight” function of such attributes is a continuous function, which also serves as the measurement scale. However, matters become more complicated when constructing a scale for a nonmonotonic distribution of attributes, that is, those that have a pronounced extremum.

In our case, these are data on punishments and rewards of young engineers, as well as management’s opinion of their immediate production activity.

If an attribute is nonmonotonic, then the “weight” function of the various points of the attribute will have a discontinuity at the point corresponding to the modal grouping. In essence, in this case we are dealing with two scales of one attribute, one of which measures its positive values and the other its negative values. In order for one scale to be a continuation of the other, it is necessary to take into account the discontinuity of the function at the point corresponding to the modal grouping.

When determining the form of an attribute’s distribution, when the scale consists of more than three points, it makes sense to smooth the empirical distribution line by the method of least squares, which undoubtedly makes it possible to avoid random deviations to some extent. In our study, such smoothing was carried out for the distribution of wage growth and advancement in position.

The arithmetic means of the sums of the “weights” of each respondent, composed of the values of the scale points to which corresponded his wage growth, advancement in position, the presence of punishments and rewards, as well as the opinion of his production activity held by his immediate management, constitute a single, general scale for measuring the immediate production activity of young engineers.

Thus, each young engineer, depending on the quality of his production activity, receives a unified assessment that makes it possible to distinguish him, in a positive or negative sense, from the population under consideration. On this basis it is possible to form several groups reflecting the different quality of the production activity of institute graduates, with their subsequent correlation with the specific factors influencing the process of forming the engineer’s personality. Other concepts of the study were measured by a similar method.

It is logical to assume that errors made in such an interpretation and measurement of social concepts somewhat distort the real picture; at the same time, however, even a first acquaintance with the research procedure gives the reader an opportunity to draw a conclusion (at the qualitative level) about the reliability of the primary information of the study. As for sampling errors, we shall dwell on certain points, important in our view, concerning the application of sampling theory in the specific study that confronted the members of our group during the planning of the study of the activity of TPI graduates.

By its very nature, the sampling method is a study of a part of the “universe.” The question, therefore, is how, on the basis of studying a part, to obtain an idea of the entire population with minimal error. When the population is homogeneous, as, for example—

for example, a sack of wheat or a piece of cloth from the standpoint of milling and tailoring a suit, respectively, this is done quite simply. It is enough to take any handful of grain or any piece of fabric, and we shall have an idea of the sack of wheat and the roll of cloth as a whole.^24 It is another matter when the aggregate is heterogeneous and, moreover, possesses elements that differ in quality. There are various types of samples designed to determine, with the desired degree of approximation to the population, the average indicators of interest to us and the distribution of a characteristic. Despite the multiplicity of types of sampling, their different classifications, and the different terminology used by different authors, “all of them are organized according to the principle of random selection in accordance with schemes considered in probability theory.”^25 In random selection, each unit of the quality under study is provided with an equal probability of falling into the number of units studied, which makes it possible to avoid bias in selection. Proceeding from this, one may distinguish three main tasks that require solution when planning a sample in any concrete sociological study. These are the determination of the qualitative structure of the population; the determination of a sample size sufficient for representative conclusions; and the development or choice of methods for selecting units of observation.

Let us consider each task separately. In determining the qualitative structure of the sample, the decisive role obviously belongs to the goals and hypotheses of the study. Indeed, in order for a sample to be representative, it is necessary that, with some given degree of accuracy, it reflect not the structure of the population under study in general, but only those features of this structure that are essential for our task. Here it is difficult to work out any specific rules and techniques. How the researcher finds it necessary to define the structure of the object being studied, and how successfully he is able to reflect it in the structure of the sample, depends mainly on the theory on which he will rely in doing so and on his personal abilities as a scholar. Traditionally this question is dealt with in the theoretical part of the program. This creates the impression that it has no relation to the sample. We shall note here, however, that the slightest inaccuracy in determining the structure of the population is fully carried over into the sampling error and, unfortunately, is not recorded anywhere. It is regarded rather as an error of the study as a whole than as an error of representativeness. For our study it is important that the sample population reflect the structure of the population in terms of the groups, identified in the theoretical analysis, with different contents of engineering labor. The most suitable for such purposes is a typical random sample, when the unit of selection coincides with the unit of observation and the sample is drawn from homogeneous parts of the population according to the principle of random selection.^26 It is also called a stratified sample, and when the goal is to achieve a more even representation in the sample of different geographical districts, an area sample.

For us, however, it is important to study not the selected groups in themselves, but those of them that are represented by a substantial majority of institute graduates. Such a need to determine the proportions, in terms of numerical composition, of the selected groups cannot be satisfie—

^24 F. Yates, Sampling Methods for Censuses and Surveys. Moscow, 1965, p. 23.

^25 N. M. Vinogradova, V. G. Evdokimov, E. M. Khitarova, N. I. Yakovleva, General Theory of Statistics. Moscow, 1968, p. 339.

^26 Ibid., pp. 357–360.

a typical sample. A simple random sample, which we used in the study, best serves this purpose.

When the qualitative structure of the sample has been compiled, the question arises of how many units of the general population must be selected for observation in each qualitatively homogeneous element of the identified structure in order for the conclusions to be representative. Or, in the language of mathematics, what should the sample be so that, with a given probability, the error in estimating the sample characteristics does not exceed the permissible (specified) limits. The problem is, in essence, the inverse of determining the magnitude of the sampling error. Therefore the formulas for determining the required sample size are merely formulas for determining its error, solved with respect to the sample size. This causes great difficulties, because it is necessary to know in advance the magnitude of the population variance and the size of the general population. In connection with this, the literature contains recommendations to conduct small pilot studies in order to obtain at least a general idea of certain parameters of the general population.^27 However, in practice this is not always convenient, and often simply impossible.

Moreover, one must not forget that the formulas for determining sampling error differ both depending on its type and depending on the type of parameters whose error the researcher is calculating. Strictly speaking, the calculation of representativeness should be carried out for each parameter that the researcher will use at the stage of data analysis. The maximum error of representativeness set by the researcher determines the reliability of the data from the sample observation, their suitability for judging the general population. It is usually meant that these errors arise as a result of random causes; therefore, when calculating representativeness, one proceeds from the normal law of distribution. In the general case, however, it is impossible to give an unambiguous answer to the question of what the sample size must be in order for the estimate of a characteristic to be regarded as normally distributed. This size, among other things, depends on how the given characteristic is distributed. “If we are dealing, for example, with the arithmetic mean of a sample from the values of a variable \(X\), whose distribution is not too asymmetrical, then the approximation to the normal law may be considered practically sufficient already at 30 units of observation. For such characteristics as variance, skewness, kurtosis, and a correlation coefficient less than 0.2, for practical normality the sample should be brought to sizes of no fewer than 100 units. If the correlation coefficient is close to one, then even at 300 units the normal approximation is still not sufficient.”^28 Such an increase in the size of the sample population can be avoided if one turns to the theory of small samples. In this case, in calculating sampling errors, we proceed not from the normal distribution of the estimates of the parameters of the sample population, but from the distribution proposed by Student and Fisher. In addition, the sample size can be substantially reduced if the probable bounds within which the error of estimating sample characteristics will not exceed the specified limits are expanded. The establishment of such a boundary is to some extent always arbitrary. Usually in social research

^27 Social Studies. Moscow, 1965.
^28 N. V. Smirnov, I. V. Dunin-Barkovsky. A Course in Probability Theory and Mathematical Statistics. Moscow, 1969, p. 211.

confidence intervals for estimates of the parameters of the sample population are taken at a probability level equal to 0.95 or 0.99.

In our view, this is not always justified and is not always dictated by the objectives of the study. Often it is important for the researcher only to confirm certain emerging tendencies in development, while he by no means intends to model one or another social process in all its complexity and precision. In this case, reasoning with a probability of 70–80%, in our view, may be regarded as quite sufficient.

At the same time, one may reason about a large sample and ignore the calculation of errors of representativeness, being confident in advance that they will not exceed permissible limits. However, there is always a danger (especially if the necessary sample size was calculated only on the basis of an estimate of the mean) that some parameter will go beyond permissible limits. And the same reasoning can be carried out in parallel with the calculation of sampling errors, which makes it possible in each specific case to indicate the level of their significance. For this it is not necessary to bring the size of the sample population up to astronomical figures. It is possible to estimate one or another parameter, using the apparatus of small samples, with as few as 10–30 units of observation. Increasing the sample size will lead to more cautious estimates, but will not change the essence of the proofs.

If a researcher rejects a hypothesis with a probability of 75%, then he is unlikely to be able to prove it at a probability level of 95 and 99%. Acceptance of a hypothesis, even with 75% probability, increases the validity of our assertions, although it does not make them true.

Taking the above into account, it is possible to calculate approximately the size of the sample population for our study. To determine differences between the groups identified in the theoretical analysis according to the content of engineering labor, if one uses mean indicators and percentages for comparison, the minimum size of each group should be 10–30 units of observation. Within this size it is also possible to calculate pair coefficients of rank correlation within each group, indicating the levels of their significance. With 9 groups, the total minimum size for a typical sample will be 90–270 units of observation. For a simple random sample this number will be the larger, the more disproportionately these groups are represented in the general population. This kind of sample makes it possible to extend the conclusions to the object of the study as a whole, that is, to the process of training engineers at the Tomsk Polytechnic Institute. As for their broader dissemination, it may probably be considered provisionally admissible, taking into account that degree of commonality in the training of specialists which is observed at the country’s polytechnical higher educational institutions.

After the researcher has solved the first two tasks, only one remains: in accordance with the size and qualitative structure of the sample, to select, by one method or another, the units to be studied for analysis. As we have already noted above, this selection must be random, in order to avoid subjectivity and bias.

The essence of random selection consists in giving each unit of observation an equal probability of being selected. There are various ways of carrying out random selection. The simplest of them are connected with the researcher’s having at hand a list of all units of the general population. Then the sample is obtained by applying the method of drawing lots (lottery) or by using a table of random numbers. Usually, by drawing lots

determine the number in the list of one unit of observation, and then the subsequent units are determined with the aid of a table of random numbers.

Often these methods are replaced by mechanical selection or by mechanical selection with a random start. In mechanical selection, the units subjected to observation are those located at a certain (equal) distance from one another in the order in which they are arranged according to a characteristic in no way connected with the characteristic under study (for example, alphabetical order). It is more difficult to implement the principle of random selection when there is no list of the general population at hand. In such cases, the organization of the selection depends both on the problem itself and on the resourcefulness of the researcher. Thus, in identifying public opinion regarding the dynamics of the standard of living, B. A. Grushin used an interesting device. On one day, passengers on long-distance trains departing from all nine railway stations of Moscow were surveyed.

In this case, the sampling scheme was based on a number of assumptions. The principal ones consisted in the supposition that 1) from the point of view of the problem under study, precisely these, and not other, respondents enter the sample by chance, and 2) all categories of working people are evenly represented among passengers traveling through Moscow. As the author himself writes, the sample “was crowned with complete success in the sense of representing the country’s various economic-geographical regions... The spontaneous sample organized in this way yielded an ensemble in which the population of all, without exception, Union republics proved to be represented; 15 of the 19 autonomous republics, all 6 territories and 49 regions of the RSFSR, 19 regions of Ukraine, 10 regions of Kazakhstan, 6 regions of Belorussia, etc.”⁹

Something similar was used in our study. The object of our study was graduates of the Tomsk Polytechnic Institute working at industrial enterprises, in design organizations, at higher educational institutions, and in research institutes of Western Siberia. We did not have at hand a list of the general population; therefore random selection was carried out in the following way. First, large cities of Western Siberia were selected, in which only large enterprises of Union significance were taken, where graduates of TPI worked. In essence, we surveyed all graduates of the institute who, at the time the study was conducted, were at these enterprises. Even without analyzing the reasons for their absence, one may say that any given engineer entered the sample purely by chance, since their absence was explained by production-related or personal reasons, naturally in no way dependent on the aims of the study. Strictly speaking, in our view, there is no essential difference between selecting units of observation by a table of random numbers and what was used by us.

1. O. Morgenstern, On the Accuracy of Economic-Statistical Observations. Moscow, 1968, p. 15. ↩

2. Man and His Work. Moscow, 1967, p. 17. ↩

3. Ibid., p. 16. ↩

4. V. I. Lenin, Collected Works, vol. 42, p. 289. ↩

5. Ibid., p. 290. ↩

6. V. A. Yadov, Methodology and Procedures of Sociological Research. Tartu, 1968, p. 74. ↩

7. H. Etschmann, G. Callabis, R. Schulz, H. Taubert. Introduction to Sociological Research. Berlin, 1966 (in German). ↩

8. Mathematical Methods in Contemporary Bourgeois Sociology. Moscow, 1966. ↩

9. B. A. Grushin, Opinion about the World and the World of Opinions. Moscow, 1967, p. 372. ↩