Original Articles
 

By Dr. Shivalingappa B Javali , Dr. Nagaraj V Gudaganavar , Dr. Shodan M J
Corresponding Author Dr. Shivalingappa B Javali
Department of Biostatistics, SDM College of Dental Sciences, Dharwad, Karnataka - India 580009
Submitting Author Dr. Shivalingappa B Javali
Other Authors Dr. Nagaraj V Gudaganavar
Anjuman Degree College, Department of Statistics, Anjuman Degree CollegenDharwad, Karnataka, India - India 580001

Dr. Shodan M J
SDM College of Dental Sciences,Department of Community Dentistry,, Department of Community Dentistry, SDM College of Dental Sciences and Hospital, Dharwad, Karnatak, India - India 580009

BIOSTATISTICS

Sample Size, Reliability Coefficient, Internal Consistency, Measurement Scale

Javali SB, Gudaganavar NV, J SM. Effect Of Varying Sample Size In Estimation Of Reliability Coefficients Of Internal Consistency. WebmedCentral BIOSTATISTICS 2011;2(2):WMC001572
doi: 10.9754/journal.wmc.2011.001572
No
Submitted on: 21 Feb 2011 10:57:30 AM GMT
Published on: 23 Feb 2011 06:40:44 PM GMT

Abstract


Reliability refers to accuracy and precision of a measurement instrument or scale. Reliability of test scores are estimated through measures of internal consistency has been characterized mathematically in many ways that appear, on the surface at least, to be very different to one another. The coefficient alpha is the most widely used measure of internal consistency for composite scores. In this article, the inferential statistics for three coefficients of internal consistency i.e. alpha coefficient with theta coefficient and omega coefficients are estimated. These indices of reliability are extremely important in health research (medical and oral health). The estimation of alpha, theta and omega coefficients and analytical effects under different sample sizes are examined and described. But computations of these coefficients are easily put forwarded by statistical software programs like SPSS, STATA, SYSTAT, STATISTICA etc. The good quantity of reliability estimates is observed in the sample size of 50 and more. Therefore, the researcher claims that for calculation of reliability coefficient for five points scale or any, the sample size should be at least 50 and more is enough. It is also concluded that with a prescription that every time a researcher reports any one of alpha coefficient, theta coefficient and omega coefficients.

Introduction


Precision of measurement is one of the major issues, which is frequently quantified by the reliability coefficients. The coefficient alpha, developed by Cronbach (1951), is the most commonly used index for estimating the reliability of measurement instruments such as scales, multiple item tests, questionnaires, or inventories (Raykov 1997) in all the fields of research such as psychology, education, statistics, sociology, medicine, counseling, nursing, political science, and economics (Cortina 1993).
Measurement of concrete characteristics required a scale, which must be reliable and valid. But, it is an important problem in the case of measuring conceptual characteristics. There is a great deal of interest in developing sensitive, reliable and valid measure for clinical populations and for aggregates such as organizations and families. It is an appropriate to begin the series with a discussion of reliability of measurement, required some statistical procedures.
Reliability estimates provide essential information regarding one aspect of soundness of the measurement of variables, items or attributes etc. Thus, the reliability coefficients that are suitable in obtaining the reliability of the scale is important. In this article we used three reliability coefficients were introduced and calculated under various sample sizes, in which coefficient of alpha in particular is important, but there are circumstances it is inappropriate and another estimates should be generated.
Further, the sample size is playing an important role on estimation of the reliability level of the measurement scale. Thus, cronbach, theta and omega are as three measures of internal consistency are examined and an example of the calculation of each is provided.

Methods


Analysis of Reliability

It is a property that measurement scale must have, is an indicator of consistency of measurement values obtained from the measurements repeated under the uniform circumstances (Ilker, et al. 2007; Gay, 1985; Carmines and Zeller, 1982; Arkin and Colton, 1970; O’ corner, 1993; Carey 1988).  It is an important concept when there is no gold standard.  Using different ways, the reliability of the scale can be examined by applying the scale once, twice or applying the parallel scales once.  The reliability of internal consistency can be examined, when scale is examined once.  

The coefficient of reliability takes the range 0 to 1.
Methods of measuring internal consistency
If the reliability can be examined by applying the measurement scale once, the error of the estimated reliability will be smaller than the other estimation procedure of reliability, wrong management, incorrect methodology of scoring, unexceptional or temporary changes in personal performance affect the internal consistency (O’ corner, 1993).
The split half method is another method denotes the homogeneity indices of the items in the scales. It pertains to the relationship between the responses of the items and the total scale score (Oncu, 1994). O’conner, 1993 claimed that an increase in homogeneity in the set of items increases the reliability estimates. The idea that the internal consistency methods depend upon is that every measurement tool is constructed to realize an objective (Karasar, 2000). The methods of internal consistency are widely and frequently preferred (Oncu, 1994). Although there are many estimates of reliability that can be used, but internal consistency is one of the most frequently generated estimates or scales or instruments composed of number of items or variables that will be formed into a linear composite.
Cronbach Alpha
Of the measures of internal consistency, the most frequently preferred is cronbach alpha (cronbach, 1951), can be used for three, four, or five point likert scale items. But Oncu (1994) claimed that, it is not limited to the true-false or correct-incorrect format of scale. It is equivalent of the average of all split-half correlations that could be generated for the scale 2N items long, where N equals the number of indicants (Novick and Lewis, 1967) and where each item is scored on more than two points (Nunnally, 1978). Alpha is easily interpreted and ranges from 0 to 1 gives low to very high internal consistency. At the lower extreme, the inter-item correlations are zero; and at the upper extreme, there is a perfect correlation among the items. Many researchers consider an alpha coefficient at least 0.070 or more to be adequate for the scale (Nunnally, 1978).
When interpreting alpha as a measure of internal consistency, every researcher should needed to understand several key factors. Initial and most basic thing is that there is an assumption that internal consistency is expected. Secondly, although the estimate is labeled internal consistency, that is not completely accurate. The nature of increase of alpha coefficient is depends on change in number of items while holding the average correlation among the items constant (Nannually, 1978). Finally, alpha is a best estimate of reliability given the assumption that the items are parallel. Otherwise it is a lower bound estimate of reliability. Parallel items are items that have identical true scores.
Although cronbach alpha is easily obtained from several packaged statistical programs, it is worthwhile understanding the formula. It becomes clear that alpha uses the average inter item correlation as part of the formula. When the items have equal variances, averaging presents no problem. The formula according to Carmines and Zeller (1979). If the items are standardized, the alpha is estimated by using the items “correlation mean or variance-co variances” mean is (Carmines and Zeller 1983; Ozdamar 1999a). Alpha due to the correlation mean is and alpha coefficient due to the variance-co variance mean is . If the correlation between the items is negative alpha coefficient will be negative because this will spoil the scales additive property, it also causes a spoil in the reliability model and scale is no more additive (Ozdamar, 1999a).
Theta coefficient
Because alpha coefficient may not be a optimal estimator of reliability, when the items are heterogeneous or when the number of items is small. The other two measures of scale should be interested (i.e. theta and omega). The theta is based on a principle components model analysis, in which components are in descending order due to the variances of each of constructions (Carmines and Zeller, 1982). Theta coefficient depends on the property like; the first component is the linear component with the maximum variance. The second component is the linear component with the second maximum variance. Components can be explained by percentage values to explain the variance of the original data set in order. The theta is computationally much simple, takes into account the eigen value that maximum explains the event than that of alpha coefficient.
Omega coefficient
Heise and Bohrnstedt (1970) introduced the third estimate of internal consistency i.e. omega (W). It is based on the factor analysis model. Factor analysis is a computationally complex approach to examining the inter relationships among the items and substes of items. In this type of modeling, in calculating the coefficient, before factoring ‘1’ values on diagonal in the correlation matrix are replaced with the communality values.
Methodology
To know the efficiency of three reliability coefficients of internal consistency, the data set has been used from dental environment stress scale with five points developed by Garbee (1980) to measure stress of dental professionals including graduates, interns and postgraduate from SDM College of Dental Sciences and Hospital, Dharwad, Karnataka, India. The example that follows is based on analysis of a 33 item scale administered to 316 dental health professionals, on convenient basis. The response rate was 98.74% (n=312). After receiving the scale from each professional, the response of each item is scored and entered in MS office-2003, then converted into SPSS 16.00 format. The three reliability coefficients i.e. alpha coefficient, theta coefficient and omega coefficient were calculated under different sample sizes starting from n=33 (because of total number of items are 33) and so on. In order to obtain theta and omega coefficients, the eigen values were calculated from a principle component analysis and communalities were estimated from a factor analysis. The mean item correlation and Split-half reliability coefficient were calculated to know the reliability of dental environment stress scale under different sample size. Further, the three reliability coefficients for internal consistency were compared under different sample size.

References


1. Raykov, T. (1997). Scale reliability, Cronbach's coefficient alpha, and violations of essential tau-equivalence with fixed congeneric components. Multivariate Behavioral Research, 32, 329-353.
2. Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78 (1), 98-104.
3. Ercan Ilker et al. (2007). Examining Cronbach Alpha, Theta, Omega, Reliability Coefficients According to the Sample Size, Vol. 6, No. 1, 2-354
4. Gay, L. R. (1985). Educational evaluation and measurement. London: A Bell & Howell.
5. Carmines, E. G., & Zeller, R. A. (1982).  Reliability and validity assessment. Beverly Hills: Sage Publications.
6. Arkin, H., & Colton, P. R. (1970). Statistical methods. New York: Barnes & Noble Books.
7. O’Connor, R. (1993). Issues in the measurement of health-related quality of life, working paper 30. Melbourne: NHMRC National Centre for Health Program Evaluation ISBN: 1-875677-26-7.
8. Carey, L. M. (1988). Measuring and evaluating school learning. London: Allyn and Bacon.
9. Oncu, H. (1994). Egitimde olcme vedegerlendirme (Measurement and evaluation in education). Ankara: Matser Basim.
10. Karasar, N. (2000). Bilimsel arastirma yontemleri (Research methods in science). Ankara: Nobel Yayin Dagitim.
11. Cronbach, (1951). Coefficient alpha and the Internal Structure of tests. Psychometrika--VOL. 16, NO. 3, pp.297-334.
12. Novick, M. R., & Lewis, M. R. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32, 1-13.
13. Nunnally, J. C.  (1978). Psychometric theory (2nd ed.).  New York:  McGraw-Hill
14. Carmines, Edward G., and Richard A. Zeller (1979). Reliability and Validity Assessment. Newbury Park, CA: Sage Publications
15. Carmines, E. G., & Zeller, R. A. (1982). Reliability and validity assessment. Beverly Hills: Sage Publications.
16. Thorndike, R. M., Cunningham, G. K., Thorndike, R. L., & Hagen, E. P., (1991). Measurement and evaluation in psychology and education (5th ed.). New York: Macmillian Publishing Co.
17. Rosenthal, and Rubin (1996). Comparing Correlated but Non-overlapping Correlation Coefficients, , Psychological Methods, 1, 178-183
18. Mendoza, J. L., Stafford, K. L., & Stauffer, J.M. (2000). Large-sample confidence   intervals for validity and reliability coefficients. Psychological Methods, 5, 356–369.
19. Van Zyl, J.M., Neudecker, H. and Nel, D.G. (2000). On the distribution of the maximum likelihood estimator of Cronbach’s alpha, Psychometrika 65(3), 271-280.
20. Ozdamar, K. (1999a). Paket programlarla istatistiksel veri analizi-1 (Statistical data analysis by custom softwares-1). Eskisehir: Kaan Kitabevi.
21. Heise, D. R., & Bohrnstedt, G. W. (1970). Validity, invalidity, and reliability. Sociological Methodology, 2, 104–129.

Source(s) of Funding


Not assisted any financial support

Competing Interests


none

Disclaimer


This article has been downloaded from WebmedCentral. With our unique author driven post publication peer review, contents posted on this web portal do not undergo any prepublication peer or editorial review. It is completely the responsibility of the authors to ensure not only scientific and ethical standards of the manuscript but also its grammatical accuracy. Authors must ensure that they obtain all the necessary permissions before submitting any information that requires obtaining a consent or approval from a third party. Authors should also ensure not to submit any information which they do not have the copyright of or of which they have transferred the copyrights to a third party.
Contents on WebmedCentral are purely for biomedical researchers and scientists. They are not meant to cater to the needs of an individual patient. The web portal or any content(s) therein is neither designed to support, nor replace, the relationship that exists between a patient/site visitor and his/her physician. Your use of the WebmedCentral site and its contents is entirely at your own risk. We do not take any responsibility for any harm that you may suffer or inflict on a third person by following the contents of this website.

Reviews
4 reviews posted so far

Good content
Posted by Dr. Shreyas Tikare on 13 Mar 2011 01:46:08 PM GMT

Title is appropriate
Posted by Dr. Rajesh Rao on 11 Mar 2011 04:10:01 AM GMT

Structure
Posted by Dr. Thomas F Heston on 26 Feb 2011 04:23:04 PM GMT

Thank you very much for your advise for some corrections in our article "Effect Of Varying Sample Size In Estimation Of Reliability Coefficients Of Internal Consistency". WebmedCentral BIOSTATISTICS 2... View more
Responded by Dr. Shivalingappa B Javali on 28 Feb 2011 05:08:37 AM GMT

Thank you very much for your advise for some corrections in our article "Effect Of Varying Sample Size In Estimation Of Reliability Coefficients Of Internal Consistency". WebmedCentral BIOSTATISTICS 2... View more
Responded by Dr. Shivalingappa B Javali on 28 Feb 2011 05:08:27 AM GMT

Comments
0 comments posted so far

Please use this functionality to flag objectionable, inappropriate, inaccurate, and offensive content to WebmedCentral Team and the authors.

 

Author Comments
0 comments posted so far

 

What is article Popularity?

Article popularity is calculated by considering the scores: age of the article
Popularity = (P - 1) / (T + 2)^1.5
Where
P : points is the sum of individual scores, which includes article Views, Downloads, Reviews, Comments and their weightage

Scores   Weightage
Views Points X 1
Download Points X 2
Comment Points X 5
Review Points X 10
Points= sum(Views Points + Download Points + Comment Points + Review Points)
T : time since submission in hours.
P is subtracted by 1 to negate submitter's vote.
Age factor is (time since submission in hours plus two) to the power of 1.5.factor.

How Article Quality Works?

For each article Authors/Readers, Reviewers and WMC Editors can review/rate the articles. These ratings are used to determine Feedback Scores.

In most cases, article receive ratings in the range of 0 to 10. We calculate average of all the ratings and consider it as article quality.

Quality=Average(Authors/Readers Ratings + Reviewers Ratings + WMC Editor Ratings)