Thursday, September 22, 2011

CRITICS ON CRONBACH -Alpha

CRITICS ON CRONBACH ALPHA
: The concept of reliability originates from Spearman’s early work with factor analysis and measurement errors over hundred years ago. However, the importance of the reliability of measurement scales has been partially obscured because of poor estimators, such as Cronbach’s alpha is widely applied despite the poor estimate of the measurement error variance used. Subsequently Cronbach’s alpha underestimates the reliability and may even give absurd, negative estimates, it remains to be the most widely applied estimator of reliability for reason of easiness; a quick method for practical needs—long before the era of computers. [Vehkalahti,Puntanen &Tarkkonen, 2006]The availability of hi-speed computing technology, easily available in our lap top will change such necessity in the name of accuracy and precision. Cronbach Alpha always exceeds the maximum reliability possible for the measures underlying for a given dataset. This misleads the test-user into believing a test has better measurement characteristics than it actually has. It overstates the reliability of the test-independent, generalizable measures the test is intended to imply. For inference beyond the test, Rasch reliability is more conservative and less misleading. [Linacre, 1997]
Nunally (1978) definition of Cronbach-alpha has been grossly misused just like Krejcie & Morgan (1970) for 'random' sampling size; but many keep on citing it NOT knowing the precision is NOT in place as compared to other more current methods as discussed above. Cohen (1992) Statistical.Power Analysis shows an alternative method of sampling size for smaller size depending on d stats. test to be employed; correlation or multiple regression. (see:http://www.ipbl.edu.my/bm/penyelidikan/jurnalpapers/jurnal2006/chua06.pdf) .
Linacre (1994) employs JMLE to determine d sample size and meet sufficient statistics principles. see:http://www.rasch.org/rmt/rmt74m.htmThe fundamentals has to be observed n CANNOT simply be breached for convenience of a given case. Thus, Rasch reliability, MNSQ, z-std, PMC, eigenvalue ratio, item indepencence etc. are stats. values dat must be in place before an item is considered a fit.

RASCH : Smaller sample size with lesser error..

In determining a practical sample size, one have to understand the method of analysis to be employed. In general practice, sample size by random sampling is done using the table developed by Krejcie &Morgan, (1970) Determining sample size for research activities. J.Educational and Psychological Measurement, 30, 607-610. 


According to Salant and Dillman (1994), the size of the sample is determined by four factors: (1) how much sampling error can be tolerated; (2)population size; (3) how varied the population is with respect to the characteristics of interest; and (4) the smallest subgroup within the sample for which estimates are needed. One of the common reference is Cohen Statistical Power Analysis (1992) being one of the most popular approaches in the behavioural sciences in calculating the required sampling size. In Krejcie and Morgan (1970), the estimated random sampling size for a population of 500 is 217. However, the estimated sampling size calculated using Cohen (1992) differs according to the type of statistical tests employed by the researcher. The sample size that is required for a correlational study is 85 while a multiple regression analysis requires 116.
 http://www.ipbl.edu.my/bm/penyelidikan/jurnalpapers/jurnal2006/chua06.pdf


Rasch statistical analysis offers a better mathematics with even smaller sample size but of sufficient stability. see http://www.rasch.org/rmt/rmt74m.htm, "Sample Size and Item Calibration Stability. Linacre JM. Rasch Measurement Transactions 1994 7:4 p.328"Rasch analysis can handsomely handle a sample size of 25-30 to generate a sound 95%CL statistics and 50-60 for a 99% CL.

Another achievement...

Rasch application in Malaysian education scenario has made another achievement at the international arena. Pn.Nazlinda Abdullah from the Faculty of Education, UiTM was awarded the Best Student Paper in recent PROMS 2011 in Singapore. see: http://accounting-research-institute.blogspot.com/2011/08/congratulations-nazlinda.html

Monday, August 16, 2010

What Is Good About Rasch Measurement?

To answer this question requires a specification of what is meant by measurement. Two main approaches to defining measurement are the traditional approach and the representational approach. The traditional approach has been widely accepted in the physical sciences since its development by Holder (1901, as cited in Michell, 1997), who synthesized the approaches of Euclid, Newton, and Dedekind. One feature of the traditional approach is that it entertains an empiricist account of number. Measurement, then, becomes the exercise of establishing a correspondence between quantitative variables in the world and numerical instruments (Mill, 1843/1973). Mill's empiricist conception of number was criticized and made to seem untenable by Frege (1884/1984), and the traditional approach to measurement was criticized by Russell (1903), who developed a representational theory of measurement. The representational theory was further advanced by Krantz, Luce, Suppes, and Tversky (1971), Suppes, Krantz, Luce, and Tversky (1989), and Luce, Krantz, Suppes, and Tversky (1990), who are its most sophisticated contemporary proponents. Michell (1994, 1997), on the other hand, has emerged as the most sophisticated contemporary proponent of the traditional theory of measurement.

On both the traditional theory and the representational theory, Rasch measurement is good because it is an example of additive conjoint measurement. Rasch measurement satisfies two conditions that are necessary in order for an attribute to be quantitative. First, the attribute must possess additivity. Second, the attribute must possess ordinality. The Rasch model possesses additivity because the difference between the manifest level and the latent level involves the additive measurement of two different latent variables -- one for the person, one for the item. The Rasch model possesses ordinality because person and item variables can be explicity compared at the latent level as being higher or lower than one another.

Rasch measurement is good partly because it stands in contrast to a ridiculous version of the representational theory that has gained ascendance within psychology: namely, operationism -- that is, the idea that a variable is completely defined by the operations or measurements used to recognize it. There may be some ontological differences between the traditional theory and the Krantz et al. representational theory regarding the state of the world, whether the variables to be measured are quantitative or qualitative, but neither of these theories is completely subjective and idealistic (in the Berkeleyan sense) in the way that operationism is. Operationism permits quantification of anything whatsoever, albeit in a wholly arbitrary way. Operationism thus exemplifies a strong Pythagorean tendency within psychology, supposing as it does that numbers can be applied to anything. The operations used to generate the numbers, however, may represent nothing other than themselves. Operationism justifies applying a rule--any rule--to empirical reality. Applying a different rule may result in a different result, but both rules are right by fiat, because they define what they purport to represent.

It may be a mistake to claim that Rasch measurement is an idealization. According to the traditional theory, an idealization is not measurement. Within any given application, however, measurement may be impossible. Indeed, Kant (1786/1970) and Searle (1994) seem to think that psychological variables such as consciousness are inherently non-quantitative. For the representational theory, applying numbers to a qualitative reality in a systematic and rigorous way is the model for measurement. Thus, the representational theory entails no quivering reflections on whether psychology can ever be a quantitative science. The traditional theory does entail such reflections, however, because, within this theory, whether any given attribute is quantitative is an empirical question to which the answer may be "no".

If psychological variables turn out to be non-quantitative, this does not entail that psychology cannot be a science. First, psychological variables will continue to have predictive power and thus practical utility. The correlation between any number of psychological traits and criterion variables, for example, ranges from 0.3 to 0.5 (Mischel [1968] has called this the personality coefficient). Explanations for these regularities, however, will have to be acknowledged as being speculative and theoretical, bringing psychology into close alliance with philosophy. The search for quantitative variables, however, may represent the wave of the future for a potentially quantitative scientific psychology.

Sunday, August 15, 2010

How do you establish a Scale Construct in Measurement ?

A scale must have a defined construct. e.g. a thermometer

  1. any measurement must have a unit, e.g 'C, 'F.
  2. a unit must be defined; 0'C is when water turns into ice at mean sea level. This called for a standard. It is the property of water; i.e. the property of the measured object.
  3. a scale of an instrument must be of equal interval.
  4. an instrument must be known for purpose; e.g clinical thermometer.
  5. You must know how to use the instrument i.e. a nurse would shake it before taking your temperature
  6. an instrument must be initiated to zero; i.e. 0'C hence calibrated.
  7. a reliable instrument is not bias; a nurse use the thermometer repeatedly irrespective of man or women of all size.
  8. a scale shall be able to predict an outcome; nobody has ever been to the sun but we can predict the temperature of the sun.
Using the correct instrument for purpose; i.e. mass spectrogrammeter we can predict the temperature of the sun. Similarly, nobody has ever been into some one's mind and measure their intelligent, a person's ability to think but we can infer by measurement just like we measure the sun's surface temperature provided we use the correct instrument and know how to use it correctly.


In education, the instrument is the exam paper, assignment, evaluation by observation of a student's performance for a given task. But how do you calibrate an exam paper ? What is the unit of measurement ?
Rasch Measurement Model offers us an intelligible explanation.

Friday, August 13, 2010

Meaningfulness of Measurement

Often researchers were stunned when asked about the meaning of their data analysis findings; whether they are measuring what is supposedly to be measured.
These issues of meaningfulness and measurement are explored at length by Fisher. The basic point is that the content of tests and surveys ought to be used to illustrate mathematical relationships between abilities and difficulties visually and conceptually without confusing them with the mathematical relationships themselves. We need to look right through the sample of items used to illustrate the construct and keep the pure thought of the construct in mind, in the manner of the numerical and geometrical figures that are understood to paradigmatically define meaningful representation by, again, a wide range of diverse philosophers. Meaningful measurement requires what Rasch called parameter separation, what Ronald Fisher called statistical sufficiency, and what Luce and Tukey called conjoint additivity.
There is a basic principle of meaningfulness accepted by a wide cross-section of different philosophical viewpoints that justifies the use of fundamental measurement models like Rasch's Measurement Model. Mundy's general theory of meaningful representation states that: "The hallmark of a meaningful propositions have truth-value independent of the choice of representation, within certain limits. The formal analysis of this distinction leads, in all three areas viz; measurement theory, geometry, and relativity, to a rather involved technical apparatus focusing upon invariance under changes of scale or changes of coordinate system." The same focus on the independence of figure and meaning, or scale and proportion, emerges in a wide variety of other works on the creation of qualitative mathematical meaning.
Hence, reseachers must understand the needs what is to be measured and knows whether they are measuring what is supposedly to be measured. Rasch offers a new paradigm in making a measurement more meaningful.