Language Educators

  • 1.  Rosetta Stone: The study

    Posted 11-19-2010 01:23

    There has been one evaluation of Rosetta Stone  that I know of. It isn't published, to my knowledge, but you can find it on the internet: "Measuring the effectiveness of Rosetta Stone" by Roumen Vesslinov, Queens College.


    The claim is that 55 hours of Rosetta Stone, doing nothing else, is worth a semester of college Spanish.


    1. The subjects were not typical high school or college students: Average age was 41. 42% had a college degree and another 36% had an advanced degree (MD or PhD). Thus, 78% had college or more (table 2 of the study).
    2. 22% were false beginners. This is confirmed from the pretest scores: The mean pretest score on the WebCAPE test was 52.6, and of the ACTFL oral interview, 89.6 of the subjects were categorized not as zero but "one" (novicelow). ONLY 4% (N = 6) were zeros (no proficiency).


    The mean posttest score was 237.7, a gain of 185.1 points on the WebCAPE test. Vesslinov says that the average gain was 4.3 points per hour. That's based on the 237.7 final mean score. Based on gains, the average gain was 3.37 points per hour.  This suggests that a genuine beginner would score about 185 points at the end of 55 hours.


    According to Vesslinov, a college Spanish program that lasts four semesters requires at least 270 on the WebCAPE to be placed in the second semester (apparently four unit classes). One that lasts six semesters (apparently three unit classes) requires 204 to be placed in the second semester. The 237.7. average final score falls between these two, and considering the pretest score, the average final score of 185 falls short of both.


    On the ACTFL oral interview, about 59% improved one level, on a 1-5 scale. But about one-third did not improve at all.


    Does RS work?  I we accept the 3.37 points per hour figure, it would take 80 hours to reach the second semester threshold.


    No comparison group was used. It would be important to have a comparison group with the same background, especially level of education. And of course it would be important not to include false beginners.


    There was also no a clear description of the RS program. I have heard their ads, however, about how we shouldn't "learn" a language but we should "absorb" it. I think I've seen something like this before ...


    Here is the Rosetta Stone spin on this study:  The article claims that 55 hours of RS is as good as 45 hours of class plus 39 hours of homework (84 hours). But remember that the study used a highly educated group, and had false beginners. My estimate is 80 hours, not 55.





    Stephen Krashen

  • 2.  RE:Rosetta Stone: The study

    Posted 11-22-2010 12:18
    This conversion is extremely, extremely important.  The more data we can have about the levels of proficiency of students who do rosetta stone K-16 (and online courses) versus the proficiency of students with classroom instruction by a certified language teacher, the better.  There are external assessments like STAMP, AAPPL, and others that we can use to measure proficiency in the 3 modes of communication.  Data is what administrators understand most if you are trying to prevent elimination of language programs.  If you teach a language course, I highly encourage you to use external assessments to gather the data.  For those of you have firsthand knowledge of computerized language courses, please push for external testing for those students.  Here is the link to ACTFL's new assessment that will be ready new fall.  This school year we are going to use the OPIc through Language Testing International to get started.

    Wendy Brownell
    Camdenton High School Camdenton MO