I’ll be honest, I’m getting a little tired of just delivering descriptive statistics about my Metadata MOOC. I’d like to delve a little deeper, do some hypothesis testing (for example, students in the developing world are more likely to complete the course than students in the first world), or do some comparison across courses. But for the moment, descriptive stats are what I have for you. My old stats professor used to say that the stages of science were: description, prediction, explanation, and control. Research on online education is definitely in the realm of prediction/explanation, maybe getting to control. Research on MOOCs, though, is new enough that we’re mostly still in description, maybe just moving into prediction, as we recognize that 15 years of prior work on online education also applies to us. So bear with me, and my descriptive statistics, for just a little longer.

What I want to write about here is completion rate in the MOOC. This is one of the hot hot hot topics of discussion about MOOCs lately. The figure that often gets bandied about for “dropout rate” is 90% (or more) of students who register for a MOOC. But as I’ve written, Total Registered Students is not a very meaningful statistic, since there’s no penalty for an individual to register for a MOOC and then never participate. Total Registered Students is a measure of interest in the course topic, not much more.

A far more meaningful number is Total Active Students, “the number of (unique) students that have logged in at least once to the session site.” Better still would be the total number of students who watched at least one video, or the total number of students who completed at least one assignment. In other words, the number of individuals who could actually be considered to be students. I’d argue that individuals who register for my MOOC and then never log in, are the equivalent of students who register for my course on campus, but then drop before the semester begins. I’d argue that individuals who log into the course site but don’t complete any assignments aren’t students, they’re auditing.

So with that as preface… How many students completed the course? Answer: 1418. A total of 1418 students earned a Statement of Accomplishment or Certificate of Achievement.

Now, the first thing I want to say about that number is this… Before my MOOC launched, I did a quick back-of-the-envelope calculation of how many students I’ve ever had in the classroom, since I started teaching in grad school. And the number I came up with was, approximately 1400. The number of students who completed my MOOC is approximately equal to the number of students I’ve had in the classroom in my entire career. The number of students who were active in the MOOC (Total Active Students) turned out to be approximately an order of magnitude more than the number of students I’ve had in the classroom in my entire career. Contemplate that.

Anyway… What percentage of students in the MOOC completed the course? Well, let me start with Total Registered Students. Yes, I know, I just said that’s not the right number to use. But it gets used a lot, so I’ll use it here, just so I have some comparable data. The problem is, the value of Total Registered Students changed over the duration of the course. But let’s use the largest value, from the start of the course — which will, of course, make the percentage completed look worst.

Total Registered Students at the end of Week 1 = 27623
Students who completed the course, as a percentage of Total Registered Students = 5%

That’s a 95% dropout rate. But as I said, Total Registered Students is not the right denominator to use. Instead, let’s use Total Active Students. Again, let’s use the largest value, from week 8.

Total Active Students = 14130
Students who completed the course, as a percentage of Total Active Students = 10%

And hey, what do you know! A 90% dropout rate!

But again, Total Active Students is probably also not the right denominator to use, since there were clearly individuals who logged into the course site, but then didn’t actually do anything. (Why, is what I want to know.) So instead, let’s use the total number of unique students who’ve watched at least one video since the start of the class, either streaming or downloaded. Again, the largest value, from week 8.

Total video viewers = 9321
Students who completed the course, as a percentage of total video viewers = 15%

An 85% dropout rate.

Now, let’s use the total number of students who completed at least one assignment. At this point, I must pause and thank Marla Sullivan, a SILS graduate, who is working with the MOOC production team at Carolina, to support me and all of the MOOCs offered by UNC. She’s awesome, and I don’t know how we would have been able to do this without her. Also, she generated this stat for me. This is the total number of students who completed homework 1, as of 4 November, with a score > 0.

Total students who completed the unit 1 homework = 2938
Students who completed the course, as a percentage of homework-doers = 48%

A 52% dropout rate.

Naturally, there are many other values that you could choose to be the denominator for calculating completion rate. But I think these are the most meaningful ones. (Well, actually, as I said, I think Total Registered Students is not really very meaningful, but it’s useful as a basis for comparison with the ongoing discussions in the higher ed press.) If you, dear reader, have any suggestions for other ways of calculating completion rate of a MOOC, let’s hear it.