selection bias leading to inflated tease ratings
Posted: Sat Jun 07, 2008 11:41 pm
It occurs to me while browsing the archives that many teases have higher ratings than they deserve. While I like the idea of helping the community by rating teases, often when the first few pages of a tease are awful I just move on to the next one without doing so. I'm probably not alone in this. Thus we see many[1] teases with comparatively few votes averaging 3.something with painfully bad prose and repulsive models.
This makes it harder than it should be to find the good teases of yesteryear.
Suggestions:
Allow voting from the front page list, preferably by clicking the stars (like on IMDB).
-or-
Use something a little bit more sophisticated than an average to sort the teases. The number of times a tease has been opened is available, but mostly useless for this purpose[2]. Presumably the number of unique (registered?) viewers can be obtained though. This should make it possible to guesstimate how many people left in disgust after the first page.
What to do with that number is another question. The most straightforward way would be to add (viewers - voters) votes of "1" (or "0" if you are feeling extra mean (no pun intended)) to the average. But that might be a bit unforgiving.
In particular it doesn't address the fact that presumably lots of people who have completed a tease (and liked it) can't be bothered to rate it.
One might try to check for this (only add proxy votes for people who have seen the first page of a tease but not the last one, for example), but this is very inelegant.. Anyway the first approach is likely the best one, my inner computer scientist trying to find an algorithmic solution to the problem should probably be ignored.
[1]
In fact right now there are only _two_ published teases in the entire database with ratings below 2, and a few dozen between 2 and 3. The vast majority of teases that people have voted for[3] are between 3 and 4. This makes sense because people will currently only vote on the teases that are good enough to watch in their entirety.
[2]
Many of the really bad teases have views/votes ratios of about 10, suggesting that lots of people do quit before getting to the end of them. But some of the most highly rated teases have almost twice as bad (high) ratios. This is probably because people watch these many times. Which is why I suggested adjusting the averages using unique viewers instead of total views. Of course total views (and the ability to sort by it) is useful in itself, but that's somewhat beside the point.
[3]
Incidentally letting us rate teases from the list would also make it really easy to solve the problem of rating flash teases.
TLDR: let us vote on teases without having to click through to the end of them plz
This makes it harder than it should be to find the good teases of yesteryear.
Suggestions:
Allow voting from the front page list, preferably by clicking the stars (like on IMDB).
-or-
Use something a little bit more sophisticated than an average to sort the teases. The number of times a tease has been opened is available, but mostly useless for this purpose[2]. Presumably the number of unique (registered?) viewers can be obtained though. This should make it possible to guesstimate how many people left in disgust after the first page.
What to do with that number is another question. The most straightforward way would be to add (viewers - voters) votes of "1" (or "0" if you are feeling extra mean (no pun intended)) to the average. But that might be a bit unforgiving.
In particular it doesn't address the fact that presumably lots of people who have completed a tease (and liked it) can't be bothered to rate it.
One might try to check for this (only add proxy votes for people who have seen the first page of a tease but not the last one, for example), but this is very inelegant.. Anyway the first approach is likely the best one, my inner computer scientist trying to find an algorithmic solution to the problem should probably be ignored.
[1]
In fact right now there are only _two_ published teases in the entire database with ratings below 2, and a few dozen between 2 and 3. The vast majority of teases that people have voted for[3] are between 3 and 4. This makes sense because people will currently only vote on the teases that are good enough to watch in their entirety.
[2]
Many of the really bad teases have views/votes ratios of about 10, suggesting that lots of people do quit before getting to the end of them. But some of the most highly rated teases have almost twice as bad (high) ratios. This is probably because people watch these many times. Which is why I suggested adjusting the averages using unique viewers instead of total views. Of course total views (and the ability to sort by it) is useful in itself, but that's somewhat beside the point.
[3]
Incidentally letting us rate teases from the list would also make it really easy to solve the problem of rating flash teases.
TLDR: let us vote on teases without having to click through to the end of them plz