January 9, 2007 at 4:02 am #251215
As the WWRD nominee for Minister of Film Ratings, I would like to address my opinions as to how I see the current situation on film ratings, and how I would like to modify the current system.
The first topic I will address is the rating format that I wish to see take form, which is a bit different from Ladon’s idea. It seems that the main point of Ladon’s system is a “rating rubric.” In his proposal, Ladon shows how ratings could be much easier to give and understand if they were to follow strict guidelines as to what certain ratings meant. This could definitely sort things out in terms of different opinions on what an actual film rating means. The problem with this, however, is that the definition of a rating is someone’s opinion on how good or bad a film is in different respects. With one set rubric that everyone has to follow, wide interpretations of each definition of a rating are possible. Potentially, if the rubric were to go into effect, we could be seeing dozens of films with very similar ratings, which defeats the purpose of seeing ratings on a film in the first place.
The purpose of ratings should be to be able to get a general yet precise idea about a film before one watches it. If a film has a low rating, then one could expect it to be badly animated, scripted, etc. If Ladon’s rubric were used, too wide a range of ideas for the same rating might be used. Nos notes that “it could become too easy to choose 7’s for most films” with Ladon’s system. I understand that Ladon’s intention is to eliminate confusion, but it seems that it might potentially cause more. Instead, there should be a compromise between opinion and set guidelines. This way, an ultimate happy medium can be achieved, so that an overall consensus can be met with both raters and general members of the Brickfilms community. How can this be achieved? The format is really quite simple. The actual ratings system must be precise, so that the true quality of a film can quickly be recognized by looking at a rating. With the current ratings system, ratings go down to the decimal, which would seem like an “exact”-enough rating to suit even the pickiest of filmgoers. However, this wide range of ratings is due to a difference in interpretation. With my proposed system, raters could rate a film with their own opinions in mind, while still complying to an exact system.
Here’s how it works. First, a rater (which under my system could be anyone who has qualified under the Rater Qualification Standards [see Section II]) goes to the film page. Before being able to rate a film, the rater must set his/her own rating preferences, which could be accessed after passing the Rater Qualification test (see Section II). Here, a scale from 1 to 10 is presented with text boxes next to them. The person, at his own discretion, may enter what he thinks each rating deserves, depending on which category (animation, story, etc.). He may choose to fill in as little as two (1 and 10) or as many as 10 (1 through 10). This allows ample amounts of flexibility. The same rating categories would be used that are in effect today. Next, the person would submit his rating preferences to a special Rating Preferences Approval Panel, which would be just as simple as submitting a brickfilm to the directory. This panel would consist of a few highly regarded individuals of the Brickfilms community and myself, and whose primary purpose would be to make sure that each person’s own rating preferences comply with what the overall rating standard should be. The panel may be allowed to give tips on how to revise a person’s rating preferences so that it would better comply with the overall rating status that will eventually be achieved. This overall rating standard would have to be decided upon amongst panel members, but it seems this could be easily accomplished, even perhaps with a poll voted upon by approved raters. The Rating Preferences Approval Panel would be similar in form to the current Film Review Panel, basically approving people’s requests. The two panels may even be able to work together in the future, especially with the new position of Minister of Film Reviews. After a member’s preferences have been approved, the member can rate any film he wants with his own system, and it will count just like the real thing (it will be the real thing, actually). With this system, people can rate films as they like – with their own personal rating systems – while agreeing with the general community at the same time! It’s a win-win situation. This scenario – a more personal preference/you’re-in-control approach – will in time encourage the rating of films by a larger majority of Brickfilms members, which is the ultimate goal for brickfilming and brickfilmers alike.
The appearance of the ratings on a film page would be similar to the way it is now. The different categories would be listed, but the ratings would not be in decimal form, since this is indeed rather confusing. Instead, the ratings would be rounded to the nearest whole number. The “rated x times” part would still be in effect, although generally the numbers would be much higher. Maybe even a link to “Who has rated this film” would be available, listing the members (but not their individual ratings) that have rated that film. I don’t like the idea of an “Admin rating” and “User rating” as Ladon has suggested, either. With the Rater Qualification Standards, one rating will be adequate.
II. Who Can Rate
Under my system, anyone who is approved under the Rater Qualification Standards can rate a film. This will not limit the rating of films to paying patrons. These qualification standards will be fairly lenient, yet weeding out those who want to rig the polls by purposely giving people bad ratings. The general outline is as follows (which may be revised as needed):
1. The rater must pass the Rater Qualification test (which, if completed successfully, will automatically redirect the member to the aforementioned rating preferences setup page). The test will consist of a link to a film, and the candidate rater must rate the film appropriately according to the preset overall rating standards (see above). This should prevent most of the n00bish people from gaining access to the ratings.
2. If the candidate rater is an admin or patron, the Rater Qualification test will not be required.
3. Any rater who is discovered to have been giving purposely bad ratings (and not just being critical) will be banned from rating indefinitely. This applies to those groups mentioned in both rules 1 and 2.
So, the outline is fairly simple. It will be stressed that the matter be taken seriously as to keep as much honesty and reliable ratings in the directory. I and whomever I choose to be on the Preference Approval Panel will have the authority to alert people of activity that may be causing them a ban from the Rater Qualification Standards usergroup, which would be created with the new overhaul of the ratings system.
III. Old Films with Unfair Ratings
Another issue that had been brought to my attention is the unfair rating of older films. When standards were lower for the acceptance of films into the directory, films were often rated much higher than they would be now. To remedy this problem, I suggest a complete removal of all “old” ratings and the emphasis on rerating old films with current standards. I like to call this Old Film Rerating Project, or OFReP. This is another new addition to the ratings system that I would like to put in place, but I’m not going to emphasize it as much as everything else, for the sheer reason that sometimes there are too many things going on at once. Nevertheless, with OFReP, approved raters (see above) could rate not just new films, but long-forgotten oldies that need attention. Still using his/her own personal rating system, the OFReP could quickly put older films in the right place in the rating pool. Not to say that all older films are bad, because some of them are really good (e.g. “The Barber of Seville” by Yellowhead Studios). Nevertheless, most of the films that fall under the category of “released before RevMen came into power” will probably be the ones that will be focused on with OFReP. This way, older good films will get more exposure and worse old films will put in their proper places. It may seem a bit drastic, but the OFReP will probably pay off in more ways than one might think in the future.
- - Anyone who passes the Rater Qualification Test will be able to rate a film (Brickfilms membership will be required to take the test)
– Film ratings will be based on a personal preference scale, but will still conform to a general community consensus
– A small Ratings Preference Approval Panel will be formed and will work alongside the Film Review Panel
– An “overall rating standard” will be drafted and possibly approved by vote from qualified raters
– The Old Film Rerating Program (OFReP) will, if enacted, help to rerate older films that have irrelevant ratings
So that sums up what I am planning for. Please support WWRD and Schlockading’s terminus ad quem as we revamp and improve not only film ratings, but the entire Brickfilms community. Remember, every vote counts!January 9, 2007 at 4:17 am #251217
This is exactly how an election should be. For the record, I don’t entirely agree with either of the new proposed systems, but the amount of thought that has been put into them and the amount of effort that has been put forward to explain them brings a tear to my little eye.
Thank you Schlockading. Thank you Ladon. I think this is a win-win situation for the voters of brickfilms.January 9, 2007 at 4:23 am #251219
How does this system benefit the director? And how long do you anticipate there will be films in the directory without ratings?
Also, by getting rid of the ‘decimal form’, what do you propose would happen if one person rates a 5 and another rates a 6? Without a decimal to even that out, who would decide who is ‘right’? It’s hardly precise, as you indicated in your post.January 9, 2007 at 4:43 am #251224
It benefits the director in several ways. It is an easy way for the director tell what the general community thinks of his film, and not just the opinions of a few select voters. Additionally, it’s a lot easier to see a number rating than the wide and varied opinions in word form in the P&R forums. Nevertheless, ratings systems aren’t necessarily made solely to benefit the director, so I don’t regard it as the highest priority.
As for the no ratings thing, I’m not completely sure about that part of the campaign yet. But if it were to go into effect, I’m sure that there could be a way to keep the old rating until a new one is posted. The point of the OFReP is to promote the ratings of old films. With as many brickfilmers who will probably want to participate, I’m sure many of the old films would be rated immediately if they were out in the open, so to speak.
On the decimals, there would probably be enough ratings to tip it over the end to one number or the other instead of it being exactly in the middle. I don’t think the case you’re referring to would occur very often. Plus, you yourself said that you didn’t like the decimal form in the ratings. How is your view any different?January 9, 2007 at 5:10 am #251229
“Schlockading” wrote: But if it were to go into effect, I’m sure that there could be a way to keep the old rating until a new one is posted.
This does seem like the perfect remedy to bringing old films to a new rating standard, but let’s keep in mind the work that would be required. I really don’t know how difficult this would be (I imagine it might just be a simple if statement or something… I know nothing of coding), but whoever wins this position should not overload Z’s capacity. Unless you have someone else in mind who can come in to help with the programming.
I do disagree with your opinion that many old films would immediately get new ratings; we now have over 1000 films in the directory, and a lot of them are bad. Sure, your Gauntlets, your Out of Timers, your Chase of Horrors will get rerated quite quickly, but I can’t imagine several members going through and downloading the hundreds of low quality films we have accepted over the years.
I also strongly disagree with abolishing decimals. I can see requiring raters to use integers (I’ve never seen the need to rate something a 7.2 or 4.6), but I think the final, calculated rating should have a decimal.
CometgreenJanuary 9, 2007 at 5:16 am #251231
I don’t recall saying I wanted to get rid of the decimal rating. I fully intend to keep it as it makes averaging easier.January 9, 2007 at 5:42 am #251235
I find this approach somewhat troubling. In the first paragraph of your outline, you say this:
“Schlodkading” wrote: The problem with this, however, is that the definition of a rating is someone’s opinion on how good or bad a film is in different respects. With one set rubric that everyone has to follow, wide interpretations of each definition of a rating are possible. Potentially, if the rubric were to go into effect, we could be seeing dozens of films with very similar ratings, which defeats the purpose of seeing ratings on a film in the first place.
But then you explain that,
“Schlodkading” wrote: the [rater] would submit his rating preferences to a special Rating Preferences Approval Panel, which would be just as simple as submitting a brickfilm to the directory. This panel would consist of a few highly regarded individuals of the Brickfilms community and myself, and whose primary purpose would be to make sure that each person’s own rating preferences comply with what the overall rating standard should be.
I don’t see how you can encourage the individuality of the ratings and say that they have to conform (somewhat) to a set standard. I think that’s going the main problem with this position, and will largely determine my vote. For the record, I want each rater to have complete power to rate however he/she wants, regardless of how the rest of the community views the film.
I would suggest that you scrap the panel who decides how the rating preferences are set. Then, show an overall score (in decimals), but list how each rater scored the film. That way, the person who esteems story over animation can have his/her voice heard, and the same goes for the one who only likes good special effects, cinematography, etc.January 9, 2007 at 7:14 am #251241
An Old OreMember
I feel that it would be counter-productive to go back and re-rate all the older films.
I’d fully support upgrading or renovating the current system and having it apply to new films, but to ask members to go back and re-rate several hundred films would be an exercise in futility.
Unless you appointed a “task force” who had a quota of films to rate each week. Then it would get done, but who would want to be in such a group?
You could go the other way, let everyone rate, except that you change it so that there is an accountability. Just like anyone can see who has made changes in a Wiki, set the ratings so that everyone can see who voted which way. If people vote “fairly” (read: accurately) then there wouldn’t be a problem.January 9, 2007 at 7:27 am #251243
Eliminating decimals is, in my opinion, a very bad idea. We have A LOT of films in the directory… to have only 10 levels of rating resolution eliminates a lot of information. I agree with Cometgreen that the rating process could be reduced to integers, but the ratings in the directory should remain decimals. With so many films, I think keeping our current level of rating resolution is a good idea.
You could go the other way, let everyone rate, except that you change it so that there is an accountability. Just like anyone can see who has made changes in a Wiki, set the ratings so that everyone can see who voted which way. If people vote “fairly” (read: accurately) then there wouldn’t be a problem.
At first this does seem like a good idea, and it’s actually a thought I’ve had in the past. While it would eliminate the concept of “malicious” votes, it would introduce the concept of peer pressure in voting, as people would want to make sure they voted in a way deemed acceptable by their peers and not how they truly felt about the film. I think the former is the lesser of two evils, as it would affect far fewer films.
Also, I tend to frown when any of the candidates proposes giving Z a higher workload. He does an immense amount of work and receives recognition for almost none of it. Obviously any change suggested by candidates is going to require some changes to the site, it’s unavoidable and it’s the way of progress. It’s the more efficient ideas, however, that are more attractive to me.January 9, 2007 at 8:13 am #251253
I feel that a Rater Qualification Test would be overly cumbersome (and if I understand the proposal correctly, it goes against my sensibility that there be no preconceived notion of what a “proper” rating might be).
I do think that Schlockading is on the right track in that the most important thing is to have more rating taking place. The only two things that would effect this result are 1) increasing the number of raters and/or 2) having raters rate more films. I would like to, in the spirit of brainstorming, suggest a modification of Schlockading’s proposal which would attempt to promote both those activities.
Rather than have a test administered to determine a candidate’s fitness to rate, I propose that non-patron raters be nominated to become a rater by an active rater (patrons would be able to rate just as in the current situation). The thing that determines when a rater gets to nominate a non-patron to become a rater would be the number of films he (or she) has rated.
For example, as a patron I am able to rate films. Once I have rated a certain number of films (a hundred, maybe), I earn the right to nominate a non-patron to be added to the group of raters. After another hundred ratings, I could nominate another. The Minister of Film Rating would be afforded the opportunity to reject a nomination but hopefully this would only be necessary in extreme cases. The same incentive to rate films would be applied to the new rater; i.e. after he (or she) rated a certain number of films, he (or she) would be permitted to nominate someone.
I haven’t really thought through all of the ramifications of this system but it would, hopefully, not be too much of a burden for Z to modify the code to handle a “rating privileged group” where currently the site probably just uses patronage as the criteria. There is some potential for abuse, but that would require a person expend quite a bit of effort (and the Minister of Film Rating could veto the nomination or revoke a non-patron’s rating privilege if foul play was suspected).
- - Anyone who passes the Rater Qualification Test will be able to rate a film (Brickfilms membership will be required to take the test)