ANNOUNCEMENT BOOK | Pokémon Watty Awards

12/22/17- An Explanation of the Judging Process

With the recent changes to the judging process, we'd like to explain how it now works so you know exactly what happens when we determine the results.

1. Inspection

This part is performed by an admin. Either Scarlet or Trevor will look at your entry and read the first chapter or two. They will look to make sure you understand basic grammar, formatting, spelling, and punctuation. If your book does not meet the basic quality requirements for one or more of these categories, you will not pass inspection. It is your job to go through and fix whatever your story is lacking before asking us to inspect your story again.

2. Judging

Each judge in a given category will thoroughly read and analyze your book, looking to see how strong your writing ability is and how good the book is overall. Each judge will rank every story based on which ones they liked the most. Once every book is ranked, they send their rankings to an admin.

3. Comparison

Once every judge for a category gives an admin their results, the admins will compare every set of rankings. No two lists will be exactly the same, but the admins look to see if one judge has wildly different rankings than the other judges. If something seems to be off in someone's rankings (for example, if a book that barely passed inspection is ranked in first place when a book by an experienced author is last) the admin will contact the leader for that category. The leader will talk to the judge, asking them why they ranked the stories how they did. If they have a reasonable response, the admin will keep their results. If not, the admin will throw their rankings away, and that judge will be removed from the PWA team.

4. Ranking

Once the admins get all the rankings, they give points based on how books placed in each judge's list. For example, if one judge ranked a story first place (one point), and another ranked it in third place (three points), the book would have four points overall. The admins do this for every entry, then rank the stories from lowest score to highest.

The scores are similar to scores in golf: the fewer points your story gets, the better your story did. So, the story with the fewest points wins the competition in that category.

5. Special Awards

We want the Special Awards to remain community driven. Therefore, we've been doing everything we can to eliminate popularity bias in the awards. We've listed these before, but we'd like to emphasize the steps we've taken to eliminate a popularity-based win.

- One story can only be nominated for a maximum of three Special Awards. Even if they win everything they're nominated for, they've only won 15% of the total awards.

- We will use our discretion when nominating two books by the same author. We don't want one author to take the majority of the awards, but we want to nominate a book for an award if it truly deserves it. We won't let one author be nominated for every award, but we don't want to discredit them just because they've achieved a lot in their stories.

- We will require both an email and username when someone fills out the Special Awards survey. This will prevent both voting multiple times and voting for yourself.