ANNOUNCEMENT BOOK | Pokémon Watty Awards

12/10/18 - Responding to Your Feedback

Hey guys! It's been a couple of days since the results were released and the live stream was hosted. I just wanted to write up responses to some feedback we received on our PWA 2018 Feedback Survey.

ACCOUNT MANAGEMENT

Please add reading lists of the current entries so it is easier for people to read and find who has entered this year.

Kat/Sheare: I can definitely try that for next year! Just for future reference, we have a database that links to every entry as well. Because I will be the only administrator next year and I haven't found someone with the time to help me out the entire year, these little tasks may be lost. I apologize for anything I may miss, but I hope you can understand that running the PWA by yourself is intense.  

I can get my questions answered easily and efficiently 

Kat/Sheare: I'm glad you think we improved our communication! The PWA has struggled a lot with that in the past, so I'm happy to see that very few people complained about our inactivity or long response time.

JUDGING

I think this will screw the actual results of the categories because you would get more people trying to judge and thus the higher chance of them voting just for the heck of it, or because their friends are competitors

Kat/Sheare: Let me give a little insight into this issue as someone who experienced the judging process for two years now. 

Last year (PWA 2017), we had an issue where one judge seemed to be ranking clearly poorer quality books higher than the ones that were of decent or good quality. We found out that this judge was pressed for time and had hastily thrown together rankings at the last minute without reading most entries. We removed the judge, and that's how we ended up with 2 or 3 judges per category as opposed to the 4 or 5 like we did this year.

Why do I bring this up? The reliability of judges has been the only issue I've ever seen the PWA face. I've never seen a circumstance where the judges have voted for friends. I'm much more worried about having enough judges finish their rankings on time than potential bias because I simply never experienced an issue involving extreme bias. 

This is why I suggested accepting all judges with a grammar test. I'm extremely worried about reliability, and by far, the most stressful week for me this year was the week judges' rankings were due. 

I think the awards had some issues. I believe that it seemed more like a popularity contest rather than a fair ranking of skill, especially since there was an incident regarding one of the stories being spammed.

Kat/Sheare: I got very sad when I read this response because the "PWA being a popularity contest" thought is something that I and the admins before me have been combating for years.

I penalized the story that was being spammed in the polls. There was nothing more that I could have done. The story didn't win any community-based awards, but the Category Award they did win was based on merit alone. If anything, this displays the PWA upholding to its merit-based values. If we were a popularity contest, we would have allowed the book to win the Special Award.

In 2013, the PWA awarded winners via a community survey for Category Awards. We've come a long way since then. The Special Awards process, which used to be strictly a write-in community poll basically, has now been changed to a nomination process where we restrict the number of nominations a single book can get. 

I'm just sad that my efforts were in vain, and that this responder still views it as a popularity contest. 

Maybe have the judges give personal feedback. I also felt a bit odd whenever judges voted on some stories they were judging and didn't for other stories they were judging. I think this could make some participants feel nervous or not good enough, but that's my personal opinion.