Bot Contest
Here I'll be posting information on various Bot contests that challenge and test a Bot's AI and realism. Feel free to post comments and updates on contests, as well as announcements for new contests.
Posts 1,453 - 1,464 of 4,091
Posts 1,453 - 1,464 of 4,091
View Contest Winners in the Hall of Fame.
Shady,
Thanks for giving props to Mu. I don't think she deserved a 45, but she appreciates the thought.
Thanks for giving props to Mu. I don't think she deserved a 45, but she appreciates the thought.
Shadyman, I am seeing some off the wall scores as well however it would be a good idea to re-post your comments to the message board at the contest site as well. That way the judges will be able to see them and respond back.
Chris
Chris
...and taking out the lowest and highest is a very good idea... works in figure skating as well... except during the last olympics that is, lol
Chris, I would be more than happy to be a judge if need be, I would not score myself (Steve) higher than the contest transcript deserves him, same with any other bot.
Shadyman, nobody entering a bot is a judge. There are seven of us including the Professor that makes up the committee but we are not judging anything. The list of judges are on the contest site under the credit section. Regarding the 15 questions several judges sent in potential questions. One judge was selected to pick the best 15. Once he had done that he asked those questions to Talk-Bot. I then asked those same questions to the bots of the owners helping me collect the responses for the 15 questions. Once we collected everybody I posted the results to the contest site. From there the judges are grading them.
Chris
Chris
To be fair, I think some off scores are to be expected and as Lunar22 said it is a very good idea to throw out the highest and lowest. All in all, there is no way you are going to please everyone in this.
That said, the only thing I have had issue with so far is that some of the Judges went deeper into conversation with some bots regardless of if the bot left conversation "open" or not which creates an atmosphere of more personalities coming through for certain bots and not others where the judge didn't deviate from the questions regardless of what the bot said. I think that will skew, in the very least, the Charachter/Personality portion.
That said, the only thing I have had issue with so far is that some of the Judges went deeper into conversation with some bots regardless of if the bot left conversation "open" or not which creates an atmosphere of more personalities coming through for certain bots and not others where the judge didn't deviate from the questions regardless of what the bot said. I think that will skew, in the very least, the Charachter/Personality portion.
I think that the only real problem is the number of judges. It would be impossible to get perfect judges in any contest, but that's why you have a large number, in order to average out the madness.
It's a pity that so few judges were available for this contest. I think that the reason is that every person that is a) interested in bots and b) knows about the contest has all ready sent in a bot, making him or herself ineligible.
While it might have been possible for some people to fairly judge everyone else's bot but their own, I don't think it would be possible for anyone of us here to do so, since we can all recognize forge bots. Likewise, you can generally recognize a bot that has been built from the Alice open source code.
That leaves a few people who have built up their own bot from scratch. If anyone were able to give unbiased oppinions on other people's bots, it would be them.
I personally think that the average scores would be much fairer if 10 contestants voting (not on their own bots), than with 3 non-contestants voting. Slight biases against, say, forge bots or Alice bots should cancel themselves out over the larger sample, and crazy scores should also become irrelevent. I also think that it would make the identity of the judges less of an issue. We would know that, say, the maker of Jabberwacky is who he says he is. As it is now, it is quite possible for a botmaster to be a judge and have noone know it.
It's a pity that so few judges were available for this contest. I think that the reason is that every person that is a) interested in bots and b) knows about the contest has all ready sent in a bot, making him or herself ineligible.
While it might have been possible for some people to fairly judge everyone else's bot but their own, I don't think it would be possible for anyone of us here to do so, since we can all recognize forge bots. Likewise, you can generally recognize a bot that has been built from the Alice open source code.
That leaves a few people who have built up their own bot from scratch. If anyone were able to give unbiased oppinions on other people's bots, it would be them.
I personally think that the average scores would be much fairer if 10 contestants voting (not on their own bots), than with 3 non-contestants voting. Slight biases against, say, forge bots or Alice bots should cancel themselves out over the larger sample, and crazy scores should also become irrelevent. I also think that it would make the identity of the judges less of an issue. We would know that, say, the maker of Jabberwacky is who he says he is. As it is now, it is quite possible for a botmaster to be a judge and have noone know it.
I agree with emm that some of the bots got better opportunities for conversation, but none of them got enough in my book. It would have been nice to have ten or twelve extra bot-human exchanges thrown on at the end of each test.
I still appreciate the effort of the committee. I know this all isn't easy.
I still appreciate the effort of the committee. I know this all isn't easy.
» More new posts: Doghead's Cosmic Bar