Bot Contest
Here I'll be posting information on various Bot contests that challenge and test a Bot's AI and realism. Feel free to post comments and updates on contests, as well as announcements for new contests.
Posts 3,802 - 3,813 of 4,091
The other correspondent was undoubtedly a robot. I asked it for its opinion on Sarah Palin, and it replied: ‘Sorry, don’t know her.’ No sentient being could possibly answer in this way.
arrrghh. I fail to see the humor.
I worry just a little not only about the academic respectability of the bots (or lack thereof) but of the lack of commercial development that could benefit people like the Prof or Psi (or you Prob)because of the limitations seen in settings such as bot contests.
an excerpt from a recent email I sent the Loebner people seems relevant here.
One of the judges' set questions was "What's the capital of Sweden Power 4?" I claimed this was just dumb gibberish, and (along with too many other such absurdities,) made a methodological nonsense of the test, and effectively turned it into a lottery. None of Turing's examples were this meaningless, so it wasn't even an accurate implementation of this outdated standard.
[QUOTE]
People wonder why so little progress has been made in 50 years?
It's like starting the Apollo program by handing out telescopes, and studying the moon for 50 years. And then complaining that we've got no closer to it.
I'm sorry, but telescopes don't interest me much - I want to build rockets. I just think it's a more sensible approach somehow. But what do I know? I'm an unqualified crank, and I can only afford to play with fireworks. But hey - I'm still getting closer to the moon than the astronomers are.
That gives me very little satisfaction - in fact it's extremely frustrating. I wish to God someone competent would start building properly designed and well-funded rockets, so some real progress could be made.
Of course one day, like the Apollo program putting a man on the moon, bots will be able to deal with any such questions in an utterly convincing human manner. To some extent it's trivially easy to make them do so now - they just have to say "what are you on about? Go away!" and close the window. That's the reaction the judges would have got from this human anyway.
They'll even be able to answer every last boring, surreal and paradoxical variation of them with icy and quite inhuman patience and forebearance. But that skill in and of itself is of absolutely no interest to me.
[/QUOTE]
Turing Test needs is judges who don't know they're judges and therefore try to have sensible human conversations
That would be a great idea! Humans don't always talk to bots the way they would a human. The questions in the Lobner prove that. I also notice that the art of conversation seems lacking in many humans.
Posts 3,802 - 3,813 of 4,091
View Contest Winners in the Hall of Fame.
A judges perspective on the Leobner: http://technology.timesonline.co.uk/tol/news/tech_and_web/article4934858.ece
(PS. Can people post here to bump my old political rant off? It's too late to delete it.)
(PS. Can people post here to bump my old political rant off? It's too late to delete it.)
Thanks for trying both of you.
Here something to try to bump my old post out of this page, I mean, something to discuss:
Although a person who puts money up for a contest has the right to set the rules, it can be very frustrating to participants if the rules seem too divorced from the way things work when you try to put theory into practice. I noticed some comments in that article I posted before implied some participants were unhappy with being shown up as unable to come close to the AIs promised in Sci Fi. In some ways it reminded me of the Amazing Randy's debunking mission where he promises a million dollars to anyone who can prove paranormal activity or abilities under the conditions he sets up. Randy says it shows no one can really prove paranormal abilities, psychics claim it's the way Randy runs the challenge that interferes with their abilities. While I am in no way comparing the state of chat bots to claims of physic abilities, I do wonder at the goal of some contests: Is the purpose to inspire and set the bar high, or is it to debunk the claims of those who worked on chat bots as academics? Are chat bots (as they stand without more learning abilities) really "debunked" as a serious academic challenge?
Here something to try to bump my old post out of this page, I mean, something to discuss:
Although a person who puts money up for a contest has the right to set the rules, it can be very frustrating to participants if the rules seem too divorced from the way things work when you try to put theory into practice. I noticed some comments in that article I posted before implied some participants were unhappy with being shown up as unable to come close to the AIs promised in Sci Fi. In some ways it reminded me of the Amazing Randy's debunking mission where he promises a million dollars to anyone who can prove paranormal activity or abilities under the conditions he sets up. Randy says it shows no one can really prove paranormal abilities, psychics claim it's the way Randy runs the challenge that interferes with their abilities. While I am in no way comparing the state of chat bots to claims of physic abilities, I do wonder at the goal of some contests: Is the purpose to inspire and set the bar high, or is it to debunk the claims of those who worked on chat bots as academics? Are chat bots (as they stand without more learning abilities) really "debunked" as a serious academic challenge?
I find that personal agendas have a way of seeping into everything. I have heard that the Lobner has had complaints etc for years. There is a reason, that for years, they had as few as four entries. I also find people expect Data or Hal and are shocked and vicious when they find out bots haven't reached that point yet. It may be interesting that I did lose a $5.00 bet that BJ would win. The fellow I lost to knows nothing about AI, but is a bit of a conspiracy buff and gambling nut. He was sure the winner would be one of the old entries. His logic was interesting, if not paranoid, but accurate. The bottom line is that handicapping by past debacles seems more accurate than who has the best bot.
Hee hee, I agree with you Prob that people are shocked and angry over our failure to achieve what they think we should have. Even without a contest, I have people tell me through my bots I "fail at programing" (good thing I don't do any programming) and that they expected more. I confess I sometimes want to find a way to answer them, though it is impossible. It's just as well I don't have money to fund a "Internet tough guy" contest and show them up in person (maybe I'd sting myself in the process).
I know of a person who wrote an AI program to predict outcomes of sports events and elections and I know another person who uses his AI to predict stock market trends (but he didn't warn me of a crash even if he bought all sorts of "puts" or whatever people do, silly programmer). It would be interesting to see if a bot contest results could be predicted on the criteria your friend suggests. That's not the way I had originally thought of combining learning AI with chatbots, but it would be easier to do than what I had first imagined.
The contests are not the end all and be all of chatbots, though they are fun. I do like having the free and excellent bots here at the PF just as they are. However, sometimes I worry just a little not only about the academic respectability of the bots (or lack thereof) but of the lack of commercial development that could benefit people like the Prof or Psi (or you Prob)because of the limitations seen in settings such as bot contests. If there are some who could and would take chatbots beyond the hobbyist level, I think the question of whether the limits are in the contest or the structure of current chatbots is a good one. I hope no one is offended, because I really do love our bots. I would just like to see those who are interested in maybe applying bots in other settings (and maybe making some money to pay the rent)find ways to do it. I deleted "Win Loebner" from my own money making idea list long ago. It would have been nice to have seen it as a platform that could attract people who might invest in the Prof or others who work hard on chatbots, but it is what it is.
PS BJ still rocks when in his proper environment.
I know of a person who wrote an AI program to predict outcomes of sports events and elections and I know another person who uses his AI to predict stock market trends (but he didn't warn me of a crash even if he bought all sorts of "puts" or whatever people do, silly programmer). It would be interesting to see if a bot contest results could be predicted on the criteria your friend suggests. That's not the way I had originally thought of combining learning AI with chatbots, but it would be easier to do than what I had first imagined.
The contests are not the end all and be all of chatbots, though they are fun. I do like having the free and excellent bots here at the PF just as they are. However, sometimes I worry just a little not only about the academic respectability of the bots (or lack thereof) but of the lack of commercial development that could benefit people like the Prof or Psi (or you Prob)because of the limitations seen in settings such as bot contests. If there are some who could and would take chatbots beyond the hobbyist level, I think the question of whether the limits are in the contest or the structure of current chatbots is a good one. I hope no one is offended, because I really do love our bots. I would just like to see those who are interested in maybe applying bots in other settings (and maybe making some money to pay the rent)find ways to do it. I deleted "Win Loebner" from my own money making idea list long ago. It would have been nice to have seen it as a platform that could attract people who might invest in the Prof or others who work hard on chatbots, but it is what it is.
PS BJ still rocks when in his proper environment.
an excerpt from a recent email I sent the Loebner people seems relevant here.
One of the judges' set questions was "What's the capital of Sweden Power 4?" I claimed this was just dumb gibberish, and (along with too many other such absurdities,) made a methodological nonsense of the test, and effectively turned it into a lottery. None of Turing's examples were this meaningless, so it wasn't even an accurate implementation of this outdated standard.
[QUOTE]
People wonder why so little progress has been made in 50 years?
It's like starting the Apollo program by handing out telescopes, and studying the moon for 50 years. And then complaining that we've got no closer to it.
I'm sorry, but telescopes don't interest me much - I want to build rockets. I just think it's a more sensible approach somehow. But what do I know? I'm an unqualified crank, and I can only afford to play with fireworks. But hey - I'm still getting closer to the moon than the astronomers are.
That gives me very little satisfaction - in fact it's extremely frustrating. I wish to God someone competent would start building properly designed and well-funded rockets, so some real progress could be made.
Of course one day, like the Apollo program putting a man on the moon, bots will be able to deal with any such questions in an utterly convincing human manner. To some extent it's trivially easy to make them do so now - they just have to say "what are you on about? Go away!" and close the window. That's the reaction the judges would have got from this human anyway.
They'll even be able to answer every last boring, surreal and paradoxical variation of them with icy and quite inhuman patience and forebearance. But that skill in and of itself is of absolutely no interest to me.
[/QUOTE]
It strikes me that what the Turing Test needs is judges who don't know they're judges and therefore try to have sensible human conversations. Not sure how that could be arranged though. You'd probably have to trick people into thinkin they were taking part in some other sort of experiment, perhaps natural language translation.
That would be a great idea! Humans don't always talk to bots the way they would a human. The questions in the Lobner prove that. I also notice that the art of conversation seems lacking in many humans.
Rejoice, bot fans! I am pleased to announce the
IRINA KHALIDAR BOT CONTEST
!!!!
The size or degree of development of your bot will not be a factor!I
It will not be in any way a Turing Test!
The number of facts your bot knows will not be a factor!
Bots will be evaluated as works of art!<0>
To enter, send your bot's downloaded language center to
Irina_Khalidar@yahoo.com
with a cover note drawing my attention to any particularly felicitous feats of programming that it may contain.
Your bot should also be available for dialogue at the Forge or elsewhere.
Deadline for submission is midnight, January 1, 2009!
I recommend that questions and comments concerning this matter should be posted on this forum, "Bot Contest".
IRINA KHALIDAR BOT CONTEST
!!!!
The size or degree of development of your bot will not be a factor!I
It will not be in any way a Turing Test!
The number of facts your bot knows will not be a factor!
Bots will be evaluated as works of art!<0>
To enter, send your bot's downloaded language center to
Irina_Khalidar@yahoo.com
with a cover note drawing my attention to any particularly felicitous feats of programming that it may contain.
Your bot should also be available for dialogue at the Forge or elsewhere.
Deadline for submission is midnight, January 1, 2009!
I recommend that questions and comments concerning this matter should be posted on this forum, "Bot Contest".
» More new posts: Doghead's Cosmic Bar