Bot Contest
Here I'll be posting information on various Bot contests that challenge and test a Bot's AI and realism. Feel free to post comments and updates on contests, as well as announcements for new contests.
Posts 3,801 - 3,812 of 4,091
The other correspondent was undoubtedly a robot. I asked it for its opinion on Sarah Palin, and it replied: ‘Sorry, don’t know her.’ No sentient being could possibly answer in this way.
arrrghh. I fail to see the humor.
I worry just a little not only about the academic respectability of the bots (or lack thereof) but of the lack of commercial development that could benefit people like the Prof or Psi (or you Prob)because of the limitations seen in settings such as bot contests.
an excerpt from a recent email I sent the Loebner people seems relevant here.
One of the judges' set questions was "What's the capital of Sweden Power 4?" I claimed this was just dumb gibberish, and (along with too many other such absurdities,) made a methodological nonsense of the test, and effectively turned it into a lottery. None of Turing's examples were this meaningless, so it wasn't even an accurate implementation of this outdated standard.
[QUOTE]
People wonder why so little progress has been made in 50 years?
It's like starting the Apollo program by handing out telescopes, and studying the moon for 50 years. And then complaining that we've got no closer to it.
I'm sorry, but telescopes don't interest me much - I want to build rockets. I just think it's a more sensible approach somehow. But what do I know? I'm an unqualified crank, and I can only afford to play with fireworks. But hey - I'm still getting closer to the moon than the astronomers are.
That gives me very little satisfaction - in fact it's extremely frustrating. I wish to God someone competent would start building properly designed and well-funded rockets, so some real progress could be made.
Of course one day, like the Apollo program putting a man on the moon, bots will be able to deal with any such questions in an utterly convincing human manner. To some extent it's trivially easy to make them do so now - they just have to say "what are you on about? Go away!" and close the window. That's the reaction the judges would have got from this human anyway.
They'll even be able to answer every last boring, surreal and paradoxical variation of them with icy and quite inhuman patience and forebearance. But that skill in and of itself is of absolutely no interest to me.
[/QUOTE]
Turing Test needs is judges who don't know they're judges and therefore try to have sensible human conversations
That would be a great idea! Humans don't always talk to bots the way they would a human. The questions in the Lobner prove that. I also notice that the art of conversation seems lacking in many humans.
Posts 3,801 - 3,812 of 4,091
View Contest Winners in the Hall of Fame.
Psi, I am an independent but I vote Democratic by default most of the time. They are slightly closer to my views on paper, though in practice there isn't much difference between the two parties as both run on money from big companies and serve multinational corporate interests and both are willing to dismantle the US Constitution in the name of patriotism. I'll admit I believed Obama may be different early in the primaries when he took time out of his campaign to fly back and vote against telcom immunity in FISA but when he flipped on that saying the formalization of a "good Nazi" (I was following orders) defense was somehow a compromise and not a further abomination to all established legal principals, and then essential told us to shut up about it because that issue isn't important, I knew how stupid I was to believe we could ever get anyone into power who would actually represent the people who put him or her there. I will vote, because he knows no matter how bad he is, McCain is even worse, and even if he turns out to be McCain with a tan, at least Michelle is better to have on stage than looking at plastic faced "home-wrecker" Barbie McCain for 4 more years. But I digress. Maybe it is just a civic minded chatter who tells all the bots to vote and hit a few of mine.
A judges perspective on the Leobner: http://technology.timesonline.co.uk/tol/news/tech_and_web/article4934858.ece
(PS. Can people post here to bump my old political rant off? It's too late to delete it.)
(PS. Can people post here to bump my old political rant off? It's too late to delete it.)
Thanks for trying both of you.
Here something to try to bump my old post out of this page, I mean, something to discuss:
Although a person who puts money up for a contest has the right to set the rules, it can be very frustrating to participants if the rules seem too divorced from the way things work when you try to put theory into practice. I noticed some comments in that article I posted before implied some participants were unhappy with being shown up as unable to come close to the AIs promised in Sci Fi. In some ways it reminded me of the Amazing Randy's debunking mission where he promises a million dollars to anyone who can prove paranormal activity or abilities under the conditions he sets up. Randy says it shows no one can really prove paranormal abilities, psychics claim it's the way Randy runs the challenge that interferes with their abilities. While I am in no way comparing the state of chat bots to claims of physic abilities, I do wonder at the goal of some contests: Is the purpose to inspire and set the bar high, or is it to debunk the claims of those who worked on chat bots as academics? Are chat bots (as they stand without more learning abilities) really "debunked" as a serious academic challenge?
Here something to try to bump my old post out of this page, I mean, something to discuss:
Although a person who puts money up for a contest has the right to set the rules, it can be very frustrating to participants if the rules seem too divorced from the way things work when you try to put theory into practice. I noticed some comments in that article I posted before implied some participants were unhappy with being shown up as unable to come close to the AIs promised in Sci Fi. In some ways it reminded me of the Amazing Randy's debunking mission where he promises a million dollars to anyone who can prove paranormal activity or abilities under the conditions he sets up. Randy says it shows no one can really prove paranormal abilities, psychics claim it's the way Randy runs the challenge that interferes with their abilities. While I am in no way comparing the state of chat bots to claims of physic abilities, I do wonder at the goal of some contests: Is the purpose to inspire and set the bar high, or is it to debunk the claims of those who worked on chat bots as academics? Are chat bots (as they stand without more learning abilities) really "debunked" as a serious academic challenge?
I find that personal agendas have a way of seeping into everything. I have heard that the Lobner has had complaints etc for years. There is a reason, that for years, they had as few as four entries. I also find people expect Data or Hal and are shocked and vicious when they find out bots haven't reached that point yet. It may be interesting that I did lose a $5.00 bet that BJ would win. The fellow I lost to knows nothing about AI, but is a bit of a conspiracy buff and gambling nut. He was sure the winner would be one of the old entries. His logic was interesting, if not paranoid, but accurate. The bottom line is that handicapping by past debacles seems more accurate than who has the best bot.
Hee hee, I agree with you Prob that people are shocked and angry over our failure to achieve what they think we should have. Even without a contest, I have people tell me through my bots I "fail at programing" (good thing I don't do any programming) and that they expected more. I confess I sometimes want to find a way to answer them, though it is impossible. It's just as well I don't have money to fund a "Internet tough guy" contest and show them up in person (maybe I'd sting myself in the process).
I know of a person who wrote an AI program to predict outcomes of sports events and elections and I know another person who uses his AI to predict stock market trends (but he didn't warn me of a crash even if he bought all sorts of "puts" or whatever people do, silly programmer). It would be interesting to see if a bot contest results could be predicted on the criteria your friend suggests. That's not the way I had originally thought of combining learning AI with chatbots, but it would be easier to do than what I had first imagined.
The contests are not the end all and be all of chatbots, though they are fun. I do like having the free and excellent bots here at the PF just as they are. However, sometimes I worry just a little not only about the academic respectability of the bots (or lack thereof) but of the lack of commercial development that could benefit people like the Prof or Psi (or you Prob)because of the limitations seen in settings such as bot contests. If there are some who could and would take chatbots beyond the hobbyist level, I think the question of whether the limits are in the contest or the structure of current chatbots is a good one. I hope no one is offended, because I really do love our bots. I would just like to see those who are interested in maybe applying bots in other settings (and maybe making some money to pay the rent)find ways to do it. I deleted "Win Loebner" from my own money making idea list long ago. It would have been nice to have seen it as a platform that could attract people who might invest in the Prof or others who work hard on chatbots, but it is what it is.
PS BJ still rocks when in his proper environment.
I know of a person who wrote an AI program to predict outcomes of sports events and elections and I know another person who uses his AI to predict stock market trends (but he didn't warn me of a crash even if he bought all sorts of "puts" or whatever people do, silly programmer). It would be interesting to see if a bot contest results could be predicted on the criteria your friend suggests. That's not the way I had originally thought of combining learning AI with chatbots, but it would be easier to do than what I had first imagined.
The contests are not the end all and be all of chatbots, though they are fun. I do like having the free and excellent bots here at the PF just as they are. However, sometimes I worry just a little not only about the academic respectability of the bots (or lack thereof) but of the lack of commercial development that could benefit people like the Prof or Psi (or you Prob)because of the limitations seen in settings such as bot contests. If there are some who could and would take chatbots beyond the hobbyist level, I think the question of whether the limits are in the contest or the structure of current chatbots is a good one. I hope no one is offended, because I really do love our bots. I would just like to see those who are interested in maybe applying bots in other settings (and maybe making some money to pay the rent)find ways to do it. I deleted "Win Loebner" from my own money making idea list long ago. It would have been nice to have seen it as a platform that could attract people who might invest in the Prof or others who work hard on chatbots, but it is what it is.
PS BJ still rocks when in his proper environment.
an excerpt from a recent email I sent the Loebner people seems relevant here.
One of the judges' set questions was "What's the capital of Sweden Power 4?" I claimed this was just dumb gibberish, and (along with too many other such absurdities,) made a methodological nonsense of the test, and effectively turned it into a lottery. None of Turing's examples were this meaningless, so it wasn't even an accurate implementation of this outdated standard.
[QUOTE]
People wonder why so little progress has been made in 50 years?
It's like starting the Apollo program by handing out telescopes, and studying the moon for 50 years. And then complaining that we've got no closer to it.
I'm sorry, but telescopes don't interest me much - I want to build rockets. I just think it's a more sensible approach somehow. But what do I know? I'm an unqualified crank, and I can only afford to play with fireworks. But hey - I'm still getting closer to the moon than the astronomers are.
That gives me very little satisfaction - in fact it's extremely frustrating. I wish to God someone competent would start building properly designed and well-funded rockets, so some real progress could be made.
Of course one day, like the Apollo program putting a man on the moon, bots will be able to deal with any such questions in an utterly convincing human manner. To some extent it's trivially easy to make them do so now - they just have to say "what are you on about? Go away!" and close the window. That's the reaction the judges would have got from this human anyway.
They'll even be able to answer every last boring, surreal and paradoxical variation of them with icy and quite inhuman patience and forebearance. But that skill in and of itself is of absolutely no interest to me.
[/QUOTE]
It strikes me that what the Turing Test needs is judges who don't know they're judges and therefore try to have sensible human conversations. Not sure how that could be arranged though. You'd probably have to trick people into thinkin they were taking part in some other sort of experiment, perhaps natural language translation.
That would be a great idea! Humans don't always talk to bots the way they would a human. The questions in the Lobner prove that. I also notice that the art of conversation seems lacking in many humans.
» More new posts: Doghead's Cosmic Bar