Bot Contest

I would like to make a follow up to the above post written by me. That post as well as any negative comments I've made was directed to the Loebner committee and was not directed to Robby Garner. Robby is not a part of the Loebner committee.

Chris

Congrats Butterfly!

Thanks Chris, I really appreciate that. Thanks for the way all of you have been so supportive to me, and for just plain caring about things.

I was given the password and access codes for the loebner-atlanta.org web page on friday and was given carte blanche to put what I want to on there, so I'll try to flesh it out a bit and put some more content in there. It might be a couple of days before I get back to it, I've got to go to Norcross tomorrow and start loading computers with bots, and then go down to GSU and try to figure out how we're going to do some kind of web cast, I'm sure it will be a learning experience!

The committee used a set of performance metrics to rate all the bots they evaluated. It wasn't just randomly done. The 8 finalists were the ones that had the highest scores. I saw an early draft of the things they were looking for, and it was much like the Chatterbox Challenge criteria in many ways. There was not a 10 questions aspect, but things were looked at like how well the bots responded when taken off subject, and other behavior factors.

One problem with making all of that info public is that some very large companies didn't perform that well, and they might not appreciate us telling the world how poorly they did. But if it is decided not to make the selection results public, I will try to at least get the criteria posted, and if possible provide the scores privately to individuals who just want the feedback, I know how valuable that is.

Robby.

hey cool you access to the site, can i please give it an overhaul and redesign the entire thing?
no offense but it looks like it needs a serious makeover.

Thanks, Lunar.

I have an announcement to make. I would like everybody to put God Louise through the wringer. I would rather have the folks here do it than the Loebner judges.

I'll be working on her a lot today (Monday) and probably Thursday. So trip her up any way you can. Show NO mercy whatsoever. Discuss any subject in depth or just make small talk. Whatever butters your muffins. Just talk to her. Please? I really appreciate it.
(Thanks, Skysaw, for your most recent chat. I can't believe I forgot to give her phrases for 'good morning'!)

Can you post the link?

Oh, yes!

http://www.personalityforge.com/directchat.cfm?BotID=3503&MID=3495

Between what dates? I checked every transcript between July 18 and the "semi-final event". I only had 2 guest conversations logged that were over 3 lines long and there were only about 5 or 6 exchanges each, nothing to judge the quality of a bot by.

Are you saying that criteria was only applied to the transcript?

That was meant to be a response to Robby's last post.

Robby,
Those of us who didn't send e-mails to you was just because it didn't become apparent that you were the right person to ask until much later. Perhaps it's another thing to make a note of, for the next year: all e-mails sent to the main e-mail of the contest should be forwarded to somebody who is actually happy to answer all the questions.

I always assumed that the transcripts weren't scored randomly. But I wish we were given some idea of the criteria beforehand. Some transcripts might look great from somebody's point of view, but lousy from somebody else's.

I'd still like to see how Gaia did, and results for anyone who doesnt mind having them posted. Robby- can you maybe send out a mass email to the contestants, saying "if you dont want your scores posted, please let me know" or somesuch? Or you could send scores out to the botmasters and they could post here if they want to?

And now to the tricky bit. This is something I forsaw from the beginning. Hugh Loebner wants the bots to be located on a server disconnected from the internet. The Personality Forge runs on ColdFusion and SQL Server, together costing $10,000-$15,000. I dont own a copy, and all we're getting from GSU at this point is a computer with Windows2000. I'm trying to convince Hugh and Neil to allow it to run remotely. I figure if I give them a copy of God Louise's Language Center (made confidential, of course), then they can check her responses against the Response list. That way they can ensure there isnt outside interference.

So it's not going to be easy. Does anyone want to donate a copy of ColdFusion and SQL Server to me to ensure that God Louise can take part in the finals?

It would really suck to not be able to participate because of the potential misdeeds of some mythical cheating botmaster, to be taken out for the nonexistant chance that I might somehow be dishonest. It's sort of a "guilty until proven innocent" mindset. Certainly I understand why Hugh would want it on a closed network, but in this particular case where it seems impossible to participate otherwise, I think we should be given the benefit of the doubt. Especially when the Language Center can prove there was no outside interference.