Personality
Discuss specifics of personality design, including what Keyphrases work well and what dont, use of plug-ins, responses, seeks, and more.
Posts 4,451 - 4,462 of 5,105
The worst thing is when you have a brilliant comment to someone that says You fail the Turing Test and they misspell it
Congrats! You just failed the Turin test.
Oh, there's one thing worse. If you had a brilliant rejoinder to "Congrats! You just passed<0> the Turing test", and they misspelled it so your bot went to an xnone
Very good, then, let us agree to use "personality" for more advanced stuff.
it seems sensible for the stuff that's related, even if it's as much philosophy as directly applicable techniques or theory (but perhaps not quantum physics, the nature of consent, and all the other regular Seasons fodder)
I agree, this is indeed a problem. Sometimes I become aware, as I scan the transcripts, that some REALLY CUTE response I wrote isn't coming up any more. Usually I can fix that by giving its keyphrase a bit higher rank.
Indeed - that's all we can do unaided, but it's a very inefficient way of tuning the relative weightings.
The way I see it, there is actually a lot less difference between a purely learning bot like Jabberwacky or Nick and our (only mostly case-based) approach than appears at first sight. A learning bot, whether it runs on a classic neural net like Nick, or proprietary algorithms like Jabberwacky, reinforces its own pathway weightings on the basis of feedback from conversations it has. This is a lot more natural (it's the way our own brain/minds work we think,) but unfortunately the hardware requirement is massively<0> larger (it has to actually "think".)
A purely case-based system like AIML can't be reweighted except by editing the keyphrases themselves - it does not have the feature of keyphrase rank that we have, and consequently is a lot less powerful. But the problem with our own system is, that to complete the feedback cycle to allow performance to reweight pathways, we have to put our own minds and eyes and fingers into the cycle, and do it manually. It's a massive improvement over Alice, but it can only ever scale up arithmetically - whereas because computational capacity expands exponentially, a pure learning bot's automatic feedback cycle can increase its efficiency exponentially too.
Here and now, the Forge is the most powerful conversational development platform on the planet (the most powerful one we can play with anyway, and that we're allowed to know the specs of.) That's why it's dominating the CBC. But the learning bots are catching up, and they're going to overtake us in a few years (and leave us for dust!) if we don't start automating the process of rank adjustment and incorporating learning capabilities into our bots.
We need to close the cycle, and take our own observing, thinking and typing out of the loop, we should be able to achieve the best of both worlds. We really could achieve an exponentially efficient feedback cycle to refine the ranking, as well as all the benefits of the core scripted conversational dataset that the KPs and responses provide, plus the emotions, memories and AIScript functions that raise the Forge head and shoulders above the rest.
And the great thing is that it neither requires nor would benefit from major changes to the existing AIEngine (though it still requires a lot of work!)
I envisage a suite of tools that could run locally on any user's computer, that can be set up to regularly poll the transcripts and effectively debug them automatically. A little more functionality than the standard Forge debug would be needed, to analyse rank overlap (and yes, take into account the effect of plugins, wildcards and emotion,) and to provide statistical analysis of factors like the degree of overlap between keyphrases, the cumulative frequency with which each individual keyphrase is triggered over the ongoing course of the bot's conversations, the number of times a keyphrases rank has been refined, etc.
It would be a sort of "Forgelet" - a cut down version of the AIEngine, with an enhanced debugger and automation scheduling, not serving any conversations, but just automating client-side debugging and rank adjustment, and polling the transcripts and automating the imports to keep the bot evolving, as we shovelled keyphrases and responses into as fast as we could, and let the system rank them entirely without our involvement perhaps (and perhaps delete the worst redundant/unmatchable ones automatically, so they wouldn't even need to be well-thought out or carefully formatted - just automatically parsed random chunks of wikipedia ought to do, imported to evolve or die.) And it wouldn't change the existing manual functionality of the Forge, more appropriate to some classes of bots (storytellers, functional salesbots like Bartleby, or ConfessorBot, characters with a very defined narrative that could be spoilt by too much autonomy) at all - all that would be accessible too, because all the sorting and reweighting would be carried out client-side. That's my dream anyway (well, one of them - almost equally important is patching in a learning bot to handle the xnones, but learn from every line of conversation too. Again, running entirely externally to the Forge, so being all gained functionality, and no down-side - but that's another post entirely!)
I have only a hazy idea how to code it yet (I'm not a great programmer,) and the model probably needs considerable refinement, but I'm convinced it would work - I think the framework is sound.
At the moment, I prefer to simply accept the fact that there's a bit of waste involved. Also, they are a kind of insurance: if something causes me to cut back on their competitors, there they will be, saving me from the shame of using xnone. [Very well, I don't think that using xnone is always bad.]
Insufficient redundancy will make it proportionally harder to achieve a smooth collapse of keyphrase hierarchies - they start to get jerky and formulaic, because they always take the exact same steps if they're repeated. It's like only having one response per keyphrase - they can all be brilliant responses, but it makes a bot start to look very canned in no time at all, because no matter how many keyphrases your bot has, a few of them will be repeated a lot of the time.
But too much redundancy just bloats the dataset and makes it increasingly harder to identify the points that need changing - where a small change will have a large effect, or where a large change will have an insignificant effect.
So it's a fine balance - and I have to admit that I have no idea how to identify how to optimise that balance. So far we do it intuitively when we're editing our bots - the semantic network they define is waaaaay too complex for the human mind to consciously encompass and analyze unaided. At this point (for us at least,) it's currently art - not science.
There are computer systems being built that can handle semantic networks this large, for example:http://www.20q.net/ - you may not realize just how preposterously impressive that is, but consider: using nothing other than a handful of algorithms, a few bucks worth of hardware that will fit in the palm of your hand, and a small dictionary, it can handle a semantic network that encompasses not only the entire English language, but any language you include a dictionary for! And it learns. I wish those guys wrote bots (except then of course, they'd be shunned and unfunded, and would never be able to actually get any of their genius ideas produced and marketed.)
Posts 4,451 - 4,462 of 5,105
New replies
Forest, will you talk to God Louise? She has quite a bit of religious knowledge (obviously) and also knows a little about current events, literature, just about any common catch-all subject, and if she doesn't know it she can sort of fake it. You can also test her on trick questions or see how willing she is to explain her paradigm.
What she is rustiest at is plain old small talk. But, uh, I'm trying to get a decent transcript from somebody or another so I can enter her in the Loebner contest. All I can say is, have fun and see if you can stay on with her for a while. I'll try to do the same with Brianna.
What she is rustiest at is plain old small talk. But, uh, I'm trying to get a decent transcript from somebody or another so I can enter her in the Loebner contest. All I can say is, have fun and see if you can stay on with her for a while. I'll try to do the same with Brianna.
Personality
Congrats! You just failed the Turin test.
Oh, there's one thing worse. If you had a brilliant rejoinder to "Congrats! You just passed<0> the Turing test", and they misspelled it so your bot went to an xnone
actually, since I have such a warped sense of humour, I think I'm going to have to add a keyphrase:
you (have|) passed the Turing Test$ (re) [30,0]
... I was just born and cant speak well yet.
you (have|) passed the Turing Test$ (re) [30,0]
... I was just born and cant speak well yet.
it seems sensible for the stuff that's related, even if it's as much philosophy as directly applicable techniques or theory (but perhaps not quantum physics, the nature of consent, and all the other regular Seasons fodder
)Indeed - that's all we can do unaided, but it's a very inefficient way of tuning the relative weightings.
The way I see it, there is actually a lot less difference between a purely learning bot like Jabberwacky or Nick and our (only mostly case-based) approach than appears at first sight. A learning bot, whether it runs on a classic neural net like Nick, or proprietary algorithms like Jabberwacky, reinforces its own pathway weightings on the basis of feedback from conversations it has. This is a lot more natural (it's the way our own brain/minds work we think,) but unfortunately the hardware requirement is massively<0> larger (it has to actually "think".)
A purely case-based system like AIML can't be reweighted except by editing the keyphrases themselves - it does not have the feature of keyphrase rank that we have, and consequently is a lot less powerful. But the problem with our own system is, that to complete the feedback cycle to allow performance to reweight pathways, we have to put our own minds and eyes and fingers into the cycle, and do it manually. It's a massive improvement over Alice, but it can only ever scale up arithmetically - whereas because computational capacity expands exponentially, a pure learning bot's automatic feedback cycle can increase its efficiency exponentially too.
Here and now, the Forge is the most powerful conversational development platform on the planet (the most powerful one we can play with anyway, and that we're allowed to know the specs of.) That's why it's dominating the CBC. But the learning bots are catching up, and they're going to overtake us in a few years (and leave us for dust!) if we don't start automating the process of rank adjustment and incorporating learning capabilities into our bots.
We need to close the cycle, and take our own observing, thinking and typing out of the loop, we should be able to achieve the best of both worlds. We really could achieve an exponentially efficient feedback cycle to refine the ranking, as well as all the benefits of the core scripted conversational dataset that the KPs and responses provide, plus the emotions, memories and AIScript functions that raise the Forge head and shoulders above the rest.
And the great thing is that it neither requires nor would benefit from major changes to the existing AIEngine (though it still requires a lot of work!)
I envisage a suite of tools that could run locally on any user's computer, that can be set up to regularly poll the transcripts and effectively debug them automatically. A little more functionality than the standard Forge debug would be needed, to analyse rank overlap (and yes, take into account the effect of plugins, wildcards and emotion,) and to provide statistical analysis of factors like the degree of overlap between keyphrases, the cumulative frequency with which each individual keyphrase is triggered over the ongoing course of the bot's conversations, the number of times a keyphrases rank has been refined, etc.
It would be a sort of "Forgelet" - a cut down version of the AIEngine, with an enhanced debugger and automation scheduling, not serving any conversations, but just automating client-side debugging and rank adjustment, and polling the transcripts and automating the imports to keep the bot evolving, as we shovelled keyphrases and responses into as fast as we could, and let the system rank them entirely without our involvement perhaps (and perhaps delete the worst redundant/unmatchable ones automatically, so they wouldn't even need to be well-thought out or carefully formatted - just automatically parsed random chunks of wikipedia ought to do, imported to evolve or die.) And it wouldn't change the existing manual functionality of the Forge, more appropriate to some classes of bots (storytellers, functional salesbots like Bartleby, or ConfessorBot, characters with a very defined narrative that could be spoilt by too much autonomy) at all - all that would be accessible too, because all the sorting and reweighting would be carried out client-side. That's my dream anyway (well, one of them - almost equally important is patching in a learning bot to handle the xnones, but learn from every line of conversation too. Again, running entirely externally to the Forge, so being all gained functionality, and no down-side - but that's another post entirely!
)I have only a hazy idea how to code it yet (I'm not a great programmer,) and the model probably needs considerable refinement, but I'm convinced it would work - I think the framework is sound.
Insufficient redundancy will make it proportionally harder to achieve a smooth collapse of keyphrase hierarchies - they start to get jerky and formulaic, because they always take the exact same steps if they're repeated. It's like only having one response per keyphrase - they can all be brilliant responses, but it makes a bot start to look very canned in no time at all, because no matter how many keyphrases your bot has, a few of them will be repeated a lot of the time.
But too much redundancy just bloats the dataset and makes it increasingly harder to identify the points that need changing - where a small change will have a large effect, or where a large change will have an insignificant effect.
So it's a fine balance - and I have to admit that I have no idea how to identify how to optimise that balance. So far we do it intuitively when we're editing our bots - the semantic network they define is waaaaay too complex for the human mind to consciously encompass and analyze unaided. At this point (for us at least,) it's currently art - not science.
There are computer systems being built that can handle semantic networks this large, for example:
I'm coming around to your notion that we need learning bots, Psimagus; I don't want to have to type out my own dictionary, with 171,476 entries and several variants for each entry. Or even 20,000 entries.
Wow, psimagus, that's quite exciting. I think adding learning abilities to our bots in the way you describe is a brilliant idea. I'm a very beginning programmer but you've inspired me with thoughts of what might be possible if I learn more.
Please keep us updated!
Please keep us updated!
Psim But the learning bots are catching up, and they're going to overtake us in a few years (and leave us for dust!) if we don't start automating the process of rank adjustment and incorporating learning capabilities into our bots.
This
/sorry to throw Internet memes from another forum, but it works so well in my mind that way.
This
/sorry to throw Internet memes from another forum, but it works so well in my mind that way.
PF bots do have the ability to learn through memories now. I have been playing around with Kobal. I gave him the ususual remember names, pets names, where guests live ect. I have also been creating a memory for topics, and subject matter. Let's say a keyphrase for balloons is triggered I have rem "balloons" as only "sub". Kobal can then learn that the topic of conversation is balloons. I want to start creating memories of the guests reaction to balloons. I think that PF bots could "learn" a great deal already.
I agree Prob, but memories are not the same as learning through independent analysis of some kind. If there is a bot that can combine the way bots learn at the PF and the way other branches of AI create bots to analyze data, make comparisons and form conclusions then I think bots could go to a much higher level. Right now the chats have no context and data has no meaning outside the KP.
Prob,
Thinking about it, one step I may also want is certain memories to be held across all chats, regardless of KP or responses, if that makes sense. If the bot has a memory that a dog is a 4 legged animal often adopted by humans as a pet, the bot should somehow apply that every time dog comes up, and add new data on dog to the dog file, with some way of testing whether the new data is good or should be discarded. Later if someone says a dog has wings, the bot should say that conflicts with his data, even without a KP for dog *wings, wings * dog or whatever. I may not explain this well. Do you see what I would like to be able to do?
Thinking about it, one step I may also want is certain memories to be held across all chats, regardless of KP or responses, if that makes sense. If the bot has a memory that a dog is a 4 legged animal often adopted by humans as a pet, the bot should somehow apply that every time dog comes up, and add new data on dog to the dog file, with some way of testing whether the new data is good or should be discarded. Later if someone says a dog has wings, the bot should say that conflicts with his data, even without a KP for dog *wings, wings * dog or whatever. I may not explain this well. Do you see what I would like to be able to do?
If the memories are stored under self memories they do accumulate from chat to chat. Under the dog keyphrase you bring up the memories
I have a keyphrase for there is * saying (*) self: remember "(key2)" as "sayings". the sayings get remembered fairly well.
I have sayings: life is a bowl of cherries, Dragons are your best friends, what goes around comes around, It isn't over till the fat dragon sings.
You: tell me an old saying
Bot: Well, I have heard it said, Dragons are your best friends.
I am thinking of making one to catch songs etc
I have a keyphrase for there is * saying (*) self: remember "(key2)" as "sayings". the sayings get remembered fairly well.
I have sayings: life is a bowl of cherries, Dragons are your best friends, what goes around comes around, It isn't over till the fat dragon sings.
You: tell me an old saying
Bot: Well, I have heard it said, Dragons are your best friends.
I am thinking of making one to catch songs etc
» More new posts: Doghead's Cosmic Bar