The Book of AI

How To Build Chatbots with The Personality Forge
Chapters
Chapter 12: Advanced Keyphrase Functionality
Regular Expressions

Regular Expressions are powerful pattern-matching statements that can match just about any pattern you can think of. As such, they can greatly expand the reach of your Keyphrases.

Keyphrase: i will have ([0-9,.]+)

Matches "i will have " followed by any possible number, which is assigned to (key1)

Keyphrase: why did (np:1) (vp:2) ([0-9,.]+:3) p[ya]jamas"

Matches "Why did the dwarf carry 400 pajamas?" You can use Key Mapping to store keys from Regular Expression matches in parenthesis. The final word will match the US or British spelling of "pajamas".

Keyphrase: ([a-z0-9_\.-]+)@([a-z0-9\.-]+)\.([a-z\.]{2,6})

Matches an email address. But is hard to read!

Be aware that Keyphrase Wildcards may not work when being modified by or pressed up against certain Regular Expresions. You'll want to test your Keyphrases that use Regular Expressions thoroughly. Due to the way Keyphrase Wildcards work, you might get unexpected results or unusual Ranks.

Regular Expressions can be tricky to work with and hard to read. There are many great tutorials on regular expressions on the web, but be prepared for some challenges if you aren't used to writing code.

Raw Mode

Once you've mastered Keyphrases, you'll notice that there are certain things that you are unable to match. Since every incoming sentence is processed, spell-corrected, and changed into as much of a complete sentence as possible, it is different from the raw, unaltered message coming in. If you want access to this raw, unaltered version of the message, then use the AIScript "raw" statement with your Keyphrase.

RAW MODE

Where: Keyphrase

Statement: raw;

What It Does: Tells the AI Engine to match this Keyphrase to the raw, unprocessed input the chatbot received.

When Is This Useful?

  • Matching Proper Nouns: The AI Engine knows most names but doesnt recognize unusual proper nouns and will attempt to correct their spelling. Use raw mode to match a proper noun that is being spell-corrected, or turn off Spell Correction in your Chatbot's Settings.
  • Matching Foreign Words: Most foreign words are unknown to the AI Engine. To match these, use raw mode.
  • Misspelled Words & Poor Grammar: The AI Engine will correct misspelled words and poor grammar. If you want a Keyphrase to match these things, use raw mode.
  • Shorthand Spelling: If you want to respond on the shorthand in phrases like this: "u r 2 kewl 4 my warez", use raw mode
  • Interjections: The AI Engine strips out interjections such as "ahh" and "hmm". Raw mode allows you to respond to these things.
  • Punctuation: If you want your Keyphrase to match a period, question mark, or exclamation point at the end of the Keyphrase, use raw mode. Matching punctuation is only rarely needed, but in those rare situations, it can be useful.

What Works in Raw Mode

You can only use the following Keyphrase features in raw mode:

Keyphrase Lists: einstein, bohr, planck
Word Lists: como (se|te) (llama|llamas)
WildCards: * or (*)
Plug-ins: (p:dance)
Limited Regular Expressions: o+h+

NOTE: Raw Mode does not support all Regular Expression syntax. Experiment to see if what you are trying to accomplish works.

Drawbacks of Raw Mode

While raw mode is useful in rare situations, there are many drawbacks that you must keep in mind.

  • Word Type Wildcards: Word type wildcards will not work: this includes (np), (vp), and the rest.
  • Grammar Issues: Any spelling or grammatical issues in the incoming message will not be corrected, and may prevent a raw Keyphrase from matching.
  • Sentence-Spanning: The second is that the * and (*) wildcards can span sentences. This is typically not something you want. For example, "i * like you" in raw mode would match "I am hungry. Bonnie is too, just like you."
  • Unpredictable Keys: Possibly the biggest drawback to raw mode is that (key)s can have unpredictable results. (postkey) will contain everything in the message following the match, including punctuation and any following sentences. You can use the Regular Expression [.,?!] to cut it off at the end of the sentence. (prekey) has similar extended reach.

So it is best to only use raw mode when you need to match something that gets changed during preprocessing.