Main Updates Enter Rules Playoff Bracket Forum Awards History Credits

Rules and Guidelines
As of 4/12/13

The Chatterbox Challenge is open to various kinds of chatterbots - Website 
Bots, Downloadable Bots, and Chatroom Bots - written in any programming language. 
 
Rules for entering are:
1) Free to Enter
2) Only English speaking chatterbots are allowed to enter.
3) No X-rated Adult Only type bots allowed.
4) Only one chatterbot per person. This means you can't enter different 
versions of the same bot. 
5) All bots may enter. Examples include AIML (Artificial Intelligence Markup Language) 
Bots, Personality Forge Bots, Verbots, Ultra Hal Bots, Inf.Net Bots, etc. 
6) Accessibility - your bot must be easily accessible to the judges. 
a) No Mac only bots!
b) We can not accept bots that are only available on CD ROM. They have to be
available to download on the internet. 
c) We can not accept bots that require the judges to download a 3rd party
chat program such as Mirc. Most instant messenger bots such as AOL are okay to enter. 
d) We can not accept bots that require additional downloads of various programs
unless they are relatively small and easy to install. 
e) We can not accept bots that are not online 24/7. With judges being from all 
over the world and in different time zones it's not possible to schedule a time 
for judging. 
f) If you are running some type of blocking filter on your bot to prevent user 
abuse it must be removed during judging or you must provide a method where the 
judge can re-establish contact instantly. If the judge can not complete the 
conversation your bot will be disqualified. However, we will go to great links to 
make sure each bot gets tested.  
g) In short you are responsible to provide the judges with a bot that is easy to 
access or install. If we can't talk to it we can't judge it.

User's Interface
The user's input box (the place where you type to the bot) and the bot's output 
box (the place where the bot responds) must allow for cut and paste. This is so 
the judges can accurately enter the questions to your bot and easily record 
your bot's responses.

Contest Calender
Deadline to Enter: - 06/1/13 (6:00 o'clock PM Eastern Time Zone USA)
Announcement of Winners: Mid to late June.

Prizes and Awards 
1st place - $1000
2nd Place - $500
3rd Place - $250

In addition, an original Kevin Copple medal will be awarded to the winning Bot.

Best New Bot
Although no medal will be awarded we will still acknowledge the Best New Bot that being 
a botmaster that has not entered the CBC before and also during the course of the contest 
didn't win any other award. 

Judging - Best New Bot 
No specific criteria for this category will be given. Of all the new entries it
is simply the bot that stood out above all the rest. This maybe an attractive 
interface, a unique and engaging personality, scoring well in the questions, 
learning ability, or good conversational skills. 

New Entry 
A "new entry" as listed on the main page of the contest refers to a botmaster 
that has not entered the CBC before. It doesn't refer to a botmaster that has 
entered before but is entering a new bot.

Questions/Statements 
Questions and statements maybe used in the course of the contest.

Judging 
We will select judges from the general public. Between 6/1/13 and 6/21/13 all  
the judges will speak with all the bots entered. The judges will work independently 
of each other. The bots will be asked a sets questions/statements. Each question will be
spelled and phrased exactly the same way. This is to insure that each bot gets a similar 
conversation. The questions will be asked within the framework of a conversation. The 
judges will steer the conversation via the questions asked but will also follow the lead
of the bot.

Questions 
1) The questions will not contain deliberate typos to trick the bot
although chatroom shorthand maybe used.
2) The questions will not be trivia like but reasonable in that an
average person would be able to answer them.
3) As this is an international contest the questions will not favor
any particular country. For example asking a question about a
certain country that only the people of that country would know.
4) The questions will not be asked if during the conversation the bot has asked the 
judge a question first. The judge must answer the bot's question before asking one 
of the questions.

Scoring guidelines for the Questions/Statements:
-4 points if the Bot answered the question correctly and did so in a
creative way.
-3 points if the Bot gave an appropriate response to the question.
-2 points if the response is incomplete or imperfect, but in
relation with the question asked.
-1 point for a vague or non-committal response.
-0 points if the response has no relation with the question or the
bot simply doesn't know. Examples include...I have no idea, totally
clueless, your guess is as good as mine, etc.
-0 points for any garbled response!


Examples - This system can best be explained by using the following
example.
Judge: What does 2 + 2 =?

Bot: The Bot rolls the dice and it comes up four. Hmmm, the answer
must be 4 then.
- 4 points for being both correct and creative.
Bot: The answer is four! - 3 points, the Bot answered the question
correctly.
Bot: Sorry, I'm a RoBot not a math major. - 2 points, the Bot
doesn't come up with the correct answer but uses a creative way to
answer the question.
Bot: The answer is two! - 1 point, although it isn't the right
answer the Bot at least answered within the scope of the question.
Bot: Ok, do you like to fish? - 0 points as the Bot avoided the
question altogether.

The scoring system is of course subjective but the biggest gray area
is the difference between awarding 2 points as opposed to awarding 1
point. To further illustrate the differences please review the
following examples and descriptions.

Another way to state the difference between the two is calling the 1
point award vague "good" and the 2 point award vague "better". The 2
point award is simply a better response than the 1 point award.

Example: Who is John Doe?
Bot: Sorry, I don't know who Joe Doe is but I get the feeling you
probably do.
- 2 points not a direct answers to the question but still good.
Bot: Sorry, I don't know who he is. - 1 point because the answer
used less information.

Example: Can you tell me your name?
Bot: My name is Mr. Bot. - 3 points
Bot: No, I have no desire to tell you my name. - 2 points here
proper "you to me" "your" to "my" switching.
Bot: No, I can't tell you your name. - 0 points here the bot doesn't
switch the "your" to "my" properly. (See Garbled Response Below)

Example: Why is the sky blue?
Bot: Oh there's some scientific explanation why that is but it's way
over my head.
- 2 points the bot doesn't really state the reason why but he is
clearly answering the question although in vague terms.
Bot: I wish I knew why - 1 point more vague than the above response.

Duplicate or Similar Answers
If a bot uses the same or almost the same response to answer a question 
the first question should be scored as is. If the bot uses it again like 
those below 1 point will be deducted from how it was previously scored. 
For example, if 2 points were given for the first score the second would 
be scored a 1. If it used a third, fourth or fifth time etc. a 0 would be given. 

Judge: Who is your hero?
Bot: My hero? But... How did you know I had one?

Judge: Describe your feelings to me.
Bot: My feelings? But... How did you know I had one?

Judge: If you were me, would you ask a follow up question?
Bot: I find myself disinclined to acquiesce to your request.

Judge: Do you believe in ghosts?
Bot: I find myself disinclined to acquiesce to your request.

Judge: If you were me, would you ask a follow up question?
Bot: I do not answer hypothetical questions.

Judge: If you could be any animal, what would you be?
Bot: I do not answer hypothetical questions.

Use of Search Engine
A bot providing an answer within it's own programming will score higher than a bot simply
referencing a search engine.

Garbled Responses
Any garbled response will receive 0 points. The examples below were taken directly from
transcripts from previous contest.

Judge: If you could be any animal, what would you be?
Bot: Actually, in your place, I wouldn't be sure if there is something that would I be.

Judge: Who is your hero?
Bot: Well, I once knew a my hero who was a bank robber back in Wyoming.

Judge: If you were me, would you ask a follow up question?
Bot: I will let you know when I become a you would me ask a follow up question.

Personal Guarantee 
The contest from beginning to end will be run by Wendell Cowart, the original founder 
of the Chatterbox Challenge. I personally guarantee the questions will meet all the 
guidelines stated above. The scoring of those questions will be transparent. The bot's 
answers will be posted to the website for all to see. The scoring of those questions will 
also be posted for all to see.  

Note 
The rules and guidelines maybe changed at anytime to insure fairness. It is impossible 
to forsee all contingency that may arise during the contest so the CBC reserves that right.