KeXu

A Few Tips on How to Lead a Proposal Review Team

Ke Xu

March 28, 2009

I. Careful Planning and Early Prevention

1. Interest Section Leaders Guide to the Academic Program

Prepared by TESOL Central Office and IS Leadership Council, this is the most important instruction handbook that guides all IS leaders on how to conduct proposal review. It tells you everything you need to know about proposal review, from the structure of the Convention Programs, types of academic programs (academic sessions and intersections), to the schedule and procedures for proposal review including deadlines for different phases of the proposal review. Read it carefully and mark important dates and deadline

2. Recruiting reviewers
In order to successfully complete the proposal review within the prescribed period of time, it is important to recruit an adequate number of reviewers

1) Requirements

• Experience in teaching, research or administration in related field
• Availability and commitment (Must be able to read 15 or more 150-word abstract during the period of June XXX-XXX). Each reviewer must sign at the bottom of the application stating that they understand the time constraint and the above-mentioned commitment (see attached sample application form).
• TESOL member, preferably primary member of your IS
• Convention experience
• Internet Acess

2) Importance of a valid email address

It is of essential importance to ask each reviewer to provide on their reviewer’s application form a valid email address which you as review team leader will later enter into the online Precis system. Reviewers will later have to use exactly the same email address to create a new account. So they must remember this email address. It is also recommendable that reviewers should provide a second email address in case the first one doesn’t work.

3) Ways of recruiting

• IS booth interview during TESOL Convention
• email interview
• phone/skype interview

4) Emergency readers

To effectively deal with the incomplete assignments left behind by drop-out or slow readers, building up a strong force of emergency readers becomes extremely important. Emergency readers can be recruited either before the review starts or during the review process (recruiting those who finish their assignments early and are willing to read more). Emergency readers must be willing to work under pressure, start late but still be able to finish their new assignments before the TESOL deadline.

3. IS deadline (3 days before TESOL deadline)
To successfully meet the deadline set by TESOL, it is highly recommendable for each IS to set up its own deadline so that the leader of the review team could have sufficient time to re-assign to emergency readers the incomplete assignments left by those reviewers who dropped out due to unexpected emergencies or those who lagged behind.

4. Professional guidance
Give the Proposal Rating Rubric (prepared by CO) to your reviewers at least one month in advance.

• Provide a list of the proposals accepted by your IS last year
• Provide the table of contents of recent issues of TQ and other professional journals related to language teaching
• Recommend some Web sites that may assist reviewers in keeping them abreast with the latest development of the theories and practice in related fields
• Form an online discussion group and launch a discussion forum on how to apply the rating criteria to the rating process so that a workable grading scale system can be established. To prevent reviewer’s true identity from leaking out, each reviewer can be given a code number when they participate in the discussion. In order to ensure the objectivity and fairness of the rating, reviewers are not encouraged to discuss on specific cases once the review starts.

Good preparation will reduce errors and mistakes, prevent accidents and problems, and therefore guarantee the success of the review.


II. Close Monitoring and “Leave No Reviewers Behind” Policy

Once the reviewers are recruited and the review process starts, it is essential to watch the progress of the review progress closely, as closely as you watch your stock investment.

Before the review starts, contact your team members at least once a month so as not to lose contact with any of your reviewers.

After the review starts, make sure you are constantly aware of the progress status of each reviewer at each stage of the 10-day process:

Shortly after the process starts, check in the Precis system and find out:

• how many reviewers have started reading
• how many reviewers have started grading

By the 3rd day after the review starts, it is necessary for review team leader to check to make sure the majority of the reviewers have started review process. To those who haven’t started reading by then, send an email to find out whether they have problem signing into the Preci system or finding the reviewer’s page. If they do have these problems, tell them how to solve the problems, or get help from the Precis tech support team. If more than 3 reviewers report the same problem, email the answer to everyone on your team so that you and tech support team don’t have to deal with each reviewer with the same problem later. To those who haven’t started the review due to other reasons, remind them that 3 days have elapsed and they have only 4 days left before the IS deadline.

By the 5th day, reviewers should have finished over 70% of their assignments. Check in the Precis system to find out:

• how many abstracts each reviewer has read
• how many abstracts each reviewer has rated
  
  

To those who are lagging behind, send an email to find out whether they are experiencing any difficulties and therefore need help. If they do have difficulties and cannot complete their assignments due to unexpected emergencies, take over those left-over abstracts and re-assign them to emergency readers. Considering the time constraint, it is advisable not to give each emergency reader over 10 abstracts.

Most of the reviewers grade the abstract they have read right away, but some reviewers may read the abstracts without rating them immediately. To those who leave their abstracts unrated by the 5th day after you start the process, send them an email and alert them that they are only 2 days away from the IS deadline, urging them to start grading the abstracts they have read.

As a review team leader, you have to know:

• how to log into the Precis system
• how to switch to the reviewer’s page
• how to enter reviewer’s data
• how to edit reviewer’s data
• how to add or delete reviewer’s data
• how to check the progress of each reviewer
• how to find out which abstract has been assigned to whom

If you don’t, contact the Precis tech team and get step-by-step instruction.


III. Effective Communication

Timely, accurate and effective communication is the lifeline of the proposal review.

• Make sure all communication channels are open and every reviewer has access to you at any time during the review process. Give your team members both your primary email and your secondary email addresses just in case you have problems with your primary email address.
• Tell reviewers before the review starts that it is important for them to reply to you to acknowledge receipt of each of the email message you send to them so that you can be kept posted on the ongoing progress and have a full control of the process.
• Conduct a test-run before the review process starts to ensure all the email addresses provided by the reviewers on their application form is accurate, updated, and that the recruited reviewer has not dropped out.
• Clearly lay out the review process to your team members so that they know what to expect and what they are expected to do at different stages of the review process.
• Be precise, clear, brief and straight forward when giving instructions. Avoid vague, redundant, misleading words and phrases.
• Be quick in communicating changes and responding to reviewers’ queries. Check your email at least twice a day during the first 4 days, and every 6 hours during the last 3 days. Reply to every email within 12 hours.
• Pick the right time to send your email. Timing is very important in effective communication. If an important email message is sent out too early, the recipients may forget it. If sent too late, the recipients may not have enough time to respond, to prepare or execute the command.
• Be efficient in responding to the queries. If more than 3 reviewers report the same problem, email the answer to the whole team. If condition permits, set up a Wiki site or online forum and let all reviewers post their questions and answers there. But again, make sure the true identity of reviewers should not be revealed and the questions should be limited to technical ones and should not involve specific details of the abstracts being read and graded.
• Be polite, friendly and personal in communicating with your team members. Don’t cram all your team members email addresses into the “To” box when you try to send them an email. Put them into the “BCC” box instead so that the recipients will only see your email address alone in the “From” box. This will make them feel this is one-to-one communication and thus feel personal and warm.
  
  

IV. Technology Concern

To successfully complete the proposal review, a good review team leader should form a strong partnership and maintain close contact with the Precis tech support team. However, the most efficient way to use the Precis tech support team is not to encourage every reviewer to communicate directly with the tech team members, but rather to identify the most common problems early, work with the tech support team to find out a solution, and then send the solution to everyone on your team. So it is very important for the review team leader herself/himself to understand the nature of the problem and has the solution to the problem. This will save time, reduce confusion and therefore increase efficiency.

Here are a couple of common problems many reviewers are likely to come across during the review:

1. Log-in problems when reviewers try to log on into the Precis system

Reviewers are often confused about which email address they should use when trying to log on into the Precis system. This happens most frequently to those reviewers who has more than one email addresses but forgets which one or ones they put on their reviewer’s application form. If they put only one email address on their application form, email them and remind them of the email address they gave you. Tell them this is the one they should use to create an account. If they put two email addresses on their application form, tell them which one they should use. They should use the one you picked and entered into the Precis system. Reviewers must understand that the email address they use to log on into the Precis system must match the one you, the review team leader, entered into the system. Otherwisw thaey are unable to log into the Precis system. As to the password they should use to log into the Precis system, it doesn’t matter which one they choose so long as they can remember it. So they can pick any password they like when they create their account, but the email address should be the same one they provided on their application form. And they have to make a note of it and store it in a safe place just in case they forget it in the future. Once they forget their password, they have to create another account and email you, the review team leader so that you can delete her/his old account, and add a new account i nthe Precis system. This will certainly increase your workload. Therefore it is necessary to tell your reviewers before they create their account that it is very important for them to take a note of their password and put it in a safe place just in case they will forget it later.

2. Switching to the reviewers’ page from the submitters’ page

By default, when reviewers successfully log on into the Precis system, they automatically land on the submitter’s page. This happens because the Precis system is set up for both submitters and reviewers. Since submitters by far outnumber reviewers, so the system is set up first of all for the convenience of the submitters. That’s why anyone who enters the system first land on the submitter’s page.

Tell your team members not to worry. Find a button on the black menu bar that runs across the page which is titled “Close” (This may sound weird since you don’t want to close and exit), click it, and they will be taken to a new page, the reviewers’ page, where they can start to read and grade abstracts.

The above are a few points I summed up from my own experience in leading EFL-IS’s proposal review team last year. Due to the difference in IS contexts and IS leaders’ individual preferences and styles, what worked with me may not necessarily work with you, but I do hope they can be of some assistance to you in leading your proposal review team.

I wish everyone of you good luck with your proposal review!

(This is a speech I made at TESOL's Interest Section Leaders Planning Meeting which was sponsored by TESOL Central Office and TESOL's Interest Section Leadership Council on Saturday, March 28, 2008)