The Art of Assessment

One of the biggest challenges that faces modern society is in selecting individuals for an extremely small group of available positions from a large number of interested applicants. A similar challenge is also faced when evaluating or assessing people for the purpose of determining that a certain learning objective has been achieved.

I was pleasantly surprised when I co-incidentally found in a matter of a week how 3 different organizations had managed to meaningfully solve the problem in an effective, impartial and exceedingly fair way.

The 3 organizations were the Indian Institute of Technology (IIT’s), BITS (Birla Institute of Technology) and ToI Group (Times of India). And all 3 were for completely different purposes. The IIT’s was for their entrance exam to the undergraduate program. BITS was for their assessment for their post graduate distance education examinations, and the ToI was for selection as trainee journalists.

The IIT’s have over 500,000 students applying for under 5000 open category seats. And they need to have a way to select the best and brightest. To make it fair it needs to be objective and not dependent on a human examiner, who is likely to have personal biases and convert responses based on subjective interpretations. When the difference between students is minuscule this minor aberration is likely to be unfair to some.  The solution is an MCQ examination. And the level of the questions is such that the difference in marks at the highest level could be as much as 20+ or 5%. This means that ability and knowledge start splaying a far larger role than luck. The IIT’s have gone several steps ahead in terms of transparency and fairness. The students are allowed to take away a carbon copy of their answer sheets. The answer key is published on the IIT site. And to top it all, in what I think is a first in the world, the answer papers along with the assigned marks will be published online. Giving every candidate an opportunity to verify that no errors have been made and to rectify them, should they have been made.
http://www.jee.iitb.ac.in/

The BITS postgraduate distance education courses have a slightly different challenge. How does one assess that the students, many of them working professionals on whether they have adequately learnt and understood the course. Since the expectation is not to know things by rote, but to have the ability to understand they conduct an open book exam. And the questions are on the application of the lessons learnt. As an example, in “Algorithms” the question could be about what the end result would be if a combination of defined algorithms were used. The books can provide clues, but if you haven’t understood each of the algorithms in the question, the books in the given time will be unable to help you solve the problem. This approach ensures that the evaluations and assessment methodology ensures that only students who have understood and learnt go to the next level.
http://www.bits-pilani.ac.in/index.aspx

The Times of India group recently wanted some junior editors. And used what seemed like a simple test. All applicants were required to submit an original article on one of a few given topics. These were evaluated by the editorial team and those selected were then asked to answer a test at their offices. The paper was interestingly created. It had several articles to be written on the spot, with themes as varied as interviews to obituaries. The next levels were Group Discussions and then Personal Interviews. The complete assessment was based on the ability to do and demonstrate that which would be required by the job profile.
http://timesofindia.indiatimes.com/

What’s interesting about all the 3 processes was that they were simple, transparent and ensured fairness to all participants. The processes implemented ensured that “If you weren’t  good, you definitely didn’t make it.”. More importantly, “If you were good the probability of you making it, was as close to 100% as it get”.