Setting a concise, thorough, and fair question paper for any exam is a daunting task. Not only does the paper setter has to adhere to the weightage and syllabus guidelines, but they also have to make sure that there are no errors, whatsoever. They have to create the question paper such that there is no repetition of questions in every exam session.
Once the paper is set, the moderators check the question papers, and then the final draft is sent for printing.
Even with so many people involved, there have been various occasions where the errors in question papers have not only led to unfair evaluation but also have generated a ruckus. The recent CBSE question paper controversy is one such example.
The Central Board of Secondary Education (CBSE) is one of the most reputed education offices in the nation and has millions of educational institutions affiliated with it. With a mammoth workforce and unparalleled resources, the Board has certainly no dearth of moderators and paper evaluation resources.
Still another controversy regarding Class X and XII question papers has raised quite a few eyebrows in the world of education.
The Class XII question paper had a question – “The unprecedented scale and spread of anti-Muslim violence in Gujarat in 2002 took place under which government?”
And, the Class X English question paper had a comprehension passage that received flak because of being “retrograde” and offensive in nature to women.
Both these question paper controversies highlight some of the core pain points of the traditional question paper setting process – human errors, cognitive bias, and improper moderation and review.
However, these are not the only pain points of the traditional question paper setting process. It is tedious, time and effort consuming, and strenuous for the educators as well.
Here, we discuss some of the best solutions to avoid errors in question paper setting process and how using smart and innovative technologies can change the exam paper setting process altogether.
Question Paper Setting Process: Pain Points and Their Effects
The traditional question paper setting process is tedious and time-consuming. It relies entirely on the potential of the paper setters and their diligence in following the by-laws while setting a new paper in every exam session. After all, there are too many factors to consider when you set an exam paper. Missing a few important points or being biased can be easier than intended.
While the paper setters are generally experts with a rich experience in education, they tend to become biased towards specific topics. They might also end up choosing similar or identical questions multiple times because of the mundane nature of the paper-setting process. They might also end up missing the questions that have a high level of complexity but at the same time might contribute to factual knowledge of the students.
Finally, the lack of readiness, diligence, and the required understanding of the sociological and holistic effect of certain questions related to sensitive topics is also another area of concern with far-reaching consequences.
So, we can all agree that the traditional paper setting process is no longer relevant in its entirety and needs revamping. Also, it needs to utilize the emerging trends in education and EdTech to stay futuristic.
But, before we examine the solutions, it is important to understand the impact of errors in the question paper on the other stakeholders of the education industry – the students.
The students get demotivated and might also get carried away during an exam when they encounter a wrong question or any controversial question.
This can affect their exam performance as well.
Likewise, when the exam controller or governing body, like CBSE, opts for awarding full marks for a controversial or wrong question, the move affects the overall grading, leading to unfair results.
There are many scenarios where the marks obtained by a candidate in specific exams are of extreme importance and are counted for the next step in academics. For example, the board marks are considered for calculating the final merit of entrance exams, like JEE.
In such cases, everybody getting extra credit just because a paper moderator was unable to do their job properly, becomes a consequential event for the other candidates.
So, how to tackle these pain points, and what is the most effortless and seamless solution for ensuring error-free and bias-free paper setting?
We explore a set of the most convincing and effective solutions in the following section.
Tips to Avoid Errors in Question Paper Setting Process?
1. Third-Party Review Committee
Most exam bodies, such as CBSE, have a specific moderator or group of moderators to ensure the integrity, fairness, correctness, and the overall adherence of a question paper to the paper setting by-laws.
But, the errors can creep in without much ado, as evident from the recent controversy.
Hence, it is important to have a third-party review committee that offers credible and thorough reviews of every question paper.
Now you might ask – “Why third-party?”, and the answer is simple:
Third-party review committees are bias-free, more aware of the job responsibilities, and review the paper without any specific organizational interest.
Hence, the papers are always reviewed with a fair perspective and a thorough understanding of the by-laws.
2. Two-Level Checking of Question Paper and More Effective Moderation
Adding another expert review after a third-party review will solve two pain points:
- It will ensure the effectiveness and thoroughness of the third-party review
- It will allow the incorporation of the suggested changes and corrections from the third-party review committee easier with an expert team presiding over the changes.
This two-level checking will also make the moderation process stricter and more thorough that will certainly fill any wiggle-room for errors in the question paper.
Apart from having multiple review committees and iterations, it is important to have more effective moderation as well. This can be done by broadening the problem statements and the scope of use cases for moderation.
The moderators should not only look at the common mistakes but also process the presented information in comprehension topics. This will ensure that no public sentiment and human emotions are hurt.
Avoiding religious controversies, political and gender bias while setting a paper is a must!
Finally, the moderators must have a specific set of by-laws to adhere to and must be given the right to remove such questions from the paper after proper presentation before the committee members.
3. Creating a Pool of Questions and Answers and Then Compiling the Question Paper
The traditional paper setting process calls for the paper setters to compile the entire paper keeping the syllabus, topics, weightage of specific units, and organizational by-laws in mind.
However, this suffers from a severe flaw – the paper comes out as what the teacher wants it to be!
And, this can make the papers redundant as the paper setters have to simply rely on their memory and overall understanding of the subject.
A perfect resolution for this problematic issue is to ask every paper setter for questions instead of an entire paper. They can formulate questions and indicate weightage against them.
So, the moderators can have a pool of questions to choose from and then compile the paper with the correct and apt questions.
This will also allow the paper setters to experiment with questions and look for innovative ways to draft questions.
4. Use Technology – The Paper Creator Tools
EdTech trends are revolutionizing the world of education all over the globe and smart question paper creation solutions, like PrepAI, are one of them.
PrepAI is a smart AI-powered question generation platform that helps educators and paper setters to set bias-free question papers from academic content that can be chosen from various sources within minutes. The paper setters can create different types of questions, such as MCQs, True/False, Descriptive Questions, and Fill-ups.
Another plus point is the option to set the complexity level of the exam as Easy, Medium, and Hard for multiple choice questions.
With no monotonous academic content and no strenuous compiling of questions, the PrepAI platform makes the job of paper setters easier and better.
As the paper setters can create a pool of questions of different complexity levels, they can easily compile different question papers for each candidate. Further, the platform ensures a fair and bias-free paper setting, even with different questions in each paper.
Hence, it is a smart and highly intuitive software that solves a number of pain points in the question paper setting.
Also, CBSE has now announced the new paper scheme where the papers will be MCQ-based and will be held twice a year. So, smart AI-powered question paper generators like PrepAI are going to be a bankable solution for paper setters in the near future.
5. Using Open Educational Sources
The open educational sources such as Google and Open Educational Resources (OER) from UNESCO will allow the paper setters to find new and relevant academic content that is not controversial, and regressive or demeaning to any human.
Using these sources for compiling papers will also allow them to keep the student evaluation standards high and relevant to the emerging trends in employment and education.
Apart from helping the paper setters in enhancing their skills and knowledge, the open resources of education and academic content also prepare students with a more global understanding of various events, perceptions, and educational content.
6. Keeping the Exam Scope and Recent Societal Trends in Mind
While setting a question paper, it is important to ensure that the scope of the question paper and syllabus is strictly followed. For instance, CBSE could have easily avoided the recent controversy, if the by-laws strictly prohibited the use of sensitive political and ethnic topics or mentioned them as “out of the exam scope”.
Also, it is important to keep the recent societal trends in mind while drafting a question paper. It is a must to steer clear of all the topics that are regressive and offensive in nature to women.
It is alright for all of us to have our own perceptions and understanding of racial or ethnic or religious groups. But, subjecting the students to our perceptions, especially when they are fighting for every single mark, is not at all advisable.
General Rules for Setting a Question Paper
Setting a good question paper is a skill that can be enhanced by relying on technology. Though the specific guidelines are personalized for each educational institution, there are a few general rules you should follow to create high-quality question papers.
Design the Exam Framework
Before you start setting exam papers and collecting questions, think about the framework for the test. This includes questions like what type of test it is (summative, formative, certification, semester, annual, etc.), the possible outcomes (how many students are likely to pass the test and what’s the ideal pass percentage), what skills, knowledge, and competencies to test (theoretical, practical, or a combination), etc.
The process of designing the exam framework can further be categorized into the following steps:
- Define the exam construct by using a standard (CBSE, SSC, ICSE, etc.) that acts as the benchmark for the test paper’s quality.
- Detail the elements of the framework, such as sub-domains (for content), learning outcomes, student cognitive levels, key indicators, and so on.
- Review and revise the framework to ensure the domains represent the standards and the learning outcomes are distributed across all cognitive levels.
For example, the questions should not be so difficult that only a small section of students can clear the test unless it is the defined benchmark (this is mostly used for nationwide competitive exams).
Create a Blueprint of the Test Paper
Another rule to follow for question paper setting is creating a blueprint that contains all the relevant information about the test. Simply put, this blueprint is a guide that lists the subject topics, the types of questions, the distribution of marks across sections, the learning outcomes you want, and the indicators/ skills that should be measured.
The blueprint is not fixed but can be edited as you prepare the question paper. You may have to adjust the marks distribution, remove or add some topics, etc., to get the desired outcome. For example, the exam setter might reduce short answer questions from fifteen to ten and add a new section for three five-mark questions where students should provide answers in a page or less.
Add Questions that Align with the Blueprint and Framework
Once the framework and blueprint are ready, the exam setter can collect and add questions to the test paper. This can be done manually or using a question generation platform like PrepAI. Here, it’s important to follow a checklist for language, syllabus, etc., to avoid errors.
- Use clear language and simple terminology.
- Don’t include unwanted or irrelevant information in the question.
- Avoid bias in sentence formation (don’t hint what kind of answers students should write).
- Keep the vocabulary content and age-appropriate.
- Take the questions only from the approved syllabus and recommended source material.
- Cross-check that the statements made are accurate and factual.
At this point, exam setters should also align the question with the marks allotted to it. For example, a short answer question carries fewer marks and should not require a multi-page answer.
Validate the Questions and Make Corrections
The validation step should not be skipped, as this is where exam paper setters can spot errors and rectify/ remove them. Depending on the type of test, the validation can be done by other teachers, the regulatory board/ authorities, or even a selected batch of students. Sometimes, a combination of validation methods ensures the approved question paper meets the framework and benchmarked standards. Using a tool like a question generator from text, you can conduct sample tests online and get feedback.
At this point, the scoring/ grading methods should also be validated and reviewed. Is the question paper inclusive? Does it test a diverse range of learners irrespective of their learning styles? Are there any duplicate and similar-sounding questions? Is there any ambiguity in the question paper? After validation, necessary corrections have to be made to the question paper. This step will take less time if you use a question answer generator tool.
Leverage the Advanced Technology: Give Paper Setting a Much-Needed Make Over
Technological disruptions are rampant in all sectors, including the education industry. And, in order to ensure that the education stays relevant to the upcoming employment demands and business market, it is important for the stakeholders to leverage technology for giving a make-over to the education and academic processes.
Weeding out the lopsided topics, modernizing the instruction medium in all the educational institutions, leveraging the internet, open educational sources, and improving the evaluation process for a bias-free and fair judgment – there is immense scope in the education industry!
Embracing AI-powered smart solutions like PrepAI is one such crucial step that can revolutionize the paper setting process, make it better, fairer, and free of errors.
How PrepAI Helps in Question Paper Setting
The purpose of question paper generators is simple. The tool creates question papers based on the input instructions. PrepAI does the same using NLP (Natural Language Processing) algorithms to accurately read the input content and convert it into different types of questions. Copy-paste the text from more than one source directly into the tool’s textbox or onto a document. Save the document as Word or PDF and upload the file to PrepAI. You can also convert videos into questions by uploading a video file or providing the link to YouTube.
Select the question types you want in the test paper along with the number of questions and click on the Generate Questions button. Naturally, you may want to edit a few questions or delete a few others. With this AI question answer generator, customizing the test paper to meet your requirements and eliminating errors is easy. Each question can be edited (as well as the answer). Manually add more questions if necessary by clicking on the Add Questions button in the left panel.
Furthermore, you can generate questions that test the higher order thinking skills like analysis, evaluation, and creation using PrepAI as it is supported by Bloom’s Taxonomy framework. With this, you can not only reduce the risk of errors when setting a question paper but can also measure a range of students’ skills using a standardized framework.
To kick-start the transformation, and to take the first step towards embracing the technology for improving education, get in touch with our team to schedule a demo today!
