Evaluating
a Foreign Language Program;
TESOL
as a Context
by:
Mohammed Alkhamali
2013
Evaluating a foreign
language program might take different shapes according to its objectives. The program
evaluation should touch upon the three types of learning, especially for some
audiences. In other words, it should consider the cognitive, affective and
psychomotor domains. Breaking these domains down into smaller components, shows
that TESOL program evaluation will take into consideration different issues
related to the aforementioned domains. Moreover, such evaluation should follow
the basic criteria like reliability and validity. This paper discusses the
foreign language evaluation in a TESOL context. It tries to answer two related
questions:
-
What are the things that we evaluate?
-
Why do we evaluate the program?
Evaluating a foreign
language program results affect the whole teaching / learning processes. The
evaluation can affect the program inputs, the students, the teachers, the
content; it also can affect the processes of the teaching / the learning as the
methods and activities. It has an effect on the outputs as well.
Before answering the
aforementioned questions, some related terms will be defined.
Terminologies
In this section, some
terms are being defined. These terms either related directly to the discussion
or indirectly as they parts of the TESOL context.
-
Evaluation: MacDonald (as
cited in Wise, 2008) Evaluation is the process of conceiving, obtaining
and communicating information for the guidance of educational decision-making
with regard to a specified program.
Genesee F. defined evaluation in TESOL settings as "a process of collecting, analyzing and interpreting information about teaching and learning in order to make informed decisions that enhance student achievement and the success of educational program."
Evaluation is the systematic collection of information about activities, characteristics, and outcomes of projects to make judgments about the project, improve effectiveness, and/or inform decisions about future programming (adapted from Patton, 1987, as cited in NOAA). Evaluation is not merely the accumulation and summary of data and information about a project. Importantly, evaluation provides managers with well-documented and considered evidence to support the decision-making process.
Genesee F. defined evaluation in TESOL settings as "a process of collecting, analyzing and interpreting information about teaching and learning in order to make informed decisions that enhance student achievement and the success of educational program."
Evaluation is the systematic collection of information about activities, characteristics, and outcomes of projects to make judgments about the project, improve effectiveness, and/or inform decisions about future programming (adapted from Patton, 1987, as cited in NOAA). Evaluation is not merely the accumulation and summary of data and information about a project. Importantly, evaluation provides managers with well-documented and considered evidence to support the decision-making process.
-
Assessment: Brindley G. defined assessment as "an umbrella term
encompassing measurement instruments administered on a one-off basis such as
tests, as well as qualitative methods of monitoring and recoding student
learning such as observation, simulation or project work."
He also distinguished evaluation from assessment as that evaluation is concerned with the overall language program while assessment is concerned with what individual students have learnt.
He also distinguished evaluation from assessment as that evaluation is concerned with the overall language program while assessment is concerned with what individual students have learnt.
-
Tests: is "one of the procedures that can be used to
assess child's performance. A test has a certain objective, for example, to see
to what extent a child understand a written text." (Loannou and Pavlou,
2003, p.4, as cited in socyberty.com). The aforementioned definitions show that testing
is one of many instruments that help us to answer many questions that are
related to the teaching/learning process.
-
Cognitive domain: involves knowledge and the development of intellectual
skills. There are six major categories starting from the simplest behavior to
the most complex as following Knowledge, Comprehension, Application, Analysis,
Synthesis and Evaluation (nwlink.com).
-
Affective domain: includes the manner in which we deal with things emotionally, such as
feelings, values, appreciation, enthusiasms, motivations, and attitudes. The
five major categories are listed from the simplest behavior to the most
complex: Receiving Phenomena, Responding to Phenomena, Valuing, Organization
and Internalizing values (nwlink.com).
-
Psychomotor domain: includes physical movement, coordination, and use of
the motor-skill areas. The seven major categories are listed from the simplest
behavior to the most complex: Perception, Set, Guided Response, Mechanism,
Complex Overt Response, Adaptation and Origination (nwlink.com).
-
Reliability: extent to which there are
consistent results on independent measurements.
-
Validity: extent to which factor of
interest is successfully captured.
What
are the things that we evaluate? and Why?
This
section presents things that we are after in evaluating a program in relation
to the three domains as following:
1-
The objectives: objectives are the end
that we all work to achieve. They need a high concentration while sitting them
as all of us start their work in any educational processes based on objectives
to achieve them. When evaluating a foreign language program, we try to find out
whether this program and its settings are suitable to achieve the set ends. The
program objectives should be evaluated in terms of being; specific, measured,
achieved, realistic and timed. The objectives should take into account the
students' natural and abilities, the society religion, culture and
expectations, job market demands, learning, teaching and technological
development ... The program objectives should concentrate on enhancing the
students' higher levels of the three learning domains.
2-
The content: It includes textbook, methods and activities, and assessments.
2.1- Textbook: When evaluating the textbook we focus on the sequencing and grading of units and subjects whether they are logical or not. The evaluation of textbook should take into consideration the objectives of the program whether they could be achieved by such content or not. The evaluation should pay attention to what extent that the textbook fulfills students' needs and interests. The textbook should be evaluated in terms of the extent that it develops the students' abilities in critical thinking and creativity. The textbook should be evaluated about how it facilitates adaptation and origination of the language in new language use situations.
2.2- Methods: in terms of teaching methodologies, evaluators need to know which methods are more effective. Knowing the affective methods helps in designing the program. It also helps in suggesting these methods for other teachers. Evaluating methods and strategies can be the answer of the students' laziness and hesitancy. Methods should be evaluated in relation to their appropriateness to the students' age and level. The methods should also take into account integrating all skills and areas of the language within the same lesson. The same should be considered about the used teaching activities. The activities should be related to the given lesson and it should facilitate it. Teaching activities, especially the ones that are designed by the teacher, should engage all of the students. It should also take into account the students' differences and needs.
2.3- Assessment: evaluators need to evaluate assessment tools for many reasons. Assessment tools are the direct and, in the most times, documented means of measuring both teachers' and students' performance and achievement. Assessment tools are important for both teachers and students in terms of satisfaction. The evaluators of the program need to make sure that appropriate tools are used in the correct way. The evaluators also need to confirm that these tools are reliable and valid.
2.1- Textbook: When evaluating the textbook we focus on the sequencing and grading of units and subjects whether they are logical or not. The evaluation of textbook should take into consideration the objectives of the program whether they could be achieved by such content or not. The evaluation should pay attention to what extent that the textbook fulfills students' needs and interests. The textbook should be evaluated in terms of the extent that it develops the students' abilities in critical thinking and creativity. The textbook should be evaluated about how it facilitates adaptation and origination of the language in new language use situations.
2.2- Methods: in terms of teaching methodologies, evaluators need to know which methods are more effective. Knowing the affective methods helps in designing the program. It also helps in suggesting these methods for other teachers. Evaluating methods and strategies can be the answer of the students' laziness and hesitancy. Methods should be evaluated in relation to their appropriateness to the students' age and level. The methods should also take into account integrating all skills and areas of the language within the same lesson. The same should be considered about the used teaching activities. The activities should be related to the given lesson and it should facilitate it. Teaching activities, especially the ones that are designed by the teacher, should engage all of the students. It should also take into account the students' differences and needs.
2.3- Assessment: evaluators need to evaluate assessment tools for many reasons. Assessment tools are the direct and, in the most times, documented means of measuring both teachers' and students' performance and achievement. Assessment tools are important for both teachers and students in terms of satisfaction. The evaluators of the program need to make sure that appropriate tools are used in the correct way. The evaluators also need to confirm that these tools are reliable and valid.
3-
The Teachers: One of the essential roles in any TESOL
program is the role of the teachers. They need to be judged to determine their
proficiency, ability in managing class, capability in facilitate learning. In
certain phase of the learning / teaching process, the teachers themselves are
considered as evaluators. Therefore, as program evaluators, we need to evaluate
the teachers' abilities in planning, implementing and evaluating the
assessments within their sphere.
Santiago P. and Benavides F. (2009) names two major purposes for teachers evaluation as following.
I - The Improvement purpose: identifying teachers' strengths and weaknesses, for further improvement in their professional development. This purpose helps teachers to learn about, reflect on and improve their practice." It seeks to set incentives for teachers to perform at their best".
II - The accountability purpose: making sure that the teachers do their best to enhance students' learning.
The evaluators should also consider teachers' training as it plays a significant role in keeping teachers up-to-date. The evaluators should also account peer teachers evaluation as a prime resource of information. The evaluation should include something about the abilities of the teachers in using technologies especially in the classroom settings.
while evaluating teachers, evaluators should use variety of tools to guarantee reliable results. Some of these tools are observations, tests, workshops, self evaluation, peer evaluation, students' evaluation and feedback about teacher and his teaching style... One element that must be emphasized in teachers evaluation is that their ability in helping the students in establishing self-learning, learners' autonomy.
Santiago P. and Benavides F. (2009) names two major purposes for teachers evaluation as following.
I - The Improvement purpose: identifying teachers' strengths and weaknesses, for further improvement in their professional development. This purpose helps teachers to learn about, reflect on and improve their practice." It seeks to set incentives for teachers to perform at their best".
II - The accountability purpose: making sure that the teachers do their best to enhance students' learning.
The evaluators should also consider teachers' training as it plays a significant role in keeping teachers up-to-date. The evaluators should also account peer teachers evaluation as a prime resource of information. The evaluation should include something about the abilities of the teachers in using technologies especially in the classroom settings.
while evaluating teachers, evaluators should use variety of tools to guarantee reliable results. Some of these tools are observations, tests, workshops, self evaluation, peer evaluation, students' evaluation and feedback about teacher and his teaching style... One element that must be emphasized in teachers evaluation is that their ability in helping the students in establishing self-learning, learners' autonomy.
4-
The Students: they are the center of
the teaching / learning process. All the steps of planning, designing,
implementing and evaluating any language programs consider the students as the
core of the processes. In case of evaluating students within the circle of the
program evaluation, there are some components should be accounted. These
components are related to the learning domains. For cognitive domain,
evaluation should more concentrate on how the program enhances the students'
abilities in the higher-level stages of this domain. Higher-level stages in
cognitive domain; analysis, synthesis and evaluation; help in developing the
students' capabilities in critical thinking and creativity. In terms of
affective domain, the evaluation has to develop the concept of responding to
the phenomena that occur in the students' environment. The students should be
active in receiving and responding to surrounded events. The evaluation should
consider, in terms to the same domain, the behavior of commitment towards his
beliefs and society. It should also, the evaluation; pay attention to what
extent the program fulfill the students' needs and interests. In relation to
the third domain, the psychomotor domain, the evaluation should measure the
program efficiency in making the students able to use perception, adaptation and origination of the language.
The evaluation will also provide evidence of the students' learning and of the
ways that affect their learning.
5-
Time: "it is a crucial
factor in any phenomenon". Evaluators should pay attention to the allotted
time; weeks and periods. The given time should be enough for the teachers to
cover the content. It should be also enough for the students to master skills
and behavior.
6-
Funding: another crucial factor in any phenomenon is
the money. Program evaluation helps to ensure that we spend money in the right
way. Evaluation also help in setting budget to other program or to the program
development. Funding evaluation helps in dividing budget according to the
requirements.
7-
The society /
environment: the evaluation has to do with the society or the environment in which
the program takes place. Evaluators need to measure and seek the attitude of
the society towards the program. Does it fulfill the expectations of the
society? Does it cover the defect of languages exchange?
8-
Clarifying program
theory: NOAA
states that "when the project was designed initially, it was based either
explicitly or implicitly on a project theory that explained how things work or
how people learn or even how organizations change. An evaluation asks those
involved to revisit that project theory. Based on experiences with the project
and information taken from research literature, the evaluation provides an opportunity
to revise the project theory. By making the project theory explicit, the
underpinnings of the project and what makes it work will be better understood
and thus, better implemented. Staff members and volunteers who understand why a
particular set of teaching methods was selected or why the project activities
were sequenced the way they were will be more likely to actually follow the
plan. They will also feel more ownership in the project if they understand the
theory behind the project more fully."
Getting results of the foreign language program evaluation is not the end of the process. Evaluation results show that a new phase of work will begin. These results provide feedback to the all dimensions within the program. Either by strengthening pros or handling cons. Evaluation results affect all the dimensions of the program. They affect the planning phase as program planners change, replace or restate plans and objectives. The program designers design and redesign the program in accordance with these results. The teachers get these results as feedback about their teaching performance and about their students learning. Evaluation results help the educational policy makers in tracing and observing the development of the program. Program evaluation shows how the aforementioned components work and how they interact together.
Types of program evaluation
To get accurate results we should use different types of
evaluation as they complete each other and as they play different roles in the
program evaluation. NOAA suggests three types of evaluation that would be
integrated as following.
A)
Front-end evaluation (needs assessment): it takes place prior to undertaking program and it aims to:
- gathering information about the gap between the current and desired level of the students' skills, attitudes, knowledge and behavior.
- helps confirm or negate assumptions of the students characteristics and appropriate content, define goals and objectives and identify stakeholders and potential collaborators.
- gathering information about the gap between the current and desired level of the students' skills, attitudes, knowledge and behavior.
- helps confirm or negate assumptions of the students characteristics and appropriate content, define goals and objectives and identify stakeholders and potential collaborators.
B)
Formative Evaluation:
it is conducted throughout the program, during program design, development and
implementation. It is used to test out methods and materials. It can be used as
a decision making tool. It aims to:
- gathering information about the students' reaction to and learning from the program's pilot or prototype products / materials.
- gathering information about problems with project delivery, and assesses progress towards outcomes of a program during implementation.
- helps provide information that can be used in making decisions about modification, continuation or expansion of the program.
- gathering information about the students' reaction to and learning from the program's pilot or prototype products / materials.
- gathering information about problems with project delivery, and assesses progress towards outcomes of a program during implementation.
- helps provide information that can be used in making decisions about modification, continuation or expansion of the program.
C)
Summative Evaluation:
It
is typically conducted at the end of the project. A formal report is
developed that can be submitted to decision-makers and other stakeholders. It aims to:
- Gathers information about the students' skills, knowledge, attitudes, and behaviors at some point in time after program's implementation begins. Results of a summative evaluation are used to inform stakeholders about the value of a program.
- Informs decision-makers about the value or worth of the project.
- Helps provide the information necessary to make decisions about the continuation, revision, or expansion of the program.
- Gathers information about the students' skills, knowledge, attitudes, and behaviors at some point in time after program's implementation begins. Results of a summative evaluation are used to inform stakeholders about the value of a program.
- Informs decision-makers about the value or worth of the project.
- Helps provide the information necessary to make decisions about the continuation, revision, or expansion of the program.
Conclusion
Evaluation is a systematic way in gathering and interpreting
information. When it comes to evaluate a foreign language program, we need to
take into consideration that evaluation affects the all phases. We should also
consider the components of these phases as we evaluate the program such as the
objectives, the content (the textbook, teaching methods and activities and
assessment), the teachers, the students, time, funding, the society/environment
and to clarify the program theory. These phases and components should be
centered on the students as being the core of educational process. For that
reason the program should be evaluated in terms of fulfilling the students'
needs and interests as it, the program evaluation, takes into consideration the
three learning domains; cognitive, affective and psychomotor. The evaluators
should integrated more than one type while evaluating the program. This helps
in getting full advantage of the program evaluation.
REFERENCES
Brindley
G.
(2001). The Cambridge Guide to Teaching English to Speakers of Other Languages
Genesee F. (2001). The
Cambridge Guide to Teaching English to Speakers of Other Languages
John m. (n.d). Roles and
Responsibilities for Evaluation in Foreign
Language Programs
TUNÇ F. (2010). EVALUATION OF AN
ENGLISH LANGUAGE TEACHING PROGRAM AT A
PUBLIC UNIVERSITY USING CIPP MODEL. A
THESIS SUBMITTED TO THE GRADUATE SCHOOL OF SOCIAL SCIENCES OF MIDDLE EAST
TECHNICAL UNIVERSITY
Wise D. (2008). Curriculum Evaluation.
University of Exeter
No comments:
Post a Comment