Development and Validation of an Inventory to Evaluate Teaching Strategies for Promoting Higher-Order Thinking Skills in the Teaching of Islamic Education

The purpose of this study is to develop and assess the validity and reliability of an inventory on teaching strategies involved in promoting HOTs as well as to identify the most and the least popular teaching strategies used. The inventory is adapted from Gulistan et al. (2016). It is in the form of questionnaire distributed to a sample of 220 primary school teachers teaching Islamic education subject. The content validity is assessed by the experts and the construct validity is measured by Exploratory Factor Analysis. The reliability of the instrument is measured using internal consistence reliability, which is alpha coefficient reliability or Cronbach Alpha. Results of exploratory factor analysis suggested that 5 items need to be removed due to their non-dimensionality as they have more or less equal loadings on several factors. The instrument developed yielded high values of internal consistency as reflected by the Cronbach alpha values. Thus, the final draft of the instrument contains 26 items which are valid and reliable. Even though the validity and reliability of the instrument are within the acceptable range, more data need to be gathered using a bigger sample size, and further analysis using confirmatory factor analysis could be used to explore deeper into the psychometric characteristics of the items before the instrument can be finalized. Furthermore, the findings of the study indicated that the most popular strategy used among teachers teaching Islamic education was the strategy for acquiring knowledge which introduces students more on memorizing basic concepts, while the least used strategy was the higher cognitive skills teaching strategies such as problem solving, collaborative learning and inquiry strategies.


Introduction
Malaysia foresees the importance of education in producing the kind of manpower needed to achieve Vision 2020.By the year 2020, Malaysia is aiming to be a fully industrialized country in its own mould with highly knowledgeable and skilful workers.In order to achieve this, all we have to do is to prepare the country with an education system which provides the most appropriate and high quality curriculum, teachers, delivery system, infrastructure, teaching strategies and assessment system which could meet the present and future demands.The basis for the success of an education system depends on the quality of the National Curriculum which will affect the human capital quality we are aiming for (BPK, 2014).Hence, the Education Ministry is trying to come out with a curriculum which is at par with the international standards focusing in aspects such as creative skills, problem-solving and innovation.The objective of the National Curriculum is to produce a balanced future generations which master the 21 st century skills.Students who master the 21st century skills with the ability to think creatively and critically are able to compete globally.These skills are in line with the six students' aspirations as stated in the Malaysian Education Blueprint, in which each student will have the knowledge, thinking skills, leadership skills, bilingual proficiency, ethics and spirituality and also national identity.
A study conducted by the Academy of Leadership for Higher Studies (AKEPT) found that fifty percent of the teachers observed failed to deliver their lessons effectively especially in their ability to inculcate HOTs (Ministry of Education, 2013).Teachers consequently make erroneous decisions while using various methods in their assessment practices (Shepard, 2000).Haris and Hofer (2009) believe that learning activities influence the daily instructional development, and the planning has to focus on students' standard and curriculum which is related to learning process, the outcome of learning and also the existing technology.So, to produce an effective teaching, a curriculum approach based on technology has to be produced which includes teachers' technique in planning instruction and also their knowledge in planning instruction.Furthermore most teachers, especially new teachers are having problem in planning teaching and fail to manage teaching effectively especially when it comes to teaching HOTs (Nurasyikin, 2016).Teachers feel that when planning teaching, it is difficult to transform it into words to write it in their daily lesson plan book.Some teachers do not understand rational in planning teaching and the use of teaching objective (Orstein & Lasley, 2000).Most senior teacher are focusing more on the content and instructional activity rather than planning for teaching objective.This is quite worrying because teaching objective is an important component in the curriculum as it will give a quite a big impact to the teaching in the classroom.
Previously, Critical and Creative Thinking Skills (CCTS) have been introduced in 1994 through Primary School New Curriculum (KBSR) and Secondary School New Curriculum (KBSM).Primary School Standard Curriculum (KSSR), which is introduced in 2011 is an effort to strengthen thinking skills with more emphasis on reasoning skills.Various thinking skills have been combined for the purpose of making judgments and assessments in problem solving processes.In facing global competition in the economic environment which is driven by innovation, MOE has emphasized HOTs in the school system.By definition, HOTs is an abstract thinking which integrates informational systems and follow rules of logic and judgment (Ivie, 1998).Norman (2009) defines HOTs as a non-algorithmic and complex mode of thinking which could generates various solutions to the proposed problem.In general, there are two types of thinking skills or cognitive skills which are higher and lower-order of thinking skills.If the thinking skills involve acquiring and understanding knowledge, it is called LOTs whereas if it involves applying and evaluating knowledge, it is called HOTs (Ozgelen, 2012).Or, HOTs is the potential use of mind in dealing with the new challenges by understanding, interpreting, analyzing and manipulating information (Anderson et al. (2001).
According to BPK (2014), basically the implementation of HOTs in the Malaysian education system context is using a comprehensive and systematic approach which includes three main elements (curriculum, pedagogy and assessment) together with the supporting elements (co-curriculum, community and private support and also resources and capacity building factors).All these elements are supposed to support each other.HOTs is applied in the school system so that students do not just memorize, but also understand and know what they are learning, and they are using common sense at a high level by mastering skills like evaluating, application, analyzing and innovating.HOTs enable students to apply knowledge, skills and values in making reflection to solve problems, innovating and able to invent something (BPK, 2014).
In improving education, there are many aspects which have to be considered because all aspects are important in improving students learning.It includes assessment, well-prepared teachers, well-designed and coherent curriculum and also a skilful instruction which is adapted to students' needs and personalized learning environments (Darling-Hammond, 2000).Providing students' with these key features of a sound education in instilling HOTs in students is a major foundation of an accountability system.
Teaching strategies play a vital role in enhancing students' acquisition of HOTs (Constantinou & Kuys, 2013).According to Rajendran (2016), there are six strategies which could be used in enhancing thinking; questioning, metacognitive approaches, componential approaches, heuristic based approaches, critical thinking approaches and creative thinking approaches.Some of the characteristics of effective strategies for developing HOTs are activating students' prior knowledge, using classroom activities such as hand-on inquiry, grouping approach or using different forms of assessment such as alternative assessment or evaluation approaches (Gulistan et al., 2015).Miri et al. (2007) suggest that teachers could create an environment to give opportunities to students to explore more about the complex problems or conduct group activities to promote metacognition.Next is on assessment practices related to HOTs.Classroom assessment should support instruction and also increase students' learning (Shepard, 2000).Thus, teachers need to be able to assess the acquisition of thinking skills by their students.However, this is not an easy task (Rajendran, 2016).Why?Because there is no single definition of thinking and the fact that there is no multiple choice tests that effectively tests for higher-order thinking disposition.However, the study by Suah et al. (2009) found that the form of assessment frequently used by school teachers was multiple-choice objective test.This is something to think about seriously by the educationist.Few characteristics have been listed out if teachers were to assess the acquisition of thinking skills in students.First, students are required to develop responses rather than just select predetermined options, to elicit HOTs in addition to basic skills, to evaluate holistic projects directly, to synthesize with classroom instruction, to do portfolios over an extended period of time, to allow for the possibility of multiple peer judgments.
Concerning teaching strategy to teach thinking skills in Islamic education subject, it is improving nowadays.Previously in primary schools, students are taught the lists of words on an ongoing basis to improve their vocabulary and other lexical unit (Rahimi & Normeza, 2014).But, students still fail to remember the words.In general, teachers in Malaysia tend to focus on students mastering the basic contents just preparing for examination rather than focusing on understanding and applying HOTs (Zailani & Dahlan, 2005).Currently, MOE is trying hard to influence teachers to inculcate HOTs by using variety of techniques and activities to enhance the teaching and learning of Arabic language proficiency among students.Teachers could conduct debates, brainstorming, ask students to search for proof or evidence and develop justifications which require students to synthesize or make judgments based on facts (Wahab, 2014).If we were to compare teaching of thinking skills between Malaysia and United Kingdom in general, there are some similarities and some differences as well (Rajendran, 2016).The education system in Malaysia today is actually rooted from the British education system so it still maintains the centralized system and conducting examinations.
Finally, the need to have a valid and reliable instrument to evaluate the implementation of teacher practices in instilling HOTs in Islamic education teaching is becoming increasingly important.When talking about HOTs people use to relate it to science and mathematics subject only.Validity (measuring what it is supposed to measure) and reliability (the extent to which scores are free of measurement error) of the questionnaire are the most important things to consider when dealing with measurement (Barroon & Rahman, 2015;Muijs, 2011).In this study, content validity is checked by the experts in this field.Construct validity is the extent to which a set of items actually reflect the theoretical latent construct those items are designed to measure (Hair et al., 2006) and in this study it is measured using EFA.And, internal reliability is a concept referring to the degree to which all of the items are measuring the same underlying construct (Pallant, 2007) whereby it is measured using Cronbach Alpha value.When a questionnaire is valid and reliable, a researcher will have confidence in the results obtained using those questionnaires during data collection.
The purpose of this study was to develop and assess the validity and reliability of an inventory for assessing teaching strategies used by teachers in the context of teaching HOTs in Islamic education subject.Specifically, this study sought to (1) develop an inventory for evaluating teaching strategies in instilling HOTs in Islamic education subject, (2) establish the validity and reliability of the inventory, and (3) identify the most and the least popular teaching strategies used.

Methodology
This survey was piloted to 220 primary school teachers teaching Islamic education subject.Thirty primary schools in Perak were selected through purposive sampling.Initially, the instrument developed was tried out to 10 teachers to check for the appropriateness of the language and content.Analysis showed that most items are appropriate.However, some minor changes have been made to phrases which are a bit ambiguous.

Development of the instrument
Basically, the instrument is developed based on the three main elements (curriculum, pedagogy and assessment) which are supporting each other in the implementation of HOTs in schools as suggested by the Malaysian government (BPK, 2014).The instrument for teaching strategy is adapted from Gulistan et al. (2016) who developed the inventory known as Strategies Used Survey Questionnaire (SUS-Q) to determine the strategies used by 7 th grade secondary science teachers in teaching science.It consists of 34 items in the form of 5-point Likert scale (1=never to 5= always) based on the constructs of cognitive development.After going through validity and reliability processes, it leaves with 31 items only.The final SUS-Q instrument by Gulistan et al. (2016) consisted of 31 items that were further validated and critiqued by 11 experts in science education, measurement, and evaluation.Then, based on the experts' feedback, minor modifications were made to the questionnaire items.The reliability value of the instrument achieved by Gulistan et al. (2016) was shown in Table 1.The sample for their study was 212 7th grade science teachers in the Iraqi-Kurdistan region.Data were analysed by adopting descriptive and inferential statistics using t-test and one-way ANOVA.Findings of the study indicated that the most popular strategy among the 7th grade science teachers was the strategy for acquiring knowledge which focused more on memorizing basic concepts in science, while the least used strategy by science teachers was the strategy for applying knowledge such as problem solving and hands-on activities.Items in Gulistan's et al. (2016) instrument are organized into three main constructs; i) strategies used for acquiring the knowledge (14 items); ii) strategies used for applying knowledge (8 items); and iii) reflection on knowledge strategies (9 items).The first construct is focusing on lower cognitive skills teaching strategies such as students' memorizing of basic concepts, while the other two constructs are focusing on higher cognitive skills teaching strategies such as problem solving, collaborative learning and inquiry strategies which could initiate students to explore, reflect and share ideas.Why is this instrument chosen for this study?Firstly, it is an instrument to determine strategies used by science teachers to teach HOTs in science education so it is suitable with this study in determining teaching strategies concerning HOTs in Islamic education subject.Secondly, this instrument is developed to suits the schools in Iraqi Kurdistan region which is not a developed country and still in the process of developing its HOTs.What changes have been done?The instrument is adapted to suits this study by referring the Curriculum and Assessment Standard Document for Islamic education subject for Year 4, 5 and 6 and also Administration Guidelines for Islamic education subjects produced by MOE in 2015.Changes are made to suits the teaching of Islamic education subject at primary level.Respondents are given sufficient time to complete the questionnaire since it is not a test, but their perception on the issues only.The newly developed instrument for this study contains Section A on Demographic Profile (5 items) and Section B on Teaching Strategy (31 items).Respondents are given sufficient time to complete the questionnaire since it is not a test, but their perception on the issues only.The questionnaires were then gathered, and then data were analysed concerning characteristics of the items using factor analysis and internal consistency measure in order to validate the inventory.

Findings and Discussion
The findings discussed in this paper are organized around two important aspects, namely validity and reliability of the instruments

Validity of the instrument
Firstly, the validity of the instrument is established.The adequacy of the data is checked using KMO Bartlett test.A statistic value of 0.947 is obtained which indicates that 94.7 percent of the variables properties are explained by the data thus, factor analysis would be meaningful.Then, EFA is performed according to section (except for demography section) to identify the number of constructs and to group the items for each construct.For the 31 items in Section B (Teaching Strategies), EFA has yielded three constructs.Items related to strategies used for acquiring the knowledge are grouped together as one factor, strategies used for applying the knowledge fall into the second factor and items for reflection on knowledge strategies fall into the third factor.However, five items (C7, C10, C14, C15 and C19) are grouped into both constructs with more or less equal loadings and thus are removed from the final instrument.Factor loadings for the remaining items are greater than 0.6.Details of the results are presented in Table 2.

Reliability of the instrument
Analysis of validity using EFA by construct for all sections in the instrument has yielded quite a high reliability measures.After all the 12 items have been removed, the remaining items are checked for reliability.The reliability values are greater than 0.770 for all constructs.Details of reliability index by construct are shown in Table 3.

The most and the least popular teaching strategies used
In order to identify the most popular strategy among primary school Islamic education teachers and which construct they focus on, the item frequency and percentage was computed for each construct as in Table 4.The teachers' responses to the strategy used indicated that Islamic education teachers focus more on teaching students basic concepts by using strategies for acquiring knowledge (M = 39.78 ± 8.23) while focus less on strategy concerning applying knowledge (M = 19.19 ± 4.99) and reflection on knowledge (M = 18.46 ± 4.24).

Conclusions
Analysis of by EFA and internal consistency on the data showed that the instrument seemed to be valid and reliable and could be used to measure the implementation of teaching strategies in promoting HOTs in Islamic education teachings.The result of the analyses suggested that 5 items should be removed from the instrument and thus, 26 items were retained in the final draft of the instrument.Even though the reliability and validity of the final draft of the instrument were within the acceptable range, some of the items suggested to be removed in this analysis could be important to be used.Therefore, a more detailed analysis with a larger sample (>1000) using Item Response Theory (IRT) model needed to be conducted before the instrument can be finalized, if possible.The use of IRT will allow the researchers to explore deeper into the psychometric characteristics of each item, and thus provide them with a higher sense of confidence to keep only important items in the instrument.The results also indicated that the low processing strategies were mostly used by the teachers.The high cognitive skills strategies were not popular among teachers.Thus, these results contributed to the body of knowledge concerning teaching strategies used in promoting HOTs among Islamic education teachers.
This study implied that teachers teaching Islamic education subject should realize their weakness in implementing high cognitive skills teaching strategy regarding HOTs during teaching.They should make an effort to improve their teaching methods and techniques.Teachers can bring in cognitive engagement more into the classroom activities rather than using text books if teachers were to emphasis more on evaluating and analyzing skills (Wenglinsky, 2002).Similarly, teachers could also use cognitive strategies which help students to think reflectively, train them to solve problems and make decisions wisely (Zohar &Schwartzer, 2005).Furthermore, MOE should also come out with more extensive hands-on training and courses which focuses on teaching strategies concerning HOTs in Islamic education teachings.Curriculum design and assessment techniques especially formative assessment should also be improvised in terms of practices if MOE were to ensure that their mission in improving HOTs is developed. http://jurnal.radenfatah.ac.id/index.php/tadib http://jurnal.radenfatah.ac.id/index.php/tadib http://jurnal.radenfatah.ac.id/index.php/tadib http://jurnal.radenfatah.ac.id/index.php/tadib

Table 1 .
Reliability analysis for SUSQ instrument

Table 3 .
Values of overall cronbach's alpha for each construct