'reliability analysis' Search Results
Validity and Reliability of Instrument Competencies Framework of Agriculture Educators: The Rasch Measurement Model
agriculture instructors pilot study rasch measurement model validity and reliability...
This study aims to produce empirical evidence of the validity and reliability of instrument items for the competency framework of agricultural teaching staff in Malaysian agricultural vocational colleges. The validity and reliability of the framework were analyzed using Rasch Model Measurement assisted by Winsteps 3.72 software. This research instrument contained 116 items, which was distributed to 30 instructors at the Teluk Intan Agricultural Vocational College, Malaysia. The selection of respondents was made by strata random where the researcher makes the strata of the population according to the percentage and then selects randomly based on the desired percentage. Validity analysis of the instrument was done through four functional testings. For reliability and separation of respondents, it was found that the individual reliability value was very good and acceptable. The results of the item polarity analysis detected no negative value (-) in the Point Measure Correlation value. Item matching analysis found that 11 items had to be dropped as they failed to meet the required conditions. From the analysis on local dependence that determines dependent items based on the standardized residual correlation value, it was discovered that the correlation value for the items used was detected; 13 items need attention. The results of the data analysis checking the functionality of the items suggested that some items should be dropped. The omission of these items has provided evidence that the instrument of competence of agricultural instructors is crucial to have a high level of validity and reliability for use in actual studies.
Measuring Thinking Styles of Pre-service and Early Career Teachers: Validation of a Revised Inventory
early career teachers pre-service teachers thinking styles validation...
The early phase is the decisive period for the teachers’ career. Intellectual styles (particularly thinking styles) play a vital role in teachers’ professional development. With the purpose of measuring thinking styles of teachers at the early phase, this study was designed to validate a revised inventory of thinking styles (TSI-R2) for pre-service and early career teachers. A sample of pre-service teachers (n = 248) were invited to join the pilot study and a sample of teachers with one year work experience (n = 252) participated in the main study. Exploratory factor analysis, confirmatory factor analysis and reliability analysis were performed. The results suggested that with slight modification, TSI-R2 was a reliable and valid instrument to measure thinking styles of pre-service teachers and early career teachers, which has some implications to enhancing teacher education and development.
A Study of Secondary School Students’ Perceptions of Fictional Characters
children’s books children’s literature fictional character identification students’ perceptions...
Fictional characters give literary works a sense of reality. The actions of fictional characters play a crucial role in children's personality development. Young readers who lack critical reading skills are more likely to incorporate fictional characters into their lives because they have a hard time telling reality from fiction. Therefore, we should determine how children perceive fictional characters and teach them that they are imaginary figures. In this way, we can help them approach those characters' actions from an external and critical perspective. This study adopted a qualitative research design (case study) to investigate secondary school students' perceptions of fictional characters. The sample consisted of 45 secondary school students (28 female and 17 male). Data were collected through interviews and document review techniques. Data were analyzed using content analysis. Results showed that participants were more likely to be interested in and identify with characters with appealing personality traits. They had four types of approaches to fictional characters: (1) Wanting to change the storyline depending on what the fictional character goes through, (2) being influenced by them, (3) seeing them as role models, or (4) ignoring them. They wanted to change the storyline, especially when the villain got what he wanted or when the hero or the victim was unhappy, suggesting that they mostly took the protagonist's side (the good guy). While most participants attributed an ontological meaning to anthropomorphic characters, the symbolic meaning became of secondary importance. They were more interested in and identified more with characters with good living conditions and no death experiences.
Data in the Educational and Social Sciences: It’s Time for Some Respect
authentic data examples carrying capacity of data data analysis framework quantifying uncertainty teaching data analysis...
This article introduces the concept of the carrying capacity of data (CCD), defined as an integrated, evaluative judgment of the credibility of specific data-based inferences, informed by quantitative and qualitative analyses, leavened by experience. The sequential process of evaluating the CCD is represented schematically by a framework that can guide data analysis and statistical inference, as well as pedagogy. Aspects of each phase are illustrated with examples. A key initial activity in empirical work is data scrutiny, comprising consideration of data provenance and characteristics, as well as data limitations in light of the context and purpose of the study. Relevant auxiliary information can contribute to evaluating the CCD, as can sensitivity analyses conducted at the modeling stage. It is argued that early courses in statistical methods, and the textbooks they rely on, typically give little emphasis to, or omit entirely, discussion of the importance of data scrutiny in scientific research. This inattention and lack of guided, practical experience leaves students unprepared for the real world of empirical studies. Instructors should both cultivate in their students a true respect for data and engage them in authentic empirical research involving real data, rather than the context-free data to which they are usually exposed.
Influence of Acculturation in Yunnan’s Ethnic Minority College Students on Their Academic Achievement: The Moderating Role of Learning Motivation
ethnic minority acculturation academic achievement learning motivation yunnan...
This study examined the effect of Yunnan’s ethnic minority college students’ acculturation on their academic achievement under the risk of the Matthew effect. Additionally, the role played by learning motivation in the relationship between ethnic minority college students’ acculturation and academic achievement was explored. A total of 403 valid questionnaires were collected from four areas in Yunnan province, China. Consequently, the Acculturation Scale, Academic Achievement Scale, and Learning Motivation Scale were used for measurement materials. These items of scales were evaluated on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). SPSS (statistical package for the social sciences) and AMOS (analysis of a moment structures) softwares were used for data analyses. In addition, items were analyzed through item analysis, confirmatory factor analysis, reliability analysis and regression analysis. These results indicated that ethnic minority college students with low acculturation and learning motivation or high acculturation and low learning motivation can become objects of the Matthew effect. However, this study also observed that in certain students, high acculturation and high learning motivation can prevent the Matthew effect. Thus, high acculturation is crucial for improving academic achievement in ethnic minority college students. A level of high learning motivation is a powerful moderator promoting the academic achievement of students with high acculturation.
Scale of Attitudes Towards Online Formative Assessment: Teacher’ Attitudes during COVID-19 Pandemic
covid-19 pandemic formative assessment online assessment scale development teacher attitude...
Due to the pandemic in many countries, schools were closed in 2020. Therefore, education was suspended, and distance education was started. During the Coronavirus disease (COVID-19) pandemic, teachers gave lessons online in virtual classrooms. In this study, Scale of Attitudes Towards Online Formative Assessment (S-AOFA) for teachers conducting online and distance courses was developed, and the teachers' attitudes were examined with respect to demographic variables. In the study conducted in the mixed-design method, qualitative and quantitative data were collected for the scale development and survey. Data were obtained from 369 teachers (science teacher, mathematics teacher, classroom teacher, and teachers in other fields) working in school in Turkey. S-AOFA was made up of 20 items and two factors as a five-point Likert-type. When the teachers' attitudes towards online formative assessment (AOFA) was examined, it was found that the mean for the factor of "Assessment Systems" was lower than that of the "Assessment Approaches". In addition, the findings revealed that there was no significant difference in the teachers' AOFA in terms of gender and that no significant difference existed in AOFA with respect to the school levels of the teachers (elementary, secondary and high school). Moreover, the results demonstrated that there was a low negative significant relationship between the teachers' AOFA and the number of students in which virtual lessons were given online. Lastly, there was a low level positive significant relationship between the teachers' AOFA and the in-class participation percentages of the students who were taught online in virtual classrooms. S-AOFA could be used by researchers in different studies in future.
Effect of Intrinsic Motivation on Junior High School Students’ Creativity: Mediating Role of Cognitive Flexibility
cognitive flexibility creativity intrinsic motivation...
This research’s aim was to explore the role of cognitive flexibility in mediating the effect of intrinsic motivation on the creativity of junior high school students. A creativity scale, an intrinsic motivation inventory and a cognitive flexibility inventory were utilized to investigate a sample of junior high school students in Shaanxi Province in western China. 765 valid questionnaires were collected and analyzed using the structural equation model (SEM). The SEM analysis showed that intrinsic motivation and cognitive flexibility have significantly positive impacts on junior high school students’ creativity. Furthermore, a full model revealed that cognitive alternatives and cognitive control mediate the relationship intrinsic motivation and creativity, and the mediation effect of cognitive alternatives is significantly greater than that of cognitive control. In general, the current research suggests that the impact of intrinsic motivation on junior high school students’ creativity is mediated by cognitive flexibility, and its results amplify prior scholars’ research results and give educators an inspiration how to cultivate creativity for the middle school students.
Developing Competency Evaluation of Pre-service Science Teachers in Industrial Revolution 4.0: Revealing Pedagogic and Professional Competencies
industrial revolution rasch model teacher competence test development...
This study aimed to develop an instrument to assess science teacher candidates’ professional and pedagogical competence in the Industrial revolution 4.0. The instrument consisted of 30 questions and was used in the main study, which was analyzed using the Rasch model to unravel the reliability and empirical validity. The questions were developed based on a predetermined grid including the skills and literacy aspects used in the industrial revolution 4.0. The 30 revised questions were then tested on 60 students of Science Education at two State universities in Indonesia who were in their third year of study times. The Rasch model was employed to test its reliability and empirical validity that included Wright map analysis, item difficulty level, distractor analysis, and item suitability with the Rasch model. The results showed that the reliability value of the professional aspect was .95 and classified as an excellent category. Moreover, the pedagogic aspect obtained .93 and was classified as a very good category. Only one question was the most difficult and did not fit the Rasch model, while the others already fitted. This reliable and valid instrument is suggested to be useful in assessing pre-service science teachers’ competence.
The Development of Life Skill Education Evaluation Model at Life Skill Training Centre
confirmatory factor analysis evaluation model life skill education...
This research aims to develop the product of the life skill education program (LSEP) which is accurate, credible, and effective. This research used the Plomp model. The model covers the input, process, output, outcome and consists of instrument, scoring guidance, and good or bad criteria. The instruments used in the model are the questionnaire, observation sheet and interview guide. The content validity of the questionnaire and observation sheet was proved by expert judgement and continued by using the Aiken Formula, the construct validity of the questionnaire was proved by the construct validity using the exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). The content validity of the questionnaire and observation sheet involved 9 validity experts. The questionnaire construct was done in two steps; the first step involved 65 students analyzed using EFA and the second step involved 199 students analyzed with CFA. The reliability of the observation sheet and questionnaire was estimated by using the Cronbach Alpha technique. The result of the trial model and its analysis shows that all the instruments are good. LSEP model is tested by involving 15 students in the course and training institute. The result of the trial model shows that the model is effective because the users explains that the model is: a) comprehensive, covering many components and sub component programs such as the input, process, output, outcome, b) practical, simple and easy in usage, c) economical, not needing much cost, energy, and time, also d) supported by valid and reliable data collection instrument.
How Critical Thinking Skills Influence Misconception in Electric Field
critical thinking skills electric field misconception physics learning...
This study aimed to determine the influence of critical thinking skills on misconceptions using a five-tier instrument in mixed-method research. The sampling technique used is simple random sampling. The data collection instrument used a critical thinking skills questionnaire, a misconception test of electric field material, and interviews. Data collection begins with quantitative data, providing a misconception test sheet and a critical thinking skills questionnaire. After that, the researcher took qualitative data in the form of interviews to strengthen data that had been obtained previously. Then from the results of the regression coefficients, there is an influence of critical thinking skills on misconceptions. The descriptive results of critical thinking skills data show that the mean of critical thinking skills is 68.50, which means that students' critical thinking skills are in a good category. Then from the results of the regression coefficients, there is an effect of critical thinking skills on a misconception, with the probability number obtained being significant. The limitations of this study are only to identify and see the impact.
Mathematical Reflective Thinking Process of Prospective Elementary Teachers Review from the Disposition in Numerical Literacy Problems
disposition numeracy problems reflective thinking process...
The purpose of this qualitative research is to analyze the reflective thinking process of prospective elementary teacher in numeracy problems in terms of their mathematical disposition. The subjects in this study were 26 prospective elementary school teachers who had attended elementary mathematics lectures. The focus in this research is the process of mathematical reflective thinking in solving story problems of a two-variable linear equation system in terms of the level of mathematical disposition. The research instrument consisted of a disposition questionnaire, a reflective thinking ability test and the researcher himself as the main instrument. Good mathematical reflective thinking skills are supported by disposition, by constantly monitoring one's own performance, reflecting on one's own performance, reasoning on one's own performance, considering the overall situation, the habit of analyzing the relationship between variables, being flexible in various alternative solutions to mathematical problems and trying to solve mathematical problems. From the results of this study, lecturers can develop learning media, scaffolding, or teaching materials that accommodate different dispositional abilities of prospective teachers that can be used to improve the reflective thinking process.
Design and Study of the Psychometric Properties of a Professors’ Expectations of Virtual University Education Questionary
expectations toward virtual education higher education professors psychometric properties validation...
This work describes the design and validation of a questionnaire to assess the expectations of higher education professors regarding virtual education (CEDVES). The sample included 546 professors, 299 men (54.66%) and 247 women (45.23%), from different scientific disciplines of a university in Chile. The final version consisted of 38 items answered using a five-point Likert scale. Nine factors were identified from the exploratory factor analysis. This configuration accounts for 75% of the variance. The structure of the instrument was studied using confirmatory factor analysis. It was found that nine factors produced a good fit, derived from a hierarchical solution in which all these factors depend on a factor of second general order. Each of the scales, like the general factor, present good indicators of reliability. The analysis indicates that this questionnaire has adequate validation and could be broadly used in higher education.
Development of the 4C Teaching Model to Improve Students’ Mathematical Critical Thinking Skills
mathematical critical thinking skill teaching model the 4c skills...
This study aims to develop a learning model based on 4C skills to improve high school students’ mathematical critical thinking skills. Research & development is the design used in this research by applying Plomp’s development theory which consists of three phases, namely needs analysis, design and implementation, and evaluation. This research was conducted at Madrasah Aliyah Negeri 2 Parepare, South Sulawesi, Indonesia involving five experts, four teachers, and 20 students. Data collection and analysis were carried out qualitatively and quantitatively. Analysis of interviews’ data and observations qualitatively with the thematic analysis technique. We also analyse data from validation sheets, questionnaires, and mathematical essay tests with the help of SPSS 20.00. The results show that students and lecturers need a learning model to improve mathematical critical thinking skills. In addition, other findings stated that the learning model developed was proven to be valid, practical, and effective to be used in general. Developing your learning model can further improve students' learning outcomes and mathematical critical thinking skills because they are designed based on the actual needs and problems. For this reason, a teacher must be able to design his learning model.
Number of Response Options, Reliability, Validity, and Potential Bias in the Use of the Likert Scale Education and Social Science Research: A Literature Review
likert scale literature review potential bias reliability and validity...
This study reviews 60 papers using a Likert scale and published between 2012 – 2021. Screening for literature review uses the PRISMA method. The data analysis technique was carried out through data extraction, then synthesized in a structured manner using the narrative method. To achieve credible research results at the stage of the data collection and data analysis process, a group discussion forum (FGD) was conducted. The findings show that only 10% of studies use a measurement scale with an even answer choice category (4, 6, 8, or 10 choices). In general, (90%) of research uses a measurement instrument that involves a Likert scale with odd response choices (5, 7, 9, or 11) and the most popular researchers use a Likert scale with a total response of 5 points. The use of a rating scale with an odd number of responses of more than five points (especially on a seven-point scale) is the most effective in terms of reliability and validity coefficients, but if the researcher wants to direct respondents to one side, then a scale with an even number of responses (six points) is possible. more suitable. The presence of response bias and central tendency bias can affect the validity and reliability of the use of the Likert scale instrument.
Capturing School Context Characteristics Using the Job Demands-Resources Model: Adaptation and Validation
cross-validation job demands-resources thinking styles in teaching work environments...
Work environments are pivotal to teachers’ professional development. However, effective instruments assessing teachers’ perceived work context are still lacking. The principal aim of the present study was to validate a Work Environments Inventory for Teachers (WEIT) constructed using the Job Demands-Resources (JD-R) model. Two studies were implemented on two independent samples totaling 696 (232 for Study 1 and 464 for Study 2) schoolteachers in Mainland China. Both exploratory and confirmatory factor analyses were employed to cross-validate the WEIT. The external construct validity of the WEIT was estimated through investigating its connection with the Thinking Styles in Teaching Inventory. Findings indicated that the WEIT is an adequate measure to map the characteristics of work environments for schoolteachers. Limitations are discussed and implications for future studies are suggested.
Rethinking the Components of Regulation of Cognition through the Structural Validity of the Meta-Text Test
metacognition performance-based testing regulation of cognition structural validity...
The field of studies in metacognition points to some limitations in the way the construct has traditionally been measured and shows a near absence of performance-based tests. The Meta-Text is a performance-based test recently created to assess components of cognition regulation: planning, monitoring, and judgment. This study presents the first evidence on the structural validity of the Meta-Text, by analyzing its dimensionality and reliability in a sample of 655 Honduran university students. Different models were tested, via item confirmatory factor analysis. The results indicated that the specific factors of planning and monitoring do not hold empirically. The bifactor model containing the general cognition regulation factor and the judgment-specific factor was evaluated as the best model (CFI = .992; NFI = .963; TLI = .991; RMSEA = .021). The reliability of the factors in this model proved to be acceptable (Ω = .701 & .699). The judgment items were well loaded only by the judgment factor, suggesting that the judgment construct may actually be another component of the metacognitive knowledge dimension but having little role in cognition regulation. The results show initial evidence on the structural validity of the Meta-Text and give rise to information previously unidentified by the field which has conceptual implications for theorizing metacognitive components.
Involvement of Teachers, Parents, and School Committees in Improving Scientific Attitudes of Elementary School Students: Application of Rasch Model Analysis
parents rasch model scientific attitude school committee teachers...
This research analyzed the involvement of teachers, parents, and school committees in improving scientific attitudes in science learning using Rasch model analysis. A survey method was used in this quantitative study. Participants in the study were selected using a purposive sample of 174 teachers, parents, and school committees in Sleman and Kebumen Regencies, Indonesia. A questionnaire was used in data collection to determine the involvement of teachers, parents, and school committees in improving scientific attitudes toward science learning. The questionnaires were completed using a Likert scale of 1-4, and the data were then analyzed using the Rasch model. The result showed that all participants were the average logit items (+1.03 logit). The reliability was 0.89, indicating a positive response to improving students' scientific attitudes. The results of the Rasch model analysis suggested that the involvement of parents, teachers, and school committees in improving scientific attitudes differed according to their roles. Each instrument element was analyzed in more detail in the Rasch model. Participants' roles were reflected in the specific involvements of teachers in learning, parents at home with children, and school committees participating in school policy-making.
Knowledge, Skills and Attitude of Pre-Service Mathematics Teachers Towards Higher-Order Thinking Skills
exposure higher-order thinking skills teaching...
This research was aimed to determine the level of knowledge, skills and attitude of pre-service Mathematics teachers in Universiti Pendidikan Sultan Idris (UPSI) towards higher-order thinking skills (HOTS) and whether the elements of HOTS were translated into the teaching courses. The research was also designed to analyze the relationship between the pre-service Mathematics teachers’ exposure to HOTS through the teaching courses and their level of knowledge, skills and attitude towards HOTS. The study utilized a quantitative approach by administration of a survey to a simple random sample of 110 pre-service Mathematics teachers from UPSI. A questionnaire with five different parts was used as the research instrument with a reliability value of .979. The collected data were then analyzed using descriptive and inferential statistics. The descriptive statistics were used to determine the mean scores of the level of knowledge, skills and attitude among the pre-service Mathematics teachers, whereas inferential statistics using Pearson-r correlation was implemented to describe the relationship between the studied variables. The findings demonstrated that the knowledge and skills of pre-service Mathematics teachers towards HOTS were at a moderate level as opposed to their attitude and exposure to HOTS which were found to be at a high level. The data analysis using Pearson correlation illustrated a significant positive relationship with r = .727, r = .757 and r = .667 between the exposure to HOTS through the teaching courses and the level of knowledge, skills and attitude regarding HOTS of pre-service Mathematics teachers at UPSI, respectively. In conclusion, the research here indicates that pre-service Mathematics teachers in general, are positive towards the implementation of HOTS but they are lacking in terms of knowledge and skills. This research is useful in the field of Mathematics education as it can serve as a guidance to further enhance the knowledge, skills and attitudes especially for future Mathematics teachers in implementing HOTS effectively and improving the quality of the teaching program offered by the university indirectly.
Factor Structure and Dimensionality of an Instrument designed to Measure the Metacognitive Orientation of Thai Science Classroom Learning Environments
classroom learning environments metacognition science education...
The purpose of this study was to establish the factor structure and dimensionality of the Metacognitive Orientation Learning Environment Scale – Science (MOLES-S) in the Thai context. The metacognitive orientation of a science classroom learning environment is defined as the extent to which psychosocial conditions that are known to enhance students’ metacognition are evident in a specific science classroom. This study builds on earlier work in the research areas of science education, metacognition, and learning environments. A sample of 5418 Thai science students in grades 10 to 12, from 40 schools across Thailand, completed the MOLES-S that had been translated into Thai. Exploratory factor analysis was undertaken and Rasch analysis was used to calibrate the scale and explore its dimensionality. The results suggest that the MOLES-S(T), where (T) represents Thailand, has the same factor structure as the original MOLES-S, is reliable, and can be used with confidence in research into metacognition in Thai high school science classrooms.
Psychometric Properties of Online Adolescent Anger Instrument
adolescent anger expressions exploratory factor analysis online instrument...
Anger is a topic that requires intervention from teachers, counsellors, psychologists, parents, and all communities. The expressions of anger are subjective and sometimes hard to identify. Thus, anger should be measured more objectively, while the expressions need to be examined closely. The purpose of this study is to provide valid confirmation for development of an online instrument to measure the types of anger expression among adolescents. Data were collected from 935 adolescents from nine schools in northern Malaysia and the theoretical literature search. The data were analysed to provide evidence of construct validity in terms of item factor analysis, reliability estimates, and correlation between the types of anger expressions. Findings were used to develop an online Adolescent Anger Instrument. It measures five types of anger expressions, namely, physical, verbal, intrinsic, extrinsic, and passive. The results showed that the instrument is internally consistent with high evidence of construct validity. Exploratory factor analysis, with varimax rotation, suggested the existence of five distinct types of anger as conceptualised. Meanwhile, the correlation between types of anger expressions indicates the strength of the relationship between them. Discussions on the findings are provided, while suggestions for future research are also described.