Free Essay

The Intelligent Essay Assessor: Applications to Educational Technology

In: Other Topics

Submitted By ahmethno
Words 4151
Pages 17
The Intelligent Essay Assessor: Applications to Educational Technology
Peter W. Foltz, New Mexico State University
Darrell Laham, Knowledge Analysis Technologies
Thomas K. Landauer, University of Colorado

Abstract
The Intelligent Essay Assessor (IEA) is a set of software tools for scoring the quality of essay content. The IEA uses Latent Semantic Analysis (LSA), which is both a computational model of human knowledge representation and a method for extracting semantic similarity of words and passages from text. Simulations of psycholinguistic phenomena show that LSA reflects similarities of human meaning effectively. To assess essay quality, LSA is first trained on domain-representative text. Then student essays are characterized by LSA representations of the meaning of the words used, and they are compared with essays of known quality in regard to their degree of conceptual relevance and the amount of relevant content. Over many diverse topics, the IEA scores agreed with human experts as accurately as expert scores agreed with each other. Implications are discussed for incorporating automatic essay scoring in more general forms of educational technology.

1. Introduction

While writing is an essential part of the educational process, many instructors find it difficult to incorporate large numbers of writing assignments in their courses due to the effort required to evaluate them. However, the ability to convey information verbally is an important educational achievement in its own right, and one that is not sufficiently well assessed by other kinds of tests. In addition, essay-based testing is thought to encourage a better conceptual understanding of the material and to reflect a deeper, more useful level of knowledge and application by students. Thus grading and commenting on written texts is important not only as an assessment method, but also as a feedback device to help students better learn both content and the skills of thinking and writing. Nevertheless, essays have been neglected in many computer-based assessment applications since there exist few techniques to score essays directly by computer. In this paper we describe a method for performing automated essay scoring of the conceptual content of essays. Based on a statistical approach to analyzing the essays and content information from the domain, the technique can provide scores that prove to be an accurate measure of the quality of essays.

The text analysis underlying the essay grading schemes is based on Latent Semantic Analysis (LSA). Detailed treatments of LSA, both as a theory of aspects of human knowledge acquisition and representation, and as a method for the extraction of semantic content of text are beyond the scope of this article. These topics are fully presented elsewhere (Deerwester, Dumais, Furnas, Landauer, & Harshman, 1990; Landauer & Dumais, 1997; Landauer, Foltz, & Laham, 1998). LSA has been successfully applied towards a number of simulations of cognitive and psycholinguistic phenomena. These simulations have shown that LSA captures a great deal of the similarity of the meaning of words expressed in discourse. For example, LSA can account for the effects of semantic priming (Landauer & Dumais, 1997), can be used to select the appropriate level of text for readers (Wolfe, Schreiner, Rehder, Laham, Foltz, Kintsch, & Landauer, 1998), and can model the effects of the coherence of texts on readers' comprehension (Foltz, Kintsch & Landauer, 1998).

Based on a statistical analysis of a large amount of text (typically thousands to millions of words), LSA derives a high-dimensional semantic space that permits comparisons of the semantic similarity of words and passages. Words and passages are represented as vectors in the space and their similarity is measured by the cosine of their contained angle in the semantic space. The LSA measured similarities have been shown to closely mimic human judgments of meaning similarity and human performance based on such similarity in a variety of ways. For example, after training on about 2,000 pages of English text, it scored as well as average test-takers on the synonym portion of TOEFL—the ETS Test of English as a Foreign Language (Landauer & Dumais, 1997). After training on an introductory psychology textbook, it achieved passing scores on two different multiple-choice exams used in introductory psychology courses (Landauer, Foltz, & Laham, in preparation). This similarity comparison made by LSA is the basis for performing automated scoring of essays through comparing the similarity of meaning between essays.

About the authors...

2. Automated Scoring with LSA

While much of the emphasis on evaluating written work has examined mechanical features, such as grammar, spelling and punctuation, there are other factors involved in writing a good essay. For example at an abstract level, one can distinguish three properties of a student essay that are desirable to assess; the correctness and completeness of its conceptual knowledge, the soundness of arguments that it presents in discussion of issues, and the fluency, elegance, and comprehensibility of its writing. Evaluation of superficial mechanical and syntactical features is fairly easy to separate from the other factors, but the rest—content, argument, comprehensibility, and aesthetic style—are likely to be difficult to distinguish because each influences the other, if only because each depends on the choice of words.

Although previous attempts to develop computational techniques for scoring essays have focused primarily on measures of style (e.g., Page, 1994), indices of content have remained secondary, indirect and superficial. In contrast to earlier approaches, LSA methods concentrate on the conceptual content, the knowledge conveyed in an essay, rather than its style, or even its syntax or argument structure.

To assess the quality of essays, LSA is first trained on domain-representative text. Examples of domain-representative texts include textbooks, articles or samples of writing that a student would encounter during learning in that domain. Based on this training, LSA derives a high-dimensional semantic representation of the information contained in the domain. Words from the text are represented as vectors in this semantic space, with the semantic similarity between words characterized by the cosine of the angle between the vectors for those words. Similarly, student essays can be characterized by LSA vectors based on the combination of all their words. These vectors can then be compared with vectors for essays or for texts of known content quality. The angle between the two vectors represents the degree to which the two essays discuss information in a similar manner. For example, an ungraded essay can be compared to essays that have already been graded. If the angle between two essays is small then those essays should be similar in content. Thus, the semantic or conceptual content of two essays can be compared and a score derived based on their similarity. Note that two essays can be considered to have almost identical content, even if they contain few or none of the same words, as long as they convey the same meaning. This feature of the comparison illustrates that LSA is doing more than just keyword matching; it is matching based on the conceptual content in the essays.

2.1. Evaluating the Effectiveness of Automated Scoring

Based on comparing conceptual content, several techniques have been developed for assessing essays. Details of these techniques have been published elsewhere and summaries of particular results will be provided below. One technique is to compare essays to ones that have been previously graded. A score for each essay is determined by comparing the essay against all previously graded essays. The scores from some number of those pre-graded essays (typically 10) that are most similar to the essay are weighted by their cosine similarity to the essay and used to determine the score for the essay. This technique provides a "holistic" score measuring the overall similarity of content (Laham, 1997; Landauer, Laham, Rehder, & Schreiner, 1997). The score is holistic because the grade for an essay is determined based on how well the overall meaning matches that of previously graded essays. In this way, it is similar to the holistic scoring approach used by human graders who judge each essay as a whole, rather than providing scores on the essay's individual components.

The holistic method has been tested on a large number of essays over a diverse set of topics. The essays have ranged in grade level, from middle school, high school, college to college graduate level essays. The topics have included essays from classes in introductory psychology, biology, history, as well as essays from standardized tests, such as analyses of arguments, and analyses of issues from the Educational Testing Service (ETS) Graduate Management Achievement Test (GMAT). For each of these sets of essays, LSA is first trained on a set of texts related to the domain. Then the content of each of the new essays is compared against the content of a set of pre-graded essays on the topic. In each case, the essays were also graded by at least two course instructors or expert graders, for example professional readers from Educational Testing Service, or other national testing organizations. Across the datasets, LSA's performance produced reliabilities within the range of their comparable inter-rater reliabilities and within the generally accepted guidelines for minimum reliability coefficients. For example, in a set of 188 essays written on the functioning of the human heart, the average correlation between two graders was 0.83, while the correlation of LSA's scores with the graders was 0.80. A summary of the performance of LSA's scoring compared to the grader-to-grader performance across a diverse set of 1205 essays on 12 topics is shown in Figure 1. The results indicate that LSA's reliability in scoring is equivalent to that of human graders.

In a more recent study, the holistic method was used to grade two additional questions from the GMAT standardized test. The performance was compared against two trained ETS graders. For one question, a set of 695 opinion essays, the correlation between the two graders was 0.86, while LSA's correlation with the ETS grades was also 0.86. For the second question, a set of 668 analyses of argument essays, the correlation between the two graders was 0.87, while LSA's correlation to the ETS grades was 0.86. Thus, LSA was able to perform at the same reliability levels as the trained ETS graders.

View a demo hosted at the University of Colorado. (Under Demonstrations, click the Intelligent Essay Assessor link.)

Figure 1. Summary of reliability results (N = 1205 Essays on 12 Diverse Topics)

While the holistic technique relies on comparing essays against a set of pre-graded essays, other techniques have been developed that also effectively characterize the quality of essays. A second technique is to compare essays to an ideal essay, or "gold standard" (Wolfe, et al., 1998). In this case, a teacher can write his or her ideal essay and all student essays are then judged on the basis of how closely they resemble the teacher's essay. In two final techniques, essays can be compared to portions of the original text, or compared to sub-components of texts or essays (Foltz, 1996; Foltz, Britt & Perfetti, 1996). In this final componential approach, individual sentences from a student's essay can be compared against a set of predetermined subtopics or sections of a textbook. This permits the determination of whether an essay sufficiently covers those subtopics. By determining whether a subtopic or section of a textbook is sufficiently covered in an essay, the computer can then provide feedback about which subtopics have been sufficiently understood and/or which sections of the textbook a student might need to review. The LSA derived scores based on this measure of the degree of coverage of subtopics in essays show equivalent correlations with human graders as graders correlate with each other

2.2. Anomalous Essay Checking

While it is important to verify the effectiveness of computer-based essay grading, it is also important that such a grader be able to determine if it cannot grade an essay reliably. Thus, a number of additional techniques have been developed that can detect "anomalous" essays. If an essay is sufficiently different from the essays for which it has been trained, the computer can flag it for human evaluation. The IEA can flag essays that are highly creative, off topic, use unusual syntax, or violate standard formats or structures for essays. In addition, the computer can determine whether any essay is too similar to other essays or to the textbook. The program is thus able to detect different levels of plagiarism. If an essay is detected as being anomalous for any reason, the essay can be automatically forwarded to the instructor for additional evaluation. This permits teachers to remain aware of students who may be having difficulties or are cheating.

3. Experiences in the Classroom: Grading and Feedback

Over the past two years, the Intelligent Essay Assessor has been used in a course in Psycholinguistics at New Mexico State University. Designed as a web-based application, it permits students to submit essays on a particular topic from their web browsers. Within about 20 seconds, students receive feedback with an estimated grade for their essay and a set of questions and statements of additional subtopics that are missing from their essays. Students can revise their essays immediately and resubmit. A demonstration is available at http://psych.nmsu.edu/essay.

To create this system, LSA was trained on a portion (four chapters) of the psycholinguistics textbook used in the class. The holistic grading method was used to provide an overall grade for any essay. In this method, each essay was compared against 40 essays from previous years that had been graded by three different graders. A score was assigned based on the grades of the previous essays to which an essay was most similar. To verify the effectiveness of this approach for providing accurate grades, the average correlation among the three human graders was 0.73 while the average correlation of LSA’s holistic grade to the individual graders was 0.80.

To provide feedback about missing information in each essay, two teaching assistants first identified the seven primary subtopics that should be covered for that essay topic. They then provided example sentences that corresponded to each of the subtopics. These example sentences were then represented in the semantic space. When a student submitted an essay, the individual sentences of the essay were compared to the subtopic sentences. If no sentence in the essay closely matched sentences for a particular subtopic, the computer provided additional feedback in the form of a question or comment to the student about the particular subtopic. For example, in their essays, students needed to describe whether a particular psycholinguistic model was a serial or parallel processing model. If the computer did not detect any mention of this, they would receive feedback asking them " Does the processing occur serially or in parallel?"

Students were permitted to use the system independently to write their essays and were encouraged to revise and resubmit them to the computer until they were satisfied with their grades. After students received their score and feedback, they also received a text box in which they could edit and resubmit their essay for grading. In addition, the web page had links to the class notes so that students could get additional help with writing their essays. The average grade for the students’ first essays was 85 (out of 100). By the last revision, the average grade was 92. Students’ essays all improved from revision to revision, with the improvements in scores ranging from 0 to 33 points over an average of 3 revisions.

An additional trial with a similar system took place in a Boulder, Colorado middle school (Kintsch, Steinhart, Matthews & Lamb, in press). In this system, called Summary Street, students were asked to summarize texts on alternative sources of energy, ancient civilizations, and the functioning of the heart.. Students received feedback about whether they had sufficiently covered subtopics from the texts they were to summarize. They could then revise and resubmit their essays. The results indicated that LSA's scores of the summaries were comparable to those given by the teacher and that LSA performed almost as well as the teachers at detecting which texts were the sources of the students' knowledge

In both the undergraduate and middle school trials, students and teachers enjoyed and valued using the system. A survey of usability and preferences for the system in the psycholinguistics course showed that 98 percent of the students indicated that they would definitely or probably use such a system if it were available for their other classes. Overall, the results show that the IEA is successful at helping students improve the quality of their essays through providing immediate and accurate feedback.

View a demo hosted at the New Mexico State University.

4. Implications of Computer-Based Essay Grading for Education

IEA can be applied to a variety of applications within education. At a minimal level, it can be used as a consistency checker, in which the teacher grades the essays and then the IEA re-grades them and indicates discrepancies between the two grades. Because the IEA is not influenced by fatigue, deadlines, or biases, it can provide a consistent and objective view of the quality of the essays. The IEA can further be used in large scale standardized testing or large classes, by either providing consistency checks or serving as an automatic grader.

At a more interactive level, the IEA can help students improve their writing through assessing and commenting on their essays. The IEA can provide comments regarding missing information, questions that a student should address, or suggestions about where to go back in a text to find the appropriate information. By providing instantaneous feedback about the quality of their essays, as well directed comments, students can use the IEA as a tool to practice writing content-based essays. Due to a lack of sufficient teachers and aides in many large section courses, writing assignments and essay exams have often been replaced by multiple choice questions. The IEA permits students to receive writing practice without requiring all essays to be evaluated by instructors. Because the IEA's evaluations are immediate, students can receive feedback and make multiple revisions over the course of one session. This overall approach is consistent with the goals of "Writing Across the Curriculum" because it makes it easier for faculty outside of English or the humanities to introduce more writing assignments in their courses. Thus, the IEA can serve to improve learning through writing.

Finally, the IEA can be integrated with other software tools for education. In most software for distance education, tools for administering writing assignments are often neglected in favor of tools for creating and grading multiple choice exams. The IEA permits writing to be a more central focus. For example, in web-based training systems, the IEA can be incorporated as an additional module, which would permit teachers to add writing assignments for their web courses. Essays can be evaluated on a secure server and scores and comments can be returned directly either to the students or to the teachers. Textbook supplements can similarly use the IEA for automated study guides. At the end of chapters, students can be asked to write essay questions addressing topics covered in the chapter. Based on an analysis of their essays, the software can suggest sections of the textbook that the students need to review before they take their exams.

5. Conclusions

To summarize, the IEA is first trained on some domain representative text such as a textbook, samples of writings, or even a large number (300 or more) of essays on a topic. Second, the IEA is provided with one or more pre-graded essays on the topic to be graded. Once these pieces of information are processed, the IEA provides accurate characterizations of the quality of essays on the same topic. Domain representative texts are easily available in electronic form, and essays can now be easily converted to or directly entered by students in electronic form. Thus, it is highly feasible to develop automated essay graders for a large number of topics across many domains. Although the IEA requires a large amount of computer processing power, it can be run from a centralized server, allowing students to access it from any web browser.

The IEA is currently available as a web-based grading service through Knowledge Analysis Technologies, LLC (KATech). Organizations wishing to have automated scoring of or evaluative comments added to essays can work with KATech personnel to have essay graders developed for their particular needs. The IEA is not sold as stand-alone software due to the fact that the hardware requirements are beyond the capabilities of standard desktop systems and some expertise is involved in the development of the semantic spaces. Instead, the graders are set up on secure KATech web servers which permit students and teachers to submit essays and receive back scores and comments. This approach further permits instructional designers to incorporate essay grading into their existing web-based software by just adding a link which forwards essays to the essay grader server. The servers can return scores and comments within just a few seconds. Demonstrations of the IEA and information about having graders developed may be found at: www.knowledge-technologies.com.

The Intelligent Essay Assessor presents a novel technique for assessing the quality of content knowledge expressed in essays. It permits the automatic scoring of short essays that would be used in any kind of content-based courses. Based on evaluations of the IEA over a wide range of essay topics and levels, the IEA proves as reliable at evaluating essay quality as human graders. Its scores are also highly consistent and objective. While the IEA is not designed for assessing creative writing, it can detect whether an essay is sufficiently different and, therefore, should be graded by the instructor. The IEA can be applied for both distance education and for training in the classroom. In both cases it may be used for assessing students’ content knowledge as well as providing feedback to students to help improve their learning. Overall, the IEA permits increasing the number of writing assignments without overly increasing the grading load on the teacher. Because writing is a critical component in helping students acquire both better content knowledge and better critical thinking skills, the IEA can serve as an effective tool for increasing students' exposure to writing.

6. Acknowledgements

The web site http://www.knowledge-technologies.com provides essay scoring demonstrations as well as information on the commercial availability of the IEA for educational software. The web site http://lsa.colorado.edu provides other demonstrations of the essay grading, additional applications of LSA and links to many of the articles cited. This research was supported in part by a contract from the Defense Advanced Research Projects Agency-Computer Aided Education and Training Initiative to Thomas Landauer and Walter Kintsch, a grant from the McDonnell Foundation’s Cognitive Science in Educational Practice program to W. Kintsch, T. Landauer, & G. Fischer, and an NSF KDI Learning and Intelligent System's grant to Peter Foltz and Adrienne Lee.

The "Intelligent Essay Assessor" has a patent pending: Methods for Analysis and Evaluation of the Semantic Content of Writing by P. W. Foltz, D. Laham, T. K. Landauer, W. Kintsch & B. Rehder, held by the University of Colorado.

View more information and demonstrations hosted at Knowledge Technologies.

7. References

Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing By Latent Semantic Analysis. Journal of the American Society For Information Science, 41, 391-407.

Foltz, P. W. (1996) Latent Semantic Analysis for text-based research. Behavior Research Methods, Instruments and Computers. 28(2), 197-202.

Foltz, P. W., Britt, M. A., & Perfetti, C. A. (1996) Reasoning from multiple texts: An automatic analysis of readers' situation models. In G.W. Cottrell (Ed.) Proceedings of the 18th Annual Cognitive Science Conference.(pp. 110-115), Hillsdale, NJ: Lawrence Erlbaum Associates.

Kintsch, E., Steinhart, D., Matthews, C., & Lamb, R (in press) Developing summarization skills through the use of LSA-based feedback. Interactive Learning Environments.

Laham, D. (1997). Automated holistic scoring of the quality of content in directed student essays through Latent Semantic Analysis. Unpublished master’s thesis, University of Colorado, Boulder, Colorado.

Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato's problem: The Latent Semantic Analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104, 211-240.

Landauer, T. K., Foltz, P. W., & Laham, D. (In preparation). Latent Semantic Analysis passes the test: knowledge representation and multiple-choice testing. Manuscript in preparation.

Landauer, T. K, Foltz, P. W. & Laham, D. (1998) An introduction to Latent Semantic Analysis. Discourse Processes, 25, 2&3, 259-284.

Landauer, T. K., Laham, D., Rehder, B., & Schreiner, M. E., (1997). How well can passage meaning be derived without using word order? A comparison of Latent Semantic Analysis and humans. In M. G. Shafto & P. Langley (Eds.), Proceedings of the 19th annual meeting of the Cognitive Science Society (pp. 412-417). Mahwah, NJ: Erlbaum.

Similar Documents

Premium Essay

Automated Grading System Review

...Journal of Information Technology Education Volume 2, 2003 An Overview of Current Research on Automated Essay Grading Salvatore Valenti, Francesca Neri and Alessandro Cucchiarelli DIIGA - Universita’ Politecnica delle Marche, Ancona, Italy valenti@inform.unian.it neri@inform.unian.it alex@inform.unian.it Executive Summary Essays are considered by many researchers as the most useful tool to assess learning outcomes, implying the ability to recall, organize and integrate ideas, the ability to express oneself in writing and the ability to supply merely than identify interpretation and application of data. It is in the measurement of such outcomes, corresponding to the evaluation and synthesis levels of the Bloom’s (1956) taxonomy that the essay questions serve their most useful purpose. One of the difficulties of grading essays is represented by the perceived subjectivity of the grading process. Many researchers claim that the subjective nature of essay assessment leads to variation in grades awarded by different human assessors, which is perceived by students as a great source of unfairness. This issue may be faced through the adoption of automated assessment tools for essays. A system for automated assessment would at least be consistent in the way it scores essays, and enormous cost and time savings could be achieved if the system can be shown to grade essays within the range of those awarded by human assessors. This paper presents an overview of current......

Words: 7241 - Pages: 29

Premium Essay

Business Psycology

...[Pick the date] [Pick the date] Business Psychology Individual Assignment Business Psychology Individual Assignment Prepared by – Dashny Sarvaloganayagan Lecturer – W.M.S. Rambadagalla Unit code – F/601/1027 Batch No – 13 Edexcel No – GI 18931 American College of Higher Education, Kandy Prepared by – Dashny Sarvaloganayagan Lecturer – W.M.S. Rambadagalla Unit code – F/601/1027 Batch No – 13 Edexcel No – GI 18931 American College of Higher Education, Kandy Contents Plagiarism Acknowledgement Introduction 1.1 Major theoretical approaches 1.2 Assess the contribution of a scientific approach to investigating workplace behaviour 1.3 Assess strength & limitations of qualitative & quantitative approaches to understand the workplace behaviour 2.1 Describe the type of individual differences which have been the subject of assessment 2.2 Assess the usefulness of psychometric instruments with particular references to reliability and validity 2.3 Make justified communications for the use of two types of measures of individual differences in making business decisions 3.1 Use the theory to explain human reactions to change 3.2 Make justified recommendations for implementing change in selected organization 3.3 Make......

Words: 13240 - Pages: 53

Premium Essay

Quality Managment

...___________________________________________________________________ 6 Professor Mohamed Zairi, Chairman, Quality Congress Middle East 2 ______________________ 6 Research Papers ______________________________________________________________ 7 TQM and its Implementation in Higher Education of Iran _________________________ S.A. Siadat _____________________________________________________________________ M. Mokhtaripour _________________________________________________________________ R. Hoveida _____________________________________________________________________ 8 8 8 8 Quality: From Where to Where? ___________________________________________ 12 Alan Brown ___________________________________________________________________ 12 The Impact of Educational Quality Models on Schools’ Performance in Dubai ________ 20 Kalthoom Al Balooshi ____________________________________________________________ 20 Wafi Dawood __________________________________________________________________ 20 Management Education and Development in the United Kingdom _________________ 25 Daniel O' Hare _________________________________________________________________ 25 Global Quality Management Systems and the Impact on Service Quality and Brand ___ 35 Jonathan M. A. Ward ____________________________________________________________ 35 Resilience: From Product to Corporate Strategy _______________________________ 42 Mehran Sepehri...

Words: 137918 - Pages: 552

Premium Essay

Mba Applications Essays

...School (MBA) Essays Nancy L. Nolan, Ph.D. Ivy League Admission: 180 Successful Business School (MBA) Essays Nancy L. Nolan, Ph.D. First Edition Magnificent Milestones, Inc., Florida Copyright 2006. Nancy L. Nolan, Ph.D. All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording or by any information storage and retrieval system without written permission from the author, except for the inclusion of brief quotations in a review. Electronic and CD-ROM versions published by: Magnificent Milestones, Inc. Post Office Box 100582 Palm Bay, Florida 32910 www.ivyleagueadmission.com CD ROM Edition 10-digit ISBN 0977376443 13-digit ISBN 9780977376445 PDF Version 10-digit ISBN 0977376494 13-digit ISBN 9780977376490 Printed in the United States of America Disclaimers: (1) This book is a compilation of successful admission essays; it does not claim to be the definitive word on the subject of MBA admission. The opinions expressed are the personal observations of the author based on her own experiences. They are not intended to prejudice any party. Accordingly, the author and publisher do not accept any liability or responsibility for any loss or damage that have been caused, or alleged to have been caused, through the use of information in this book. (2) Admission to business school depends on several factors in addition to a candidate's essays (including......

Words: 89385 - Pages: 358

Free Essay

Empoyment

...susceptible jobs are to computerisation. To assess this, we begin by implementing a novel methodology to estimate the probability of computerisation for 702 detailed occupations, using a Gaussian process classifier. Based on these estimates, we examine expected impacts of future computerisation on US labour market outcomes, with the primary objective of analysing the number of jobs at risk and the relationship between an occupation’s probability of computerisation, wages and educational attainment. According to our estimates, about 47 percent of total US employment is at risk. We further provide evidence that wages and educational attainment exhibit a strong negative relationship with an occupation’s probability of computerisation. Keywords: Occupational Choice, Technological Change, Wage Inequality, Employment, Skill Demand JEL Classification: E24, J24, J31, J62, O33. We thank the Oxford University Engineering Sciences Department and the Oxford Martin Programme on the Impacts of Future Technology for hosting the “Machines and Employment” Workshop. We are indebted to Stuart Armstrong, Nick Bostrom, Eris Chinellato, Mark Cummins, Daniel Dewey, David Dorn, Alex Flint, Claudia Goldin, John Muellbauer, Vincent Mueller, Paul Newman, Seán Ó hÉigeartaigh, Anders Sandberg, Murray Shanahan, and Keith Woolcock for their excellent suggestions. † Oxford Martin School, University of Oxford, Oxford, OX1 1PT, United......

Words: 26582 - Pages: 107

Free Essay

Haha

...Jeremy Harmer how to teach english new edition r ----------------------------------------- ^ - 1 \I with DVD _ "The How to... series is written by teachers and teacher trainers, people who know the reality of the classroom and the support teachers need to get the most out of their students. Our aim is to build teachers' confidence, knowledge and classroom abilities - and inspire them to try out new ideas." Jeremy Harmer, Series Editor How to Teach English is a practical guide for teachers who are at an early stage in their careers and for those studying for the CELTA, Certificate in TESOL and TKT exams. This new edition has been fully revised to reflect recent methodological developments and includes a DVD with clips from actual classes demonstrating good teaching practice a comprehensive glossary of teaching terminology, including terms required for the TKT exam • a new chapter on testing a Task File of photocopiable training tasks www.longman.com/methodology i Jeremy Harmer has taught in Mexico and the UK, and has trained teachers around the world. As well as editing the How to... series of books, he is also the author of the highly acclaimed The Practice of English Language Teaching. Pearson Education Limited Edinburgh Gate Harlow Essex CM20 2JE England and Associated Companies throughout the world. www.longman.com © Pearson Education Limited 2007 All rights reserved; no part of this publication may be reproduced, stored in a retrieval......

Words: 19054 - Pages: 77

Premium Essay

Effect

...STABILITY AND ACADEMIC PERFORMANCE OF STUDENTS IN SENIOR SECONDARY SCHOOL PHYSICS IN CROSS RIVER STATE OF NIGERIA ........................................................................35 MODEL JOB ANALYSIS AND DESCRIPTION FOR PUBLIC RELATIONS PRACTITIONERS IN KENYA..............................46 NOLLYWOOD, NEW COMMUNICATION TECHNOLOGIES AND INDIGENOUS CULTURES IN A GLOBALIZED WORLD: THE NIGERIAN DILEMMA ......................................................................................................................................... 62 2 This Page is deliberately left blank 3 THE EFFECTS OF INQUIRY-BASED AND COMPETITIVE LEARNING STRATEGIES ON ACADEMIC PERFOMANCE OF SENIOR SECONDARY SCHOOL STUDENTS IN PHYSICS AFOLABI, FOLASHADE DEPARTMENT OF TEACHER EDUCATION FACULTY OF EDUCATION UNIVERSITY OF IBADAN, IBADAN OYO STATE, NIGERIA. afolabigrace@yahoo.com +2348033682979 ABSTRACT The study investigated the effects of inquiry-based and competitive learning strategies on academic performance of students’ in physics. Quasi-experimental design was adopted for the study. A purposive sampling technique was used to select 2 schools out of 13-co-educational secondary schools in...

Words: 31554 - Pages: 127

Premium Essay

Paper

...Unit - 1 : Introduction to Human Resource Management Structure of Unit: 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 1.11 1.12 Objectives Introduction Opening Case What is Human Resource Management? Nature of HRM Scope of HRM Objectives of HRM Functions of HRM Role of HRM HRM in the New Millennium Summary Self Assessment Questions Reference Books 1.0 Objectives After studying this unit, you will be able to:      1.1 Understand the basic concepts of human resource management (HRM). Explain what human resource management is and how it relates to the management process. Provide an overview of functions of HRM. Describe how the major roles of HR management are being transformed. Explain the role of HRM in the present millennium. Introduction Human beings are social beings and hardly ever live and work in isolation. We always plan, develop and manage our relations both consciously and unconsciously. The relations are the outcome of our actions and depend to a great extent upon our ability to manage our actions. From childhood each and every individual acquire knowledge and experience on understanding others and how to behave in each and every situations in life. Later we carry forward this learning and understanding in carrying and managing relations at our workplace. The whole context of Human Resource Management revolves around this core matter of managing relations at work place. Since mid 1980’s Human......

Words: 92068 - Pages: 369

Premium Essay

Total Quality Management

...TOTAL QUALITY MANAGEMENT AND SIX SIGMA Edited by Tauseef Aized Total Quality Management and Six Sigma http://dx.doi.org/10.5772/2559 Edited by Tauseef Aized Contributors Aleksandar Vujovic, Zdravko Krivokapic, Jelena Jovanovic, Svante Lifvergren, Bo Bergman, Adela-Eliza Dumitrascu, Anisor Nedelcu, Erika Alves dos Santos, Mithat Zeydan, Gülhan Toğa, Johnson Olabode Adeoti, Andrey Kostogryzov, George Nistratov, Andrey Nistratov, Vidoje Moracanin, Ching-Chow Yang, Ayon Chakraborty, Kay Chuan Tan, Graham Cartwright, John Oakland Published by InTech Janeza Trdine 9, 51000 Rijeka, Croatia Copyright © 2012 InTech All chapters are Open Access distributed under the Creative Commons Attribution 3.0 license, which allows users to download, copy and build upon published articles even for commercial purposes, as long as the author and publisher are properly credited, which ensures maximum dissemination and a wider impact of our publications. After this work has been published by InTech, authors have the right to republish it, in whole or part, in any publication of which they are the author, and to make other personal use of the work. Any republication, referencing or personal use of the work must explicitly identify the original source. Notice Statements and opinions expressed in the chapters are these of the individual contributors and not necessarily those of the editors or publisher. No responsibility is accepted for the accuracy of information contained...

Words: 105584 - Pages: 423

Premium Essay

Paper

...Organizational Behavior, 15e (Robbins/Judge) Chapter 14 Conflict and Negotiation 1) ________ is defined as a process that begins when one party perceives another party has or is about to negatively affect something the first party cares about. A) Problem solving B) Assessment C) Conflict D) Negotiation E) Collective bargaining Answer: C 2) Conservationists have had a perpetual conflict with the government of the United States over the fast and rampant depletion of the earth's natural resources. They argue that the United States must reduce its consumption level significantly to rectify this problem. Which of the following, if true, would most strengthen the conservationists' argument? A) The United States has been conserving forests for several years. B) Most countries have not taken any measures to reduce their consumption levels of natural resources. C) There are several countries that have more requirements for resources than the United States. D) The United States accounts for one-third of the total world resource consumption. E) New resource deposits are constantly being discovered. Answer: D 3) According to the traditional view of conflict, all conflict is ________. A) harmful and must be avoided B) natural and helps generate discussion C) necessary and improves creativity of a group D) healthy and improves productivity E) rational and cannot be avoided Answer: A 4) According to the ________ view of conflict, conflict can be a positive force in a group and some...

Words: 41308 - Pages: 166

Premium Essay

The Gold One

...PART 1 THE STUDY OF HUMAN DEVELOPMENT R esearching the process of human development across cultures provides us with an opportunity to improve the human condition as well as, hopefully, to acquire the knowledge needed to optimize life satisfaction. We therefore begin with an overview of how diverse social science and life science researchers (collectively known as developmentalists) approach the monumental task of studying humans over the course of the life span. Our discussion includes the goals of the scientific community, the recognized framework for studying the life span, what aspects of development warrant extensive examination, and what scientific methods are used to conduct research with humans. Chapter 2 discusses the main developmental theories over the past 100 years, when social scientists, biologists, and chemists focused on studying discrete aspects of human development. Earlier introspective methods about subconscious experience and contemporary measurable evidence about microscopic genetic codes, neurons, and hormones all contribute to our understanding of the human condition. Contemporary researchers are focusing on how to integrate scientific findings and theory from across cultures into a more meaningful whole about human development. CHAPTER 1 Introduction Critical Thinking Questions 1. Developmental change takes place in three fundamental domains: physical, cognitive, and emotional-social. Which domain has been most important for...

Words: 20056 - Pages: 81

Premium Essay

Ldr 531 Final Exam Prep

...LDR/531 isn’t available until Sep. 03 and has 30 questions. LDR 531 FINAL EXAM PREP 1) Mintzberg concluded that managers perform 10 different, highly interrelated roles. Which of the following is one of the broad categories into which these roles might be grouped? A) intrapersonal B) institutional C) decisional D) affective E) reflective 2) Over the past two decades, business schools have added required courses on people skills to many of their curricula. Why have they done this? A. Managers no longer need technical skills in subjects such as economics and accounting to succeed. B. Managers need to understand human behavior if they are to be effective. C. These skills enable managers to effectively lead human resources departments. D. A manager with good interpersonal skills can help create a pleasant workplace 3) Which of the following is best defined as a consciously coordinated social unit, composed of two or more people, which functions on a relatively continuous basis to achieve a common goal or set of goals? A. Party B. Unit C. Community D. Organization 4) Which of the following is not one of the four primary management functions? A) controlling  B) planning  C) staffing  D) organizing  E) leading 5) Determining how tasks are to be grouped is part of which management function? A. Leading B. Planning C. Controlling D. Organizing E. Contemplating 6) Which of the following is least likely to be considered a manager? A. A lieutenant leading an infantry...

Words: 96939 - Pages: 388

Free Essay

Spa & Wellness Industry

...and Gerard Bodeker AMSTERDAM • BOSTON • HEIDELBERG • LONDON • NEWYORK • OXFORD • PARIS • SAN DIEGO • SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO Butterworth-Heinemann is an imprint of Elsevier http://www.nckvietnam.com Butterworth-Heinemann is an imprint of Elsevier Linacre House, Jordan Hill, Oxford OX2 8DP, UK 30 Corporate Drive, Suite 400, Burlington, MA01803, USA First edition 2008 Copyright © 2008 Elsevier Ltd. All rights reserved No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise without the prior written permission of the publisher Permissions may be sought directly from Elsevier’s Science & Technology Rights Department in Oxford, UK: phone ( 44) (0) 1865 843830; fax ( 44) (0) 1865 853333; email: permissions@elsevier.com. Alternatively you can submit your request online by visiting the Elsevier web site at http:/ /elsevier.com/locate/permissions, and selecting Obtaining permission to use Elsevier material Notice No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Library of Congress......

Words: 153860 - Pages: 616

Premium Essay

Paper

...Assistant: Meghan Orvis Vice President, Career and Professional Marketing: Jennifer McAvey Marketing Director: Deborah S. Yarnell Senior Marketing Manager: Erin Coffin Marketing Coordinator: Shanna Gibbs Production Director: Carolyn Miller Production Manager: Andrew Crouth Senior Content Project Manager: Andrea Majot Senior Art Director: Jack Pendleton Cover illustration: Image copyright 2009. Used under license from Shutterstock.com Production Technology Analyst: Tom Stover © 2010 Course Technology, Cengage Learning ALL RIGHTS RESERVED. No part of this work covered by the copyright herein may be reproduced, transmitted, stored, or used in any form or by any means graphic, electronic, or mechanical, including but not limited to photocopying, recording, scanning, digitizing, taping, Web distribution, information networks, or information storage and retrieval systems, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without the prior written permission of the publisher. For product information and technology assistance, contact us at Professional Group Cengage Learning Customer & Sales Support, 1-800-354-9706 For permission to use material from this text or...

Words: 229697 - Pages: 919

Premium Essay

世界是平的

...The World is Flat Thomas L Friedman Kq p K To Matt and Kay and to Ron Kq p K Contents How the World Became Flat One: While I Was Sleeping / 3 Two: The Ten Forces That Flattened the World / 48 Flattener#l. 11/9/89 Flattener #2. 8/9/95 Flattener #3. Work Flow Software Flattener #4. Open-Sourcing Flattener #5. Outsourcing Flattener #6. Offshoring Flattener #7. Supply-Chaining Flattener #8. Insourcing Flattener #9. In-forming Flattener #10. The Steroids Three: The Triple Convergence / 173 Four: The Great Sorting Out / 201 America and the Flat World Five: America and Free Trade / 225 Six: The Untouchables / 237 Seven: The Quiet Crisis / 250 Eight: This Is Not a Test / 276 Developing Countries and the Flat World Nine: The Virgin of Guadalupe / 309 Companies and the Flat World Geopolitics and the Flat World Eleven: The Unflat World / 371 Twelve: The Dell Theory of Conflict Prevention / 414 Conclusion: Imagination Thirteen: 11/9 Versus 9/11 / 441 Acknowledgments I 471 Index I 475 Kq p K :::::How the World Became Flat ::::: ONE While I Was Sleeping Your Highnesses, as Catholic Christians, and princes who love and promote the holy Christian faith, and are enemies of the doctrine of Mahomet, and of all idolatry and heresy, determined to send me, Christopher Columbus, to the above-mentioned countries of India, to see the said princes, people, and territories, and to learn their disposition and the proper method of converting them to......

Words: 170179 - Pages: 681