Note - this page is no longer being maintained, and has been superseded by the Shell Centre Publications website (www.mathshell.com).
Assessing Problem Solving: Characteristics Of Student Performance On Paper-Based And Computer-Based Tasks
Malcolm Swan, Alan Bell
Given the wide-ranging set of tasks described in the paper Domain Frameworks in Mathematics and Problem solving, it is of interest to identify what general student competencies are evoked by these tasks. This paper describes some characteristics of student performance on the Problem-solving element of the World Class Tests of Mathematics and ‘Problem Solving in Mathematics, Science and Technology.
Assessment Tools for Implementing the Standards
Hugh Burkhardt, 2010
Hugh Burkhardt's talk on Assessment Tools for Implementing the Standards, with experiences from the Shell Centre at the conference on Curriculum Design, Development, and Implementation in an Era of Common Core State Standards Washington, DC, August 1-3 2010
Computers in Mathematics Assessment
Daniel Pead (2007,2010)
Computers have the potential to present a wide range of mathematics assessment tasks to students, but the complications of responding to tasks via a keyboard and mouse present many difficulties, as does the analysis of the responses obtained. Furthermore, few projects will have free reign to reinvent the curriculum or make drastic changes in the style of tasks used. This work details recent research and development undertaken at the Mathematics Assessment Resource Service, University of Nottingham, and focusses on three different computer-based assessment projects.
Summary paper (2007) - PDF (840K)
D. Pead PhD thesis On Computer-Based Assessment of Mathematics (2010) and live examples
Domain Frameworks in Mathematics and Problem Solving
Alan Bell, Hugh Burkhardt
In many fields there is an essential complementarity between the analytic and the holistic – for example in music, between the rules of melody and harmony and musical compositions. In assessment, the holistic aspect is represented by the assessment tasks themselves, which provide students with “the opportunity to show what they know, understand and can do”; the complementary analytic framework is provided by the specification of the domain of performance to be assessed. While MARS sees the richness of the task set as the key factor in the quality of an assessment, the domain framework is essential for the explanation of the assessment, and for the balancing of the tests. The challenges in designing such a framework are substantial, when you move beyond short technical exercises assessing knowledge and skills to the assessment of substantial reasoning involving higher level skills.
Firmer Foundations for Policy Making
Hugh Burkhardt (2008)
This paper outlines a development in evidence-based policy making that will yield outcomes closer to intentions in education and, perhaps, some other policy areas. For known or predictable challenges, the approach offers ministers a choice of well- developed solutions that have been shown to work well; these can replace often-hurried responses that are inevitably speculative and thus unreliable. The key new weapon is a programme of inexpensive, small-scale developments using the kind of “engineering research” methodology that is standard in successful research-based fields.
High-stakes Examinations that Support Student Learning
Paul Black, Hugh Burkhardt, Phil Daro, Glenda Lappan, Daniel
Pead, and Max Stephens
for the ISDDE Working Group on Examinations and Policy
The recommendations in this paper arise from meetings of this Working Group of ISDDE, the International Society for Design and Development in Education. The group brought together high-level international expertise in assessment design. It tackled issues that are central to policy makers looking for tests that, at reasonable cost, deliver valid, reliable assessments of students’ performance in mathematics – with results that inform students, teachers, and school systems. This paper describes the analysis and recommendations from the group, with references that provide further detail. It is designed to contribute to the conversation on “how to do better”.
There are currently two versions of this paper, each addressed to the two US consortia developing assessments for the new Common Core State Standards. A version intended for a more general, international audience is in preparation for a forthcoming edition of Educational Designer.
Recommendations for the design, development and implementation of the PARCC assessments (PDF,270K)
Recommendations for the design, development and implementation of the SBAC assessments (PDF, 270K)
Making Functional Mathematics Happen
Hugh Burkhardt, Alan Bell, Daniel Pead and Malcolm Swan, 2006
These comments are in response to the following recent QCA documents:
- Functional skill subject definitions and On Developing Functional Skills Qualifications;
- Functional Mathematics Standards, along with those for ICT and English;
- Ken Boston's recent speech on Mathematics to ACME.
We will focus on Functional Mathematics (FM), with which we have been creatively involved for more than two decades of research, development and classroom practice.
Styles of Research: Insight and Impact
Hugh Burkhardt
In this paper I look at the roles of different approaches to research in improving the performance of education systems. I compare the approaches characteristic of different traditions – the humanities, the sciences, engineering and the arts – all of which are recognisable in Education. I suggest that, if impact on the quality of education provided to most children, rather than just insight into it, is to become a primary research goal, the engineering approach needs greater emphasis in the balance of research effort, and research credit. Such a shift is likely to have other positive effects. The different characteristics of research, and the roles of strong and weak ‘theory’ are discussed.