The public and the private: pupil entry for National Curriculum Mathematics tests.

Pupil entry for National Curriculum Mathematics tests: the public and private life of teacher assessment Máiréad Dunne Institute of Education University of Sussex

Abstract

This paper brings together data from two research projects concerned with assessment in school mathematics. It is an attempt to combine data and analysis focusing on the selection of pupils for certain mathematical experiences within school classrooms and their subsequent test entry levels. Highlighting the teacher, connections are made between the external system of national testing and internal formative assessment in continual process during schooling. Data from a recent ESRC is used to examine different school practices for pupil test entry. Behind the resulting official statistics of examination performance - the public, lie the everyday practices in the maths classroom - the private. Insights from an ethnographic study of secondary school mathematics classes are used to elucidate some aspects of this private life. In bringing these two elements together this paper highlights both the overt and covert influence that teachers still have on the test results of their pupils.

Introduction

The historical context of the two research studies in this paper is a period in which a conservative agenda, initially sponsored by the New Right was set up to gradually displace liberal progressive educational ideology (Ball, 1990; 1994). The emphasis behind this effort to reconstitute state educational provision has continued into the late 90’s despite a change of national government. The incremental establishment of this conservative educational agenda has been effected by changes in the detail and emphases of teachers’ work, a changing balance of activities and priorities (Brown, 1992) supported by the introduction of a different educational discourse around the problematics of everyday life in schools (Dunne, 1994). The pressures on teachers to make the transition from progressive to conservative are both coercive and obvious in terms of particular innovations and legislation eg the National Curriculum (NC) or the Teachers Pay and Conditions Act 1986, but they are also pervasive and subtle.

The National Curriculum (NC) and Key Stage (KS) Assessment at ages 7, 11, 14 and 16 were introduced through the Education Act 1988. This has led to the establishment of annual national testing of pupils at each of these four Key Stages. School and teacher accountability measures, other features of the educational reform, have given rise to enormous interest in school effectiveness. The publication of school league tables is an important element in the government efforts to raise standards and make schools more effective. With the emphasis on measurable school outcomes, these tables report the relevant KS test results for all state schools. As the teachers in our research studies observed, this focus on the national paper and pen test results has affected their pedagogy and teacher assessment practices. In relation to the latter, the teachers noted a diminished contribution of teacher assessed attainment levels to each pupils final KS result.

In this context, in which teachers appear more distanced from the formal processes of pupil assessment, this paper is concerned with the ways in which teachers still have overt and covert influence on their pupils’ test results. Constraints of space permit only a glossing of some of the most significant elements of this argument which otherwise might be more elaborated and developed. After a brief overview of the two research projects informing this paper, I first look at the implications of particular school practices for pupil entry to the different NC mathematics test levels at KS3. Following this, I examine the ways in which teachers describe their formative assessment practices taking place continually during mathematics classes with their pupils. The focus here will be an analysis of mathematics teachers’ explanations of the processes by which certain pupils are selected for particular classroom experiences and ultimately particular levels of the National Curriculum tests. The final picture that emerges shows how, despite major educational reforms, teachers still have a highly significant influence on the public grading of mathematics performance of their pupils.

Research overview

The data used in the next section derives from an ESRC study concerned with pupil interpretation and performance on KS2&3 NC tests in mathematics (Cooper and Dunne, 1997). At the KS3 level we collected three Nelson Cognitive Ability test scores (CATs) and the 1996 Mathematics National Curriculum test scores of 473 Year 9 children from three secondary schools. We also interviewed 15 teachers, concentrating on the school’s approach to mathematics and on teachers’ perspectives on the NC assessment and their pupils in their schools. The selection of schools was based upon providing a cross social class sample of children and a willingness of the school and mathematics teachers to be involved in the research project.

The second research project was an ethnographic study undertaken from between 1991 - 1994 in four state secondary schools. The data and analysis presented here were obtained from a year of intensive school-based field work, followed by continual periodic contact with four mathematics teachers from each school. The use of formal and informal interviews were central to the data collection. In the larger study a wide variety of data collection methods were employed. The data presented and analysed in this paper derive predominantly from the individual and group interviews with four mathematics teachers. Transcriptions of each of these interviews were circulated to the teachers for comment. The selection of the participating schools was based upon providing a broad range of school contexts, in terms of social class and ethnic mix. All school were co-educational to provide a gender mix. The final selection was made to maximise the representation of different sex and ethnic groups among the teachers. Both the schools and the teachers were volunteers.

Public performance

The tables and figures presented below are derived from a section of a recently completed ESRC project on mathematics assessment (Cooper and Dunne, 1997). In this paper I want to draw attention to school practices in relation to pupil KS3 mathematics test level entry. Decisions about examination entry are informed by mathematics teachers’ assessment of their pupils’ abilities which in turn are mediated by the mathematics department and/or school policy (whether formal or informal) in this regard. There are several factors above and beyond a concern for the individual child that might influence school policy regarding pupil entry. School examination results have implications for school / teacher accountability, the position of the school in the league tables and in the educational market place. Given that a pupils’ NC level is capped at the top of each test’s level range a school’s attitude to risk as well as its expectations of children will affect test entry decisions. Whatever the practice within each context, decisions on test level entry are underpinned and rationalized by teacher assessments of their pupils’ mathematical abilities relevant to the test.

It was evident from this research that schools had different practices in relation to these pupil entry decisions. Taking measured ability as an heuristic baseline, schools differ in their allocation of children with given ability scores to levels of the May 1996 tests. The example of non-verbal ability is shown in Table 1 where it can be seen that the first school ‘requires’ a higher CAT score for entry to the three major test levels than the other two.

	SAT 3-5 taken		SAT 4-6 taken		SAT 5-7 taken		SAT 6-8 taken
	Mean NV CAT score	Count	Mean NV CAT score	Count	Mean NV CAT score	Count	Mean NV CAT score	Count
School
D	94.74	84	106.70	110	113.65	51	120.67	10
E	85.46	39	99.60	50	109.09	11	126.50	2
F	89.30	54	98.31	35	107.25	28	n/a	0

Table 1: Mean non-verbal CAT score by school and tier of entry for May 1996 test It is possible to see the effects of this from another perspective. First a variable is constructed by defining 3-5 entry as 4, 4-6 as 5, 5-7 as 6, and 6-8 as 7. A ratio is then created by dividing SAT level actually achieved by this measure of SAT level taken. The ratio obtained which will give some idea of the differences between the schools in respect of levels of entry. The distributions of this ratio by school, are shown in Table 2

The higher ratios in the first column bear out the tendency of School D to ‘under-enter’ children relative to the others in our sample.

	School D		School E		School F
All pupils	Ratio	Count	Ratio	Count	Ratio	Count
	1.03	255	0.85	102	0.93	117

Table 2: Ratio of SAT level achieved / SAT taken: Means by school

It is possible to push this analysis further through an exploration of the considerable overlap of scores on baskets of common items across neighbouring levels (e.g. 3-5, 4-6) of the May 1996 tests. As an illustration, the distribution of marks on the 61 common items are shown for the two groups entered for 3-5 and 4-6 in Figures 7 and 8. Ten children appear at the right of Figure 7 who were entered for the 3-5 test but who score better than the mean achieved by children entered for the 4-6 test (52.1). . This difficult area of level placement, critical to both school and pupil profiles, raises a threat to valid assessment inherent in the testing arrangements for KS3

Figure 1: Distribution of children’s scores on items common to the May 1996 3-5 and 4-6 tests for children who took the 3-5 tests in May 1996

Figure 2: Distribution of children’s scores on items common to the May 1996 3-5 and 4-6 tests for children who took the 4-6 tests in May 1996

Although they can hardly be expected to get selection for test levels ‘just right’, teacher judgements are key to the limits of their pupils’ possible KS level attainment. It should be noted, however, that these judgements are circumscribed by structures for the national testing from which, on the whole, mathematics teachers are distanced. Mediation at the institutional level, through school and mathematics department policy/practices are processes in which mathematics teachers have more direct but variable influence. Nevertheless, at the fundamental level, in the classroom, teachers clearly have an overt and pivotal role in the entry of their pupils to specific test levels.

Having touched upon some of the problematics of the public life of test entry, I now move, in the next section, to consider the private life of the classroom as the context within which pupils are framed in terms of their mathematical ability and teachers have a more covert role in their pupils’ NC test level entry.

The private life of teacher assessment

An element of the educational reforms was the introduction of systems of school accountability, one net effect of which has been to highlight examination results as a measure of school effectiveness. Interviews with teachers from both projects elicited descriptions of resultant changes in pedagogy and assessment. The overwhelming majority of these teachers reported increased tendency to teach whole class lessons in a formal style, give tests and continually to reinforce basic mathematical operations. They also noted the diminished importance and independence of the Teacher Assessment level recorded for each pupil in the official NC test results. Despite these latter observations concerning the greater circumscription of teachers’ control over the mathematics curriculum and its assessment, there are still significant ways in which teachers make crucial decisions about the school mathematics experiences of individual pupils and the limits of their achievement level in public examinations. The previous section considered this influence in terms of the apparent school policy for examination entry. The rest of this paper will look at the processes of teacher assessment through a preliminary exploration of the social relations of the classroom. At issue here is an attempt to understand how teachers explain their part in grading their pupils. It is important to note here that this not a claim that mathematical ability is only constructed within the confines of the classroom or to suggest that there are no extra-situational components at work in the shaping of individual mathematical achievement.

In the initial phases of this research with teachers it became evident that despite the divergence in their responses to the changing material and ideological conditions of their work, brought on by the educational reforms, there were also implicit continuities. Hidden sets of social relations, part of the covert curriculum, structure the teaching and learning context. The local conditions that frame school teaching and learning settings are assumed across different educational ideologies and often remain unproblematic. As such they represent certain continuities underlying ideological conflicts emergent at different junctures in educational debate. This basic argument is made succinctly by Davies et al (1990) in reference to the most recent educational reforms.

"The broad dichotomy between traditional and progressive education can serve to hide the variability which exists within the framework of progressive education. Forms of progressive education can be class, race and gender biased and depress the performance of working class children, blacks and girls. These biasses intrude from a range of sources - the middle class assumptions upon which schooling itself is predicated, teacher expectations, the impact of hidden curriculum - which operate in both traditional and progressive educational environments. " (Davies et al, 1990: 26)

The evident poor achievement and participation in mathematics of the same particular social groups (Apple, 1992; Dowling, 1991; Ernest, 1991; Dubberley, 1988) despite educational and pedagogical reform lends support to the assertion of fundamental continuities. The assumptions teachers make about pupils and schools are more part of the hidden agenda than the official rhetoric of an educational or political party line. Importantly, these assumptions directly influence the assessments teachers’ make of individual pupils’ mathematical abilities.

In efforts to understand the process of teacher assessment, this study began by exploring how teachers described the ways they made judgements about their pupils’ mathematical capacities. The initial discussions revealed the teachers shared confidence around the way they assess the 'ability' of pupils.

MD: What will happen next year to this group?

R: . . . we're going to have setting within two streams - an A and B stream. For maths a top, middle and lower group in both streams. . . .

MD: Do you already envisage where people in this class will be?

R: If I went through a list I think I'd get 90% right without looking at marks. I haven't given it any thought. But I'd probably be able to say what group they'd be in. . . . I've always felt I've known kids. I've known them well enough and a lot of my assessment is mental, stored in my braincells and not down on paper.

Such confidence in an implicit and largely unarticulated process was also extended in some cases to pupils who were not in the teachers’ mathematics classes

R: . . . Looking at them, just knowing them as people, I would have said that half of the kids that put their hand up were in the lower half of the group.

MD: As you don't teach them what gives you that impression?

R: I would have said if we weren't talking about this, that they are the less able kids within the group.

MD: How do you know?

R: Their written work is not very good, they are not the quickest at thinking if you ask them questions. Making decisions, not very quick at making decisions, not always the best decision. The comments I hear from other staff.

MD: One pupil I talked to felt she could never be good at maths because she was shy.

R: Well, I mean certainly that would be true of somebody whose left my group. You would think that he was very able because he'd talk a lot.

Through critical reflection attempts were made to make explicit those factors that informed teacher assessments.

H: As a teacher of maths . . . the balance of what I do as a teacher changes with different groups. . . . I don't think it's an active decision . . . . It just so happens if individuals talk to you in a particular way, you respond in a particular way so if you think they're being particularly aggressive then you in turn, might be aggressive back and therefore you've automatically got a confrontation situation which means that you're not gonna get as far as you should. Whereas an individual that will ask you a question and just seems to be what a teacher perceives as the way children ask questions, then you answer them and you think this is a good pupil because he's doing what I expect.

In an account of his own experience of classroom social relations from the other side of he desk, albeit in a selective school, Furlong (1991) uses a notion of 'class cultural affinity' to describe how some of his class mates had closer relations with their teachers. "It was a common value system that was both intangible and powerful, producing a bond which transcended the day-to-day conflicts of classroom life." The mathematics teachers recognised this:

H: I've seen pupils in the street some usually say 'Hello!', others just walk past. I think it depends on the individuals or the class. There are some that you do talk about your home especially when there is some common ground between you. There are some individuals that their home life is so alien to what I've experienced it would be so inappropriate. I'm not saying they can't talk to us [teachers] because there is no common ground and I suppose you do say a few things.

Teacher P traces a connection between this cultural affinity and teacher assessment, explaining how such inter-personal relations influence teacher judgements of pupils capabilities.

P: Again they're all labelled aren't they, just depending on where they live or their family background. We do that all the time. I think a lot of the time, even at school, we treat pupils differently because of their background. You can pick out straight away which ones are sort of from a background who's parents are going to support your actions, lets say. . . . And you as a teacher do that straight away. By doing that you've already limited their success in your subject haven't you? I don't know if you do it consciously or unconsciously but if you look I think you do it.

However, as Teacher H describes below, the structurally ascribed power positions of pupils and teachers is not sufficient to describe classroom social relations. Experience inside schools will quickly demonstrate social interactions as a complex of resistance and collusion of pupils and teachers and between them.

MD: So when we go on to that automatic pilot stage what tells you that it's okay to behave in that way?

H: Because you get away with it and there's no one else around to say that that's not acceptable. It's only when you get an adverse response from the pupils that you know that you've done something particularly wrong.

The significance of the pupils in the school and the classroom has been acknowledged within several studies that have focused upon pupil perspectives or responses within social institutions (See for example Mac an Ghaill, 1992; William’s, 1988; Griffin, 1985 and Willis, 1977). The various ways in which the pupils’ identity affects teachers responses are described by the mathematics teachers by focusing upon how gender structures social interactions in their classrooms.

R: I do think there are quite a lot of boys comments and girls comments, hurtful comments and again do the staff help at times? I don't know whether we do. I hope kids realise I'm joking, but that is just as bad. . . . I certainly heighten it rather than letting it sleep. I think they are very different in the way they react and boys get me to react a bit silly. Comment more regularly whereas the girls don't get me like that, they don't make the same comments.

MD: I wonder with people like Sofna for example, if she were a boy, how different would the school or teachers response be to her?

A: Yeah, because she's very demanding yet. . . . If it was a boy I wouldn't let them nag me so much, I'd just say go away. I say go away enough to Sofna, but I'd say it more to a boy.

MD: In a way, talking about her being very demanding, she's actually always on task.

A: Yeah, she's always trying, she tries but yeah, I mean she is always on task. That is amazing, she never really strays off. . .

These teachers’ comments clearly indicate largely unexamined sets of their inter-subject relations, integral to school and classroom routine, that inform the process of teacher assessment. Culturally and contextually specific expectations and codes of behaviour, although highly significant, remain unarticulated and hidden. Other research has focused on some of these factors in relation to mathematics test items (see Cooper and Dunne 1998; Cooper, Dunne and Rodgers, 1997) and school mathematics texts (see Dowling, 1998). In the case here, of the mathematics classroom, the routinisation of teacher pupil interactions, in practice, acts to normalise, de-personalise and de-politicise these processes. Appeals to fairness and professionalism are strategies that distance teachers personally from their assessment decisions, even though they are fundamentally influenced by interpersonal interactions (Avis, 1994: Grace, 1987). Such objectification of a highly interactive arena, camouflages the ways in which classroom social relations constitute teachers’ assessments of their pupils. The routinisation of these assessment activities works to normalise and even neutralise their covert power, rendering it extremely difficult to make the complexity of these relations visible.

The deeply personal effect of what is a routine teacher task is clearly described by two Year 9 pupils talking about being moved down a mathematics set:

MD: So what did you feel like when you had to go [down] to that class?

Andrea: I was really angry. So was Daniella. Well she was in there before so it's worse for her. That means she hasn't done no good during the year, so it's worse. At first I said I wasn't gonna come to school. When I just found out I told Miss, 'How come I'm in that class.' and she didn't say nothing. I don't feel like going to maths anymore. I used to love maths when I used to go to the other group.

####################

MD: You've just been put into that group. Why did that happen?

Harvinder: I don't know. I mean I was doing really okay in Miss Stanton's class. I thought I was really good and I was proud that I'm okay and then when I just went down there, I just felt ashamed of myself and didn't want to live any more. . . . When I found out. Disappointed. . . . Felt stupid. I feel dumb.. . . It's just that the work that we do now we're supposed to do it in the first year. And that's what really disappoints me.

These pupils have been subject to unexplicated judgements about their mathematical capabilities and future performance, with the explicit expectation that they accept these - interpellation. The effect of these experiences is not only immediate and limited to the specific context of the mathematics classroom, it is likely to be carried with that individual through school, into other curriculum areas and beyond. Teacher P recognises the potential effect of her assessments upon her pupils,

P: It just takes one teacher to say that you're stupid and that's it. That's practically your whole life gone. For most people it is. For other people, yes it's something for them to challenge. But there aren't a lot of people that are going to challenge it. For most, l think, it just destroys them there and then and they think 'What can l do now?'

Teacher judgements at the classroom level not only contribute to the official documented mathematics level attained by each pupil, but they also inform decisions about the test tier entry and/or appropriate mathematics class set. The subsequent differential treatment of pupils by teachers is justified predominantly by reference to individualised and essentialised notions of ability (Dowling, 1991; Dubberley, 1988; Ruthven, 1987). The reduction of ability to only a personal attribute is superficial, it diminishes the significance of the classroom as an arena for inter-subjective interaction, and ignores the constitution of an individual pupil's and teacher's subjectivity by relations of, for example, age, gender, class and ethnicity. The personally interactive context within which teacher assessment takes place is depoliticised and depersonalised through a normalisation which conceals the complexities of classroom social relations and the dominance of particular cultural forms. Such reference to individual qualities naturalises and neutralises (O'Loughlin, 1992) the 'cultural affinity' that teachers enjoy with certain pupils and the covert, though perhaps not conspiratorial, power they have over each pupils’ schooling in mathematics. The simple and well recognised deference to a clinical notion of ability conceals the social relations which are the substance of schooling. (Delpit, 1988; Dowling, 1991; Connell et al, 1982). Indeed the dominant cultural codes in schools against which individual behaviour is assessed, are made invisible or neutral.

Conclusion

This paper has attempted to highlight the various ways in which the daily work of mathematics teachers is fundamental to the limits and possibilities of their pupils mathematics education experiences. The increase of external regulation on the teaching profession has coincided with the institution of national testing arrangements that contradictorily, depend upon the teachers’ professional attitude to their work. Indeed, the validity of national KS testing rests on assumptions of such teacher attitudes. Despite the displacement of teacher assessment in favour of national paper and pen tests it is evident that teachers retain powerful influence over the processes integral to national assessment. This influence is overt in the public sphere through pupil test entry and more covert, in the private context of the mathematics classroom. Teachers connect and mediate between the local arena of classroom mathematics and the department, school and national results. Interests in the school outcomes and effectiveness have tended to focus on the public part of these processes of assessment. More research on the hidden structuring of teacher decisions about the mathematical capabilities of their pupils would undoubtedly provide greater understandings of the social relations of the classroom and the assessment process. Such developments would clearly contribute also to associated issues concerned with social justice in education

Acknowledgements

I would like to thank the teachers and children in the ten schools within which these two research projects were located. The data and analysis in the first research project was mainly funded by the ESRC (Project: R000235863, 1995 - 1997). Thanks go to my co-researchers Nicola Rodgers and Dr Barry Cooper. For the demanding task of interview transcription, thanks also go to Iestyn Williams, Sally Jeacock, Beryl Clough, Hayley Kirby and Julia Martin-Woodbridge

References

Apple, M. (1992) Do The Standards Go Far Enough? Power, Policy, and Practice in Mathematics Education. Journal for Research in Mathematics Education. Vol.23 No.5 pp. 413-431.

Avis, J. (1991) Educational practice, professionalism and social relations. In Department of Cultural Studies, University of Birmingham, Education Group II. Education Limited. Schooling, Training and the New Right in England since 1979. London, Unwin Hyman. pp.273 - 291.

Avis, J. (1994) Teacher professionalism: one more time. Educational Review, Vol.46, No.1, pp.63-72.

Ball, S. J. (1990) Politics and Policy Making in Education. Explorations in Policy Sociology. London, Routledge.

Ball, S. J. (1994) Educational Reform. Buckingham, Open University Press.

Brown, M. (1992) Elaborate Nonsense? The muddled tale of Standard Assessment Tasks at Key Stage 3. In C. Gipps (ed) Developing Assessment for the National Curriculum. London, Kogan Page and London University

Connell, R. W., Ashenden, D. J., Kessler, S. and Dowsett, G. W. (1982) Making the Difference. Schools, Families and Social Division. Sydney, George Allen and Unwin.

Cooper, B. and Dunne, M. (1997) Mathematics Assessment at Key Stages Two and Three: Pupils’ Interpretation and Performance. Final Report to the ESRC on Project R000235863 (1995-1997)

Cooper, B. and Dunne,M. (1998) Anyone for Tennis? Social class differences in children’s difficulties with ‘realistic’ mathematics testing. Sociological Review, Vol 46, No.1, pp.115-148.

Cooper,B., Dunne, M. and Rodgers, N. (1997) Social class, gender, item type and performance in national tests of primary school mathematics: some research evidence from England. Paper presented at American Educational Research Association, Chicago, March 1997.

Davies, A. M., Holland, J. and Minhas, R. (1990) Equal Opportunities in the new ERA. Hillcole Paper 2. London, Tufnell Press.

Delpit, L. (1988) The silenced dialogue: power and pedagogy in educating other people's children. Harvard Educational Review, Vol.58, No.3, pp.280-298.

Dowling, P. (1991) A Touch of Class: Ability, Social Class and Intertext in SMP 11-16. In D. Pimm and E. Love (Eds) Teaching and Learning School Mathematics. London, Hodder and Stoughton. pp 137-152.

Dowling, P. (1998) The Sociology of Mathematics Education: Mathematical Myths/Pedagogical Texts. London, The Falmer Press.

Dubberley, W. (1988) Social Class and the Process of Schooling - A Case Study of a Comprehensive School In a Mining Community. In a. Green and S.J. Ball (Eds) Progress and Inequality in Comprehensive Education. London, Routledge.

Dunne, M (1994) The Construction of Ability: A critical exploration of mathematics Teachers’ accounts. Unpublished PhD thesis, University of Birmingham UK.

Dunne, M (1995) Teacher Responses to the restructuring of education: experiences in England. Irish Educational Studies, Vol.14, pp.93-105.

Ernest, P. (1991) The Philosophy of Mathematics Education. Basingstoke, The Falmer Press.

Furlong, V. J. (1991) Disaffected Pupils: the Sociological Perspective. British Journal of Sociology of Education. Vol.12, No.3, pp. 293-307

Grace, G. (1987) Teachers and the state in Britain: a changing relationship. In M. Lawn and G. Grace (eds) Teachers: The Culture and Politics of Work. Lewes, The Falmer Press.

Griffin, C. (1985) Typical Girls? Young Women from School to the Job Market. London, Routledge & Kegan Paul.

Mac an Ghaill, M. (1992) Student Perspectives on Curriculum Innovation and Change in an English Secondary School: An Empirical Study. British Educational Research Journal. Vol.18, No.3, pp.221-234.

O'Loughlin, M. (1992) Rethinking Science Education: Beyond Piagetian Constructivism Toward a Sociocultural Model of Teaching and Learning. Journal of Research in Science Teaching. Vol 29. No 8 pp 791-820

Ruthven, K. (1987) Ability Stereotyping in Mathematics. Educational Studies in Mathematics 18. pp 243-253.

Williams, L. O. (1988) Partial Surrender. Race and Resistance in the Youth Service. London, The Falmer Press.

Willis, P. (1977) Learning to Labour: How Working Class Kids Get Working Class Jobs. London, Saxon House.

Woods, P. (1990) Teacher Skills and Strategies. London, The Falmer Press.