Annual Assessment Report for the

Mathematics Department

September, 2000

Introduction

 As part of our continuing assessment program, the Mathematics Department undertook two major initiatives in the school year 1999-2000. We assessed effectiveness of all general education mathematics courses as a general education mathematics courses, and continued to evaluate the effectiveness of our mathematics placement exam. In addition, we assessed the computer literacy of our mathematics majors by administering a competency exam.

Objectives

The Mathematics Department has set specific goals for all mathematics courses which can be used to fulfill Carthage quantitative literacy requirement.  Our objective was to determine the effectiveness of mathematics instruction in all general education courses in achieving these goals.   These courses are Math 103 Applied Mathematics, Math 104 Math For Teachers, Math 105 Functions, Graphs and Analysis, Math 112 Calculus I, and Math 113 Calculus II.  In all, data was collected from 12 sections of these courses:  6 sections of Math 103, 2 sections of Math 104, 1 section of Math 105, 1 section of Math 112, and 2 sections of Math 113.

Method

In past years, the department has experimented with various methods of assessment, including pre- and post-testing, post testing alone, and imbedded questions.  Based on those experiences, the department imbedded questions in the final exams of many sections general education mathematics in the 1999-2000 school year.  The students were not aware that these questions were to be used for assessment purposes.  The results were tabulated by the department chair and will be discussed by the department in the fall of 2000. The results will also be posted on the Mathematics Department web site.  The actual questions are not included in this report because of the public presentation of this document, but can be obtained from the Mathematics Department Chair.  The first question, the Rental Question, involved extracting data from a bar graph, and the second question involved a linear currency conversion.

Findings

 General: The Table 1 summarizes the results of the exam by question.  The numbers indicate the frequency of responses for each option.  The correct answer is indicated with a star. The percentage correct does not include those who did not respond.

Assessment Score Summary By Question


Rental
(a)
(b)
(c)
(d)
(e)
no resp.
% Correct
Part 1
233*
1
0
3
0
1
98.3
Part 2
31
0
22
4
178*
2
75.7
Part 3
13
1
3
12
208*
1
87.8
               
Currency
              
Part 1
5
129*
30
63
10
1
54.4
Part 2
27
181*
11
12
2
5
77.7

Table 1

Below is a question-by-question analysis of the data.

Rental Question, parts 1, 2, and 3: The success indicated by student responses tells us that students are learning how to read bar graphs by the time they finish any mathematics course.  Students did have more problems with part 2 than anticipated.

Currency Question, part 1:  While only 54.4% of students answered this question incorrectly, 39.2% of students seemed to fail to convert their answer to the proper units, causing them to respond with answer "c" or "d".  This may indicate that over 90% of students knew how to do the algebra involved in the unit conversion, but about over 40% of those students were not careful about units of measure.

Currency Question, part 2: Interestingly, the students were able to answer this part of the question correctly.  This is certainly due in part to the fact that no responses were even close to the number obtained if one failed to convert dollars to cents.  However, many students caught their mistake in this question, made the necessary conversion, but failed to go back and correct part 1.

Course Specific Findings:  Some very interesting trends emerged when we examined the data course-by-course and section-by-section.  In the discussion below, the section score is the percentage of students in an individual section that answered a question correctly.  Section scores are reported in Table 2.

Proportion Correct on Assessment Questions By Section


Course
Rental Ques. 1
Rental Ques. 2
Rental Ques. 3
Currency Ques. 1
Currency Ques. 2
Math 103
1.0
.77
.95
.41
.73
Math 103
1.0
.87
.96
.65
.91
Math 103
.89
.56
.67
.56
.78
Math 103
.94
.72
.94
.44
.71
Math 103
1.0
.77
.96
.50
.85
Math 103
1.0
.77
.77
.32
.81
Math 104
1.0
.78
.96
.43
.78
Math 104
1.0
.79
.76
.94
.82
Math 105
.94
.71
.72
.33
.38
Math 112
.95
.74
.84
.95
.74
Math 113
1.0
.73
.93
.87
.87
Math 113
1.0
.71
.94
.94
.82

Table 2

Rental Question, part 1:  No individual section scored lower that 89% on this question, and many sections exhibited 100% correct responses.  The low score was found in a class of 8 students, taught by an instructor who is no longer at Carthage.  Nearly all students can answer questions of this nature by the time they finish their mathematics course.

Rental Question, part 2:  Section scores ranged from 56% to 87%, and all but the high and low scores were between 71% and 78%.  Both the high and low scores were found in sections of Math 103.

Rental Question, part 3:  Section scores ranged from 67%-96%.

Currency Question, part 1:  This was by far the weakest question, with most scores below 50%.  However, a closer look at the responses brought a startling trend to light.  Consider the first two lines of data in Table 3.

A Comparison of Math 103 and Math 112



 
 
 

 
Math 103
Math 112
Percentage of students who answered currency question part 1 correctly
47.5%
92.1%
Percentage of students who probably answered currency question part 1 incorrectly because they didn't convert units
47.5%
7.8%
Percentage of students who answered currency question part 2 correctly
80.5%
80.4%

Table 3

These results seem to indicate that many students in Math 103 failed to convert dollars to cents, and had they made the conversion, they would have most likely answered the question correctly.  The data on part 2 of this question supports that conclusion.

Currency Question, part 2:  Why did so many students answer this question correctly?  As mentioned above, none of the answers  were even close to the number obtained if one failed to convert dollars to cents.  The fact that students in Math 103 scored as well as those in Math 112 is striking, particularly because there was such a significant difference in part 1 of this question.

Computer Assessment of Majors

All twelve senior mathematics majors completed a computer assessment exam which can be viewed at this link.  All twelve completed the exam reasonably well.  Eight completed it perfectly, while one could not plot a graph using Mathematica (minor syntax problems), and three others had never copied a picture out of a web page.  In finding the mean of a long list of numbers, seven students chose to use Mathematica and five chose to use Microsoft Excel to complete the task.  All students obtained the correct number.

Use of Results

The fact that students in all courses were performing at roughly the same level was encouraging.  These results will be shared will all mathematics instructors, and will be discussed at department meetings.  Adjunct instructors will also be included in the discussion.  We are particularly interested in finding out why students in Math 103 failed to go back and correct their answer to part 1 of the currency question when part 2 of that question led them to the conclusion that they needed to convert dollars to sense.  We will continue to use assessment data to improve the delivery of all mathematics courses.

The computer literacy test needs to be revised to obtain significant results. The results we obtained were pleasing, but a new exam should be written to better ascertain weaknesses in the computer literacy of Carthage mathematics majors.

Next Year

In 2000-2001, the mathematics department plans to target classes in the School of Professional Studies its general education assessment as outlined above.  We plan to use the imbedded questions all evening sections of Math 103, 104, 105, and 106.  We may also use the imbedded questions in spring sections of Math 103, 104, 105, and 106 to see if the actions based on the recommendations stated above affect performance in these courses.