Assessing the State and Outcomes of our Education through Direct Evaluations

Over the past several years, Sophia University has focused on measuring and enhancing the quality of its education. In particular, we view the Diploma Policy (DP), which defines the skills a student should have upon graduation, as a “promise to society,” and we strive to ensure that all students achieve the high standards of this promise.

First, each department reviews its curriculum to ensure that its courses offered enable students to achieve the skills in the DP. Between 2019 to 2022, we developed a curriculum map illustrating the relationship between each course and the DP, as well as a curriculum tree that illustrates the map and the difficulty of each course. Next, we conducted/began an indirect evaluation (subjective evaluation) from the students’ perspective to determine whether they felt classes contributing to fulfil the DP were being offered and whether they themselves feel that they have acquired the skills laid out in the DP. Beginning in 2022, we implemented class and student surveys in a standardized form across the entire university.

Following this, we developed the “10 Indicators of Curriculum and Academic Achievement.” As the name implies, this is a multifaceted analysis of the implementation of education and the resulting student performance, based on direct evaluation (objective evaluation) using 10 different indicators.

The University introduced GPA as a means of direct evaluation for the entire university in the 2000s, early among Japanese universities, and has utilized it in designing the admissions system and other areas. The “GPA Subcommittee” established by the Quality Assurance Committee in June 2022 examined ways to utilize GPA to improve the quality and accuracy of education, resulting in the development of 10 indicators beyond GPA.

Priority One: Easy-to-understand and easy-to-use indicators

The 10 Indicators of Curriculum and Academic Achievement are as follows:

Indicator 1: Number of credits earned

Indicator 2: Average number of credits recorded/average number of credits earned

Indicator 3: GPA by grade level

Indicator 4: GPA Distribution of Courses Offered by Departments

Indicator 5: Number of Courses Offered by Department and GPA Required by Elective Category

Indicator 6: Number of courses per numbering system and GPA per level

Indicator 7: GPA scatter diagram for university-wide general studies courses and departmental courses

Indicator 8: Number of courses by DP

Indicator 9: Number of credits earned by DP

Indicator 10: GPA average by DP

The timing of this tally is at graduation. The data is calculated for the entire university, for each faculty, and for each department as a way of understanding how that grade level has learned over the past four years.

For example, Indicator 1: Number of credits earned, showed us that many students earn the maximum number of credits in their first and second years. During their third year, the number of credits earned decreases slightly, and in their fourth year, students have more schedule flexibility, allowing them to engage in other activities and studies. I imagine this means that some students may not have been able to study thoroughly their first and second years given the number of credits they acquired.

Indicator 8: Number of Courses by DP, indicates how many courses contribute to the achievement of DPs 1 through 5, if a given department has five DPs. If there are DPs with extremely few courses linked to another, students will have difficulty fulfilling DPs. It may be necessary to adjust course offerings or redefine DPs so that students can achieve all DPs without difficulty.

Indicator 9: Number of credits earned by DP, shows on average how many credits each student has acquired in each grade by DP. If a DP is biased toward a particular grade level, it will be necessary to place courses in other grades to enhance the ability to fulfil that DP more widely.

In Indicator 10: Average GPA by DP calculates the average GPA of the courses associated with that DP. If there is a large difference between DPs, the grading criteria or the difficulty level of achievement for each DP may not be consistent. The results are displayed in a box plot that shows the median value and variations in the data at a glance.

We spent a lot of time considering how to establish each indicator and how to represent the results with this principle in mind: “make it easy to understand.” We carefully selected ten ideas for a variety of indicators that we thought would be useful for all faculties and departments. We aimed for simple ideas and visuals that would allow anyone to instantly understand what was being tabulated and how, while avoiding statistical terminology and specialized calculation methods.

Resuming the Discussion on Revising the Three Policies

In February 2023, we calculated the data for class that enrolled in 2018. Currently, we have shared data within the University on the three years through the class that enrolled in 2020.

The 10 indicators have become indispensable data for the academic assessments (departmental self-assessment) conducted by each department. Each department is currently reviewing its three policies and revising its curriculum accordingly for the 2027 academic year – the 10 indicators are a vital source of discussion for this series of reviews. In addition to Indicators 8-10, which deal with the DP, analysis of whether the expected type of students are enrolling in the program can also be used as a basis for reviewing the AP (Admission Policy), and the number of courses and credits earned per grade level are used to review the CP (Curriculum Policy).

As a university-wide initiative, the “Subcommittee for Deep Learning and Credit System Enhancement” was established to consider lowering the maximum credits that can be earned during a student’s first and second years. The committee has submitted the proposal to each faculty for discussion, and some departments are already considering implementing this change.

The current 10 indicators analyze specialized courses offered by each faculty and department, but in the future, we are considering focusing our analysis more on university-wide general studies courses. The university-wide general studies courses were renewed in 2022, and analysis by the indicators will provide material for verifying the progress of Sophia University’s educational reform.

At Sophia University, the DP and other educational goals are not merely stated in words, but are verified and substantiated through a mixture of direct and indirect evaluations. We hope you look forward to the continued enhancement of Sophia University’s education.