Composition Forum 35, Spring 2017
http://compositionforum.com/issue/35/

Closing the Loop: Strengthening Disciplinary Writing in an English BA Program

Miranda Wilcox

This program profile narrates how the Department of English at Brigham Young University (BYU) reviewed and revised the disciplinary writing requirements in the English BA program between 2006 and 2015. The story begins in 2006 with the dual problems of recognizing the lack of development in student writing in the major and of responding to accreditation pressure to do outcomes-based assessment on all degree-granting programs. Over the last eight years our department has used assessment tools to make two significant adjustments to solve these problems. First, we aligned the English BA curriculum with program learning outcomes by developing a sequence of three writing courses in which majors explicitly learn the discourse conventions of literary criticism. Next, we designed sequences of writing activities in these courses for students to practice increasingly complex integration of disciplinary knowledge. Now we are using collaboratively designed rubrics to evaluate embedded assignments in these writing courses, a process that helps us assess the impact of these changes on students’ writing development. This profile demonstrates how an English department reflected about the practice of teaching writing and applied methods from outcomes-based assessment to strengthen students’ disciplinary writing skills.

Defining Program Learning Outcomes

In 2006, the Northwest Commission on Colleges and Universities requested that BYU “identify and publish expected learning outcomes for each of its degree programs; demonstrate that students who complete their programs have achieved the stated outcomes; and provide evidence consistently across its programs that its assessment activities lead to improvement of teaching and learning” (Tanner 2). These mandates generated a number of university-wide and department-specific assessment initiatives over the last seven years at BYU, a private, research university, with approximately 30,000 students, sponsored by the Church of Jesus Christ of Latter-day Saints in Provo, Utah. The university has traditionally had a strong undergraduate focus, and administrators embraced this opportunity to “make a virtue of necessity by building an even better house of learning” (Tanner 4).

The Department of English is the largest department in the College of Humanities. In 2006-2007, the English department offered its 1,200 undergraduate majors two bachelors programs: English and English Teaching.{1} The English Teaching program has been closely integrated with the English BA curriculum and will not be discussed separately. Seventy-five percent of the English BA students are female. Approximately thirty-three percent of these students seek full-time employment in the discipline, twenty-five percent seek work outside the discipline, fifteen percent immediately attend graduate school, and the same percentage work at home. Providing writing instruction to such a large number of students with such a range of post-graduate goals has been an ongoing challenge for the English department.

The English department employs almost sixty full-time faculty, more than sixty part-time faculty, and between fifty and sixty graduate student instructors. The faculty specialize in a variety of areas, including, British literature, American literature, Rhetoric and Composition, Creative Writing, Folklore, and English Education. The centrifugal pull toward specialization sometimes challenges a sense of common identity, pedagogical goals, and larger disciplinary aims, particularly with respect to assumptions about and expectations for student writing.

In the spring of 2007, the department executive committee, comprised of the chair and three associate chairs, appointed two faculty members to form an assessment committee. The chair of the committee had some training in assessment through his undergraduate major in English Teaching. At the College of Humanities’ assessment retreat (dubbed Camp Assess-a-lot) in May 2007, the assessment committee and department chair (all tenured American literature specialists) defined six program learning outcomes. Three of the six outcomes concerned writing expectations:

• English graduates will be able to develop and communicate their ideas clearly in writing. This includes sound sentence-level mechanics and style, a clear focus, and cohesive overall development.
• English graduates will be able to identify and evaluate appropriate research resources, incorporate these sources into well-documented formal academic writing, and formulate their own arguments based at least in part on those sources.
• English graduates will write with evident skill in the uses of theory and method.

In 2013, these three learning outcomes were streamlined into one that deals with interpretive and communicative skills: English graduates employ critical reading strategies, disciplinary writing expertise, and sophisticated analytical skills in their written and oral communication.{2}

Evaluating Curricular Alignment with Program Learning Outcomes

Next the assessment committee and department chair identified which required major courses contributed to which learning outcomes using the mapping method described by Mary Allen (43). This mapping was revelatory; most of the courses that introduced students to the skills described in the learning outcomes were taught by part-time faculty. The mapping also revealed misalignment between the writing learning outcomes and writing instruction in the major. The assessment committee realized that they expected students to learn the skills outlined in the first two writing outcomes in ENGL 314 (Writing about Literature), a General Education (GE) advanced writing course that was not required for English majors. English majors had multiple options for fulfilling the general education advanced writing requirement, including taking GE classes titled Writing about the Arts and Humanities, Writing about Literature, Persuasive Writing, or Technical Writing. These courses were primarily taught by part-time faculty not actively publishing in professional venues. In addition, many English students delayed completing the advanced writing requirement until late in the major, thus limiting their ability to develop skills and transfer them to other classes. To compensate, English faculty teaching the major’s gateway course (ENGL 251: Introduction to English Studies) and the three literary history survey classes had been expected to teach students how to write literary criticism; yet, little class time could be devoted to writing instruction in these courses because of their heavy reading loads. In light of these circumstances, the assessment committee realized that it was not surprising that faculty and students were dissatisfied by the quality of writing in English major classes.

Next the assessment committee examined syllabi of courses required for the English BA program. The syllabi revealed disparity in the number, length, and complexity of the writing assignments across the major. Some courses required a significant final research paper, while others required short response papers requiring no research component. It was clear that the assigned tasks did not require increasing analytic or rhetorical complexity as the students progressed through the major. Writing instruction was very uneven.

More evidence of problems with writing instruction in the major was gathered in the fall of 2007. The assessment committee collected samples of student papers from required courses at the beginning, mid-point, and end of the major for groups of faculty to evaluate with a rubric devised from the program learning outcomes. This rubric described five traits that could each be rated along a six-point spectrum from emerging, to developing, and mastering [see the multi-year rubric in the Appendix]. The emerging descriptors characterized the average writing skills of students in 200-level courses, the developing in 300-level courses, and the mastering in 400-level courses. These writing samples confirmed that faculty had widely differing expectations about paper length, compliance with MLA bibliographic format, and incorporation of contextual and theoretical sources. During this process, many faculty admitted that they expected English majors to develop their writing skills in general education freshman writing and advanced writing classes and devoted little class-time for explicit writing instruction.

The assessment committee found that using the department’s multi-year rubric to evaluate samples of student papers from courses across the major was not an effective way to track students’ writing development. It was difficult to evaluate the sample papers without knowing the parameters of the assignment, and there was no mechanism to provide feedback to students or instructors. There were also questions about the long-term sustainability of archiving hundreds of student papers each semester and hiring faculty to spend days reading them.

Similar conclusions were reached when the assessment committee reviewed transcripts of focus group interviews conducted over a two-year period with hundreds of graduating students. Students reported that writing instruction and assignments across the major were uneven. They also noted that the course prerequisites were widely ignored by their peers; the inconsistent preparation among students in higher-level courses often resulted in the need to review basic concepts that they should have acquired in prerequisite courses. Students experienced little discernible increase in conceptual complexity as they progressed through the major.

As department leaders and stakeholders recognized that majors were receiving limited and inconsistent explicit disciplinary writing instruction, they begin considering ways of redistributing writing instruction in classes required for the major and taught by professorial faculty.

Integrating Disciplinary Writing Instruction into the Curriculum

During the 2007-2008 school year, the department executive, assessment, and curriculum committees planned curriculum changes to strengthen and spread writing instruction across the English BA program.{3} These committees felt strongly that the curriculum needed to provide students sufficient opportunities to develop writing skills to meet the program learning outcomes. In fact, the dean of the College of Humanities implored that educators have the ethical and moral imperative to use assessment principles to facilitate student learning.

In August 2007, the department chair charged all upper-division courses to require at least one significant writing assignment involving primary and secondary research, source synthesis, proper formatting and documentation in MLA style, and evident facility with the theory or method relevant to that course. Another impetus for making curricular changes was preparing for a department unit review in winter 2008. The unit review document identified the need to teach research and writing skills more systematically as one of the three priorities for improving the quality of the English BA program. The department chair concluded that we needed to strengthen the relationship of our core courses to the program learning objectives as a whole, particularly regarding how we taught writing in the major.

In response to the findings of the unit review and the assessment committee, the following curriculum changes were proposed to the faculty at a department retreat in August 2008. These changes were subsequently approved by the university curriculum council in 2009.

• The creation of two major-specific GE writing courses: English majors enroll in ENGL 295: Writing Literary Criticism before enrolling in any upper division courses. The existing ENGL 495: The Senior Course was designated as the culminating writing course in which students produce a capstone writing project demonstrating their mastery of program learning outcomes. GE grants students advanced writing credit when they complete both courses
• Expansion of the Shakespeare requirement to a writing-intensive major-authors requirement: Students spend a semester reading representative works of Chaucer, Shakespeare, Milton, or other major author and engage with critical conversations about the author. This is an intermediate writing benchmark, in which students build on skills learned in ENGL 295 as they develop advanced competence in research and in composing literary criticism while focusing on the work of a single author.

Changing the major requirements and course catalog descriptions were intermediary steps in changing the culture of writing instruction in our department. In 2008-2009 school year, the department executive committee instituted policies standardizing the quantity of writing in upper division courses. In a series of meetings, faculty discussed which genres we wanted majors to learn how to write. We decided that we wanted to model students’ class writing assignments on professional genres that we produced as scholars of English Studies. Ultimately we decided that the twenty-minute, 8-10-page papers we deliver at conferences would be an appropriate model for students’ term papers. In ENGL 295, students would learn to write 8-10-page papers, so they could write 8-10 page papers in all upper-division, non-creative writing courses. These papers would develop arguments about texts, draw on the critical conversation about these texts, and demonstrate awareness of the texts’ historical and cultural contexts

In addition, the executive committee appointed course coordinators for all multi-section courses required for the major, including the three core writing courses. The course coordinators were tasked to foster dialogue among the faculty teaching the courses about the relationship between course and program learning outcomes and how to best meet them. Most faculty changed their syllabi and writing assignments to incorporate these changes. But we discovered that simply mandating the quantity of student writing in program and class assignments was not sufficient to improve the quality of their writing. The next phase of transforming writing in the English BA program began; the assessment committee and writing course coordinators worked together to identify pedagogical practices and curricular experiences to implement in the core writing courses that would help students develop expertise in our discipline’s research genre.

Implementing Course Embedded Assessment

Our department was playing out a common trajectory in higher education in recognizing that academic writing is not a homogenous practice across all disciplines (Deane and O’Neill; Wilder and Wolfe) and that “coordinated and sequenced assignments . . . accelerate students’ growth as disciplinary thinkers and writers” (Bean, Backward Design 215). Rhetorical and composition genre research has demonstrated the abundant advantages of recognizing the situated nature of writing expertise and teaching students disciplinary writing. However, most of the faculty, who have been involved in transforming the writing curriculum in our department, have come to the same conclusion as literature specialists trying to teach students to write about literature and, in some cases, including my own, as participants in the department’s assessment efforts.{4}

I was assigned to the assessment committee in the fall of 2009, joining the chair of the committee who had been involved with department assessment since 2006. It was my third year as a faculty member at BYU, and I was pre-tenure. I teach medieval literature, and my research focuses on the intersections of religious and textual culture in Anglo-Saxon England. At the time, I had no formal training in assessment or composition. In 2011, I became the assessment coordinator, and a literary theorist and former member of the curriculum committee joined me as the assessment committee. During this time the assessment committee had been responsible for making sure that our department complied with the ongoing mandates from the university’s office of assessment in preparation for the accreditation visit from the Northwest Commission in May 2015. Accordingly, much of the focus of our committee work over the last six years has been refining methods of measuring program learning outcomes as well as gathering and interpreting evidence of student learning.

While reading books about assessment practices in higher education, I learned about course-embedded assessment, a technique in which students’ performances on designated assignments in multiple-section courses are evaluated by faculty with a common evaluation tool (see Walvoord and Anderson 151-71; Bean, Backward Design 225-28; Gerretson and Golson). The embedded approach is compelling for program assessment in my department for several reasons. When employed at multiple points in the program, embedded assessment serves as a direct measure of student learning development. Establishing the sequence of three core writing courses in the major offered an ideal curricular framework for implementing this method of assessment. In addition, the department executive and assessment committees aim for our assessment practices to be faculty generated and to strengthen our curricular coherency. I decided that the culminating assignments in the core writing courses graded by instructors with a shared rubric would be an effective direct assessment method for the writing course and our program’s learning outcomes regarding writing skills. This method has become a significant component of measuring other program learning outcomes as well.

Since the department had already adjusted the English BA program’s curriculum to facilitate students’ writing development, the next step was to design common assignments and rubrics to be used in the three core writing courses. The need for course documents and shared rubrics became even more clear as the assessment committee surveyed the students taking and the faculty teaching ENGL 295: Writing Literary Criticism, the initial core writing course. Faculty began teaching ENGL 295: Writing Literary Criticism in the fall semester of 2009 with the following criteria: 1) select an accessible, short literary text, such as a short novel, a play, a collection of poems or short stories, with a rich critical conversation for the students to use as subject of their written analyses, 2) teach students how to use the MLA Handbook and attend a library orientation with the humanities-reference librarian, 3) assign an oral presentation and an 8-10 page literary analysis paper. Two years later, the surveys revealed wide variance among the sections—some sections involved lengthy reading assignments, others were not assigning papers of the specified length, and a few were replicating the curriculum of another gateway course (ENGL 251: Introduction to English Studies). In 2010, ninety-five percent of senior students the assessment committee interviewed in focus groups reported that they were required to do research in upper-division classes; however, many wished that they had been more explicitly instructed about how to discern good sources and how to structure the argument of their research papers.

The assessment committee realized that faculty needed to identify the specific writing skills that they wanted students to learn and practice in this course and the other core writing courses. We also perceived the need for course documents in which faculty committees would outline sequenced writing assignments and rubrics that would introduce students to the conventions of writing literary criticism and provide them increasingly complex opportunities to practice this genre. Designing the curricula collaboratively for the three core writing courses was time intensive, yet productive for encouraging faculty buy-in. It took approximately eighteen months for me to organize faculty committees to produce course documents that defined learning outcomes and outlined integrated sequences of assignments as well as for the course coordinators to train instructors how to use these documents in their classes.

In May 2011, six faculty were invited to a two-day assessment retreat to compose course learning outcomes, to determine methods of evaluating these outcomes, and to draft a course document for ENGL 295. The committee was chaired by myself and included two rhetoricians, an English-teaching specialist who publishes on composition pedagogy, an American studies specialist, and the director of the university’s WAC program. Four of these faculty had taught ENGL 295. This committee employed the strategy of backward course design. We began by composing three course learning outcomes that aligned with the program learning outcomes.

• Writing: Students will construct a paper employing distinct rhetorical moves to develop a coherent and logical argument that makes a significant literary claim.
• Research: Students will: 1) find and interpret credible sources, 2) integrate them purposefully in support of their own literary analysis, and 3) document these sources correctly using MLA format.
• Professionalization: Students will effectively employ the appropriate conventions of style, form, and tone in literary scholarship in written and oral communications.

Then we discussed how we would measure students’ achievement of these outcomes. We decided that students would read and emulate models of literary criticism from student and professional journals to learn the genre’s structure and function while writing a series of three papers of increasing length and complexity. We described the sequence of writing assignments in the course document. In the first paper (4-5 pages), students would focus on closely reading a literary text. In the second paper (6-7 pages), they would demonstrate an engagement in the critical conversation surrounding the primary text by responding to literary scholarship selected by the instructor. In the third paper (8-10 pages), students would conduct their own research in order to dig deeper into the literary or critical conversation most relevant to their topics and to be attentive to the primary text’s cultural and historical contexts. To guide students through this process, the committee explained in the course document that instructors should hold regular writing conferences with students, facilitate peer writing feedback, and require students to revise drafts.

To evaluate these papers, the committee generated a detailed analytic rubric comprised of nine criteria, each with three levels of achievement. Although we worried that the comprehensive nature of the rubric might intimidate students, we wanted to break down the elements and conventions of literary criticism as explicitly as possible for novice writers. This rubric and those used in the other core writing courses are included in the appendix to this profile.

As the faculty committee completed the course document and course rubric, we articulated the purpose of ENGL 295 as follows:

Students learn a new discourse in English 295 so they can participate in and contribute to disciplinary conversations. Some students may be able to imitate literary scholarship, but most need explicit instruction about disciplinary conventions and expectations. English 295 provides this training. The course’s learning outcomes focus on the craft of writing literary criticism rather than discussing literature. The sequencing and scaffolding of the writing assignments encourage students to mature and develop confidence in their writing. The methods of evaluation aid students, faculty, and the department in assessing students’ writing progress.

Over the summer, the new course coordinator introduced the faculty who would be teaching the course during the upcoming school year to the course document and the course rubrics. In October 2011, the entire department gathered for a lunch meeting in which faculty discussed ways to implement writing assignments in 300-level classes that would give students opportunities to practice and expand the skills they learned in ENGL 295.

In the spring of 2012, I chaired another faculty committee to compose a course document for the Major Author course (ENGL 381-384) as the intermediate writing benchmark for English majors between ENGL 295 and 495. The committee included the other member of the assessment committee and several members of the department executive committee who taught the major author course at least semi-regularly. The committee imagined the major author’s course as an index of the students’ maturation as critical readers, thinkers, and writers. Because this course involves intensive engagement with the representative works of one author within his or her historical, cultural, and critical contexts, it provides an environment conducive to writing a longer culminating literary analysis of 12-15 pages. This paper pushes students beyond the 8-10 page paper they learned how to write in 295 and were producing in other 300-level courses in anticipation of the senior capstone paper they would write in ENGL 495. The committee identified the following learning outcomes.

• History, Context, Genres, Themes, and Ethics: Explain the historical and literary contexts, genres and themes, and ethical dimensions of the major author’s life and representative works.
• Secondary Scholarship: Demonstrate familiarity with significant secondary scholarship and critical perspectives on the major author’s life and works.
• Critical Thinking: Develop reading strategies that explore how texts are constructed to present a set of arguments (historical, aesthetic, etc.) and use those critical reading skills to interpret the texts’ concerns and to trace interrelationships among them.
• Scholarly Research: Perform scholarly research on the major author’s works by identifying and evaluating relevant primary and secondary sources, by formulating arguments conversant with these sources, and by articulating those arguments in formal academic writing.

The committee also composed a rubric to evaluate the final paper. They aligned this rubric with the 295 rubric by reducing the number of criteria in the Major Author rubric by focusing on five key elements: literary interpretation, framing and rhetorical strategies, scholarly engagement, argumentation, and style and mechanics. The faculty teaching the Major Author courses decided to modify the wording of some of the criteria several years later. Additional revisions were made during a norming session in 2014. I encouraged the course coordinators of the core writing class to invite faculty to discuss the efficacy of the rubric and curriculum at least once a semester and to refine the course document and rubric based on the faculty’s suggestions. In doing so, the rubrics remain dynamic learning tools in the classroom and the program.

In the fall of 2012, a faculty committee redesigned the third core writing class, ENGL 495: The Senior Course. The course coordinator, assistant department chair, and the two members of the assessment committee composed learning outcomes, the course document, and a rubric. They specified two learning outcomes.

• Disciplinary Expertise: Students will (1) gain a broader and deeper understanding of a particular disciplinary topic or issue, (2) become familiar with the critical conversation about this area, and (3) develop sufficient competency in research and writing to make a contribution to the critical conversation about this area.
• Capstone Project: Students will develop an independent research project and craft an argument with the supervision of the instructor. Their projects will produce original and sophisticated pieces of literary criticism that situate their arguments in the context of the critical conversation about the disciplinary topic and issue that are suitable for submission to an undergraduate journal. Students should develop their project with a real audience in mind--a particular journal or graduate admission committee, and for the guidelines for length (generally between 15-20 pages) listed in their submission instructions. Students should substantially revise a complete draft of their paper in personal consultation with the instructor.

In this course, students demonstrate their synthetic application of the reading, analytic, and writing skills they have learned in the major in their capstone paper. It was difficult to reach consensus on the length of the capstone paper. Many faculty believed that a capstone paper should be longer than the 8-10 page literary analysis students write for most 300-level classes. However, some were concerned that the length of student papers does not correlate with the strength of their argument or quality of their style. We eventually decided to consider genre. In our scholarship, conference papers often anticipate journal articles. We considered how students might use a journal article capstone beyond the classroom. Students could submit an article to a number of undergraduate journals published at BYU or at other institutions, including our department’s Criterion: A Journal of Literary Criticism. In addition, students applying to graduate school need a polished writing sample of usually 15-20 pages. In light of these conversations, we revised the learning outcome as it reads above to permit a certain degree of flexibility based on students’ goals and the variability of the course content.

In October 2012 as a culmination of these efforts, the assessment committee presented an overview of the sequenced writing curriculum to the entire faculty using this chart to illustrate the relations among the courses.

English Course	Skills and Methods	Interpretive Focus	Culminating Paper: Genre and Length
295: Writing Literary Criticism	Make and support an argument Locate, evaluate, and use secondary sources Use MLA citation style Write professionally (appropriate form, style, tone, mechanics, etc.) Revise	In general, students focus on an individual text in their analysis.	Conference Paper 8-10 pages
381-84: Major Author	Consider the implication of authorial intent Take into account genre conventions Compare and contrast related texts Explore the relationship between text and context	As part of their analysis, students explore connections among literary texts and historical contexts, broadly defined.	Long conference paper/short article 12-15 pages
495: Senior Course	Learn the disciplinary context within which a particular issue or topic is situated by learning: the major authorities on the topic, the relative weight given to their various positions, which critical camps have weighed in on this particular issue or topic, what Stanley Fish calls the “tacit conditions of production for criticism.” Write in ways which demonstrate an awareness of disciplinary context, by way of: a clear, weighted bibliography; effective contextualization of quotations and references; an introduction which situates the argument as part of a broader scholar discussion.	At strategic moments in their essays, students situate their analysis within the broader critical conversation about the research topic or issue.	Article suitable for submission to an undergraduate journal 15 to 20 pages

The process of designing a sequence of writing courses challenged the English department to become more self-aware of disciplinary conventions as we identified the specific skills students needed to learn to produce expert writing and mapped where the skills would be introduced, practiced, and performed. We became committed to ensuring that students would have multiple and consistent opportunities to practice these skills, and we designated the culminating writing assignments in each course to be direct measures for assessing the students’ achievement of course and program learning outcomes.

Assessing the Core Writing Courses

The writing course rubrics have proven effective in facilitating student learning, charting students’ progress, and making decisions about teaching in class and at different points in the English BA program. The English department uses the scores of the students’ final papers in the three core writing courses as a direct measure of student progress toward the program learning outcomes concerning writing and as a method for evaluating the strengths and weaknesses of the curriculums of the writing courses.

The writing course rubrics were designed in a sequence and their relations can be characterized in terms of the decreasing detail of the traits and the increasing complexity of the writing tasks. The ENGL 295 Rubric breaks down the components of a paper in most detail. The three categories of traits with accompanying descriptions outline the basic conventions and expectations for novice writers. The Major Author’s rubric focuses the developing writers’ attention on five key elements of their writing. The shorter ENGL 495 rubric assumes that writers who are mastering genre conventions are writing sophisticated papers that require greater flexibility in scope and evaluation. The assessment committee has used the following chart to explain the relation among the graduated rubrics to our students’ writing development over the course of the major.

English 295 Rubric	English 381-384 Rubric	English 495 Rubric
Writing Moves Significance of Literary Claim Organization of Argument Literary Analysis Research and Analysis Interpretation of Sources Documentation of Sources Professionalization Tone Style & Mechanics Diction	Literary Interpretation Framing and Rhetorical Strategies Scholarly Engagement Argumentation Style & Mechanics	Contextualized Literary Interpretation Engagement with Disciplinary Conversation Argumentation Professional Style

The graduated levels of emerging, developing, and mastering trait descriptors accommodate students with a range of skill levels. Instructors of the core writing courses, particularly in ENGL 295, use the course rubrics in the classroom to facilitate dialogue with students about disciplinary standards and methods of meaning-making practices. Instructors use these rubrics to help students anticipate clearly how their work will be evaluated and what is expected. Many faculty invite students to use rubrics during peer review exercises. The rubrics also foster students’ ability to self-assess. Faculty encourage students to identify their strengths and weaknesses as writers and to make specific goals to improve their writing using feedback they receive from their papers scored with the rubrics (see Harrington 52-55 and Bean, “Backward Design” 226-28, for more discussion about rubrics’ potential for “good feedback”). Ideally, students keep their papers and rubrics from each core writing class and track their development as writers through the major. At some point the department would like to develop an online portfolio system where students can upload their essays and rubrics, but the institutional logistics are prohibitory right now. We also envision that students might use this work to articulate their writing skills to employers and graduate programs.

Faculty report that they use course rubrics to identify the specific skills they teach their students. Faculty also appreciate that the rubrics help them evaluate student work by established and consistent criteria, reducing their bias. The assessment committee considered hiring faculty committees annually or biannually for double-blind scoring of the papers produced in the core writing courses, but we decided that this method would not be financially sustainable with the number of students in our program. We decided that the instructors would score their own students’ papers using the course rubrics and report the rubric scores to the assessment committee, who would archive the scores for course and program assessment analysis. To improve the reliability of the scoring, the course coordinators invite the instructors to a norming session near the end of the semester in which they score the same set of sample papers and calibrate any variant scoring. These sessions provide instructors a consistent and collective perspective while evaluating their own students’ performance. The sessions also facilitate discussions among faculty about improving teaching and clarifying expectations in the rubric descriptors.

At the end of semesters, the faculty submit the rubric scores for their students’ final papers to the assessment committee. I figure the average score for each trait and the percentage of students meeting the criteria for emerging, developing, and mastering writing performance. The committee annually reports this data to the course coordinators, to the department executive committee, to the college deans, and to the university’s assessment office. We also make a short presentation to the department faculty at our fall assessment meeting. The department reports the rubric data to demonstrate the reliability of our assessment program to the college, university, and accrediting body; the rubrics set consistent and psychometric criteria that align our standards, curriculum, instruction, and assessment tasks. We hope the rubrics will permit learners, teachers, and other stakeholders to monitor student progress over a long period of time.

It took several semesters for the assessment committee to develop an efficient method of collecting and archiving rubric scores from the course instructors. In 2012, the Office of Digital Humanities built an online database to archive the rubric scores. However, fixing problems with the database has been an ongoing problem, and I have since switched to Excel spreadsheets which I can manage with no knowledge of computer programming. The assessment committee performed a pilot collection of rubric scores with ENGL 295 classes in winter 2013, and we collected rubric scores from all sections of the core writing courses from fall 2014. Faculty compliance has been a consistent challenge; some faculty resist using the course rubric and/or reporting rubric scores. The assessment committee works with the course coordinators to encourage and train all writing instructors to use and score the rubrics. To alleviate some of the instructors' anxiety, the department does not mandate that faculty use the course rubrics to calculate students’ grades on the papers or in the courses.

With two years of rubric scores, we started looking for trends to indicate how students’ writing develops at the beginning, mid-point, and end of the major. In September 2015, the assessment committee met with the writing course coordinators to discuss the data. Even though we were interpreting quantitative data, we used Bean’s collaborative “discourse approach” to frame our discussion (Backward Design 218). The average rubric scores for all nine criteria in ENGL 295 has consistently hovered at 4 (on a scale of 1 to 6). The rubric averages for the Major Authors courses were variable, in part because the faculty did not norm every semester and not every course section reported data. The average scores for ENGL 495 dipped below 4 after the faculty decided to add a zero category for papers that did not meet the minimum standard for a 1. We decided that this data indicates that our students are consistently improving their writing skills as they move through the major, because they demonstrate their ability to meet increasingly higher expectations in each core writing course. The slightly lower scores in ENGL 495 may be due to the fact that some of the capstone projects require students to write genres other than literary analysis, such as an ethnographic folklore report or a rhetorical analysis. In addition, students may be encouraged to interpret a diversity of exhibit sources, such as oral stories, cultural artifacts, or film, in addition to literary sources. We are confident that productive faculty conversations about student writing will continue to stimulate improved curricula, assignments, and teaching strategies.

External assessment also confirms that English majors write strong papers compared to their peers in other GE advanced writing courses. In May 2015, the GE assessment team gathered eleven faculty from departments across campus to read papers from the seven advanced writing classes offered in the College of Humanities and two in the History department and score them with an analytic rubric. The average scores for papers written in ENGL 295 and ENGL 495 were 4.4 on a 5 point scale. The senior history course was the only course that scored higher with a 4.6 average. Brian Jackson, the director of the University Writing Program concluded from this data that “the two-course sequences generally score higher than the generic one-course ones. The scores confirm previous research on three points: (1) two doses of writing instruction are better than one, (2) the more a student knows about the subject matter, the better the student will perform, and (3) the more articulated the disciplinary genre (e.g. literary or historical analysis) and clear the discourse community (e.g. literary studies), the better the student will perform.” I am encouraged that English department’s investment in implementing disciplinary writing instruction within our undergraduate major has been productive.

Learning from the Experience and Looking Ahead

I learned several valuable lessons over the last six years about using learning-outcomes based assessment to address writing deficiencies in the English BA program. I have four suggestions for departments that are similarly tasked to implement program assessment measures and for departments that would like to improve disciplinary writing instruction at institutions without a WID/WEC facilitator or center to guide them through the process.

Facilitate Faculty Collaboration

Revising, re-sequencing, or integrating writing courses into a program provides many opportunities for faculty to collaborate. This collaboration builds unity and vision among the faculty about their shared responsibility to facilitate student learning through writing regardless of their research specializations. Certainly many faculty will have hesitations and reservations about implementing writing in new ways in their classroom, but if they participate in the process of developing the writing plan and instituting the changes, they will most likely feel greater investment in coordinating and sequencing assignments to “accelerate students’ growth as disciplinary thinkers and writers” (Bean, Backward Design 215).

In Backward Design: Towards an Effective Model of Staff Development in Writing in the Disciplines, John Bean describes three lunchtime workshops that help faculty realize that teaching disciplinary skills involves conveying not only subject matter knowledge but also genre knowledge, rhetorical knowledge, and writing process knowledge (216-25). These workshops would have been useful for our faculty, particularly while formulating the program learning outcomes and again when designing the curricula for the core writing courses. It took my department years to realize, what Bean’s workshops are designed to convey in three lunchtimes, that even students in writing-intensive English programs need explicit disciplinary writing instruction. Our eventual discovery, though, was not hindered by literary specialists who resisted teaching disciplinary rhetorical knowledge to novice writers, a dynamic that Laura Wilder and Joanna Wolfe (172-73, 194-98) observed in some English departments experimenting with Writing in the Disciplines (WID) programs (see also Wilder, Rhetorical Strategies, 174-201). Collectively we lacked knowledge about the body of scholarship on genre theory and its application to writing pedagogy in literary studies.

My department reflected collectively about our goals for student writing and reached consensus in making curricular changes to help students meet these expectations. The execution of the core writing curriculum depended on a conscientious and collective effort to change the teaching culture of the department. Previously faculty members enjoyed considerable latitude in designing and teaching their courses, and peer evaluation of teaching focused on particulars of classroom effectiveness. We began developing a more collaborative culture of teaching. When we viewed the strength of the whole program as a priority, we became invested in refining course documents and rubrics, involved in ongoing conversations with each other about pedagogical best practices, and committed to promoting students’ writing development. Faculty report that teaching a core-writing course changes the way that they teach and assign writing in other classes. With this in mind, the department administration has encouraged every full-time faculty member to teach at least one core-writing course on a semi-regular basis.

Employ Methods of Learning-Outcomes Assessment

The three steps of assessment provide the framework and, in my department’s case, the institutional motivation, for making curricular changes, reallocating resources, and providing faculty development to improve student learning. In Assessment: Clear and Simple, Barbara Walvoord summarizes the process with these questions: “What do we want students to be able to do when they complete our courses of study” (3), or what are the program’s learning goals or outcomes? “How well are students achieving these goals, and what factors influence their learning” (4), or what evidence indicates students’ success or deficiency? “How can we use the information to improve student learning” (4), or what changes can we make to “close the loop”? There are many books and websites that offer advice about performing learning-outcomes based assessment. I wish I had read Bean’s “Backward Design” and found the University of Minnesota’s Writing Enriched Curriculum (WEC) program website earlier. Both outline models for creating, implementing, and evaluating department writing plans that enculturate students into disciplinary discourse.

I would also have benefitted from knowing about Writing in the Discipline scholarship (for example, Dean and O’Neill, Goldschmidt, and Wilder, Rhetorical Strategies). I had gleaned some of the pedagogical issues involved in sound writing instruction, such as backward course design, sequenced assignments, and using rubrics for formative and summative purposes, while reading Harris, Beaufort, Thaiss and Zawacki, Bean 2011, and Bain with the summer book groups sponsored by BYU’s Writing across the Curriculum program. The theoretical and practical insights of how to “teach disciplinary ways of seeing, question-asking, gathering evidence, analyzing, and arguing” would have given the assessment committee, the writing course coordinators, and other stakeholders in the department reliable frameworks to justify the curricular changes and to implement and assess them more effectively and efficiently. (Bean, Backward Design 218)

Define Disciplinary Genres

One of the first steps in improving student disciplinary expertise is teaching them to write the genres that experts in the field produce. For as Anne Beaufort and John Williams observe, our students’ writing demonstrates the degree to which they are capable of “doing the analytical work of the discipline,” and so teaching a discipline’s genre is teaching the discipline (64). Likewise, Katherine Harrington argues that “through writing, and opportunities to practice writing, students learn not only to recognize the conventions used in the disciplines they are studying, but also, more fundamentally, they learn how these conventions reveal and contribute to creating the epistemological orientation and knowledge-making practices at play in the disciplinary fields they are beginning to inhabit themselves” (49). However, faculty are susceptible to expert blind spots and “may have little conscious awareness” of all the tacit skills and knowledge required for complex tasks (Ambrose et al. 112). Thus faculty may struggle to explain the discourse conventions they have so thoroughly internalized as practitioners in the field to novice students (Ambrose et al. 99). This problem is compounded in disciplines, such as literary studies or even more broadly English studies, in which experts produce writing that does not adopt a standardized structure, as scientific articles do, and that adopts methods and theories from other disciplines. It is not surprising that students express confusion about what faculty expect in their papers.

My department decided very early in the process of developing a disciplinary writing curriculum in our undergraduate program that we wanted students to learn how to write literary criticism, yet as faculty committees began drafting writing course documents and designing rubrics we discovered how difficult it is to define and even agree about the component elements in literary criticism. Being involved in these curricular conversations and teaching the core writing classes have motivated me to develop a greater awareness about the genre conventions of literary analysis and has helped me teach these conventions explicitly to my students. My students have appreciated learning James Bizup’s BEAM schema about the function of research sources in constructing disciplinary appropriate arguments and the special topoi or inventional strategies employed in literary criticism examined by Laura Wilder and Joanna Wolf. My department could facilitate more opportunities for faculty to discuss disciplinary practices in their own writing and to share how to teach these practices to students.

At some point I anticipate that my colleagues will want to rethink whether the English BA program should focus so exclusively on teaching students the genre of literary criticism. Students have opportunities to learn other genres by electing to take English classes in folklore, rhetoric, creative writing, or film; they may also minor in Creative Writing or Writing and Rhetoric. We also need to consider how transferable the writing skills of literary criticism will be for students in their varied fields of employment after graduation since only a small percentage of our students become professional literary critics. This issue is becoming more pertinent as we recently implemented another curricular change that requires English majors to complete an extracurricular professional experience (ex. internship) before graduation. If we want students to translate their skills to a professional environment and learn how to narrate this process for a prospective employer, what genres in addition to literary criticism will students need to master? In the meantime, although we are wary of claiming the universal applicability of skills developed by writing about literature, we hope that if English majors learn to navigate the discourse conventions of at least one discipline that they will have developed skills to adeptly navigate other fields in their future.{5}

Implement Course Embedded Assessment

Course embedded assessment is an effective and flexible method to assess a program’s writing outcomes with many benefits to the program, faculty, and students as I have already discussed. Like every assessment measure, though, it takes time and effort to maintain, which I did not fully appreciate when implementing it. It would be wise to consider, in addition to how many assignments will be used as program indices, what evidence will be collected from those assignments, who will collect the data, how will the data be archived and interpreted, and how will faculty or staff be trained to maintain this process.

Keeping up with the logistics of collecting and analyzing rubric scores for hundreds of students in over a dozen of sections each semester is time consuming. Ensuring consistent faculty compliance with course embedded assessment requires systematic planning on the part of the assessment committee, a committee that has for the last six years included only two full-time faculty members who manage the department’s entire assessment work without course reductions or dedicated secretarial assistance. It also requires committed course coordinators who have often been assigned other committee duties in the department. In addition, the faculty teaching the core writing courses are expected to follow the course document, use the rubric to evaluate their students’ papers, and attend a norming session once a semester. Figuring out a simple and non-time consuming method of collecting rubric scores and a reliable and secure method of archiving them is an ongoing challenge for me. Another challenge is learning how to translate the numeric rubric scores into a discussion about the strengths and weaknesses of the writing curriculum and then improvising strategies to address those weaknesses. Nevertheless, I believe that navigating these challenges will strengthen the department’s commitment to teaching writing in a disciplinary context and generate recursive reinforcement throughout the English BA curriculum that will facilitate students’ transformation from novice to expert writers.

Conclusion

In December 2014, I asked a focus group of senior students who were among the first cohort to have taken all three core writing courses whether they felt that they met the department writing learning outcome. One student said that his writing skills have “drastically improved” since high school:

I think 295 was really the transition for me; it helped me see the greater scope of academic writing and entering the conversation. Instead of having simple formats or rules you have to follow, you are now given free rein to look at what is being said and formulate your own connection and utilize what other people have said to . . . bolster your argument. I think that is the most fascinating part and what has improved most about my writing is the ability to convey very unique ideas by catapulting myself off of, or bouncing myself off of, ideas that surround me that I wasn’t aware of before.

Another student realized the importance of developing “an engaging style.” While another described the importance of taking the time to draft her paper: “when I sit down and write a rough draft and have it revised and have it looked at by other people and redo it and redo it and redo it until it gets better.” Many students learned, as one student expressed “that there are so many different styles of writing and you have to kind of know who you’re writing for and adapt your voice in different situations.” These responses and others give me hope that our department’s curricular changes are helping students develop disciplinary expertise. Best wishes to other departments that embark on this journey.

Appendix (PDF)

Notes

1. The enrollment in the English BA has declined to approximately 800 students in the last seven years. The English Department now sponsors three minors: English Literature, Writing and Rhetoric, and Creative Writing. It also has 75 graduate students enrolled in the English MA or Creative Writing MFA programs. (Return to text.)
2. The English BA program learning outcomes have since been revised several times in the last decade. In the fall of 2015 they are: Interpretive and Communicative Skills: English graduates employ critical reading strategies, disciplinary writing expertise, and sophisticated analytical skills in their written and oral communication. Interpretive and Communicative Applications: English graduates will translate these skills to professional environments and narrate the value of these skills to prospective employers. Interpretive and Communicative Contexts: English graduates know how to trace the development of literary traditions, investigate authors, and differentiate genres, and they know how to discuss disciplinary methodologies and scholarly conversations; they use these contexts to frame their written, oral, and visual work. Interpretive and Communicative Ethics: English graduates embrace literature and writing as sources of wisdom, spiritual insight, and aesthetic pleasure; as mediums for encountering and reflecting upon the diversity of human experience; and as guides for building relation and discerning value. (Return to text.)
3. In 2007-2008, the executive committee was composed of tenured faculty specializing in 19th-century American literature, English education and composition pedagogy, poetry, and British Romantic literature. The associate chair who studies British Romanticism chaired the assessment committee of two American ethnic literature specialists. The curriculum committee was chaired by a literary theorist and included a short story writer, a folklorist, and a Native American literature specialist. (Return to text.)
4. The rhetoric and composition specialists in our department were not more involved in developing the core writing courses in the English BA program because some were directing the University Writing program that oversees the GE freshmen writing and non-major advanced writing courses, some were developing the Writing and Rhetoric minor, and others were serving as deans. (Return to text.)
5. I thank my colleagues Don Chapman, Greg Clark, Kristine Hansen, and Jamin Rowan for their insightful feedback. I also thank Dennis Cutchins, Ed Cutler, Trent Hickman, Daniel Muhlestein, and Phil Snyder for providing information and support for this program profile. (Return to text.)

Works Cited

Allen, Mary J. Assessing Academic Programs in Higher Education. Anker Publishing, 2004.

Ambrose, Susan A., et al. How Learning Works: 7 Research-Based Principles for Smart Teaching. Jossey-Bass, 2010.

Bain, Ken. What the Best College Students Do. Harvard UP, 2012.

Bean, John C. Backward Design: Towards an Effective Model of Staff Development in Writing in the Discipline. Deane and O’Neill, pp. 215-36.

Bean, John C. Engaging Ideas: The Professor’s Guide to Integrating Writing, Critical Thinking, and Active Learning in the Classroom. 2nd ed., Jossey-Bass, 2011a.

Beaufort, Anne. College Writing and Beyond. Utah State UP, 2007.

Beaufort, Anne, and John A. Williams. Writing History: Informed or Not by Genre Theory. Genre Across the Curriculum, edited by Anne Herrington and Charles Moran, Utah State UP, 2005, pp. 44-64.

Bizup, James. BEAM: A Rhetorical Vocabulary for Teaching Research-Based Writing. Rhetoric Review, vol. 27, no. 1, 2008, pp. 72-86. Taylor & Francis, doi:10.1080/07350190701738858.

Deane, Mary, and Peter O’Neill, editors. Writing in the Disciplines. Palgrave Macmillan, 2011.

Gerretson, Helen, and Emily Golson. Synopsis of the Use of Course-Embedded Assessment in a Medium Sized Public University’s General Education Program. The Journal of General Education, vol. 54, no. 2, 2005, pp. 139-49.

Goldschmidt, Mary. Teaching Writing in the Disciplines: Student Perspectives on Learning Genre. Teaching & Learning Inquiry, vol. 2, no. 2, 2014, pp. 25-40.

Harrington, Katherine. The Role of Assessment in ‘Writing in the Disciplines’. Deane and O’Neill, pp. 48-62.

Harris, Joseph. Rewriting: How to do Things with Texts. Utah State UP, 2006.

Tanner, John S. Building a Better House of Learning. BYU Annual University Conference, 29 August 2006. BYU Speeches, www.speeches.byu.edu/talks/john-s-tanner_building-better-house-learning/. Accessed 1 May 2015.

Thaiss, Chris, and Terry Myers Zawacki. Engaged Writers, Dynamic Disciplines: Research on the Academic Writing Life. Boynton/Cook, 2006.

Walvoord, Barbara. Assessment Clear and Simple: A Practical Guide for Institutions, Departments, and General Education. Jossey-Bass, 2004.

Walvoord, Barbara, and Virginia J. Anderson. Effective Grading: A Tool for Learning and Assessment in College. Jossey-Bass, 2009.

Wilder, Laura. ‘The Rhetoric of Literary Criticism’ Revisited: Mistaken Critics, Complex Contexts, and Social Justice. Written Communication, vol. 22, no. 1, 2005, pp. 76-119. Sage Journals, doi:10.1177/0741088304272751.

---. Rhetorical Strategies and Genre Conventions in Literary Studies: Teaching and Writing in the Disciplines. Southern Illinois UP, 2012.

Wilder, Laura, and Joanna Wolfe. Sharing the Tacit Rhetorical Knowledge of the Literary Scholar: The Effects of Making Disciplinary Conventions Explicit in Undergraduate Writing about Literature Courses. Research in the Teaching of English, vol. 44, no. 2, 2009, pp. 170-209. National Council of Teachers of English, www.ncte.org/journals/rte/issues/v44-2. Accessed 27 April 2015.

Wolfe, Joanna. A Method for Teaching Invention in the Gateway Literature Class. Pedagogy, vol. 3, no. 3, 2003, pp. 399-425. Project Muse, www.muse.jhu.edu/article/46557.

WEC: Writing Enriched Curriculum. University of Minnesota, www.wec.umn.edu/. Accessed 15 August 2015.