Table of Contents
- Overview and Introduction to Assessment
- Assessment process and procedures at UALR
- Six steps to assessment
- Assessment Plans: Steps 1 and 2: Goals and Objectives
- Assessment Plans: Step 3: Connecting objectives to curriculum
- Assessment Plans: Step 4: Tools, Methods and Design
- Assessment Progress Reports: Step 5: Reporting Findings
- Assessment Progress Reports: Step 6: Using Results
- Reviewing Assessment Plans and Progress Reports
- Appendix A
- Appendix B
- Appendix C
I. Overview and Introduction to Assessment
â€śAssessmentâ€ť is a word that has multiple meanings, both within the educational context and outside of it. First and foremost, assessment is about student learning. As teachers, we are constantly assessing individual studentsâ€™ performances against explicit and implicit benchmarks. When we â€śassess student learning outcomesâ€ť we are stepping back to get a bigger picture of learning than is represented by one student in one class section. We are trying to assess how our students as a whole are achieving across courses and throughout our curriculum. The goal is not to determine the success or failure of an individual student, or that of an individual faculty member, in a specific course. Rather, is our curriculum as a whole producing the types of graduates that we want to be known for? If so, where are the successes and how can we build on those strengths to make our programs even more successful? If not, where are the deficiencies and how can we improve them? Assessment of student learning provides the tools and opportunity to help us achieve our educational goals.
Purpose of handbook:
Assessing student learning outcomes is a vital part of our teaching and learning mission; but how does this work? This handbook represents an effort to provide faculty with everything that they need to know to pursue a successful student learning assessment program. A â€śsuccessfulâ€ť assessment program provides faculty with the information that they need to continuously improve the teaching and learning experience for their students.
Each section of this handbook addresses a different part of the assessment process. This is not meant to be prescriptive, but rather to provide a pool of resources to help programs develop plans that best fit their needs and circumstances. But first, letâ€™s take a moment to remind ourselves of why assessing student learning is important.
Assessment and the UALR mission:
The University of Arkansas at Little Rockâ€™s mission states that, â€śâ€¦ The University has a responsibility to provide excellence in instruction to ensure high-quality education for our students. This responsibility includes developing faculty teaching skills, awareness of the ways students learn, assessing student learning outcomes, and enhancement of resources to support effective instruction.â€ť (Adopted by UALR Faculty Senate, 1988.) Assessing student learning is the key to evaluating our effectiveness in achieving this part of our stated mission. As such, assessing student learning is everyoneâ€™s
business and responsibility.
Assessment and accountability:
There are also external drivers behind the assessment process. As a state-assisted institution, we are accountable to the people of Arkansas through several state agencies and bodies. Thus assessment plays a major role in program review. We are also accountable to our peer higher education institutions through the process of accreditationâ€”as a campus, and as colleges or programs within our disciplines. All of these external stakeholders are calling for data to demonstrate that students are achieving the institutionsâ€™ learning goals; and, if they are not, that we are making progress toward creating an educational experience that should support that achievement.
Core competencies and assessment
When assessing core courses, we also need to keep in mind two different agendas. Core courses tend to be introductory or gateway courses into our majors and degree programs. As such, we have questions we want to ask about how well those courses are fulfilling goals related to fundamental skills and understandings within our discipline. Core courses also are part of the general education component of our university curriculum. They are included as ways to fulfill those general education requirements because they address one or more of the ten core competencies. As such, we have questions we want to ask about how well these courses are fulfilling learning goals related to those competencies. (A list of the core competencies can be found on this webpage).
Organization of this handbook:
In the following sections of this handbook, we hope to provide you with tools that will assist you in developing, implementing, and maintaining an assessment process that will help you reach your core course learning goals. In order to do that, this handbook has drawn on the efforts of faculty and professional staff from across campus. And in order to keep doing that, your feedback and suggestions for improvement are vital. Please contact the Provostâ€™s Office with any comments, suggestions, or corrections.
II. Assessment process and procedures at UALR
Plans and progress reports;
All assessment efforts start with a plan which lays out what we want to accomplish about assessing student learning. Plans include statements of goals, objectives, tools, methods, and how the results will be evaluated and connected with decision-making. Annual progress reports state what part of the plan was accomplished, the results obtained for that year, and how those results will be used to make decisions about the course being assessed.
Part of the approval process for courses to become part of the core or general education curriculum is a statement of how that course will help students achieve one or
more of the ten core competencies. Each core course must submit an approved assessment plan as part of the packet presented to Undergraduate Council in order to be approved to be part of the core curriculum. That plan must include how the identified core competencies for that course will be assessed as well as any course-specific learning outcomes.
Approval and review:
Plans are subject to approval and progress reports are subject to review. These processes fulfill several functions. First and foremost, they provide feedback to faculty to help them fine tune their assessment program so that it will maximize their success in answering the questions about student learning that are most important to them. Second of all, annual reviews help to insure that individual programs donâ€™t â€śdriftâ€ť away from keeping the assessment plan moving forward. Finally, annual reviews help to insure that programs will be ready when the time for program review and/or accreditation rolls around. (The year that the accreditation agency comes is not the year to start collecting student learning data!)
Assessment plans are approved at the college level first and then by at the university level by the Provost Assessment Advisory Group (PAAG). The college level evaluation typically focuses on college or discipline-specific accountability or accreditation needs. The PAAG provides feedback to insure that the plan will provide acceptable information at the institutional assessment level, meeting the accreditation standards set by the Higher Learning Commission/North Central Accreditation Association. Approval of the assessment plan by PAAG must accompany any request to the Undergraduate or Graduate Council for including a new course into the core curriculum. (More on the approval process in Chapter IX.) Core course progress reports are reviewed each fall semester before the annual Assessment Expo. Core course faculty members review each othersâ€™ progress reports electronically, and then they interact with the report authors and provide qualitative verbal feedback in a round-table setting during the Expo itself. Written versions of the reviews are distributed through the Provostâ€™s Office. A final version of the core course progress report is sent to the Provostâ€™s Office for inclusion on â€śAssessment Centralâ€ť website. The idea behind this strategy is that the core curriculum belongs to the entire campus, so the review should involve the other faculty involved in teaching the core and the results should be posted where they are accessible to the entire campus community.
Finally, the question sometimes arises about whether or not an assessment plan also needs to go through approval by our Institutional Review Board. Typically they do not, provided that the data gathered will be used for internal purposes to evaluate teaching effectiveness. It becomes less clear if part of your plan includes educational research that could be disseminated publicly. Whichever it the case, all information provided by participants in learning outcome assessment should be treated confidentially, with the identity of any given participant and his or her performance obscured and protected.
Institutional level assessment of general education:
Every ten years UALR compiles information about assessment of student learning across campus in preparation for our campus-wide accreditation by the Higher Learning Commission of the North Central Association. All degree program and core course plans and progress reports are included, as well as the annual college and PAAG reports.
Plans are submitted for approval whenever adding a course to the core is proposed. Plans can be submitted at any time of the year.
Core course progress reports are submitted in the fall and reviewed at the Assessment Expo, with written review summaries distributed afterwards. The progress reports are then posted on the Assessment Central website maintained by the Provostâ€™s Office. The data presented in progress reports are from the preceding calendar year. This is to allow faculty sufficient time to not only collect but to analyze, interpret, and make decisions based on those findings.
The assessment cycle:
Plans should be written with an assessment cycle in mind. How many years do you anticipate it will take to address all of the learning goals and objectives in your plan? You might want to, and it may be feasible to, assess every goal and objective every year. More complex plans may be more successful if they are implemented over a period of three to five years. Certain objectives are targeted in each year within the cycle, with all objectives addressed during the length of the cycle. The final year of the cycle is spent in evaluating all results obtained in reference to all of the objectives over the all of the years since the last plan was written. The â€śprogress reportâ€ť for that concluding cycle year is a revised plan which includes an implementation plan for the next assessment cycle. You could think about your assessment plan as being like a grant for a research program. Grants applications are written and approved for a finite time period, with periodic progress reports, a final report, and the last year of the grant is often spent is submitting the next multi-year proposal. Not all objectives are addressed in every year, and the results from past research are used to inform the next stage in the research program.
III. Six steps to assessment:
The following chapters are organized around six basic steps in creating an assessment plan:
- Identifying student
learning goals, including the core competencies the course will address.
- Stating student
learning outcomes or objectives.
- Connecting learning
objectives to the curriculum.
- Identifying assessment
tools, methods, and design, including implementation timeline and rubrics.
- Analyzing, interpreting,
reporting and communicating findings.
- Using findings for
improvement and decision-making.
The first four steps are included in assessment plans. Progress reports focus on the last two. At the end of each chapter you will find instructions on how to complete the corresponding section of the program or core course plan or progress report forms.
Tips for success:
#1. Make it meaningful. The purpose of assessment it to provide information for decision-making and curricular improvement. That means you must start by deciding what are the most important questions you could ask and the most useful information that you could have about your educational program.
#2. Keep it simple. Less may really be more when it comes to assessment. Faculty, staff, and students are all busy people. You are most likely to succeed if you limit your questions to the ones that really matter the most.
#3. Ask questions about factors over which you have control. It may be interesting to know that, for example, students who work more than 20 hours a week are less successful in your course; but is that anything over which you can exert control? Are there any changes or adjustments that you could really make that would address this? If not, then the information may be important but not useful for the purposes of curricular improvement.
#4. Create reasonable deadlines and assessment cycle timelines. It is important to create realistic deadlines for accomplishing your assessment plan, including all of the stated learning outcomes or objectives. You should think in terms of a three to five-year cycle. If it wonâ€™t be reasonable to accomplish everything you want during that length of time, perhaps you should re-think your approach and focus on what the most pressing concerns first. Less pressing questions could wait until the next cycle.
#5. Make it measurable.Assessment plans will be most effective when the objectives are written in a way that will make it transparent whether or not students have met your benchmark. The more guesswork or judgment is involved, the more room there will be for disagreement about what the results are, what they mean, and what should be done in response to them.
#6. Use a mix of multiple
measures. Relying mainly on one or two achievement measures can be a risky strategy. Sometimes plans place all of their faith in a single test score or survey outcome. Multiple measures increase oneâ€™s chances of getting valid information that approaches a target from different angles.
#7. Use embedded, double-duty activities. Donâ€™t add extra activities to either faculty or student workload simply for the sake of learning outcome assessments. You are more likely to get valid measures of student achievement if you use activities that already have meaning for students. Need a writing sample? Find one in class where students will be motivated to produce the best work of which they are capable.
#8. Remember that it is your course and therefore your plan. Yes, external stakeholders have provided a lot of the motivation for assessment; but in the end it is your course and your plan. You will have to live with the results, so it is important that everyone invested in the course is involved in implementing the plan and discussing the implications of the results. Plans are only useful to the extent that they are used, and they are only used to the extent that people whom they affect have a voice in them. On the other hand, remember that consensus does not mean unanimity. When people disagree about what should be in a plan, it can be an opportunity for discussion and clarification about the program mission, goals, or objectives. If the results are ambiguous, that can be an opportunity to reframe the objectives in a way that will be more measurable.
#9. Remember that the only â€śfailureâ€ť in assessment is the failure to act. Successful assessment does not mean that the findings indicate that all is â€śrosyâ€ť and nothing needs changing. Successful assessment is when the findings are meaningful and point toward action. What matters is that assessment should lead to a process of continuous quality improvement.
#10. Consolidate your assessment efforts. We all have a need to do assessment for purposes of campus-wide accreditation and accountability. But several units also have college or program level accreditation requirements that include assessing student learning; so think in terms of how one plan can fulfill both audiences. (Please contact the Provostâ€™s Office for assistance in aligning your discipline accreditation requirements with the campus-wide accreditation requirements.)
IV. Assessment Plans: Steps 1 and 2: Goals and Objectives
Before we can start collecting data, we need to decide: what are the most important questions that we want to ask? What is it that we are trying to accomplish with our students? How do we define success for our graduates? If you think of assessment as a research project, these are the hypotheses which we will be testing in the experimental design that follows. Without a clear model and set of hypotheses, a research program will break down into a series of disconnected studies. With a model and framework of hypotheses to guide our work, then our efforts are more likely to pay off in a new body of knowledge. So, we need to start by deciding what are the major assessment questions or hypotheses, i.e., what are the learning goals and objectives that we are going to test.
Definitions: goals and objectives:
Learning goals are the clearly articulated statements of the desired end results of the curriculum. Goals are the general skill or knowledge categories you want your students to achieve. Goals for core courses should also include the core competencies for which that course was approved. These are the standards for excellence for core courses, and they represent the curriculumâ€™s core competencies. Learning goals form the foundation for any assessment plan, but articulating them can be the most difficult step in writing any plan. Assessment is intended to provide information to help us determine whether or not we are meeting our goals, but our goals are more often implicit than explicit. Everyone believes that we all â€śknowâ€ť what we are trying to accomplish by having a particular requirement as part of the curriculum, but it is often surprising how much what one person â€śknowsâ€ť differs from another. Goals should be the result of a consensus of all contributors to that curriculumâ€”commonly referred to as the â€śstakeholders.â€ť Stakeholders could include faculty, professional staff, students, alumni, employers, or segments of the larger central Arkansas community affected by the programâ€™s goals. Identifying and stating the courseâ€™s learning goals can provide important guidance not only for assessment but in making many other decisions.
Finally, it is best to keep the list of learning goals short. From a practical point of view, every goal will need to be represented in your assessment plan. But from a more theoretical point of view, how many different goals does your program really have the resources to pursue? Goals should be comprehensive but also a realistic statement of what you actually intend to accomplish, not a wish list of everything that you could accomplish in an ideal situation.
Learning objectives or outcomes are the concrete, measurable manifestations of those goals. (In this document, we will use â€śobjectivesâ€ť and â€śoutcomesâ€ť interchangeably. Some college or discipline specific review bodies prefer one to the other. Please use whatever works best for your unit.) Objectives identify the specific accomplishments to be achieved and that will provide the proof that students have achieved the course learning goals. These are the outcomes of the activities students encounter in their course, not the activities themselves, and typically are not covered by a single activity in a single course. Therefore it is important to:
- Define objectives so you can recognize and measure them.
- Identify specific and observable student behaviors–not teacher behaviors.
- Describe outcomes–not processes.
Objectives or outcomes typically fall into one of three domains:
a). What do we want our students to know? These cognitive outcomes could include:
- Knowledge base
- Intellectual understanding
- Theories and models
- Subject matter
- Analysis and synthesis
b). What do we want our students to be able to do? These behavioral outcomes could include:
- Application to real problems
- Communication skills
- Technology skills
- Problem-solving strategies
- Information gathering skills
- Critical thinking
c). What do we want our students to value? These affective outcomes could include:
- Professional ethics
It is important to select a reasonable number of learning objectives for your course. Each goal should probably not be associated with more than four to six learning objectives or outcomes. Using the research program analogy again, how many different hypotheses do you really have the power to test at one time? Each objective should be stated in measurable terms. Assessment plans should, over the period of the assessment cycle, address every learning objective for that program (or core course).
Plan or progress report form instructions:
The learning goals for the core course should be entered under Section 1 on either the â€śCore Course Assessment Plan Form.â€ť Then the learning objectives that correspond to each of the program goals should be entered into Section 2. Please take care to align the objectives with their corresponding goals. (See Appendix A for forms and detailed instructions.)
For progress reports:
Copy the learning goals and objectives statements from the plan to Sections 1 and 2 of the â€śCore Course Assessment Progress Report Form.â€ť Only include those goals and objectives that are addressed in the current progress report. The purpose is to help the reader understand how this progress report fits into the overall plan; so the goals and objectives can be stated briefly. A link to the entire plan can be included for those readers who wish more information. (See Appendix B for forms and detailed instructions.)
V. Assessment Plans: Step 3: Connecting objectives to curriculum
Once you have settled on your goals and objectives, the next step is to consider how you will assess them. To do that you will need to identify at what points in the program you would get the best â€śbang for the buckâ€ť for your assessment efforts. At what point should you, for example, assess understanding of major theories in your field? An ability to apply professional ethics? In order to make these decision, you need to have the â€śbig pictureâ€ť of when different program goals and objectives are addressed in your curriculum. And then you can identify the best points at which to assess them.
Curriculum assessment map
A useful way to get the â€śbig pictureâ€ť for your curriculum is to construct a curriculum assessment map. The idea of a map is really very simple. Columns represent the learning objectives for the program and rows represent the actual course activities. Each cell, then, is the intersection of the overall objectives with the specific course activities.
Within each cell you will be asked to make which activity is designed to achieve which objective for your course. (The Curriculum Assessment Map for Course Assessment is attached to the Plan form in Appendix A.) This process can also help to identify redundancies or gaps in a course plan. Here is an example for discussion that includes the four objectives for the first program goal. What does this matrix tell us about this curriculum?
|Goal 1||Objective 1||Multiple choice exams|
|Objective 2||Multiple choice exams|
|Goal 2||Objective 1||Essay exam questions
What would you say about the status of the curriculum as outlined in the table above? Are there objectives that arenâ€™t getting the coverage that they should? Are there activities that either are not â€śpulling their weightâ€ť with respect to this goal or are being expected to cover too much? (Note: You could do one map that includes all goal by objective combinations, or a separate map for each goal.) This process of reviewing oneâ€™s curriculum with the learning goals and learning objectives in mind can be considered a form of assessment itself because it could result in changes being made to what is covered in the course. When you are finished you will have a visual aid to how each activity contributes to the overall course learning goals and objectives. How else might such a visual aid be useful to your unit?
Identifying points of assessment
The next step is to identify where in the curriculum it makes the most sense to assess each objective. Here is the above example but with possible points for assessment identified.
|ACTIVITY||USED FOR ASSESSMENT|
|Goal 1||Objective 1||Multiple choice exams||Yes|
|Objective 2||Multiple choice exams||Yes|
|Goal 2||Objective 1||Essay exam questions
For example, do you want to assess Goal 2, Objective 1 through the essay exam questions or the paper? Should Goal 1, Objective 3 be assessed through the paper or the course project? (In Section 4 you will decide exactly which tools you will use and enter those in the map instead.) Completing such a map requires input and collaboration from all faculty involved in a teaching a particular course. This discussion in itself is an important part of the assessment process: identifying just how our implicit assumptions about our curriculum are being made explicit in what we are asking students to achieve at various points.
Plan or progress report form instructions:
Each plan should have a curriculum assessment map completed and attached as part of the response to Section 3 on the plan form (see Appendix A for forms). Each activity that will be used to address each goal and objective should be noted. A more detailed description of each assessment data source or tool will be included as part of Section 4 of the plan (more on that in the next chapter).
For progress reports:
Once the curriculum assessment map is completed for the plan, it does not need to be attached to each progress report. However, you should copy and paste any information from Section 3 that is relevant to the results reported in a given progress report.
VI. Assessment Plans: Step 4: Tools, Methods and Design
At this point you should have a set of goals for your core course (Step 1); measurable objectives based on those goals (Step 2); plus a â€śbig pictureâ€ť of how those objectives are expressed in your curriculum and some ideas about at what points in the course you would like to gather data about progress toward those objectives (Step 3). Now it is time to decide on exactly what data you want to gather to help you answer your assessment questions or test your assessment hypotheses. What information will tell you whether or not a given objective has been met? How best can you obtain that information? This is what will go into Section 4 of your plan.
Assessment tool types:
Direct methods involve the actual products of student work that can be evaluated in light of the learning outcomes. These may be the same materials for which students have been evaluated on an individual level (i.e., given a grade), but in this context one is looking across students at learning outcomes of groups. These methods could include:
- Course-embedded assignments (e.g., tests, papers, reviews, presentations)
- Standardized tests
- Locally developed tests
- Portfolio evaluation
- Thesis or dissertation
- Oral exams
- Focus groups
Indirect methods are not based directly on academic or work performance. These usually involve perceptions by either the student or someone in a position to observe the studentâ€™s work. These could include:
- Student satisfaction surveys
- Exit interviews
- Focus groups
- Alumni surveys
- Employer surveys
- External reviewer
Direct measures may be more time-consuming to collect and evaluate, but they can be more closely tied to the identified learning objectives. Indirect measures are less objective, but they are easier to collect and provide important information about the extent to which the program is meeting student or employer needs. Again, the best strategy will be to select a mix of measures depending upon the questions you have identified in your plan.
Issues in selecting tools:
There are numerous tools and strategies for collecting data about student learning. The most important thing to keep in mind is: How well do these measures match goals and objectives? Which tool or strategy will supply the information relevant to the questions or hypotheses that you are trying to test?
Concern is often expressed by faculty about student motivation to do their best on assessment activities. For this reason, experts in assessment recommend that you use course embedded versus stand-alone measures. Studentsâ€™ motivation to do their best will be highest on measures that are meaningful and have face validity for students. An activity which is part of a course, and perhaps part of a course grade, is the most likely to be meaningful for students. Stand-alone measures that are administered outside of a class can be valuable and may be the best way to get the information you need. However, you will need to take care to include some contingency that will encourage and reinforce studentsâ€™ best efforts on such measures.
Validity and reliability also need to be considered when selecting assessment tools. A â€śvalidâ€ť tool is one that measures what it purports to measure. What evidence is there that the test or activity actually elicits the knowledge, behavior, or attitude that you are trying to assess? A â€śreliableâ€ť measure is one that produce consistent results, such that if there is a change in a studentâ€™s performance you can have confidence that it is the result of a real change on the part of the student and not a result of the time, place, or person collecting the information.
While it is tempting to use one measure to assess any given objective, you are better off in the long run identifying multiple measures. What if the one measure you choose turns out not to be valid or reliable? What if it snows the day of the standardized exam and half of your sample does not show up? Relying on single measures can make you vulnerable.
When selecting tools, it is also wise to select a mix of qualitative and quantitative measures. While quantitative measures may seem more â€śobjectiveâ€ť in that they result in numerical values that can be easily manipulated, it may be that qualitative measures may better capture the â€śholisticâ€ť values you want to examine about student performance. Should you adopt or adapt measures or create your own? Certainly using measures that have already been tested and found reliable, valid, and useful can save you a lot of time and energy. However, they may not be an exact match to the questions you are trying to ask or the hypotheses you are trying to test. Again, selecting more than one measure to assess any single objective is the most likely way to end up with usable and useful information.
Finally, keep in mind that you are already assessing! Most programs are already doing assessment, although they may not think of what they are doing in those terms. You may already be gathering the data you need. Before you add new assignments or tools, check out your current data resources. These would include: institutional information, course evaluation feedback, surveys of alumni, the National Survey of Student Engagement, and existing student records. What does this already tell you about student learning success?
When to assess
The curriculum assessment map will help you identify which activities you want to use in your course to gather assessment data. But are there ways in which it would also be important to your core course to sample student performance? Such points could include entry into major, â€śrising juniorâ€ť or some other landmark in the studentâ€™s progress through the curriculum, or perhaps shortly before or after graduation.
Samples and sampling methods:
Plans should also include a description of the groups from which you plan to gather data and when. Will you use data from all students in a single section, randomly selected students across all sections, or some other strategy? You should also consider sampling different groups, either randomly or according to some set of predetermined criteria. How many people should be included in your data-gathering? While the ideal would be to include everyone who falls within an identified group, this might not be practical. Again, the most important thing to keep in mind is what information you want to retrieve in order to answer your most important questions. Less can be more if the group selected and the information collected are a close match to what you will need.
What is a rubric? This word has multiple meanings, the common feature being that it consists of some set of standards for evaluating an assessment artifact. A rubric includes a clear statement of the characteristics of the performance or artifact to be evaluated. (For example, what would an example of professional communication include?) It also includes clear descriptions of various levels of the criterion being evaluated. (What would be the characteristics of an inadequate, adequate, or excellent example of professional communication in your discipline?) A good rubric is connected to the criteria that define a quality performance. Rubrics also allow one to create a quantifiable measure to summarize and evaluate a body of qualitative data.
The best time to develop rubrics is when you are creating your plan. Rubrics can help make assessment more transparent to various stakeholders. They can also help raters achieve better consistency or reliability in how they evaluate different artifacts. If you wait until after you have an assessment artifact in hand (e.g., a sample of a â€śprofessional communicationâ€ť), you may find that the qualities you are looking for are missingâ€”not because of poor teaching but because of uninformed teaching. Developing a rubric is an opportunity for faculty to discuss and reach consensus on exactly what they mean by a particular criterion, and, thus, are more likely to reach consensus on approaches to use to reach that criterion. This is not the same as teaching to the test; rather it is useful for instructors to know what is important to include when they develop their course syllabi and teaching plans. However, if rubrics will be developed at some later time, then that should be included in the implementation timeline.
The â€śCore Course Assessment Plan Review Checklistâ€ť and â€śCore Course Assessment Progress Report Checklistâ€ť are rubrics to help reviewers evaluate those documents (see Appendix C). The individual items are aligned with the parts of the plan and progress report forms. The sub-items are aligned with the components that would need to be present to be a good plan or progress report. The overall rating rubric provides an opportunity to provide a global evaluation, taking into account performance on all of the critical features listed, in a format that can be easily summarized across programs or courses.
If you have not considered costs during your planning process, this would be a good time to do that. â€śCostsâ€ť include not only materials (e.g., standardized tests and their scoring) but human resources. Who will collect, organize, analyze, and report the data? Will you need to pay for outside services such as external reviewers? Will you want to provide some form of compensation for students, alumni, etc.? Does the department or program have the resources needed to conduct the assessment research outlined in the plan? If not, does the college have resources that could be allocated for this project? While not a requirement for getting your assessment plan approved, not calculating in your resource needs on the front end may make a criticaldifference in the actual success of your plan.
The final part of the assessment plan is implementation. Plans should be written with a specific â€ślife cycleâ€ť or â€śassessment cycleâ€ť in mind. The length of the cycle will vary based on several factors. One will be the scope of the plan itself. How many years it is reasonable to expect that it will take to address all of the learning goals, objectives, and questions outlined in the current plan? At the completion of that time period, the academic unit offering a core course will need an additional year to review all of the findings over the assessment life cycle, and to then revise their old or develop a new plan for the coming cycle. Another factor could be the length of time between program and/or accreditation reviews. Assessment data are an important part of both types of review. It might be useful to complete an assessment cycle, including developing a new plan, the year before writing either a program or accreditation review report.
There is no â€śperfect cycleâ€ť that will fit all plans. Some core course faculty groups might want to assess all objectives every year. Others might feel that
three or even five years fits best. The important thing is to include a realistic implementation plan as you develop your overall assessment plan, with an explanation of why that cycle makes sense for your learning goals and objectives.
Plan or progress report form instructions:
Section 4 of your plan asks for a complete description of the design and methods you will use to answer your assessment questions and test your important student learning outcome hypotheses. In your curriculum assessment map, you should include a brief statement indicating which activity will be used to collect assessment data at which point in the course curriculum.
This section of your plan should also include an implementation timeline for the plan. How many years are covered by this plan? Which objectives or questions will be addressed each year and how many years will it take to address all of the goals and objectives? The final year of the cycle will be an opportunity to revisit the plan and make changes based on all the information gathered over the years of the cycle. Finally, how will important stakeholders be informed about and involved in the process?
For progress reports:
Once the methods and design have been completed for the plan, they do not need to be attached to each progress report. However, you should copy and paste the activities from Section 4 that are relevant to the results reported in a given progress report. Any significant changes from the plan should be included and explained in the progress report.
VII. Assessment Progress Reports: Step 5: Reporting Findings
What were your main findings? How did you analyze them? How do you interpret them? This seems like the simplest part of any progress report, but simplicity does not belie its importance. This is your chance to describe what you found out about that aspect of your assessment plan that was put into motion during the past year. Here are some things to consider when writing the results section of your progress report. You might also look at progress reports from other programs submitted for the past year by accessing them through the Assessment Central website.
Connecting findings to goals and objectives
What are the major results or findings regarding the goals and objectives addressed in the current report? These should be clearly aligned with the outcomes that they address.
Interpreting for different audiences
Your progress report may be read by several different audiences. It should be read by all of the faculty and staff involved in your program. It will also be read by your college assessment committee, by your Dean, and eventually by the Provost or his representative. Progress reports are posted on the Provostâ€™s Assessment Central website and thus are part of our public profile: evidence for our various stakeholders of our efforts to provide the best possible learning environment for our students. These reports will, at some point, be read by program review and/or accreditation bodies for whom assessment of student learning outcomes is one of the criteria for evaluation of our educational programs.
Progress report form instructions
From your plans:
The first four parts of the progress report form can be copied from the relevant sections of your assessment plan. Only the goals, objectives, methods, etc., that are relevant to the research conducted during the progress report period need be included.
On the progress report form:
Section 5 is your opportunity to report on your assessment findings for the progress report year under consideration. Graphical data representations can be included, but it is also helpful to have a narrative that summarizes the main highlights of your findings.
VIII. Assessment Progress Reports: Step 6: Using Results
With this step, we come to the final piece of the assessment puzzle. The success of an assessment plan ultimately rests on whether or not it provides you with the information
needed to make informed decisions about your core courseâ€™s future. Assessment findings must be usable and used for plotting the future of your curriculum.
Here are some things to keep in mind as when writing this section of your progress report. You might also look at progress reports from other programs submitted for the past year by accessing them through the Assessment Central website.
Connecting decisions to results
Assessment writers often talk about the importance of the â€śassessment feedback loop.â€ť The underlying idea is that results provide feedback that leads to decisions that in turn lead to new goals and objectives emerging for a given curricular program. Assessment plans that do not incorporate a feedback loop are seen as failures, no matter how much data is gather or how psychometrically meticulous that data may be.
Communicating results and inclusion in decision-making
For information to be used, it must be communicated. How was the information passed on to those who need it to inform their decision-making? Was it communicated in a way that was transparent to the user? Assessment results should never be used to lay blame on individuals, but rather should be communicated in ways that will lead to ever improving program quality.
Progress report form instructions:
What conclusions can be drawn from the data? What changes, if any, have been or will be made? This could include a revision of the objective or the assessment method to be used in the future. What improvements have been made based on assessment findings? How will the findings be used to make decisions about curriculum and instruction? Will your target for the coming assessment year change? How was feedback about the assessment results and program improvement communicated internally and externally? You should include how faculty, students and other stakeholders were involved in decision-making based on the results.
IX. Reviewing Assessment Plans and Progress Reports
UALR has an annual review process in place for progress reports and a multi-level approval process for assessment plans. Why? One reason is to provide assistance, feedback, and guidance to programs. Your best resources for developing a successful assessment plan, and getting the most out of your progress reports, are your fellow faculty members in your college. By reviewing each otherâ€™s efforts, we can share our successes and move forward as a campus. Another reason is to be sure that no program is left behind when it comes time for external reviews, such as our discipline-specific and campus-wide accreditation visits. It is easy for assessment to slide to the â€śback burnerâ€ť while we deal with immediate crises and needs. However, assessment is not something that can be done at the last minute. Annual review and monitoring helps keep the whole campus engaged and on-target.
Review versus approval:
Plans are submitted for approval and progress reports are submitted for review. The purpose of the approval process is to ensure that the assessment plan contains all of the required components and has the best possible chance of success. Are the goals and objectives clearly defined? Are they measurable? Is the plan and timeline realistic? Are all of the goals and objectives included in the implementation plan? The purpose of the review process for progress reports is to ensure that the plan is unfolding as proposed and that the results are being used appropriately for program or course improvement.
Level of approval or review:
The first formal level of approval or review for plans is at the college level. Each college has an assessment committee that evaluates new or major revisions of plans as submitted. This includes all core course assessment plans. The second level of approval for plans is at the university level. The Provost Assessment Advisory Group (PAAG) evaluates new plans, following approval at the college level. One representative from each college sits on PAAG, usually the chair of the college assessment committee. That individual presents the plan, provides feedback to the academic unit proposing the plan, and in general functions as a liaison between the program and/or college and the university committees. Approval of the assessment plan by PAAG must accompany any request to the Undergraduate or Graduate Council for including a new course into the core curriculum.
There is only one level of review for core course progress reports. This is done each fall and concludes with a review session during the annual Assessment Expo. Progress report writers review and provide feedback to each other through the mechanism of small group discussion, followed by a written review summary. The core course assessment progress reports are then posted to the Assessment Central webpage through the Provostâ€™s Office.
The review checklists found in Appendix C are designed for evaluating core course plans or progress reports. Each section lines up with one of the required sections on the plan or progress report forms.
Instructions for review checklists
There is a section on the review checklist that corresponds to each section of the assessment plan. The reviewer should check all items that apply to the plan under review. Each section also has a place for comments relevant to that section of the plan. At the end of the checklist is a place where the reviewer can give an overall evaluation of the plan.
For progress reports:
There is a section on the review checklist that corresponds to each section of the progress report. The reviewer should check all items that apply to the report under review. Each section also has a place for comments relevant to that section of the report. At the end of the checklist is a place where the reviewer can give an overall evaluation of the report.