I. Mission Test
The mission of the Department of Speech Communication at UALR is to foster the co-construction of better social worlds through positive communication. Our focus is to enable students to practice positive communication in interpersonal, organizational, and public contexts.
Consistent with our departmental mission, the department is united around five theoretic models, which represent our core values. These five models include: (a) Positive Communication, (b) Organizational Culture, (c) Diffusion and Transformation, (d) Crisis and Renewal, and (e) Experiential Learning. The pillars function as areas of inquiry, expertise, and foci for teaching and research. They reflect the department’s cohesive focus on ethical communication that is apparent in both our undergraduate and graduate programs.
Throughout our curriculum, we attempt to get students to explore and answer theoretically-driven “big questions” (from Pearce & Cronen’s Coordinated Management of Meaning theory) using these core values.
- What are we creating in our communication?
- How are we creating those things with our communication?
- What do we want to create?
- What forms/kinds of communication will help us create what we want?
- How can we implement this type of communication?
We help students learn to better analyze and develop messages, anticipate communication barriers and accomplish communicative goals, and embrace differences and influence discourse around them. Students learn to apply theory to everyday situations in order to improve the communication within those situations
Our vision is to enhance our centrality to UA Little Rock’s mission and vision related to student success, applied scholarship, and community engagement, resulting in regional and national recognition.
This vision involves the following strategic goals for our department:
- Build stronger bridges with the community to facilitate service learning, internships, collaborative grants, and joint faculty-student projects for applied research and service;
- Engage our alumni and interested community members in providing scholarships and paid internships to provide a path for student academic support;
- Increase faculty size in order to fully support our growing face-to-face and online student populations;
- Create additional pathways for and through graduate education for interested students;
- Access grant funding to develop the Communication Skill Center into a regional center of excellence for communication issues;
- Gain support for a building/space that creates professional, inviting spaces for students, alumni, community members, and faculty.
Teaching Highlights: Instructional Activities/Curriculum
Highlights of our teaching accomplishments this year include focusing specifically on the student success and curriculum areas of our programs, participating in several important high-impact practices, and receiving excellence awards.
Student Success & Curriculum Work
We continued this work via our Student Success and Curriculum sub-committee this year, which is responsible for assessing and revising curriculum in our department to improve student learning. The faculty on this committee undertook the development of a five-year assessment plan for our new undergraduate learning outcomes and curriculum this past academic year. Faculty pilot-tested and normed rubrics to be used for the first two outcomes of our newly restated Applied Communication undergraduate major, using the Blackboard Outcomes tool. In addition, graduate faculty worked to revise the learning outcomes for our master’s program, and developed specific timeline goals for students to successfully complete required master’s coursework and research.
High-Impact Learning Experiences
Students continue to be involved in working on internships within the Communication Skill Center (CSC), in which they learn to tutor students and assist in workshops across campus on improving student communication skills. Participation in this internship averages around 5-10 students per semester, and students are able to repeat this credit for consecutive semesters, if they are “re-hired” by Dr. Kristen McIntyre and the graduate assistants who provide direct supervision to the interns. These internal internships provide good experience for students who are looking to be hired for external internships in the future.
Our externship coordinator, Melissa Johnson, continued to build her contacts in the community to increase student opportunities for external internships, with 10 students taking advantage of these this past year. For example, she developed additional partnerships with UAMS in the patient advocacy program, as well as with several individual partnerships within the for-profit and non-profit arena for our students, based on their professional interests. She continued to hold students and organizations accountable for their partnerships through internship observations, student portfolios, and regular meetings.
- Study Abroad
Dr. Avinash Thombre took a group of undergraduate and graduate students to Trinidad and Tobago in Fall 2016, providing them with an immersive experience with individuals who are situated on the margins of three cultures. This high impact class helped develop in students an understanding of the complexity of intercultural communication skills in day-to-day interactions.
- Capstone Projects & Research
Undergraduate students are required to pass our department’s capstone research project and presentation before they can graduate from our program. They regularly work with faculty mentors on their projects in order to be successful. Students produce presentations that clearly discuss how a theory of their choice applies to their own case study research. These projects and presentations provide the basis for our Undergraduate Program Assessment.
- Community Service
Many of our classes require service learning, with several of our faculty members trained as service learning educators. For example, this past year faculty required the following service required in our classes.
- SPCH 1300: Four hours of required service incorporated into student class projects and speeches.
- SPCH 3320 Advanced Public Speaking: Eight hours of required service incorporated into class projects and speeches.
- SPCH 4314 Internship: Communication Skill Center interns facilitate communication skill-building activities and PowerPoint workshops in SPCH 1300.
- SPCH 7352 Organizational Communication Training: Students conduct trainings with local nonprofits.
- Graduate Student Culture Analysis Projects, approximately 10 each year, involve serving local organizations by analyzing culture and communication and providing practical implications.
As an example of the impact of this work, for this year, between the SPCH 1300 and 3320 classes alone, we had 3,432 service hours reported. Just for these two classes, this equaled a total of $82,848 donated to our communities, based on the estimated value of volunteer time (equivalent of $24.14 per hour; see http://independentsector.org/resource/the-value-of-volunteer-time/).
- Excellence Awards
Dr. Kristen McIntyre continued her two-year receipt of teaching awards, adding this year the College of Social Sciences and Communication’s “Willingness to Go Above and Beyond” award for her leadership in assessment, development and delivery of multiple communication workshops, and dissemination of best practices in service-learning. She has also been instrumental in the standardization of assessment rubrics in departmental undergraduate courses, giving the department guidance in tracking student progress from the basic course through the capstone course. Dr. McIntyre mentors graduate students in best assessment practices by teaching them how to assess their trainings. At the university level, she participated in creating speech communication learning outcomes for the university core, co-created a presentation rubric for use in assessing speeches in campus-wide classes, and developed assessment procedures for collecting artifacts and coding presentations.
Dr. McIntyre regularly conducts campus and community workshops, as well as oversees graduate assistant and undergraduate intern facilitators in workshops. Her community-based workshops impact a wide array of organizations such as AmeriCorps, the Arkansas Department of Health, Immerse Arkansas, JobCorps, and the UAMS Center for Orthopaedic Research. These workshops cover a wide array of topics such as Persuasion, Audience Engagement, Establishing Credibility, Networking, Employee Engagement, Conflict Resolution, Constructive Feedback, Transformational Leadership Skills, Team Communication, and Ethical Feedback. This sample of topics reveal the breadth of her work and commitment. Dr. McIntyre has created a relationship with “Immerse Arkansas,” a non-profit focused on providing mentoring to youth as they age out of the foster care system. She annually engages her graduate organizational communication training class in conducting trainings for this organization. Furthermore, she now has three students creating an “Academy of Communication Excellence” which will equip others to provide ongoing training.
Research and Creative Activity/Scholarly Activity Highlights
As is normal for the tenured and tenure-track faculty in our department, our faculty members were active in participating in research and scholarly activity this past year. A sample of the scholarly work this year done by our faculty include the following.
- Chatham-Carpenter, A. (in press). The future online: Instructional communication scholars taking the lead. Communication Education, 66 (4).
- Chatham-Carpenter, A. (2017, April). Online, hybrid, and face-to-face communication classes: Can they co-exist in the same program? Presentation at the annual conference of the Southern States Communication Association, Greenville, SC.
- Heistad, D., Chatham-Carpenter, A., Moser, K., & Woods, K. (2017). Educating with purpose: An integrated communication model for first-year student success. In T. Skipper (Ed.), What makes the first-year seminar high impact: An exploration of effective educational practices. Columbia, SC: National Resource Center for the First-Year Experience & Students in Transition.
- McIntyre, J.J., & McIntyre, K. A. (in press). Why are there so many crucifixes? In Brunner and C. Hickerson (Eds.), Cases in Public Relations: Translating Ethics into Action. Oxford University Press.
- McIntyre, K. A. (2016, September). To perform or not to perform: The case for a communication orientation to public speaking. Talk presented at the Arkansas Communication and Theater Arts Association Conference, Eureka Springs, AR.
- Mirivel, J. C. (2017). How communication scholars think and act: A lifespan perspective. New York: Peter Lang Publishing.
- Mirivel, J. C., & Fuller, R. (2017). Social talk at work: Speech acts that make a difference. In B. Vine (Ed.), Routledge handbook of language in the workplace. New York: Routledge.
- Mirivel, J. C. (2016, November). Positive Communication’s Civic Calling (Co-Chair and co-organizer). Pre-conference session at the Annual Meeting of the National Communication Association, Philadelphia, PA.
- Mohammed, S. N., & Thombre, A. (2016). An investigation of user comments of Facebook pages of Trinidad Tobago’s Indian music format radio stations. Top 4 papers in the Intercultural Communication Division at the International Communication Association Annual conference Fukuoka, Japan.
- Thompson, C. & Kleine, M. (2016). “Possibilities and problematics of promoting a liberatory classroom in an institutional context: The University,” published power point. Future of Education.
- Thompson, C. L., & Kleine, M. W. (2016). Varied responses as a means to the richness of discourse: Reading tough texts. International Journal for the Scholarship of Teaching and Learning, 10 (1), Article 5. Available at: http://digitalcommons.georgiasouthern.edu/ij-sotl/vol10/iss1/5
- Thompson, C. & Kleine, M. (2017). Using literature to explore interpersonal theory: Representation of rhetorical objectification and oppression. Journal of Pedagogy, 7 (2), 97-118.
- Thompson, C. & Kleine, M, & Rabalais, A. (2016). Resituating Freire in an academic context. LibreriaUniversitaria, Future of Education, 2016. 6th June, 2016.
- Whitaker, R., & Mirivel, J. C. (2016, November). Positive communication during divorce. Paper presented at the Annual Meeting of the National Communication Association, Philadelphia, PA.
Service and Engagement Highlights
Listed below are some specifics about the excellent service and engagement of faculty and students in our department.
Avinash Thombre completed the LeadAR program, which is a training program for emerging Arkansas leaders, sponsored by the University of Arkansas Division of Agriculture Cooperative Extension Services. This program has provided opportunities for Dr. Thombre to network and learn about economic and social issues facing communities in the state of Arkansas, in order to be able to address critical problems in these communities.
Every year, the faculty conducts trainings, workshops, presentations, or consulting in a wide variety of organizational contexts. Here is a sampling of their participation this past year.
- Arkansas Communication & Theatre Arts Association workshop facilitator
- Arkansas Public Administration Consortium workshop trainer
- American Heart Association emcee
- Boy Scouts of America workshop presenter
- Central Arkansas Marriage Initiative facilitator
- Dassault Falcon Jet consultant
- National Association of Social Workers (NASW) consultant for Arkansas chapter & conference presenter for national conference
- Service-Learning Academy for Community Connection Center facilitator
- Single Parent Scholarship Fund mentoring
- Preaching team for a local church
- Prayer team for a local church
- Presentations on effective presentations to multiple classes across campus
- Presentations on networking and workplace appreciation to classes across campus
- Presentations on large vs. small group communication to classes across campus
- UAMS Leadership institute workshop facilitation
In addition, students, staff, and faculty connected with the Communication Skill Center at UA Little Rock presented 31 workshops between July 1, 2016 and June 30, 2017, on topics as diverse as providing ethical feedback, effective speech structure, negotiating, networking, and elevator pitches. These workshops were presented to students in marketing, management, construction management, nonprofit management, social work, computer and information sciences, and earth sciences, as well as students across campus in venues sponsored by the Academic Skills Center and our department.
Faculty participate in key committees and initiatives on campus, such as the following.
- Academy for Teaching & Learning Excellence
- Anderson Institute on Racial & Ethnic Diversity
- CSSC ad hoc Communication Committee
- CSSC Assessment Committee
- CSSC Awards Committee
- CSSC College Policy & Governance Committee
- CSSC Core Curriculum Committee
- CSSC Dean’s Advisory Council
- CSSC Undergraduate & Graduate Curriculum Committees
- Disability Resource Center Faculty Advisory Board
- Discover UALR
- Donaghey Scholars
- Faculty Advisory Board to the Provost
- International Celebration Week Planning Team
- International Studies Advisory Committee
- UA Little Rock Core Assessment
- UA Little Rock Faculty Mentoring Program
- UA Little Rock Service Learning Certificate Committee
- UA Little Rock Sustainability Committee
- UA Little Rock Task Force on Program Coordination
Student Success Highlights: Recruitment, Retention, & Timely Graduation
We are very pleased with the outcomes of our recruitment efforts to help bring students into our major. We continue to use an appreciative and intrusive advising model to help students be retained through to graduation. Listed below are some specifics about these efforts.
In the past two years, the Department of Applied Communication has seen an almost 50% increase in the number of undergraduate students majoring in its program. Much of that increase comes from having a bachelor’s degree option that is now completely online, as part of the UA Little Rock Online program, as well as a strategic use of communication to stakeholders about our program over the past three years.
Consistent with our departmental communication plan, we have worked on a number of marketing & recruiting initiatives this past year, including:
- Rebranding our departmental website, flyers, and advising sheets to include our new name.
- Maintaining a closed Facebook group for the Department, which highlights graduate student, undergraduate student, and faculty accomplishments, as well as showcases the work of program alumni.
- Showcasing faculty, students, and alumni on our webpage.
- Maintaining a public Facebook page for the CSC that promotes the importance of communication scholarship, as well as events and workshops related to the CSC and the department.
The department continued its second year of work with two departmental committees in order to best deal with the functions of recruitment, retention, and timely graduation. Half of the faculty serve on one of the committees, and the other half serve on the other committee.
- The Student Success and Curriculum (SSC) committee focuses on building curriculum and assessing student learning, as well as improving advising across the programs of the department.
- The Building Bridges with the Community (BBC) committee focuses on recruiting students into the various programs and marketing the programs to external stakeholders.
Each faculty member is trained in how to declare majors. We continue to use the undergraduate advising form representing the Fall 2015 catalog changes, which help students know where they are in the process of obtaining their degree. In addition, we created an interactive google sheet advising form for the upcoming Fall 2017 catalog changes.
We have also worked on creating two-year and four-year advising plans for the new Degree Works university application, which will help our students when visualizing how they can finish their degrees as new first-year or transfer students.
Once a student declares Applied Communication as a major and has taken the required ACOM/SPCH 1300 course, they are advised to take three core classes (ACOM 2310, ACOM 2311, ACOM 3320) in their first 2-3 semesters of the program (or one, if they are within 30 hours of graduation). These classes focus on learning the theories of the discipline, conducting case study research, and preparing and delivering persuasive presentations. Most students follow this advice, enabling them to have a strong base that sets them up for success to take further professional and elective classes. The case study approach grounds our entire curriculum and unites efforts to assess our learning objectives in the form of our capstone class.
The department launched its third Communication Week event this past spring. The week focused on providing additional resources for our students related to communication knowledge and skills. Faculty led games and simulations for students. Faculty and guest speakers facilitated workshops related to interviewing and networking. An alumni “Lunch and Learn” event allowed students to learn from departmental alumni, and there was an alumni fund-raising event held in conjunction with our leadership lecture on Change Leadership by one of our alumni.
Our Student Awards Committee refined the criteria for our annual undergraduate and graduate Distinguished Scholar Award, as well as Making a Difference Award. We gave these awards out to five students in Spring 2017, and have a plaque displaying their names in our office area.
The department reinstituted its honors society connection to the National Communication Association’s Lambda Pi Eta student organization, inducting 18 new members, and welcoming 3 additional members from another chapter into our local chapter.
Students who took the SPCH 1300 course at UA Little Rock have had a 74.8 – 81.3% (76.4% average) retention rate for the past 10 years (based on enrolling the next academic year), while students who did not take a SPCH 1300 course at UA Little Rock or elsewhere had a retention rate of 45.6 – 57.0% (50.8% average). If students transferred in a SPCH 1300 course, their retention rate for the past 10 years averaged between 72.8 to 76.8% (74.1% average). These numbers show the importance of students taking the SPCH 1300 course to be successful in college.
Development and Alumni Relations Highlights
We established a Building Bridges with the Community (BBC) departmental committee two years ago, who worked with the Department Chair and Alumni Advisory/Development Board to put on an Applied Communication Leadership Lecture Series, beginning in Fall 2016. This four-date lecture series included both faculty and alumni as facilitators of topics such as positive communication, nonverbal communication, managing conflict for leaders in the workplace, and change management. The average attendance at each of the lectures this past year was 20 people. We will be hosting the second annual leadership lecture series this coming year.
Our Alumni Advisory/Development Board consists of eight BA and MA alumni:
● Koy Butler, House of Three
● Debbie Knight, One Banc
● Tamidra Marable, Heifer International
● Mary Cantrell Twedt, UAMS
● Alex Long, Jason International
● Bruce Trimble, Bridgeway Hospital
● Ashley McNatt, Dept. of Health
● Christy Standerfer, Clinton School of Public Service
Two years ago, we began a targeted funding campaign to promote two endowed scholarships in Applied Communication, named after our former dean Dr. Angie Brenton and emeritus professor Dr. Allan Ward. This funding campaign has also been supported by our Alumni Advisory/Development Board.
To support ongoing excellence in undergraduate and graduate education, our Department has additional funding opportunities, including Undergraduate Student Support and Graduate Student Support, as seen on the following weblink (https://ualr.edu/appliedcomm/give-to-the-department/). In addition, we added a direct giving link on our webpage for these opportunities (https://ualr.edu/giving/product/Applied-communication/).
Our assessment processes for our SPCH 1300 course, and our undergraduate and graduate programs, all involve direct measures. There are specific learning outcomes for each of these, and students’ work in their classes are compared to a rubric, with aggregated results from the assignments used to determine if specific learning outcomes are being met by the course and programs. Specific information about how we did these types of assessments in the past year are included below.
In order to have a course in UA Little Rock’s General Education Core, courses must meet certain skills and values (https://ualr.edu/facultysenate/files/2013/12/Policy503.3_v2.pdf) . For the SPCH 1300 course, we aligned our course to these skills and values, as seen below.
By the end of the SPCH 1300 course, students will be able to:
- Apply foundational communication principles to a variety of contexts. (UA Little Rock General Education Core Skills 1.2, 2.1, 2.2)
- Present speeches in an organized manner. (UA Little Rock General Education Core Skills 1.1, 2.1)
- Use technology to draft components of a presentation. (UA Little Rock General Education Core Skills 3.1)
- Integrate online research into a presentation. (UA Little Rock General Education Core Skills 3.2)
- Evaluate communication behaviors in multiple contexts. (UA Little Rock General Education Core Skills 1.2, 2.1; Core Values 1.1)
- Generate communication-based solutions for a given situation. (UA Little Rock General Education Core Skills 2.2; Core Values 1.1)
- Articulate the relationship between communication and positive community change. (UA Little Rock General Education Core Values 2.1)
In the SPCH 1300 course, students are given a “misconceptions about communication” pretest at the beginning of the class and a posttest at the end of the semester. Difference scores are used to determine if there is a significant difference in improvement of misconceptions from the beginning of the semester to the end. In addition, the following student work pieces are collected for assessment purposes: (a) self-introduction speech, (b) positive communication video, (c) informative service speech, and (d) interpersonal communication paper. Pre- and post-test speech and paper rubrics are applied to these presentations and analyses.
This past year, faculty focused on assessing Core Skills 1.1 and 1.2. They also piloted assessment rubrics for Core Skills 2.1 and 2.2. Student Informative Service speech videos were used to assess Skills 1.1 and 1.2, as well as Skills 2.1 and 2.2. In addition, Interpersonal Communication papers were used to pilot assessment rubrics for Skills 2.1 and 2.2.
A total of 337 students were enrolled in SPCH 1300 during the Fall 2016 semester. Six (6) SPCH 1300 sections were randomly selected for assessment prior to our involvement with piloting Blackboard Outcomes. These sections included 3 face-to-face sections, 1 8-week online section, and 2 16-week online sections. Due to the time frame of piloting Blackboard Outcomes, 112 paper artifacts were previously collected earlier in the semester from the six selected sections. Thirty-four (34) paper artifacts from the face-to-face sections and 33 paper artifacts from the online sections were randomly selected for a 20% sample size of the 337 potential artifacts. Of the selected artifacts, three were not scored because they were the incorrect artifact and three artifacts were removed for being incomplete submissions (parts of the assignment were missing due to copying/PDF errors), leaving a total of 61 scored paper artifacts. In addition, using the Blackboard Outcomes platform, 7 video artifacts were randomly selected for assessment of Skills 1.1 and 1.2, and the Skills 2.1 and 2.2 rubric pilot.
In the undergraduate BA program, we use our capstone SPCH 4300 course as our major assessment tool. This past year, students’ presentations continued to be evaluated by both faculty and external stakeholders, using pre-established rubrics. During and after students’ presentations, faculty and external stakeholders assess students’ capstone presentations to refine our classroom teaching. The capstone presentations undertaken by our students and the resulting evaluation by faculty and stakeholders provide us with insights about our teaching and students’ learning.
This year we modified the rubrics for our capstone presentation to have macrostructure of their presentation be a required element. This allowed us to focus more in the rubric on the application of theory to their case, as well as adapting their message to their audience. This helped us move to be in line with our revised undergraduate learning outcomes described below.
We believe the curriculum in the Applied Communication department at UALR enables students to learn how to practice positive communication in interpersonal, organizational, and public contexts to make a difference. We believe that the coursework and experiential learning in which students in our programs engage, while in our department, allow them to be able to do the following.
Communication graduates use a positive communication framework to ethically lead by:
- Analyzing messages
Communication graduates use communication theory to effectively critique messages. This message analysis is done through active listening and critical thinking, allowing communication graduates to enact mindful responses to messages.
- Developing messages
Communication graduates create and adapt their messages to different purposes and contexts, using a variety of methods. When appropriate, they co-create their messages with their stakeholders.
- Anticipating communication barriers
Communication graduates critically apply communication theory to situations involving leadership, relationships, and change. In doing so, they assess potential breakdowns in communication, and generate communication recommendations for self and others.
- Accomplishing communicative goals
Communication graduates are effective as leaders in change management. They work with others to co-create goals and collaboratively problem solve situations. They evaluate their own personal communication strengths and weaknesses to improve their own communication as change leaders.
- Embracing difference
Communication graduates embrace differences between others via active listening and positive communication. They respect diverse perspectives and the ways those perspectives influence communication. They adapt their communication in diverse cultural contexts.
- Influencing discourse
Communication graduates recognize the importance of communication in everyday life. They use their communication and research skills to frame and evaluate personal, local, national, and global issues, in order to productively respond to those issues. In doing so, they are strong advocates for applying best practices in communication within various contexts.
The Department of Applied Communication participated last year in the Provost’s Program Assessment pilot program. As part of our preparation going into the self-study for our program review, we spent the year revising our undergraduate learning outcomes, mapping our curriculum, identifying artifacts for assessment, and building learning outcome rubrics. Given the massive overhaul of our program assessment process, the small Spring 2016 Capstone class (3 students), and our involvement in piloting Blackboard Outcomes this past fall, we chose to focus our assessment efforts on the 17 student artifacts from fall 2016.
On Friday, January 13, 2017, the department held an assessment retreat. During the retreat, four tenure-track faculty, three full-time instructors, and one SPCH 1300 adjunct instructor normed program student learning outcome rubrics for Goals 1 and 2. In preparation for the retreat, faculty were asked to score past papers for both SPCH 2311 and SPCH 4300 on Goals 1 and 2. During the retreat, faculty talked through their respective scoring, and adjustments were made to the rubric descriptions. Faculty then broke out to score additional papers, regrouped, and discussed scoring, which resulted in greater inter-coder reliability.
In addition, our program underwent Program Review this past year. In preparation, we conducted a survey with our current students and alumni. In line with our program assessment of our first two undergraduate student learning outcomes this cycle, our current undergraduate students (48/120, 40% response rate) stated, on average, that they believed that their skills and knowledge have improved in all of our student outcome areas as a result of taking classes in our department, with averages ranging from 4.16 to 4.52 (on a scale of 1-5, not at all improved to very much improved).
We have three overall learning goals for our graduate program. Each year we look at one of these learning goals, rotating through them every three years. These goals are for students, by the time they graduate, to be able to:
- understand and engage in applied communication research,
- develop ethically responsible recommendations for applied communication contexts, and
- make effective presentations to shape or inform communication practices.
We collect several student artifacts over the course of their program, which we use to explore how students are learning, related to these goals. These artifacts include students’ final research projects and their portfolio self-assessments. We use pre-established rubrics for all of these. In addition, their final project is presented to a professional audience of faculty members.
This past year we were in the 5th year of our assessment plan, which focuses on an “assessment of our assessment” process. Growing from a Graduate Faculty retreat in August 2016, we determined the need to change our plan. As part of a transition into this new plan, we collected pilot data from a student portfolio self-assessment. This self-assessment required reflection on all three major program goals. In addition, we conducted an alumni survey on “course sequencing and project preparation,” a focus group with our board members on revised program goals, and an additional student survey focused on skill development as part of our 2017 Program Review Self-Study.
Our faculty participated in a “blind” review of self-assessment papers with a rubric, which we normed in norming sessions. Each paper/presentation was rated by two separate raters, with a total of 11 papers and 1 presentation rated. Ratings were within 1 point of each other except in 8/600 instances, for a 96% inter-coder reliability rating within 1 point. Ratings were the same exact rating in 85/214 instances, for a 60% inter-coder reliability rating. The areas with the most disagreement were on the program learning outcomes items and the item on whether the student identified adequately areas in which to continue growing.
Across all outcomes, the majority of students are meeting or slightly exceeding the 2.00 benchmark in most assessed evaluative elements.
Skill 1.1. Presentational structure. The majority of sampled students met (or exceeded) the benchmark for six of the nine evaluative elements: attention getters (80%, 2.30), speaker credibility (70%, 2.40), preview statements (70%, 2.30), transitions (90%, 2.40), summary statements (77%, 2.11), and clinchers (77%, 2.44). A few structural elements, speaker credibility, transitions, and clincher, were surprisingly strong (2.40+). In terms of student learning, the sampled SPCH 1300 students have a fairly strong foundational understanding of and ability to enact the majority of elements that make up effective presentational structure. However, it is clear that sampled students struggled with establishing listener relevance (30% not attempting, 1.80), stating a clear goal for the speech (40% not attempting, 1.70), and restating the speech goal (44% not attempting, 1.78).
Skill 1.2. Application of communication concepts to interpersonal contexts. The majority of sampled students met (or exceeded) both evaluative elements: selecting communication concepts for application (80%, 2.40) and applying communication concepts (80%. 2.20) to interpersonal contexts. Interestingly, the majority of the sampled students (60%) were able to identify more specific communication concepts for application represented by a 3 and 4 score. This suggests that students are developing a slightly more refined understanding of the complexity of communication than anticipated.
Skill 2.1. Adaptation of messages to audiences. The majority of sampled students met the 2.00 benchmark for two of the three evaluative elements: Use of language (82%, 2.09) and communication orientation (70%, 2.10). Overall, the sampled students are able to use language and delivery behaviors that are clear and not distracting from the message. However, as reflected in the structural outcome, sampled students struggled with establishing listener relevance (36% not attempted, with an average score of 1.56).
Skill 2.2. Making recommendations to improve communication. Three of the four evaluative elements reflected student learning at or above the 2.00 benchmark: selecting communication concepts for self recommendations (90%, 2.69), applying concepts for self (85%, 2.41), and selecting communication concepts for others in the relationship (85%, 2.52). Similarly to the Skill 1.2 results for selecting communication concepts for application in an interpersonal context, the sampled students showed a much more sophisticated ability to select specific communication concepts to guide their recommendations for improving communication. Additionally, the sampled students demonstrated a slightly more developed ability to apply the concepts to themselves, being able to explain the difference the concept would make. Not surprisingly, students struggled with applying the communication concepts to others in the relationship (30% scored a 1, with an average score of 1.98), providing only basic explanations at best, without addressing the positive difference that would be made by applying communication concepts in relationships.
Faculty were assigned student artifacts (papers and presentations) to score from the Fall 2016 SPCH 2311 and SPCH 4300 courses through the Blackboard Outcomes platform. Scoring took place over a two-week time period. The number of papers and presentations that were randomly selected to be reviewed included the following.
- SPCH 2311 Papers: 10 (27% of total students in class) scored artifacts
- SPCH 2311 Presentations: 7 (19% of total students in class) scored artifacts
- SPCH 4300 Papers: 10 (59% of total students in class) scored artifacts
- SPCH 4300 Presentations: 6 (35% of total students in class) scored artifacts
SPCH 2311 is an entry-level course in our program, so a score of 2 (out of 4) or better was expected for the assessed components for Goals 1 and 2 in both papers and presentations. SPCH 4300 is our senior capstone course, so a score of 4 (out of 4) was set as the expectation for the assessed components for Goals 1 and 2 in both papers and presentations. Means were used to determine how well each component was being met. Each artifact was reviewed by two faculty members. Overall, inter-coder reliability was strong.
Students are meeting, or closely approaching, all outcomes at the set benchmark.
- Students are exceeding the benchmark (2) in the following areas for outcomes 1 & 2: Organization-2.4 (oral & written), Use of Language (presentation)-2.8, Control of Mechanics-2.45, and Communication Orientation-2.3.
- Students are meeting the benchmark (2) in the following areas: Concepts-2.14, Use of Language (paper)-2.15, and Sources & Evidence-1.9.
- Students are performing inconsistently with regard to making Claims and Interpretation of Evidence with regard to the paper and presentation artifacts. Students are much stronger in both areas (2.07, 2.29) in their presentations versus their papers (1.55, 1.7).
Students are meeting, or approaching, the majority of outcomes at the set benchmark.
- Students are meeting, or approaching, the benchmark (4) in the following areas for PLOs 1 & 2: Organization-3.7 (oral & written), Use of Language (presentation)-3.9, Sources & Evidence (presentation)-3.7, Concepts (paper)-3.7, Concepts (presentation)-3.9, Claims (paper)-3.55, Claims (presentation)-3.67, Interpretation of Evidence (paper)-3.55, and Interpretation of Evidence (presentation)-3.67.
- Students are struggling to meet the benchmark in the following areas: Use of Language (paper)-3.2, Interpretation of Evidence (paper)-3.15, Communication Orientation (oral)-3.2, and Control of Mechanics (written)-3.1.
- Students are performing inconsistently across the paper and presentation with regard to Use of Language and Sources & Evidence. Students are much stronger in both areas (3.9, 3.7) in their presentations versus their papers (3.2, 3.15).
It was interesting to us that students in both 2311 and 4300 were much stronger in their presentations, in general, than in their papers. In discussing this with the Student Success & Curriculum sub-committee faculty members, we wondered if it is easier to use effective language when adapting to a live audience than when writing to a perceived audience. We also wondered if the difference was related to varying rater expectations for use of language in an academic paper. Also, we highly emphasize the oral citation of evidence and sources for students’ 20 minute capstone presentations, and although we stress APA in-text citations for their papers, those may not be as noticeable to a reviewer when reading 20 page papers. Students also work with a mentor for their presentation and overall have more practice giving presentations than writing research papers in our program, so that could be another reason for the differences in their scores between the two genres.
Results indicated that our students continue to demonstrate a high level of competence in presentation skills and mastery of content in the capstone course, suggesting that the changes we’ve made to both the curriculum and our teaching practices are having the desired effect. Students are clearly making progress from our 2311 course to our 4300 course, and we are pleased with their growth overall.
Using our graduate program student portfolios, we assessed the adequacy of support students provided from their portfolio related to our three graduate program goals. We also assessed other areas meant to inform our revised assessment plan. We set 2.5 as the benchmark we would like our graduates to reach, as a minimum, on a scale of 1-3. Overall patterns emerged that indicated that students were able to document learning in relation to two of the three main program goals at that level (2.5 & 2.63 respectively), but the second outcome on developing ethically responsible recommendations was lower than the expected benchmark (2.38). We also assessed their knowledge in relation to the five major applied theory areas, and again a pattern of strength emerged with mean scores ranging from 2.5 to 2.8 across their ability to document knowledge of these theories. In addition, other patterns of strength emerged in terms of documenting growth (2.5), clear writing (2.92), and organization of writing (2.75). The two primary areas of concern related to the extent their thinking about communication had evolved (2.25) and their ability to identify areas for future growth (2.0), which may have been due to the lack of clarity in the instructions given for these areas of the self-reflection.
The survey done for our program review self-study this year showed that current graduate students indicated that they believe their skills have improved in all three of our learning outcome areas, as a result of this program (scale 1 = “not at all improved” to 5 = “very much improved”), with students’ ratings averaging a 4.68 on “understanding and engaging in applied communication research,” a 4.64 on “developing ethically responsible recommendations for applied communication contexts,” and a 4.55 on “making effective presentations to shape and inform communication patterns.”
Curricular Modifications Based on Assessment Outcomes
The SPCH 1300 program is committed to the continuous improvement process and thus meet monthly to review the curriculum, norm assignment grading, and discuss ideas for activities that help students learn key concepts and skills. At the end of the fall 2016 semester, the SPCH 1300 program made two important changes. First, based on the assessment experience the previous year, the Skill 1.1 rubric was significantly revised to include the evaluative elements for the components of structure. The initial collapsed rubric, while efficient, did not provide enough detail to inform meaningful curriculum improvement. The new rubric is much more useful. Second, the SPCH 1300 program faculty decided to revise the summative Informative Service Speech assignment. With the intent to streamline the core assessment process, the assignment was revised to include content areas that would allow this one artifact to serve for assessment of all expected core outcomes.
Given the small sample size of our Blackboard Outcomes pilot this past year, we will not be making additional drastic changes to the curriculum. However, at our March 2017 SPCH 1300 faculty meeting, we devoted time to discussing strategies for providing students with activities (F2F and online) for developing their application of communication concepts for improving others’ communication in the Interpersonal Paper. In April 2017, we devoted time to discussing strategies for working with students on developing a better understanding and ability to enact establishing listener relevance, thesis statements, and thesis restatements in the Informative Service Speech.
All SPCH 1300 section modalities now use Blackboard to collect student speech video artifacts through a single attempt assignment drop box. Students may submit videos in a variety of formats; however, instructors will be encouraging students to upload video files into Kaltura or providing students with Google Drive links to their respective video files for submission. We feel that these two modes are most efficient for the variety of processes in place across section modalities for recording and uploading speeches.
As a result of revising our undergraduate learning outcomes, the departmental Student Success and Curriculum sub-committee created a curriculum map and an assessment plan that would allow us to assess our new undergraduate outcomes over a period of three years (two outcomes a year). Based on the curriculum map, we found we needed to do some adjustment to our major curriculum, with the start of that revised curriculum being Summer 2017. There will now be a dedicated class or class options for all of our learning outcomes, starting Fall 2017.
Each year we have developed a number of ways to involve faculty, students and stakeholders in our assessment efforts. Our entire faculty is heavily invested in virtually every aspect of our assessment process. We continue to fine-tune our assessment process to provide us with higher quality data. We have revised our learning outcomes and this year developed an accompanying assessment plan for those outcomes, which led us to make some changes in our undergraduate curriculum.
Given that our program has currently revised our program goals and piloted new assessment processes, we are using the results of this assessment cycle to inform our assessment processes moving forward. Specifically, we are working on ensuring that all courses mapped for curriculum have single-attempt assignment submission boxes for collecting student artifacts in course Blackboard shells. We are also working with the Blackboard Outcomes team to streamline user friendliness, as well as reporting of the types and level of data.
We identified the following areas needing follow-up faculty discussions related to assessment and curriculum, and have already started these discussions in our Student Success & Curriculum (SSC) sub-committee.
- Revisit our rubric descriptions for Use of Language and Control of Mechanics. Currently, Use of Language is written to capture both written and oral artifacts. However, the difference between the 2311 and 4300 paper and presentation means caused us to wonder if the description needs to be separated into two distinct categories (oral and written), like we do with Organization. With regard to Control of Mechanics we felt that the descriptions may not be capturing fully what we’d really like to be assessing here, specifically in relation to APA style.
- Revisit the benchmark (4/4) for all Capstone outcomes. In making sense of our results, we wondered if it wouldn’t be more realistic to set different benchmarks for each respective outcome. While a rating of 4 is ideal, a 3 on many of the outcomes would be commendable.
- Develop consistency in public presentation expectations. The presentations given in 2311 did not require an audience (students presented to their webcams); however, the presentations in 4300 were presented live, in front of faculty, external stakeholders, students, as well as family and friends of the presenters, which would most likely account for the lower score (3.2) for Communication Orientation in the 4300 presentations. Currently, we have no consistent expectations for a “live audience” for online student presentations. We feel this is an important conversation as we have observed that our face-to-face students tend to present much more confidently than our online students in a live audience situation. Conversely, we want to discuss ways to strengthen both face-to-face and online students’ abilities to present well via webcam, as this is a common form of conferencing and presenting in the workplace.
- Develop consistency in claim development. Students learn to make claims primarily in our required Advanced Public Speaking class. However, many of the students in the 2311 course have not yet had that class or are taking it concurrently with the Advanced Public Speaking class. In discussing these findings as part of our ongoing Student Success and Curriculum sub-committee discussions, we realized that we need to build in more opportunities for students in our 2310 and 2311 required classes to make claims using theory and case study research, to better prepare them for the higher expectations of doing this at our capstone level, and to support what is being done in the Advanced Public Speaking class.
- Develop consistency in source requirement expectations. In discussing our findings for Sources & Evidence, we confirmed that we do not have consistent source requirements across our entry-level and Capstone required courses. We believe it would be beneficial to develop a scaffolding approach to building students’ competencies with source identification and inclusion, particularly in written work.
For the graduate MA program, our students continue to demonstrate the ability to understand and engage in applied communication research through their required research projects and comprehensive exams and/or portfolio reflections. Students are introduced to a case study approach in the first semester, and then in the following semesters, they take courses that each include a case analysis. This process seems to be working in helping our students meet our outcomes.
We have some work we would like to do this coming year to improve students’ meeting of our learning outcomes. They are outlined and described below.
- Finalize our program goals to mirror both national level outcomes as well as a way to connect parallel outcomes in our undergraduate program. We utilized our 2017 Program Review Self -Study to reflect on and capture progress in discerning these revisions, and are working towards a draft which includes the following learning statement.
MA-ACS graduates will engage in the departmental mission of fostering the co-creation of better social worlds through positive communication by ethically (a) analyzing messages, (b) setting communicative goals, and (c) influencing discourse. In doing so, they will learn to answer three big questions in their professional and personal lives by collecting and analyzing research data: (1) what are we creating in our communication, (2) what do we want to create in our communication, and (3) what communication practices will create what we want to create.
- Develop a pre-program and post-program assessment process, including both direct and indirect measures. The student assessment of their learning proved to be a meaningful experience. They were able to explore early program writing, for example, with end of program writing. However, this assessment would be enriched with clearer, pre-program assessments. We are considering the following components to be a part of this assessment process:
- A 360% assessment of interpersonal competency
- Defining communication
- Connections between program learning outcomes and careers
In addition to self-assessment, we want to utilize student final projects for summative assessments. We are revising and developing rubrics for faculty review of final projects. Thus, beyond self-assessment, we will have faculty evaluations of these projects.
- Complete a full assessment plan with new outcomes, methods, project completion guidelines, and course sequencing. Sample new project completion guidelines we would like to be in place this coming year, include:
- Proposal finished & approved by end of first year
- IRB submitted by beginning of end of first year
- Collect data in Summer & Fall at beginning of second year
- Write & finish project by end of second year
Overall, the ongoing assessment of learning across our programs is used to strengthen our teaching. We have meetings as a faculty to discuss results from the SPCH 1300, undergraduate, and graduate assessment processes. We continue to reflect on three major questions in our ongoing efforts to enhance student learning.
- How can we enhance deep learning across our various programs?
- How can we develop student mindsets by challenging communication misconceptions?
- How can we improve our case study approach to communication analysis to ensure continuity across the program?
VI. Priorities for Next Year
Our goals for this coming year flow directly from our successful program review completed in Spring 2017.
Fully Implement Undergraduate Curriculum & Assessment Plans
The implementation for this priority is already in place, with the revised restatement of our BA major in Applied Communication coming online in Fall 2017. We have also created an updated assessment plan, in line with this new curriculum.
Address Enrollment Growth Issues
We need to assess how effective our current strategies are in addressing the proportions of online to face-to-face students in our classes. We plan on continuing discussions on this, determining what the need is for both types of sections and if we are meeting the needs of both sets of students. We want to continue exploring “best practice” solutions for both our students and our faculty. We also need to determine enrollment capacity for our undergraduate and graduate programs with our current staffing. We need to have further discussions about what optimal numbers should be in our presentation-based courses vs. theory-based courses. We need to continue to advocate strongly for more faculty members. If we are going to continue to grow, we will need additional faculty members.
Evaluate Advising Needs & Alternate Models of Advising
As we manage our enrollment growth, we may need to explore other models of advising. We have not really had many discussions about the future of advising, and these discussions need to take place. It would be helpful to look at other models across our field, as well as within our college. We also do not know yet what the effect of having Degree Works as a tool will make on advising, since this is just now being implemented at the university level, effective with our BA in Applied Communication major restatement this Fall.
Focus on Graduate Education Identity and Student Success
Along with the work we have been doing this year on articulating a clear and compelling identity and recruitment focus for our MA program, we need to finalize our mission and learning outcomes for our graduate program. We also need to implement our “start-to-finish” approach of introducing MA students to the program’s research requirement and providing needing scaffolding in their coursework to support the development of their project, with the goal of students completing their program requirements in 2-3 years from when they started the program.
Increase Outreach to Alumni & Other Stakeholders
We need to broaden and strengthen ties to our alumni. We need to continue and strengthen our efforts with relevant professional organizations. We need alumni to help us take point on this in the future and may need to add more members to our alumni advisory/development board to accomplish this.
We also need to build in intentional reporting of outcomes of outreach activities, and explore the possibility of using outreach activities for generating revenue. When students are working on projects with organizations, we would like to start asking the organizations to give to scholarship funds. We need to also work on getting word out about internships, as well, to raise scholarship funds.
We need to continue to use opportunities like our Leadership Lecture series to reach out to the public about what we do, and to raise scholarship funds. We need to work closely with the UA Little Rock Foundation office personnel to help tap potential donors in order to reach endowment level for both of our scholarship funds.
VII. Appendix: Fast Facts
Fall 2016 census: 784 (722 undergraduate classes, 62 graduate classes)
1st major headcount Fall 2016
- Applied Communication Studies MA – 18
- Speech Communication BA – 104
Student Semester Credit Hours
- Fall 2016
- Undergraduate classes – 2123
- Graduate classes - 186
- Spring 2017
- Undergraduate classes – 1753
- Graduate classes - 141
- Undergraduate - 2
- Graduate - 3
- Undergraduate - 7
- Graduate - 0
- Undergraduate - 18
- Graduate - 5
Our Fall 2014 – Fall 2015 retention rate for our Applied Communication Studies MA students was 60%, while our Fall 2014 success rate (students retained in Fall 2015 or graduated in 2014-15) from our MA program was 68%. For our undergraduate program, our Fall 2014 – Fall 2015 retention rate was 46.5%, while our Fall 2014 success rate was 65.1% (accounting for students who graduated in 2014-15 or were retained in Fall 2015).
- Instructors: 3
- Associate Professors: 1
- Full Professors: 5
- Administrative Specialist III: 1
Grants & Contracts
- Thombre, A. (2016, December). HIV/AIDS and STD Testing and Diffusion of Awareness. Arkansas Department of Health. $500.
- Thombre, A. (2017, April). Preparation of UALR Campus Stakeholders for Embracing Sustainable Practices Towards Creating a Zero-Waste Campus. UALR Sustainability Committee. $2500.
- Angela Laird Brenton Memorial Scholarship – amount raised to date of $16,497.00.
- Allan Ward Scholarship – amount raised to date of $1,787.03.