EDLI Newsletter 3 – August 2021

General Team Update

This month EDLI wrapped up the Colleges’ Online Learning Academy (COLA) and has started evaluating surveys and deliverables from the experience. Another main focus for this month has been connecting to various units across campus to better coordinate our efforts with theirs. For example, we have met with AAN separately, and then with AAN, IT, Grad School, MSU Library, and Hub to explore what faculty development opportunities are currently being offered, the overlap, and how we might structure these efforts more collaboratively and effectively in the future. In addition to these activities, we have advanced our focus on educational spaces by developing implementation and evaluation plans for HyFlex/multimodal capable classrooms and preparing for the fall semester. This project has been in close collaboration with IT and the Library. EDLI members have also worked on an additional submission to the Creating Inclusive Excellence Grants funded by the Office of Inclusion and Intercultural Initiatives and evaluating our mid-semester evaluation survey.

COLA: Colleges Online Learning Academy

The COLA wrapped up in August. Students presented their teaching portfolios and online course modules to each other and to the EDLI team through a “Virtual Gallery Walk” event or through an asynchronous recording or sharing of their portfolios. The teaching portfolios presented and shared by the fellows ranged from teaching philosophies extending their values in teaching into future careers to the development of comprehensive course modules aligned with learning objectives. Students also completed an end-of-summer reflection and were invited to complete a post-survey. (Contact: Caitlin). 

Multimodal Classroom Project

In the spring EDLI worked with IT, the Hub, and members from partner Colleges to identify 6 classrooms on campus that would be converted to Multimodal Capable Classrooms. These rooms are now set up for Multimodal Teaching and several of them have multimodal instruction happening during the fall semester. The main upgrade made to these rooms is a classroom-wide microphone and audio system that allows easy and clear two-way communication between students in the classroom and those who are virtual. 

EDLI has identified at least six courses across CAL, CANR, Broad, and NatSci that are engaging in some level of multimodal instruction. A number of those instructors will participate in a pilot study on student outcomes, faculty technology needs, and faculty satisfaction with multimodal course designs. Jun and Caitlin are working with postdocs from IT on the evaluation and will share surveys with instructors and students in the first week of September. This evaluation will be used to develop guidelines for faculty interested in multimodal instruction and will serve as pilot data for grant applications to further develop MSU’s multimodal expertise. (Contact: Caitlin and Jun). 

EDLI mid-semester course student feedback survey statistical analysis

During the first year since the EDLI collective was formed, Jeremy led the EDLI team to develop an online survey for students to provide anonymous, intermediate feedback to their courses so that the course instructors would be able to identify areas for improvement and address students’ needs and challenges in their learning process during the remainder of a semester. With an array of quantitative measures and open-ended questions, the EDLI mid-semester survey allows students to provide substantive, in-depth feedback. 

To assess whether the EDLI mid-semester survey instrument provides a sound basis for instructors to interpret collected data, Jun conducted relevant statistical analyses based on the data collected from eight different courses across our three colleges wherein the EDLI mid-semester survey was implemented during this past academic year. Results from the analyses offered support to the quality of the instrument. Specifically, the reliability measures (Cronbach’s alpha values) ranged from .798 to .812 for the set of four questions we developed for capturing student perception of the “alignment” between course activities and learning objectives, indicating high levels of internal consistency. Additionally, the corrected item-total correlation values of the questions were all higher than the .4 threshold for them to be considered internally consistent. Results from the factor analysis indicated that all four questions fit into one underlying construct of “alignment”. The Kaiser-Meyer-Olkin measure of sampling adequacy (KMO) values were higher than .6. Bartlett’s tests of sphericity yielded p values < .001. These values demonstrated the adequacy of this factor analysis. For your information, the “alignment” question set is as follows:

• The out-of-class activities (e.g. homework or readings) deepen your understanding of the course objectives and/or key concepts. (1 = Strongly disagree; 5 = Strongly agree)
• The in-class activities (e.g. lectures or discussions) deepen your understanding of the course objectives and/or key concepts. (1 = Strongly disagree; 5 = Strongly agree)
• The class activities (e.g. homework, lectures, or discussions) prepare you for the assessments (e.g. quizzes or exams). (1 = Strongly disagree; 5 = Strongly agree)
• How well are the questions on assessments (e.g. quizzes, tests, exams) aligned with class activities (e.g. homework, lectures, discussions)? (1 = Not aligned; 5 = Tightly aligned)

As conceptualized by Scott and other EDLI team members, this set of questions on the survey would allow us to offer an overall student-perceived “alignment” measure for any specific course. Here is an example based on the analysis of mid-semester student feedback data of a course: 

For this course, on average, the student-perceived “alignment” score was 3.8372 (out of 5), with the standard deviation of .95413. 

This approach reflects and is aligned with the backward course design principle that centers around course learning objectives. (Contact: Jeremy and Jun).

Creating Inclusive Excellence Grants: Peer Teaching Evaluations

Stephen and Caitlin worked with Michael Lockett of the AAN/HUB to submit an internal grant proposal to the Creating Inclusive Excellence program. The project centers on a peer teaching evaluation program for instructors in CISGS and the STEM building with the goal of developing more equitable teaching evaluation processes. This is aligned with the ongoing work in NatSci on updating teaching evaluation processes. 

AAN Discussion

EDLI convened a meeting with leadership from the Academic Advancement Network. This conversation centered around EDLI’s accomplishments to date,  the teaching & learning support that EDLI currently offers educators on campus, the manner in which that work aligns with the work and mission of AAN. This fruitful discussion led to broader strategic conversations about the possible future state of teaching & learning support on MSU’s campus, EDLI’s role in that support structure, and the optimal distribution of resources related to teaching and learning support.

Coordination among campus collaborators

EDLI continues to deepen ties with key stakeholders on campus. Over the month of July Jessica Sender, Coordinator for Online Teaching and Learning Technologies in the MSU Library became a regular attendee at most EDLI meetings. Throughout August Jessica continued to be a regular contributor and she brought additional library stakeholders into critical dialogues when appropriate.  The Libraries offer deep background and knowledge about information literacy, expertise in authentic and meaningful assessment, teaching and learning across modalities, and close collaborations with faculty, staff, and students. 

Beyond the deeper integration of the Library’s expertise, EDLI convened a “meeting of the minds” among campus thought leaders in the area of teaching & learning support. Attendants at that meeting were representatives from MSU IT, MSU Hub, Libraries, AAN, The Graduate School, and EDLI. The conversation focused on those units’ current teaching & learning support operations and their goals for the coming 12-18 months. The purpose of the conversation was to identify areas of overlap and redundancy, reveal possible points of collaboration, and to ensure that key players in these units had a more comprehensive understanding of the teaching & learning support environment on campus. A document detailing key take-aways from this important strategic meeting is located here: T&L Support Fall 2021 Meeting Summary

Additionally, Scott, Sarah, and Jessica have attended meetings on Quality Matters with stakeholders across the institution. There they have discussed a larger University-wide QM strategy, as well as what the various stakeholders can offer in terms of QM Professional Development and quality reviews. 

EDLI hopes that these are the first of many such conversations, and will continue to convene similar meetings to facilitate as much collaboration on campus as possible.