Assessment Tool to Measure Mentorship Role in Undergraduate Research

By Office of the Provost, University of Wisconsin-Madison

January is National Mentor Month and UW–Madison has reason to celebrate. For over two decades, the university has been a nationally recognized leader in evidence-based research mentor and mentee training, and has led the way in providing high-quality undergraduate research experiences. Thanks to the dedicated work of a group of units from across campus, yet another exciting, new advancement is underway.

“Undergraduate research is an integrated, high-impact learning experience that can transform the way students think and learn, and the careers they aspire to pursue, and it’s one of many ways students engage in the Wisconsin Experience,” explains Mo Bischof, associate vice provost and director of assessment, Office of the Provost. “But how do we measure the impact of these complex learning experiences, which can vary considerably from one student to the next? This is what we’re aiming to do.”

Bischof is referring to a new, collaborative effort that aims to develop a common framework and assessment tool that departments, schools and colleges will be able to use to measure undergraduate researchers’ learning experiences. The project builds on the research of Janet Branchaw, assistant professor of kinesiology and director of WISCIENCE, and will provide useful information for improving the research experiences for both undergraduate mentees and their mentors.

Members of the Collaborative for Advancing Learning and Teaching (The Collaborative) – Student Learning Assessment and WISCIENCE – are working in partnership with the Center for the Improvement of Mentored Experiences in Research (CIMER) at the Wisconsin Center for Education Research (WCER) and the College of Letters & Science Undergraduate Research Scholars (URS) program on the project.

Madison faculty members have long been committed to including undergraduate students on their research teams and to providing high-quality mentoring. Many have participated in research mentor training offered by WISCIENCE in collaboration with the Delta Program, originally developed by UW-Madison Professor Jo Handelsman and Associate Scientist Christine Pfund.

“Given UW-Madison’s notable experience and leadership in undergraduate research and research mentor and mentee training, we believe it is an ideal environment in which to develop an institutional framework and assessment tool,” says Branchaw.

Development of the framework began last spring, when drafts were reviewed and input was gathered from faculty and staff at the annual Teaching and Learning Symposium. A first iteration of the assessment tool, which includes paired surveys of student self-assessment and mentor assessment of the student, was also tested last spring with students and mentors participating in the Undergraduate Symposium.

The paired survey platform is hosted at CIMER and was originally developed as part of a NSF funded project led by Branchaw to assess learning in the NSF Research Experiences for Undergraduates (REU) site programs across the nation. The platform has the capacity to compile data from across UW-Madison’s schools, colleges and departments to produce an aggregated university wide report.

Dr. Christine Pfund, director of CIMER, notes, “I see this as an exciting opportunity. It will advance our understanding of the mentored undergraduate research experience from the perspective of both the mentor and mentee at UW-Madison, as well as align us with assessment approaches being used nationally to advance the science of mentorship.”

The UW-Madison framework and common assessment tool are expected to be available for use by departments, schools and colleges in fall 2018.

For questions or to provide input on the framework and assessment tool, please contact Dr. Amber Smith, WISCIENCE director of research mentor and mentee training. Feedback will be used to ensure that the framework and assessment tool accurately represent and capture undergraduate research learning outcomes from all units.

Join campus as it celebrates its undergraduate research and hear from students about their unique experiences at this year’s Undergraduate Symposium on April 13, 2018.

UW–Madison Researchers Contribute to NAS Report on Undergraduate Research Experiences

By Wisconsin Center for Education Research, School of Education, University of Wisconsin–Madison

Branchaw and Grodsky
Eric Grodksy and Janet Branchaw served on a NAS Committee charged with studying undergraduate research experiences.

A National Academy of Sciences committee whose members include Janet Branchaw and Eric Grodsky, both on faculty at the University of Wisconsin–Madison School of Education and principal investigators at its Wisconsin Center for Education Research (WCER), released a study examining evidence on undergraduate research experiences. Branchaw and Grodsky are among 16 experts from across the country who were invited to join the committee in 2015.

“In recent years, colleges and universities across the United States have rapidly expanded undergraduate research experiences as a strategy to increase students’ interest and persistence in STEM subjects,” states Branchaw, assistant professor of kinesiology and director of WISCIENCE, a science education and engagement institute on campus. “However, support to track the number and types of experiences, and to assess the results, have not kept pace.”

Branchaw, who directs the largest federally funded undergraduate research program on campus and chairs the National Science Foundation’s leadership committee on undergraduate research experiences in biology, has developed and studied undergraduate research programs for more than 15 years.

 “The National Science Foundation sponsored the study to learn how research experiences affect undergraduate education,” says Grodsky, professor of sociology and educational policy studies. “We reviewed a lot of literature and promising evidence that these experiences benefit students, but found the evidence far from persuasive.”

While the committee members agreed that they believe research experiences can be transformative, Grodsky’s expertise in statistical methods and causal inference helped them determine that “more comparative data from well-designed studies is needed to support that belief and to better understand which practices are most effective.”

Branchaw and Grodsky did not know each other before meeting at the Undergraduate Research Experiences for STEM Students: Successes, Challenges and Opportunities’ first committee session in Washington, D.C. “I was surprised to see somebody else from my university,” states Branchaw. “We quickly introduced ourselves and began comparing travel plans and our experiences on campus.” UW–Madison is the only institution with more than one person serving on the committee.

Though it is unclear how committee members are selected, Grodsky says selecting members of differing academic disciplines was important for the committee’s task. “Here we had scientists building and conducting science education programs for undergraduate students,” he said. “The committee needed the knowledge and tools of education experts and social scientists to analyze and understand the human impact of these programs.”

A third UW–Madison contributor to the report is Christine Pfund, also a WCER researcher and director of one of its newer projects, the Center for the Improvement of Mentored Experiences in Research (CIMER). Because mentoring was singled out by the committee for its importance in STEM education, it commissioned Pfund to write an article on mentoring that provided baseline content for a chapter in the report.

Pfund studies research-mentoring relationships and trains research mentors across science, technology, engineering, mathematics and medicine. She is one of five investigators selected by the National Institutes of Health to establish a National Research Mentoring Network. Pfund and Branchaw have a long history of collaborating on training development and the study of mentoring relationships.

“Chris is widely recognized as a leader in mentoring and has a following all her own,” says Grodsky. “We were very fortunate she was able to contribute her expertise to the report.”

According to Pfund, “Mentoring has become somewhat of a ‘sweet spot’ for UW–Madison, which is well positioned to become a national leader in the field.” Under her leadership, an important goal for CIMER is to advance the “science of mentoring” at UW–Madison and across the nation.

Enabling Effective Mentorship

By Ranjendrani Mukhopadhyay

CIMER’s director, Christine Pfund, discusses what makes mentorship successful in a special section on education. ASBMB TODAY, August

Chris Pfund
Christine Pfund, CIMER Director, Wisconsin Center for Education Research (WCER)​

What makes mentorship successful? That’s what Christine Pfund studies at the University of Wisconsin–Madison. Pfund is interested in understanding, developing and implementing effective mentor training in science, engineering and medicine.

“We’re putting our precious trainees in the hands of folks who are well-intentioned but have had no professional development in the arena (of mentoring). It leaves a lot to chance,” she says. “No matter how well-intended someone is and no matter how good they are, there is always room” to improve.

After earning a Ph.D. in cell and molecular biology, Pfund did a postdoctoral stint in the early 2000s in the department of plant pathology. She then switched her focus to improving classroom teaching and research mentoring.

These days, Pfund is one of the principal investigators of the National Research Mentoring Network that was established recently by the National Institutes of Health. She is also director of the new Center for the Improvement of Mentored Experience in Research.

Rajendrani Mukhopadhyay, the chief science correspondent for the American Society for Biochemistry and Molecular Biology, spoke with Pfund to find out more about her research in effective mentoring practices. The interview has been edited for length and clarity.

How did you become interested in mentoring?

I’d always been interested in improving teaching in the classroom and had been doing a lot of work on the side at UW Madison. About halfway through my postdoc, I started to think about what I really could do in that arena. (At the same time,) UW Madison got two big grants. One was a (National Science Foundation) grant to Robert Mathieu to establish a Center for the Integration of Research, Teaching and Learning, CIRTL, and the other was the (Howard Hughes Medical Institution) professor grant to Jo Handelsman.

I spent the next eight years working for both programs. I was an associate director of the CIRTL program at UW Madison, working primarily on professional development for future faculty in STEM. I was also a co-director of the Wisconsin Program for Scientific Teaching, which came out of Jo Handelsman’s HHMI grant. (Author’s note: Handelsman currently is the associate director for science at the White House Office of Science and Technology Policy.) The program was on improving teaching in biological sciences, faculty professional development and establishing national summer institutes. A part of that project also was to develop research mentor training.

The research mentor and mentee training continued to grow. We were able to take successes from the original HHMI grant and work with CIRTL and get an NSF grant. Then I moved over to the medical school and started working on adaptations of our approaches for clinical and translational research. I started to get some research grants and worked with social scientists and others to study interventions and start to understand mentoring relationships.

Most recently, I used all of that to become part of the leadership for the National Research Mentoring Network. It allows us to continue the work to understand interventions on a much more national scale and scale up training from evidence-based approaches.

How do you define mentoring?

The across-the-board generic definition of mentoring focuses on it being a collaborative learning relationship that proceeds through purposeful stages over time and has the primary goal of helping the mentees gain the skills and knowledge they need to move on in their chosen careers. That applies to many different kinds of mentoring relationships. It could be a classic research mentoring relationship like we know in the sciences. It can have elements of career coaching. It could be peer mentoring. It could be virtual mentoring with someone who doesn’t even have a research relationship with you.

Are there differences in mentoring between science and other fields?

When we did our adaptation work for some of our mentor–training interventions, we expected, across the STEM disciplines, for things to be very similar. What we found out was that the differences that were most salient were not between, for example, chemistry, physics and math. The differences were in projects. The kind that were theoretically based, where there was a lot of thinking, had a different nature (of mentoring) because you were working together on an idea. It was different than the “we’re doing something hands-on together.” We found the nature of the work had implications on the relationship.

Imagine when you’re in a meeting (with a mentee). You’re saying, “You need to have your own idea, and I’ll bounce off whether it’s a good theoretical idea.” Even the nature of those conversations is different from “Here, you need to master these skills, come up with an idea and implement it.”

The nature of the work influences the nature of the conversation that happens between mentor and mentee.

How do you make sure you’re not creating “mini-mes” (clones of professors) and are paying attention to diversity?

There needs to be, both at the individual mentor–mentee level as well as at a systems level, the recognition to address diversity within these relationships and acknowledge that culture plays a role. How people work, what they think is important, the motivation to do it, the vision they have for what is possible and why it matters — those are all culturally informed. If we don’t pay attention to those things within mentoring relationships, research programs and training programs, it’s going to continue to privilege the dominant cultural norms. There will not be an acceptance and a benefit from embracing different value orientations (as well as) an allowance for diversification of the workforce.

Diversification of the workforce isn’t just about embracing people who have different backgrounds. It’s about embracing that they bring different values and orientations to the table. That happens at the individual level and at the organizational level. Mentoring is the place where this needs to be addressed. If individual mentors believe that their role is to create “mini-me’s,” then who they accept, how they train, what they see as success and what ideas they accept become enormously limited.

How do you get mentors to think about what they are doing?

Mentors need to reflect on what is their motivation for taking on mentees. If their motivation for doing it is so that people can be just like them, then that has a huge influence on who they should be taking. If their idea is to inspire the next generation to do amazing things, then they need to really think about if they have set up the relationship and the environment to empower those successes.

We are putting forth this idea of culturally responsive mentoring. Cultural context matters. We are creating training to get mentors to start to work through their own assumptions — not just their biases but their own assumptions about the role that culture plays, how they can create space for that and acknowledge the impact it has. If we continue to force folks to check their culture at the door when they enter the lab, then we also choose to check all the benefits that come with it at the door.

Chris Presenting at trainingPfund says people need to reflect on their motivations to mentor.

Do you think mentorship in science has changed over time?

It may feel like there’s not been a lot of change, but there has been. The conversation alone has changed. The fact that federal agencies are calling for evidence-based mentoring to be part of the training programs — it’s a huge change. While it’s going to take a long, long time, and we certainly aren’t anywhere near where we need to be in order to capitalize on the investment, the needle has moved. I want to respect the people who feel like it hasn’t moved enough and that there is an enormous amount of work to do. (But) there has been a lot of movement in the last decade.

I should have asked this earlier: What is the payoff of mentorship?

There is a lot of research out there that has linked strong mentorship to things like enhanced scientific identity, a sense of belonging, persistence, productivity, career satisfaction and definitely enhanced recruitment of folks from traditionally underrepresented groups.

The issue is that the evaluation (of mentorship) has not been methodically rigorous. Also, often, because the definition of mentoring and the context in which it occurs is so ubiquitous, we don’t know what we’re studying or what the results are linked to. If we really want to understand the critical elements of mentoring and the roles that mentoring plays in the elements of success we want to see in diversifying the workforce, then the community has to get on board with describing what they are studying and using common metrics.

Who were your mentors?

A part of why I’m so passionate about this work is because I’ve had the privilege of having some amazing mentors! Without a doubt, Jo Handelsman has been an amazing mentor to me. My graduate adviser and folks with whom I’ve worked along the way all played different roles. One of my current mentors, Christine Sorkness, whom I work with on the NRMN, has the amazing ability to push me beyond my comfort level and make me believe that I can do it but also to say, “I’ll be here if you stumble.”

That has been a common theme — my mentors strongly believe in my potential and push me, but they also let me know that they’ll be there to help.