From service to science: NIH shifts focus of mentoring network aimed at boosting grantee diversity

By Jeffrey Mervis

Christine Pfund (left) leads the the National Research Mentoring Network’s coordination center. TODD BROWN/UNIVERSITY OF WISCONSIN, MADISON

As co-director of graduate affairs at the University of Chicago, Nancy Schwartz spent the past 4 years helping faculty members at 15 major research universities become better mentors. The project was supported by the National Research Mentoring Network (NRMN), a $23 million effort that the National Institutes of Health launched after discovering an embarrassing racial gap in who gets NIH grants.

Begun in 2014, NRMN was designed to scale up successful mentoring practices in the biomedical sciences. NIH officials hoped its efforts would boost the fortunes of minority applicants. But last summer, when NIH renewed the network for another 5 years, officials decided to spend most of the money on the science of mentoring, that is, testing different approaches to mentoring with a small, carefully chosen population. Barely 10% of NIH’s $50 million investment in phase two of NRMN is going to the type of services, including an online portal that provides one-stop shopping for a cornucopia of mentoring activities, that characterized first phase.

Schwartz’s project is a casualty of that shift in emphasis from service to research. And she is one of several researchers familiar with NRMN who wonder whether something will be lost as NIH remakes the program. “What’s wrong with more service?” she wonders. “Or at least a mix of both approaches?’”

Using what we know

NIH’s decision to refocus NRMN comes as the role of mentoring in training scientists is attracting greater attention from researchers. In fall 2019, a panel assembled by the National Academies of Sciences, Engineering, and Medicine (NASEM) released a major report that makes the case for mentorship as a worthy field of study and describes several areas where more research is needed. “Despite its important place in academic scientific culture, mentoring rarely receives the focused attention, evaluation, and recognition of other aspects of professional development such as teaching and research,” it declares.

That lack of attention is what drove NIH to pour $39 million into 11 studies of mentoring as part of NRMN’s second phase, says Hannah Valantine, NIH’s chief officer for scientific workforce diversity. An additional $5.5 million will go to a center to coordinate those studies, and the remaining $5.5 million to what is now called the NRMN resource center.

Valantine says the research projects should help NIH understand why a particular approach works. “In phase one we did very important work with mentoring and coaching workshops and they look as if they are working well,” Valantine says. “But because those activities didn’t include a randomized control trial, we don’t know if there was selection bias. We don’t want to continue spending resources on an activity if there isn’t evidence that it is working really well.”

At the same time, the authors of the NASEM study urged their colleagues not to wait for more data before making changes in how students, including those from groups underrepresented in science, are trained for academic careers in biomedical research.

“A growing body of evidence exists about how to create and sustain successful and inclusive mentoring relationships,” says Angela Byars-Winston, a professor of medicine at the University of Wisconsin (UW), Madison, and chair of the committee that wrote the report. “We hope our report can catalyze [the] use of that evidence.”

 
Jamboor Vishwanatha runs the National Research Mentoring Network resource center. ALEXIS SHORT/NATIONAL RESEARCH MENTORING NETWORK RESOURCE CENTER

Putting mentoring on the map

For 5 years, NRMN has been a mechanism to do exactly that. Nearly 15,000 people have registered through its online portal. For some students, it’s the first step toward a guided virtual mentoring experience with a faculty member who matches their interests. Thousands more have taken advantage of other types of NRMN-arranged professional development. Postdocs and early-career scientists have learned how to write better grant proposals, faculty members have improved their mentoring skills, and an elite group has been trained to share knowledge with colleagues.

The scientists involved in phase one of NRMN have also worked hard to disseminate their results so that others can put them to use. A string of publications by NRMN investigators “offer interesting evidence on what works and doesn’t work in trying to scale mentoring to a national audience,” says Christine Pfund, a UW Madison developmental biologist.

Pfund ran the mentor training core in NRMN’s first phase and now oversees the phase two coordinating center. In addition to generating results, Pfund says NRMN 1.0 has helped put mentoring on the academic map.

“NRMN brought together the entire research community to talk about where mentoring was and where it needs to go,” she says. “So, I think it’s had a huge impact on raising the visibility of mentoring.”

Pfund and her NRMN colleagues acknowledge the shift in emphasis. “Yes, NIH is going in a different direction,” says Jamboor Vishwanatha, who ran the online component of NRMN and who now leads the renamed resource center. But they say NIH’s vote of confidence is appreciated and insist their ability to provide direct services won’t be curtailed.

“NIH has told us that we were very successful,” Vishwanatha says, “And everything we were doing—the web portal, online platform, the matching process, and the network—is continuing.”

NRMN was never meant to demonstrate the type of efficacy that Valantine wants to see in phase two, according to Pfund. Phase one wasn’t “resourced sufficiently” to study why a particular approach to mentoring worked, and for which audience, she says. Phase two will allow for that type of deep dive, she adds.

Byars-Winston, who was also part of the NRMN 1.0 team, says she can’t wait to take the plunge. “There was no money for research in phase one,” she says. “Now, in phase two we can experiment.”

In her case, that means leading a project aimed at addressing cultural biases in mentoring, one of the hottest topics in the field. Byars-Winston and her team hope to train 600 faculty members at 32 universities in understanding and practicing culturally aware mentorship (CAM), then assess how that training affects both how they mentor their graduate students and institutional policies relating to diversity and inclusion.

A counseling psychologist who studies the impact of culture on career development in academic settings, Byars-Winston has spent 2 decades experimenting with various facets of CAM. She says NIH’s new approach to NRMN will give her a chance to drill down even deeper into the science of mentoring and run more experiments. “And for us, the experiment is dosage,” she explains. “Does a half-day of training have the same impact as a full day? That’s a hard-core research question.”

Taking an institutional approach

Schwartz, meanwhile, finds herself on the outside for phase two. She applied for one of the science-of-mentoring awards, but didn’t make the cut, despite the success of her phase one project.

Over a career spanning 5 decades, Schwartz has received NIH support for both her research on skeletal and brain development and disabilities and her commitment to improving graduate training in the biomedical sciences. The NRMN grant she received in 2015—a $250,000-a-year supplemental award—allowed her to expand her work with the Committee on Institutional Cooperation, a 15-school consortium that served as a Midwest hub for NRMN.

The project addressed training issues across the academic research pipeline, from bolstering the self-esteem of first-year graduate students to facilitating the mentoring activities of senior faculty members. Schwartz, who is also a professor of pediatrics, has yet to publish her results, but preliminary analyses suggest its interventions had exactly the type of impacts that NIH had hoped NRMN would achieve.

For example, the 160 faculty members who were trained as facilitators returned to their campuses and led mentoring workshops attended by 2700 students, postdocs, and other faculty—demonstrating the ability to scale up successful practices. A majority of the 137 people in the grant-writing workshops were underrepresented minorities, showing that these services can reach their target audiences. And those young scientists have put the knowledge to good use: Nearly half of their applications to NIH and other funding sources have been successful, and many of the postdocs have been hired into tenure-track faculty positions, including some at the institutions involved in the project.

“We were able to put together multiple things that had been shown to work on their own,” Schwartz says. “And combining them amplified their impact.” NRMN leaders apparently agreed: Of five supplemental awards and 10 other small projects, hers was the only one that survived a midcourse vetting.

Schwartz suspects her comprehensive view of mentoring wasn’t a good fit for the individual research projects that NIH wanted to fund. “We took an institutional approach,” she says. “So, there’s no way for us to plug into phase two of NRMN. Everybody liked what we were doing, so it’s frustrating not to be able to continue.”

Using NRMN as a springboard

Elizabeth Jakob is having a much easier time staying connected to NRMN. But she’s also worried that its revamped structure may hinder further expansion of mentoring efforts like hers.

A behavioral ecologist and associate dean for the graduate school at the University of Massachusetts (UMass), Amherst, Jakob runs a mentoring academy based on NRMN training she received. The academy’s ability to operate on a shoestring budget is a testament to what NRMN was able to accomplish by emphasizing service, she says.

“Without NRMN, maybe we could have put out a handbook, or brought in a few speakers,” Jakob says. “But definitely not the type of mentor training that we’ve been conducting.”

Jakob admits she was skeptical of the value of the first NRMN-sponsored workshop she attended. “I had been through other training that didn’t make a difference,” she says, “But its superinteractive format keeps you engaged.” After honing her skills, she began to think about what it would take to reach the 1100 tenured and tenure-track faculty members at UMass.

“I realized I couldn’t do it myself,” she admits. A dozen colleagues stepped up and took the additional training needed to become a facilitator. Armed with enough instructors, she began to advertise the course across the entire campus. That’s when she realized that marketing skills were as important as mentoring skills.

As Jakob tells it, “Having a department chair say, ‘I took it and it was worthwhile,’ is a lot more effective than saying, ‘Hey, we noticed your students are taking a very long time to graduate. Maybe you should think about ways to provide them with better guidance.’”

Those endorsements help her hold down the cost of the training. “Faculty just show up,” Jakob says. “We don’t provide any payment or incentives.” Trainers receive a small stipend, she adds, but the only other expenses are coffee, doughnuts, and the training materials.

Jakob estimates that more than 20% of the faculty have been trained since the workshops began in 2017. She thinks she needs to reach at least 40% to make a lasting impact on the quality of graduate education mentoring at UMass. But it’s a question she may never be able to answer.

“What’s difficult is measuring the impact on mentees,” she admits. “We haven’t been collecting the data ourselves.”

That’s been the role of NRMN for the past 5 years, she says. “We implement their training modules, and we just assume that they are following up.”

That assumption may have been true during NRMN 1.0, when Pfund led an administrative center than managed all NRMN-related programs. She also directed NRMN’s mentor training core, of which Jakob’s academy was an offshoot. Data were collected and passed along to an evaluation center based at the University of California, Los Angeles (UCLA).

But NRMN 2.0 doesn’t have the same reporting lines of authority. The UCLA scientists will be collecting and analyzing some follow-up data on the online mentoring and mentor training components of phase one but will not handle anything coming from the 11 research projects. And Pfund’s phase two coordinating center has no administrative authority over the 11 research projects.

“It is not appropriate for the coordination center to have any input in the study design or their recruitment effort,” Pfund explains. “Those issues are between the project and their NIH program officer.”

Members of the National Research Mentoring Network engage in a team-building exercise. KRISTIN STEINER, PUBLIC RELATIONS ASSOCIATE/UNIVERSITY OF UTAH

Helping students and faculty

Some of the projects in NRMN’s second phase put down strong roots during phase one. For example, Manoj Mishra, a biologist at Alabama State University, will be testing variations of NRMN’s online mentoring training on first-year students at his and two other historically black institutions in the South. One group of 150 students at each campus will be automatically enrolled in NRMN after arriving on campus and given instructions on how to use the online resources. A second group of equal size will not only join NRMN, but will also receive coaching, peer mentoring, and mentee training. A third group, the control, will consist of those who decline any mentoring support.

The goal is to measure the impact of the mentoring intervention on the students’ undergraduate experience—for example, did they pursue a science major, did they engage in research, and did they build a stronger support network as a result of the mentoring? The last variable is especially relevant for students at Alabama State, Manoj says.

“Some 60% of our freshmen come from rural areas, and Montgomery is the first time they’ve been in a city,” he notes. “Some of them need the motivation and mentoring just to keep from dropping out and going home.” The project adds an experimental element to what Vishwanatha’s team has been doing in serving those who come to the NRMN portal voluntarily.

In contrast, Akshay Sood’s project tackles an issue within a population that was not central to phase one. A pulmonologist at the University of New Mexico’s (UNM’s) health science center, Sood and his team will be measuring the efficacy of online versus face-to-face mentoring to reduce attrition among faculty members at UNM, Arizona State University, and the University of Oklahoma. Attrition is a major problem among women and underrepresented minorities, he says.

“Most of our faculty are clinician educators with a heavy teaching and clinical load,” he says. “But they also need to maintain an active research program.” The target audience is junior faculty members, where turnover is nearly 50% over 5 years. The project is unique in having early-career faculty members as mentees, he adds.

An uncertain future

NRMN is part of the Diversity Partnership Consortium that NIH created in the wake of a 2010 report by University of Kansas, Lawrence, economist Donna Ginther showing that African American applicants stood a much lower chance—by 10 percentage points—of receiving a bread-and-butter research project grant (R01). The bulk of NIH’s initial 5-year, $250 million investment in the consortium went to 10 projects aimed at helping minority undergraduate students taking their first step toward a career in biomedical research, and those projects have been renewed for another 5 years.

Valantine has calculated that the funding gap could be eliminated through a relatively modest increase in the absolute number of African American scientists who win NIH grants. But she’s not willing to settle for parity.

“The biggest problem is the minuscule number of African American scientists in tenure-track positions in biomedical research,” she says. “That is what leads to the tiny number who are applying for R01 grants. Of course, we want to eliminate the gap, and I am confident we will be able to do so. But that would not be victory in my mind,” because she would like to see an increase in overall numbers.

NRMN 1.0 was all about putting more minorities in a position to submit winning grant applications. That meant tackling all the components involved in training the next generation of scientists, with better mentoring and grantwriting skills at the top of the list.

But doing that takes time. NRMN investigators always assumed they would need to find another source of support after the diversity consortium went out of business. “One of the major goals of phase two is sustainability,” Valantine says.

The principal investigators on the new research projects say they have not been told what will happen when their grants end in 2024. “It’s a research grant,” Sook says. “If the results are good, we certainly hope that NIH will want to renew it.”

The stakes are even higher for those who have been with NRMN since its formation. Enlarging the pool of scientists engaged in and benefiting from high-quality mentoring remains their top priority. But they aren’t sure what sustainability means under the new format for NRMN 2.0.

“We’re all concerned,” Pfund says. “We’ve been told that funding [for NRMN] will end in 10 years, and we don’t know what NIH is planning to do after. Whatever happens, she says, “we don’t want people to feel abandoned.”

UW−Madison Mentorship Experts Anchor Federal Push to Diversify Biomedical Workforce

UW−Madison Mentorship Experts Anchor Federal Push to Diversify Biomedical Workforce

WCER investigators Christine Pfund, Angela Byars-Winston receive major grants

October 31, 2019   |   By Karen Rivedal, WCER Communications

UW-Madison expertise is key to a new $43 million federal effort to bring more women and minorities into the biomedical sciences.

UW-Madison expertise is key to a new $43 million federal effort to bring more women and minorities into the biomedical sciences.

UW−Madison will continue to play a leading role in the second and final phase of a sweeping federal investment in better research mentoring, with responsibility for two major grants in the $43 million follow-up push to boost diversity of students, staff and faculty researchers in the biomedical sciences.

The five-year program starting this fall as part of the NIH’s Diversity Program Consortium follows a $19 million first phase that sought to increase participation of underrepresented groups in the biomedical workforce through development of a National Research Mentoring Network (NRMN) starting in 2014.

That network, which remains active in Phase 2, works to enhance the training and career development of people from diverse backgrounds in biomedical research — including underrepresented populations such as women and minorities — through an online clearinghouse of evidence-based training programs for mentors and mentees, webinars and networking opportunities, including a guided mentoring tool.

UW−Madison was central to Phase 1, serving as a national hub for mentorship training through a $3 million grant awarded to Christine Pfund. Pfund directed the NRMN’s Mentor Training Core as one of five principal NRMN investigators in Phase 1, and also worked as a researcher in the School of Education’s Wisconsin Center for Education Research (WCER) and at the UW Institute for Clinical and Translational Research.

But while the Mentor Training Core gathered a comprehensive suite of programs in culturally responsive mentoring, Phase 2 goes a step further. While continuing to develop mentoring opportunities for biomedical researchers, from undergraduates through early career faculty, Phase 2 focuses on the rigorous testing of selected interventions to determine what approaches work best for whom and in what context.

National profile of mentoring rises with NAS report

The importance of better mentoring has been increasingly recognized across government and academia over the past decade as a driver for improved career satisfaction, progression and productivity in many fields of work and academic pursuits, but especially as a tool to broaden participation in the disciplines of science, technology, engineering, math and medicine (STEMM).

.

Report cover

This week, mentoring’s ascending role was recognized with the release of a comprehensive consensus study on the science of effective mentorship in STEMM by the National Academy of Sciences.

The 287-page report, plus an online guide to recommended tools and resources, was produced by a national ad hoc committee led by Angela Byars-Winston, a UW−Madison Department of Medicine professor and investigator with WCER’s Center for the Improvement of Mentored Experiences in Research. Pfund, who directs CIMER, also is among the committee’s 12 members, in a dual nod to UW−Madison’s 15-year record of working to improve research-based mentorship.

A consensus report matters in part because it acts to collate and critically review all the evidence-based findings on a given science or topic, plus identify gaps in knowledge and directions for future research.

“There’s so much we still don’t know,” Byars-Winston says. “The traditional dyad of mentoring (one faculty member/one student) is just one model. We also know that peer mentorship and having multiple faculty mentors can be effective. What has the best impact, and for who? Does it matter if we have the same race and gender? We also have a lot of wonderful studies that exist, but replication is everything.”

UW−Madison unique in Phase 2 representation

In Phase 2, the five cores* that made up NRMN’s first phase, including Pfund’s Mentor Training Core, cease to exist. In their place is a resource center, coordination center and a third arm made up of 11 research projects using experimental designs to expand the scientific scope of the NRMN initiative.

The 11 independent research projects are based at institutions across the country and must be focused around one of three primary areas of intervention determined to enhance mentoring and networking: the science of mentoring, professional networks and the navigation of critical career-transition points.

Pfund received a $5.4 million grant to lead the coordination center in Phase 2 of the NRMN initiative. In addition, Byars-Winston was awarded a $3 million grant as director of one of the 11 research projects: a nationwide intervention to train 600 faculty mentors of doctoral students in 32 biomedical programs, including one in-person training session for each participant.

That representation puts UW−Madison in rare air, as the only institution in the country to boast PIs in two of the three branches that make up Phase 2 of the NRMN initiative.

“No one else in the initiative has two,” Pfund says. “It’s an exciting time.”

Pfund leads coordination center in Phase 2

During Phase 2, the resource center, to be run by the University of North Texas’ Health Science Center with a $4.6 million grant, is tasked with managing and continuing to refine the guided mentoring tool and the NRMN website, including regular posting of public reports on the 11 research projects’ progress.

The coordination center, meanwhile, will coordinate the early stages of data collection from the research projects and provide feedback to “promote synergies” between the resource center and the 11 projects to catalyze and maximize research benefits, according to the National Institutes of Health (NIH)

.

Christine Pfund

“Specifically, the NRMN Coordination Center will create the infrastructure to support substantial coordination, collection, storage, tracking, and reporting of all NRMN data, as well as implement responsive communication strategies that will effectively and efficiently build communities of inclusive practice, foster research innovations, and disseminate key findings,” the NIH says. 

Pfund says her team is “thrilled” to lead the coordination center in NRMN’s second phase.

“Our goal is to support NRMN research projects through collaboration to seize long-term potential across the collective,” she says. “Continuing the important work started in Phase I, we will provide the infrastructure needed to help the resource center and the 11 research project investigators maximize their knowledge about which interventions work, for what populations, and under what conditions. We will strive to create synergy across diverse projects and ideas to achieve a holistic perspective, thus contributing to NIH’s ability to shift the paradigm in biomedical research workforce development.”

Pfund says she plans that the coordination center also will:

  • bring together investigators’ “diverse cultural experiences, perspectives, expertise, disciplines, and theoretical frameworks as they work to increase diversity in the biomedical workforce.”
  • catalyze collaborations to generate new knowledge over the next five years.
  • lay the groundwork for the next decade of research in the areas of mentoring, professional development and networking.

600 mentors to be trained with Byars-Winston grant

Byars-Winston says her team has confirmed about half of the 600 faculty mentors who will participate in her research project – titled “Impact of Culturally Aware Mentoring Interventions on Research Mentors and Graduate Training Programs,” – with recruiting underway starting this week for the rest.

Designed to facilitate the academic success of biomedical doctoral students by training their faculty mentors to better reach them through culturally aware mentoring practices, the project will measure both individual outcomes and any organizational changes seen in mentors’ departmental cultures.

.

Angela Byars-Winston

Department culture matters, Byars-Winston notes, because it often communicates “who belongs in science” through the messages it sends with its policies, practices and training programs.

“It’s about who gets selected to report the research coming out of a project, or who gets nominated for an award or whose work is valued in the newsletter,” she says. “Does the department communicate a climate that says, ‘We believe in every student’s success’? Or is it more about winnowing people out, telling them, ‘Take one look to your left and one to your right, and one of you won’t be here next year’?”

The basics of good mentorship include aligning expectations between mentor and mentee and having clear communications, while a key adjustment for many would-be mentors in biomedical science is just accepting that mentoring is a practice that can be learned – that it’s not just innate or not, and that it’s not the same as being competent in a particular discipline, Byars-Winston says.

“We see that attitude, especially in research mentoring, and we want to shift that,” she says. “It is a learned skill, there are competencies, you can be educated to do it better.”

Everyone wants to ‘feel valued and heard’

When teaching faculty mentors about the culturally aware aspects of research mentoring, Byars-Winston says the key is respecting differences while recognizing aspects of universality.

“We start with understanding that we’re all cultural beings,” she says. “Whether I’m a white male or a Chinese woman or some other gender/ethnicity, I start off with the idea that we all have culture, and the last common denominator is that every individual wants to feel valued and heard.”

The curriculum in her research project is to be provided online and in-person, with one testing variable focused around when the in-person intervention occurs. In Phase 1, the program was offered only in one eight-hour day, so that factor will be changed up and measured, too.

“We don’t know if we’re going to find the same effectiveness as we did before,” Byars-Winston said. “It really is a true experiment with three different formats and doses.”

Measuring two types of outcomes throughout the study’s five-year window also distinguishes it.

“This is pretty powerful stuff for us because we’ll have the first set of data that we know of in research on STEMM environments that will have this longitudinal experimental study at both the individual and institutional levels,” Byars-Winston says.

The other 10 research projects in Phase 2 include studies to:

  • build a diverse biomedical workforce through communication across difference.
  • identify the influence of psychosocial support on personal gains and objective career outcomes.
  • examine inclusive mentor networks, which includes, as one of the four principal investigators from different institutions, Jo Handelsman, a UW-Madison Vilas Research Professor and mentoring pioneer who now directs the university’s Wisconsin Institute for Discovery.

*Phase 1 also had a core for administration through Boston College; for research resources and outreach at the Morehouse School of Medicine in Atlanta; for professional development offering relevant training other than mentoring, such as grant-writing, run by the University of Utah, School of Medicine; and for mentorship and networking, at the University of North Texas Health Science Center, which handled the initial development and maintenance of the NRMN website, as well as development of the guided, virtual mentorship tool known as MyNRMN. The mentorship programs formerly gathered under the Mentor Training Core and the MyNRMN tool continue to live on the NRMN website.

The Science of Effective Mentorship in STEMM

By NASEM Board on Higher Education and Workforce (BHEW)

The Science of Effective Mentorship in STEMM

Public Release 
Wednesday, October 30, 2019
1:00pm – 3:00pm EDT

Please join us for the release of the new National Academies report, The Science of Effective Mentorship in STEMM, which systematically compiles and analyzes current research on the characteristics, competencies, and behaviors of effective mentors and mentees in STEMM. In addition to the comprehensive report, the committee will debut a practical resource guide for mentoring practitioners to create and support viable, sustainable mentoring support systems.

View additional details about this project at www.nas.edu/mentoring.

Training Opportunity: Enhance undergraduate and graduate research training programs with Entering Research

Facilitator Training workshops for Entering Research (undergraduate and graduate researcher training) will be held February 13-14, 2020 and June 4-5, 2020 at UW-Madison. In this two-day event co-hosted by the Center for the Improvement of Mentored Experiences in Research (CIMER) and the Wisconsin Institute for Science Education and Community Engagement (WISCIENCE), participants will learn about Entering Research, a training curriculum designed for undergraduate and graduate research trainees in STEMM.

 

Participants will learn evidence-based approaches to researcher training and gain the knowledge, confidence, and facilitation skills needed to design, implement, and evaluate an Entering Research training at their institutions. Participants will build their own custom curriculum based on the needs of their trainees by selecting from nearly 100 activities that address seven areas of trainee development: research communication & comprehension skills; practical research skills; research ethics; researcher identity; researcher confidence & independence; equity & inclusion awareness & skills; and professional & career development skills.

Program directors, administrators, faculty, instructors, and staff of undergraduate and graduate research training programs are encouraged to apply.  Each workshop is limited to 32 participants. Please visit the Facilitating Entering Research website for more information and to reserve your spot in one of the upcoming workshops. To join our listserv and learn about upcoming workshops, please complete this online form.

Assessment Tool to Measure Mentorship Role in Undergraduate Research

By Office of the Provost, University of Wisconsin-Madison

January is National Mentor Month and UW–Madison has reason to celebrate. For over two decades, the university has been a nationally recognized leader in evidence-based research mentor and mentee training, and has led the way in providing high-quality undergraduate research experiences. Thanks to the dedicated work of a group of units from across campus, yet another exciting, new advancement is underway.

“Undergraduate research is an integrated, high-impact learning experience that can transform the way students think and learn, and the careers they aspire to pursue, and it’s one of many ways students engage in the Wisconsin Experience,” explains Mo Bischof, associate vice provost and director of assessment, Office of the Provost. “But how do we measure the impact of these complex learning experiences, which can vary considerably from one student to the next? This is what we’re aiming to do.”

Bischof is referring to a new, collaborative effort that aims to develop a common framework and assessment tool that departments, schools and colleges will be able to use to measure undergraduate researchers’ learning experiences. The project builds on the research of Janet Branchaw, assistant professor of kinesiology and director of WISCIENCE, and will provide useful information for improving the research experiences for both undergraduate mentees and their mentors.

Members of the Collaborative for Advancing Learning and Teaching (The Collaborative) – Student Learning Assessment and WISCIENCE – are working in partnership with the Center for the Improvement of Mentored Experiences in Research (CIMER) at the Wisconsin Center for Education Research (WCER) and the College of Letters & Science Undergraduate Research Scholars (URS) program on the project.

Madison faculty members have long been committed to including undergraduate students on their research teams and to providing high-quality mentoring. Many have participated in research mentor training offered by WISCIENCE in collaboration with the Delta Program, originally developed by UW-Madison Professor Jo Handelsman and Associate Scientist Christine Pfund.

“Given UW-Madison’s notable experience and leadership in undergraduate research and research mentor and mentee training, we believe it is an ideal environment in which to develop an institutional framework and assessment tool,” says Branchaw.

Development of the framework began last spring, when drafts were reviewed and input was gathered from faculty and staff at the annual Teaching and Learning Symposium. A first iteration of the assessment tool, which includes paired surveys of student self-assessment and mentor assessment of the student, was also tested last spring with students and mentors participating in the Undergraduate Symposium.

The paired survey platform is hosted at CIMER and was originally developed as part of a NSF funded project led by Branchaw to assess learning in the NSF Research Experiences for Undergraduates (REU) site programs across the nation. The platform has the capacity to compile data from across UW-Madison’s schools, colleges and departments to produce an aggregated university wide report.

Dr. Christine Pfund, director of CIMER, notes, “I see this as an exciting opportunity. It will advance our understanding of the mentored undergraduate research experience from the perspective of both the mentor and mentee at UW-Madison, as well as align us with assessment approaches being used nationally to advance the science of mentorship.”

The UW-Madison framework and common assessment tool are expected to be available for use by departments, schools and colleges in fall 2018.

For questions or to provide input on the framework and assessment tool, please contact Dr. Amber Smith, WISCIENCE director of research mentor and mentee training. Feedback will be used to ensure that the framework and assessment tool accurately represent and capture undergraduate research learning outcomes from all units.

Join campus as it celebrates its undergraduate research and hear from students about their unique experiences at this year’s Undergraduate Symposium on April 13, 2018.

UW–Madison Researchers Contribute to NAS Report on Undergraduate Research Experiences

By Wisconsin Center for Education Research, School of Education, University of Wisconsin–Madison

Branchaw and Grodsky
Eric Grodksy and Janet Branchaw served on a NAS Committee charged with studying undergraduate research experiences.

A National Academy of Sciences committee whose members include Janet Branchaw and Eric Grodsky, both on faculty at the University of Wisconsin–Madison School of Education and principal investigators at its Wisconsin Center for Education Research (WCER), released a study examining evidence on undergraduate research experiences. Branchaw and Grodsky are among 16 experts from across the country who were invited to join the committee in 2015.

“In recent years, colleges and universities across the United States have rapidly expanded undergraduate research experiences as a strategy to increase students’ interest and persistence in STEM subjects,” states Branchaw, assistant professor of kinesiology and director of WISCIENCE, a science education and engagement institute on campus. “However, support to track the number and types of experiences, and to assess the results, have not kept pace.”

Branchaw, who directs the largest federally funded undergraduate research program on campus and chairs the National Science Foundation’s leadership committee on undergraduate research experiences in biology, has developed and studied undergraduate research programs for more than 15 years.

 “The National Science Foundation sponsored the study to learn how research experiences affect undergraduate education,” says Grodsky, professor of sociology and educational policy studies. “We reviewed a lot of literature and promising evidence that these experiences benefit students, but found the evidence far from persuasive.”

While the committee members agreed that they believe research experiences can be transformative, Grodsky’s expertise in statistical methods and causal inference helped them determine that “more comparative data from well-designed studies is needed to support that belief and to better understand which practices are most effective.”

Branchaw and Grodsky did not know each other before meeting at the Undergraduate Research Experiences for STEM Students: Successes, Challenges and Opportunities’ first committee session in Washington, D.C. “I was surprised to see somebody else from my university,” states Branchaw. “We quickly introduced ourselves and began comparing travel plans and our experiences on campus.” UW–Madison is the only institution with more than one person serving on the committee.

Though it is unclear how committee members are selected, Grodsky says selecting members of differing academic disciplines was important for the committee’s task. “Here we had scientists building and conducting science education programs for undergraduate students,” he said. “The committee needed the knowledge and tools of education experts and social scientists to analyze and understand the human impact of these programs.”

A third UW–Madison contributor to the report is Christine Pfund, also a WCER researcher and director of one of its newer projects, the Center for the Improvement of Mentored Experiences in Research (CIMER). Because mentoring was singled out by the committee for its importance in STEM education, it commissioned Pfund to write an article on mentoring that provided baseline content for a chapter in the report.

Pfund studies research-mentoring relationships and trains research mentors across science, technology, engineering, mathematics and medicine. She is one of five investigators selected by the National Institutes of Health to establish a National Research Mentoring Network. Pfund and Branchaw have a long history of collaborating on training development and the study of mentoring relationships.

“Chris is widely recognized as a leader in mentoring and has a following all her own,” says Grodsky. “We were very fortunate she was able to contribute her expertise to the report.”

According to Pfund, “Mentoring has become somewhat of a ‘sweet spot’ for UW–Madison, which is well positioned to become a national leader in the field.” Under her leadership, an important goal for CIMER is to advance the “science of mentoring” at UW–Madison and across the nation.

Enabling Effective Mentorship

By Ranjendrani Mukhopadhyay

CIMER’s director, Christine Pfund, discusses what makes mentorship successful in a special section on education. ASBMB TODAY, August

Chris Pfund
Christine Pfund, CIMER Director, Wisconsin Center for Education Research (WCER)​

What makes mentorship successful? That’s what Christine Pfund studies at the University of Wisconsin–Madison. Pfund is interested in understanding, developing and implementing effective mentor training in science, engineering and medicine.

“We’re putting our precious trainees in the hands of folks who are well-intentioned but have had no professional development in the arena (of mentoring). It leaves a lot to chance,” she says. “No matter how well-intended someone is and no matter how good they are, there is always room” to improve.

After earning a Ph.D. in cell and molecular biology, Pfund did a postdoctoral stint in the early 2000s in the department of plant pathology. She then switched her focus to improving classroom teaching and research mentoring.

These days, Pfund is one of the principal investigators of the National Research Mentoring Network that was established recently by the National Institutes of Health. She is also director of the new Center for the Improvement of Mentored Experience in Research.

Rajendrani Mukhopadhyay, the chief science correspondent for the American Society for Biochemistry and Molecular Biology, spoke with Pfund to find out more about her research in effective mentoring practices. The interview has been edited for length and clarity.

How did you become interested in mentoring?

I’d always been interested in improving teaching in the classroom and had been doing a lot of work on the side at UW Madison. About halfway through my postdoc, I started to think about what I really could do in that arena. (At the same time,) UW Madison got two big grants. One was a (National Science Foundation) grant to Robert Mathieu to establish a Center for the Integration of Research, Teaching and Learning, CIRTL, and the other was the (Howard Hughes Medical Institution) professor grant to Jo Handelsman.

I spent the next eight years working for both programs. I was an associate director of the CIRTL program at UW Madison, working primarily on professional development for future faculty in STEM. I was also a co-director of the Wisconsin Program for Scientific Teaching, which came out of Jo Handelsman’s HHMI grant. (Author’s note: Handelsman currently is the associate director for science at the White House Office of Science and Technology Policy.) The program was on improving teaching in biological sciences, faculty professional development and establishing national summer institutes. A part of that project also was to develop research mentor training.

The research mentor and mentee training continued to grow. We were able to take successes from the original HHMI grant and work with CIRTL and get an NSF grant. Then I moved over to the medical school and started working on adaptations of our approaches for clinical and translational research. I started to get some research grants and worked with social scientists and others to study interventions and start to understand mentoring relationships.

Most recently, I used all of that to become part of the leadership for the National Research Mentoring Network. It allows us to continue the work to understand interventions on a much more national scale and scale up training from evidence-based approaches.

How do you define mentoring?

The across-the-board generic definition of mentoring focuses on it being a collaborative learning relationship that proceeds through purposeful stages over time and has the primary goal of helping the mentees gain the skills and knowledge they need to move on in their chosen careers. That applies to many different kinds of mentoring relationships. It could be a classic research mentoring relationship like we know in the sciences. It can have elements of career coaching. It could be peer mentoring. It could be virtual mentoring with someone who doesn’t even have a research relationship with you.

Are there differences in mentoring between science and other fields?

When we did our adaptation work for some of our mentor–training interventions, we expected, across the STEM disciplines, for things to be very similar. What we found out was that the differences that were most salient were not between, for example, chemistry, physics and math. The differences were in projects. The kind that were theoretically based, where there was a lot of thinking, had a different nature (of mentoring) because you were working together on an idea. It was different than the “we’re doing something hands-on together.” We found the nature of the work had implications on the relationship.

Imagine when you’re in a meeting (with a mentee). You’re saying, “You need to have your own idea, and I’ll bounce off whether it’s a good theoretical idea.” Even the nature of those conversations is different from “Here, you need to master these skills, come up with an idea and implement it.”

The nature of the work influences the nature of the conversation that happens between mentor and mentee.

How do you make sure you’re not creating “mini-mes” (clones of professors) and are paying attention to diversity?

There needs to be, both at the individual mentor–mentee level as well as at a systems level, the recognition to address diversity within these relationships and acknowledge that culture plays a role. How people work, what they think is important, the motivation to do it, the vision they have for what is possible and why it matters — those are all culturally informed. If we don’t pay attention to those things within mentoring relationships, research programs and training programs, it’s going to continue to privilege the dominant cultural norms. There will not be an acceptance and a benefit from embracing different value orientations (as well as) an allowance for diversification of the workforce.

Diversification of the workforce isn’t just about embracing people who have different backgrounds. It’s about embracing that they bring different values and orientations to the table. That happens at the individual level and at the organizational level. Mentoring is the place where this needs to be addressed. If individual mentors believe that their role is to create “mini-me’s,” then who they accept, how they train, what they see as success and what ideas they accept become enormously limited.

How do you get mentors to think about what they are doing?

Mentors need to reflect on what is their motivation for taking on mentees. If their motivation for doing it is so that people can be just like them, then that has a huge influence on who they should be taking. If their idea is to inspire the next generation to do amazing things, then they need to really think about if they have set up the relationship and the environment to empower those successes.

We are putting forth this idea of culturally responsive mentoring. Cultural context matters. We are creating training to get mentors to start to work through their own assumptions — not just their biases but their own assumptions about the role that culture plays, how they can create space for that and acknowledge the impact it has. If we continue to force folks to check their culture at the door when they enter the lab, then we also choose to check all the benefits that come with it at the door.

Chris Presenting at trainingPfund says people need to reflect on their motivations to mentor.

Do you think mentorship in science has changed over time?

It may feel like there’s not been a lot of change, but there has been. The conversation alone has changed. The fact that federal agencies are calling for evidence-based mentoring to be part of the training programs — it’s a huge change. While it’s going to take a long, long time, and we certainly aren’t anywhere near where we need to be in order to capitalize on the investment, the needle has moved. I want to respect the people who feel like it hasn’t moved enough and that there is an enormous amount of work to do. (But) there has been a lot of movement in the last decade.

I should have asked this earlier: What is the payoff of mentorship?

There is a lot of research out there that has linked strong mentorship to things like enhanced scientific identity, a sense of belonging, persistence, productivity, career satisfaction and definitely enhanced recruitment of folks from traditionally underrepresented groups.

The issue is that the evaluation (of mentorship) has not been methodically rigorous. Also, often, because the definition of mentoring and the context in which it occurs is so ubiquitous, we don’t know what we’re studying or what the results are linked to. If we really want to understand the critical elements of mentoring and the roles that mentoring plays in the elements of success we want to see in diversifying the workforce, then the community has to get on board with describing what they are studying and using common metrics.

Who were your mentors?

A part of why I’m so passionate about this work is because I’ve had the privilege of having some amazing mentors! Without a doubt, Jo Handelsman has been an amazing mentor to me. My graduate adviser and folks with whom I’ve worked along the way all played different roles. One of my current mentors, Christine Sorkness, whom I work with on the NRMN, has the amazing ability to push me beyond my comfort level and make me believe that I can do it but also to say, “I’ll be here if you stumble.”

That has been a common theme — my mentors strongly believe in my potential and push me, but they also let me know that they’ll be there to help.