FORMATIVE EVALUATION

Our evaluation plan involves collection of baseline and progress data in four interrelated categories:


1. Overcoming ISU "known barriers" to retention and advancement of women and people of color in STEM;


2. Overcoming Department specific barriers to retention and advancement of women and people of color in STEM;


3. Institutionalizing change; and


4. Increasing overall retention and advancement of women faculty into senior and leadership ranks.

There is overlap among categories 1-3 because the issues are inextricable. Key to the effectiveness of our institutional transformation plan are the ADVANCE Researchers and one of our External Advisors. ADVANCE researchers, representing social sciences, statistics and women's studies, will use formative evaluation data to refine operating procedures, develop best practices toolkits, and contribute to our knowledge on women in STEM fields. Professor Sullivan (ISU ADVANCE External Advisor, Utah State ADVANCE Co-PI) will assist in establishing routine information sharing between U-State and ISU regarding department-level interventions and toolkit development. Bi-annual teleconferences will enable ADVANCE teams from each school to learn from one another's successes and challenges.

Indicators of Progress toward Overcoming ISU's "Known Barriers." Our efforts to document progress toward overcoming ISU "known barriers" to the recruitment and advancement of women in STEM fields will be enhanced by ISU's participation in the Harvard directed Collaborative on Academic Careers in Higher Education survey (COACHE) (Harvard Study of New Scholars, 2005). COACHE baseline data on the satisfaction of tenure-eligible faculty will be collected in Fall 2005, and a follow-up survey to assess progress will be done in Fall 2007. ISU will develop additional questions for the COACHE survey to enhance the usefulness of the data for the ADVANCE project. Examples of types of indicators, and the data to be collected are:

Overcoming Lack of Transparency


• Proportion of departments with clear and accessible documentation of decision-making processes in key areas (e.g., space allocation, salary increases, resource allocation, promotion and tenure).


• STEM faculty members' perceptions of transparency in decision-making.

Overcoming Isolation


• STEM faculty members' perceptions of networking opportunities within department and the college, and across campus (COACHE survey, baseline Fall 2005).


• STEM faculty's perceived levels of isolation in departments, colleges, university.


• Proportion of STEM faculty whose perceptions of isolation have improved.

Overcoming Lack of Mentoring


• Increase in quantity and quality of mentoring for women and underrepresented faculty of color.


• STEM faculty's perceptions of the effectiveness of mentoring programs on increasing access to networking and other career opportunities, research teams, promotion and tenure information, etc.


Overcoming Lack of Career Flexibility


• STEM faculty use of existing and new career flexibility policies.


• STEM faculty's perceptions of how career flexibility policies affect own and others' progress toward tenure, promotion, salary increases, teaching assignments, and leadership opportunities.



INDICATORS OF PROGRESS TOWARD OVERCOMING DEPARTMENT-SPECIFIC BARRIERS


• Proportion of department chairs who have received training in diversity and inclusive leadership.


• Changes in proportion of faculty within focal departments who support and/or are actively involved in achieving equity objectives (including participation in workshops, seminars, etc.).


• Proportion of STEM departments that have developed and implemented "best practices" guidelines for achieving equity objectives.


• Proportion of faculty within focal departments who report changing routine work activities (including teaching, research, service) to ensure positive equity outcomes.

Indicators of Progress toward Institutionalized Change (see ADVANCE Institutional Transformation Indicators Working Group, 2005)

Outcomes of institutional processes of recruitment and advancement.


• No. of faculty who submit tenure packets, and number awarded tenure, by gender and department.


• No. of tenured associate professors by department and gender with years-in-rank.


• No. of faculty hired and who leave by rank, gender, and department.


• Cohort analyses of tenure and promotion, including to full professor.

Allocation of resources for science and engineering faculty at ISU.


• Study of salaries of men and women faculty (with controls, e.g., department, rank, years in rank).


• Study of space allocation of STEM faculty by gender (with controls).


• Study of start-up packages of newly hired faculty by gender (with controls).

Indicators of Increased Overall Retention and Advancement of Women Faculty in Senior and Leadership Ranks (see ADVANCE Institutional Transformation Indicators Working Group, 2005)

Distribution of science and engineering faculty.


• Number of men and women tenured and tenure-track faculty by department, rank and gender.


• Number of non-tenured men and women faculty (e.g., Instructional, Research, Clinical, Postdoc.).

Distribution of science and engineering faculty in leadership positions in the institution.


• Proportion of STEM women and underrepresented faculty of color in leadership positions.

Data on key indicators of progress toward ADVANCE goals will be collected in each of years 1-4, enabling us to establish a baseline for assessing progress. We will submit baseline data in the year1 Annual Report, and will provide data on changes in key indicators in subsequent reports. Because demographics and dynamics in STEM and Social and Behavioral Science (SBS) departments often differ (see ADVANCE Institutional Transformation Indicators Working Group, 2005) we will examine and report baseline and progress data/analyses separately for STEM and SBS.


Back to Initiatives