By River Marquez, with contributions from Susan Rusch, Robert Bartolotta, Michelle Flores, Cyndi Lucas, and Borjan Zic
This month, we are spotlighting our Workforce Demographic Data Quality Technical Assistance Project, a national initiative to understand how program participants choose to identify their race and other demographic data to ensure equity in Department of Labor (DOL) Employment Training Administration (ETA) data quality reported by Workforce Innovation and Opportunity Act (WIOA) Titles I and III programs, also known as WIOA and Wagner-Peyser (WP), respectively. Missing demographic data could reduce the precision and accuracy of equity measures. Demographic data completeness can be improved through a self-assessment of the current processes and system, as well as targeted technical assistance. MSG’s project scope was to understand why WIOA and Wagner-Peyser program participants don’t always identify their race and/or ethnicity on workforce registration questionnaires to improve data collection processes to increase equity and accessibility for program participants. Once the reporting disparities were better understood, planning technical assistance resources to improve the quality of data collection practices within workforce programs was the next step.
ETA’s analysis showed that approximately 16% of program participants (about 1 in 6) did not respond to demographic questions about their race. However, state workforce agencies showed a broad variation in response rates. Sixteen states reported that fewer than 10% of participants did not provide their race, while nine states reported non-response rates above 20% for the race category across WIOA Title I and III programs. In contrast, only 1.4% of WIOA participants declined to report their sex. Among the different demographic categories studied, ETA found significant differences in “decline to state race” rates between states. Much, but not all the non-response can be explained by Hispanic and Latino participants identifying their ethnicity but not their race; however, it was difficult to understand the cause of the missing data.
Non-Response Rate for the Race Category for WIOA Programs
The MSG team began its research by reviewing data from other agencies, organizations, and institutions. Overall, these studies found that improvements to the completion rates for demographic questions about race and ethnicity resulted from clear messaging about the purpose of data collection, responding to concerns about how government agencies use the information, providing clear definitions of key terms within the questionnaires, offering options that allow a participant to identify with a specific race category or subcategory, and providing staff-assisted services to help customers complete applications and answer questions.
After reviewing the ETA analysis, along with data from other programs that serve similar populations, the MSG team wanted to gain more understanding and context for what was observed and began asking themselves the following high-level questions:
1. How can the non-response rate vary so much from state to state, and region to region? And what are different states and regions doing differently?
2. What can we learn from the southern border states which generally have higher Hispanic or Latino populations?
3. How is demographic data presented and received by participants with diverse cultural backgrounds in the workforce registration system?
4. How different is the workforce registration system in each of the states? What are the commonalities?
5. And, what states have been evaluating demographic data fields, and the quality and completeness of the responses?
To understand the variations in demographic data reporting between states, the project team gathered information from state workforce agencies and Regional Office representatives to learn about state data collection practices. The project team conducted consultations and discovery sessions to learn about the challenges states face regarding demographic data collection and reporting. In these sessions, state WIOA program administrators and analysts reported challenges related to technical system design, staffing, developing procedural guides to support frontline staff, enhancing cultural and socio-economic awareness, and the ability to identify trends that help to reduce non-response rates and improve equitable access to programs.
2019 Wagner-Peyser Findings for Completeness by State – Non-Response for Race
The project team hosted one regional office consultation and three group sessions with select states. The consultation’s goal was to develop a deeper understanding of the context of data collection and to identify a series of priorities to guide the State Discovery Sessions and TA product development. First, MSG summarized the available demographic data quality information and how the three groups of states were categorized. MSG then presented a series of questions to the participants. The team had a few hypotheses for what these consultations would unearth, in large part due to their research.
Key insights gathered from the consultations showed that there was no one reason for reporting disparities. There are certainly cultural and social factors, but there are also limitations each state faces when working to collect data from WIOA and WP participants. Because race and ethnicity are such sensitive topics, participants are not always comfortable sharing that. It isn’t always clear what these questions are asking, or why. For example, the differences between race and ethnicity or Hispanic and Latiné aren’t always understood. Many participants don’t understand why this information is necessary, and since it isn’t required, it’s safer for them to leave responses to those questions blank.
Internal data collection presents its own logistical challenges that must be addressed in conjunction with their external counterparts. Those challenges include, but are not limited to, program staff challenges, their available knowledge, resources and tools, technical systems, data analysis processes, and the policies and procedures under which all of this operates.
MSG took its research and all that it learned from the consultations to advance the project to its next goal: to design a technical assistance plan. The project team grounded the plan on two key considerations related to demographic data collection in the public workforce system: the workforce system participation in TA activities is voluntary, and demographic data responses from workforce participants are not required.
The project team created three TA products for dissemination via WorkforceGPS. The TA products focus on promising practices, tips, self-assessments, and recommendations in four key areas: staff processes, roles, and training; equity-based and socio-economic considerations, technical systems design; and using data to inform decisions. Each TA product was informed by state agency practices implemented to improve data quality and collection practices.
Promising Practices Brief
One of the products was a brief on promising practices that was broken into four areas. Each of the four areas was developed with a specific audience in mind, which could be shared as a stand-alone document/brief. The four themes were identified in the discovery phase of this project:
1. Promoting Effective Communication, Understanding, and Collaboration for Program Equity;
2. Determining Staffing Needs and Roles;
3. Designing Technical Processes to Improve Demographic Data Quality and Completeness; and
4. Using Data to Make Informed Decisions.
The team also developed a tips document for frontline staff when communicating with customers who are unsure how to answer demographic data questions. There is also a self-assessment tool that allows users (State Workforce Agencies and local American Job Centers) to evaluate their communications and guidance to participants and staff about demographic data collection practices and data quality, along with their technical system design, reporting capabilities, and their ability to look for trends within reports to identify errors and inaccuracies in data collection.
The complete suite of products is available on WorkforceGPS and is stored under the September 14, 2022, webinar event called Promising Practices for Improving Demographic Data Quality for WIOA Programs.