Our Process

The evaluation will be guided by the following evaluation questions:

  • What types of organizations received ARC business development grants? What are the characteristics of their beneficiaries?
  • What were the goals of the projects? What approaches were used to meet the goals?
  • What outputs and outcomes did grantees achieve upon conclusion of their grant?
    • What additional outcomes and impacts have been realized since conclusion of the grant?
  • How did beneficiaries of the grant efforts participate in the grant project and what was their experience? Did the business performance measures of beneficiaries increase after participating in a grant?
  • What factors influenced grantees’ successful implementation and sustainability? What challenges/barriers to success did projects face and how were they addressed?
  • What impacts has the pandemic had on grantees’ ability to serve their communities/beneficiaries?
  • What changes have grantees already implemented, or are planning to implement, in response to evolving needs /opportunities that emerged from the pandemic?

The evaluation will utilize the CDC’s Program Evaluation Framework, a widely respected framework for evaluating federal- and state-funded programs in the United States. The framework encompasses six steps emphasizing accuracy and ensuring that evaluation findings are useful. Recognizing the need for strong, authentic working relationships with ARC and grantees to produce a useful evaluation, the evaluation will take a participatory approach. Accordingly, we will solicit input from staff, grantees, and their partners at key milestones in the process through kick-off meetings, a survey, focus groups, and case studies.

CDC Six Step Evaluation Framework​

Six Steps

The evaluation process will unfold over three phases, where information learned from each phase will be used to inform the next phase of data collection

Phases

Phase I: October 2022 – January 2023

The evaluation will start with a thorough document review of materials related to Business Development grants. Once the team is thoroughly knowledgeable about grantees and their grant efforts, we will host two virtual kick-off meetings with grantees about the evaluation process and invite their active participation in the evaluation.

Phase II: February – June 2023

After the kickoff meetings, a survey will be sent to all grantees to update performance measures and collect information about the status of grant activities, beneficiaries served, partnerships, additional funds leveraged, factors that contributed to success, work on equity, and impact of the pandemic. Findings from the survey will be used to develop topics for a series (~10) of focus groups to be held with select grantees, partners, and beneficiaries to probe survey findings more deeply. These virtual focus groups will examine different topics, such as factors contributing to their successes, challenges encountered and overcome in implementation, partnerships, equity, and impact of the pandemic.

Phase III: July – September 2023

Following the focus groups, all data and learning collected to date will be used to select grantees/communities for ten case studies. Prior to a 2–3-day site visit for each case study, a secondary literature review on the economic conditions of the targeted communities will be conducted to add additional context to the grantee’s work. Site visit activities will include observations, additional document review, and semi-structured interviews with grantee staff, partners, and beneficiaries to learn about the implementation approach and perceived influences that either contributed to or deflected from success.

Conclusion: October 2023

Once all data is collected, evaluation findings and lessons learned will be shared with ARC and grantees to ensure evaluation use. A public-facing final report and presentation that highlights grantee efforts and outcomes and overall lessons learned will also be developed.