NEFE has delayed the release of the HSFPP Edition 9 content. Learn more about the anticipated rollout.

Is Your Financial Education Program Hitting the Mark?

Ready, Aim, Evaluate

 

This article was originally published in the Nov.-Dec. 2016 NEFE Digest.

You have designed your financial education intervention, workshop or program using trusted, vetted resources. You have created meaningful activities based on learning objectives. You have a well-trained educator ready to deliver the curriculum to your target audience. The students are ready. The teacher is ready. So, how will you know if your program is successful or not? NEFE’s financial education evaluation resources can help.

Why Evaluate Your Financial Education Program?

A financial education program without evaluation is similar to an explorer without a compass. Without a compass, an explorer is not able to decide whether he or she is on the right track. Without an evaluation, the educator can’t know for sure whether the financial program is producing successful results and meeting the audience’s needs.

What is Financial Education Program Evaluation?

The process of systematically assessing the implementation of a financial education intervention by comparing learner achievements with program goals and objectives to identify the intervention’s strengths and weaknesses.

Benefits of Evaluation

Evaluation gives you data to inform the future actions of your program. This data can help you enhance what is working and change or eliminate what is not. A successful evaluation can be used to:

  • Build a shared meaning among internal and external stakeholders about the program’s goals.
  • Inform decision making with objective data rather than just anecdotal feedback or personal perceptions.
  • Measure program performance and goal achievement.
  • Identify effective (and ineffective) practices in financial education.
  • Document findings to justify continuing or expanding your financial education program.

How to Evaluate a Financial Education Program

 

Step 1

Begin with the end in mind.

Design your evaluation based upon the desired program outcomes. What is the ultimate goal of the intervention? What exactly is it that you want learners to know and be able to do as a result of the intervention?

Step 2

Be realistic about outcomes.

What are realistic expectations given the material covered in the intervention, the time frame, the delivery method and the audience? A one-time workshop presented to learners with very low financial literacy can expect different outcomes than a semester-long course presented to learners with high financial literacy.

Step 3

Choose the right evaluator.

Some programs can be evaluated adequately by internal program staff, while other programs benefit from hiring an objective external party to conduct the evaluation.

Step 4

Include stakeholders from the start.

Build an advisory team that includes any and all stakeholders who will be interested in and/or affected by the evaluation results. This includes program staff, funders, administrators, decision makers and learners. Make sure that you understand what data each of these stakeholders hopes to see in the evaluation.

Step 5

Plan the evaluation before program implementation.

Effective evaluation covers the full spectrum of the intervention – from a pretest or survey that establishes the learners’ baseline starting point, to a post-test or survey that measures the learners’ knowledge and skill growth immediately following the intervention, and in ideal situations, a longer-term post-test or survey that assesses the lasting behavioral change for an extended time (from six months to a year or longer) after the intervention.


How Do You Know When Learners Get It?

The ultimate goal of evaluation is to collect data that helps your program achieve its mission — in financial education, that mission often involves changing learners’ behavior over an extended period of time. Learners typically don’t achieve this type of change overnight. In general, financial education out comes advance up a hierarchy.

Learning to change
Evaluations show learners’ satisfaction with the program and changes in knowledge, attitudes, skills and aspirations.

Taking action to change
Evaluations show learners’ first steps toward practicing and becoming comfortable with the changes they are working toward.

Maintaining long-term change
Evaluations show improved circumstances as a result of lasting behavioral change.

Managing Expectations of Outcomes

The outcomes your program can expect vary with the type of program, the fidelity of program implementation, and the age and financial literacy of the participants. Short-term outcomes are easier to measure and document than long-term outcomes. For example, it’s easier to conduct an immediate post-intervention test measuring learners’ knowledge of how interest rates work than it is to assess whether the learner made the “right” decision when financing a car six months after the intervention.

Type of Program Short-Term Outcomes Long-Term Outcomes

First-Time Homebuyer Education Program

  • Knowledge of how to assess affordable housing
  • Knowledge of how to save money for closing costs
  • Ability to shop for the lowest mortgage interest rate
  • Reduced stress and anxiety about home buying process
  • Purchase of a home within financial means
  • Successful payment of mortgage over time
  • Frequent use of personal budget techniques to manage home-related expenses

Debt Reduction Education Program

  • Ability to identify needs and wants separately
  • Understanding the effective spending habits
  • Knowledge of personal and household budgeting techniques
  • Knowledge of credit building strategies.
  • Reduced debt
  • Improved debt ratio (debt/assets)
  • Improved credit score
  • Frequent use of personal budget techniques to manage debt

How Age Affects Outcomes

Learners’ outcomes also are affected by their age and stage of life development. The Consumer Financial Protection Bureau (CFPB)’s Developmental Model of Youth Financial Capability offers guidelines to help aim interventions at age-appropriate target outcomes. To view the CFPB financial well-being model, visit www.consumerfinance.gov.

Ages 3-5 Ages 6-12 Ages 13-21 Adult Learners

Executive function – self-control, working memory, problem solving

Healthy financial habits and norms – Heathly money habits and norms, e.g., frugality, saving, planning ahead, considering personal values in decision making

Financial knowledge and decision-making skills – Factual knowledge, research and analysis skills, how to be an informed consumer

Financial well-being  – Control of day-to-day finances, ability to absorb financial shocks, on track with goals, financial freedom to enjoy life


Design Your Evaluation

Are you ready to design your financial education program evaluation? NEFE’s Financial Education Evaluation Toolkit (Toolkit.nefe.org) has resources to help you, including NEFE’s Financial Education Evaluation Manual and a database of evaluation forms, templates and questions to get you started.

Download the full November-December 2016 Digest(.pdf).