Measuring results: making the business case for learning

Well thought-out interviews can uncover organizational roadblocks to success, leading to team and organization interventions if needed.

Seven keys to measurement success

Our approach with our clients is to: a) identify the key measures of success, and b) structure simple, but thorough, systems to capture accurate data, draw conclusions, make adjustments as needed, and report results.

Our experience shows that there are seven keys to creating a successful measurement system.

1. Get early and substantial stakeholder alignment
Conduct a stakeholder analysis upfront. Ensure sponsors, funders, key organizational leaders and other stakeholders are aligned about the problem or opportunity for which this training is a solution and how it is connected to business strategy. Set specific expectations about success, as well as about measurement's importance, and its cost. Determine together the level of evaluation desired for the program.

Agree on the rationale for the evaluation. Typical rationales include deciding whether or not to continue a program, identifying improvements to a program, and justifying the investment.

2. Consider cost versus benefit
In our experience, many organizations plan very elaborate measurement systems at the launch of an initiative — but never implement them. This occurs primarily because the costs of developing and implementing the system aren't worth the benefits, or the program — as measured by more simple instruments — is determined successful early on.

3. Define the behaviors you want
Be rigorous in your thinking and detailed in your definition of the desired behaviors that are critical to your organization's success, and therefore must be measured.

4. Use a control group
It will give you a better sense of whether or not there has been an actual behavior change, and may limit your need to invest in deeper and broader measurement tools later.

5. Use before-and-after instruments
Short, yet skillfully developed before-and-after instruments can gather good data inexpensively, particularly when measuring skill improvement. Allow time for behavior change to take place before administering the "after" instrument.

6. Create shared responsibility
Any measurement system should be co-designed and implemented between Interaction Associates and your organization. This demonstrates your commitment to measuring results to internal stakeholders, and allows you to benefit from our thought partnership.

7. Seek "evidence of impact" before "proof of impact"
Dean Spitzer, a senior performance consultant with IBM, noted in his article "Embracing Evaluation" in the June 1999 issue of Training, that gathering "evidence" of behavior change or impact is much less expensive than statistical "proof." If structured correctly, evidence data can be effective and determinant, while being less expensive to design and implement.

A Successful Measurement Plan

What does a successful, cost-effective measurement plan look like? "Keep it simple" when you first start out.

You might start with an evaluation (Level 1, in Kirkpatrick's Levels of Evaluation) to capture the learner's reaction to the course. To measure learning (Level 2), you might use electronic pre- and post-assessments, built around the workshop objectives, for both the participant and the participant's manager.

To determine behavior change on the job and business impact (Levels 3 and 4), you might use post-assessments to report use of the knowledge and skills on the job 90 days after completing the training, and interview participants and their managers about the business impacts they have achieved that could be attributed directly to the training.

The interview is a where you can trouble-shoot organizational issues that may be impeding implementation of learned behaviors. Well thought-out interviews can uncover organizational roadblocks to success, leading to team and organization interventions if needed.