(Tijana Milosevic, Public Policy Intern for the Public Diplomacy Council, compiled this report.)

The U.S. Advisory Commission on Public Diplomacy held a public meeting yesterday to discuss measurement tools used in the evaluation of U.S. government public diplomacy efforts.

Walter Douglas, representing the Under Secretary of State for Public Diplomacy and Public Affairs’ office, underscored the importance of measurement efforts in Under Secretary Judith McHale’s Strategic Framework for Public Diplomacy, released in March. The Framework acknowledges that, in the past, “tools for evaluating short and long term impact have not been uniformly used and built into planning.” Thus, one of the objectives of the present strategy is to incorporate measurement into all public diplomacy plans and fully utilize evaluation assessment tools to measure public diplomacy’s impact.

“This is a heck of a commitment we’re making in our strategic framework as we go forward,” Douglas said.

Rick Ruth, Director of Policy and Evaluation at the Bureau of Educational and Cultural Affairs, noted that some have debated over time whether public diplomacy programs can be effectively evaluated. “You can measure public diplomacy, and you can measure the impact of exchanges –you just have to decide to do it and commit the resources to do it,” Ruth said.

Current evaluation efforts are focused on developing quantifiable measures and capturing longitudinal effects of public diplomacy activities. “Congress and the Administration have greater confidence in applying new funding to those activities where [we] can say we’ve measured it, here are the results,” Ruth noted.

Cherreka Montgomery, Director of Evaluation and Measurement Unit (EMU) in the Under Secretary’s Office of Policy Planning and Resources (R/PPR), described the current level of commitment to evaluation as unprecedented.

When she was hired, in 2005, public diplomacy had more than 898 different performance measures, most of which were merely outputs. “We really lacked strategic focus on the impact,” she said. With such weak performance measures, the Office of Management and Budget (OMB) assessed public diplomacy strategy and performance measurement as “not performing.” As a result of subsequent work on standardization, the number of performance measures declined to 21, and OMB revised its assessment to ‘performing’.

One of the landmark efforts to synthesize evaluation is the “Public Diplomacy Impact Project” (PDI). Launched in 2007, it is the first study to assess aggregate PD impact. It combines every public diplomacy tool used—from ECA exchanges and speakers to the America.gov website and E-journals, Montgomery said. PDI combines both qualitative and quantitative approaches: examples include focus groups on new media use and drivers behind negative and positive opinions about the U.S.

Montgomery repeatedly stressed the value of evaluation in understanding the target audiences that public diplomacy works to engage. Towards this goal, “electronic media engagement evaluation” seeks to identify the most influential topics and phrases used in digital atmospheres. Sentiment analyses include coding conversations as positive or negative in an effort to understand the nature of online discourse. She referred to McHale’s assessment that public diplomacy must move towards “digital campaigning” and evaluation towards “market research analysis.”