However, thanks to the use of MeetingMetrics, a new meeting management tool, organizers learned in advance that most attendees' views differed completely from those of management. Armed with this knowledge, the event could be planned with specific goals in mind - and the level of change in attendees' attitudes after the meeting could be easily measured.

"Before the meeting, 75 percent of attendees believed that the current brand strategy was good," said Tony Lorenz, president of ProActive Inc., a recently acquired Freeman company. "But it had already been decided that it was going to change; AVW-TELAV [a Freeman-owned audiovisual company] would be re-branded as Freeman. So that was a mountain to climb."

Another disconnect existed regarding Freeman's recent acquisition of ProActive, a meeting design and production company. Many Freeman employees didn't see how ProActive's services aligned with those of other Freeman companies. In fact, based on pre-meeting surveys, which garnered a response rate of 81 percent, "more than half the audience didn't understand the strategic acquisition of ProActive, so we had to attend to that," Lorenz said.

Based on the results of the pre-meeting surveys, Freeman reconsidered the content of the meeting program and made some changes. In the case of the re-branding initiative, "we had to create great emphasis on the strategic rationale for the change," Lorenz said. "We did that through a number of tactics; for instance, we brought in customer views about the Freeman brand, live and in video, underscoring the market opportunities available by being a single brand."

While the topic of re-branding certainly would have been discussed at the meeting anyway, "without understanding the ideas of our audiences, it wouldn't have happened to that extent," Lorenz said. As a result of the focused programming, post-meeting surveys showed that 95 percent of attendees believed the new brand strategy reflected the company's best interests.

In an effort to communicate management's reasons for the acquisition, "we showed by doing," Lorenz said. The newly acquired company took responsibility for the event's communications strategy, creative direction, content development, and production, which showed attendees "what we do and what we brought to the meeting," Lorenz added. "In the post-event survey, only 4 percent didn't understand the strategy behind the acquisition."

Design Tool and Report Card

For Freeman, MeetingMetrics served as both a "design tool and a report card," Lorenz said, automating the processes of both research and evaluation and making results easily measurable and reportable. "Our approach is that in order to begin the creative process, we need to understand the expectations of both the executive team and the attendees so that we can develop the event according to the full environment," Lorenz said. "We have been using this methodology for a long time, but what used to cost $75,000 to $80,000 now costs $2,000 with this new online tool."

MeetingMetrics, developed by GuideStar Research, is an easy-to-use online service. It is based on 25 years of meeting measurement experience and expertise. For more than two decades, GuideStar has made a name for itself as a pioneer and leader in measuring large group meeting results with its Meeting Productivity Process (MPP), which measures the psychological and behavioral impacts of large group meetings. Provided by GuideStar's staff of research professionals, Ph.D.-level organizational psychologists, the MPP has delivered dramatic, measurable results for many Fortune 500 companies looking to get more from their corporate meetings and events. MeetingMetrics, offered in partnership with PCMA and Starwood Hotels and Resorts, puts the same methodology and research into a simple online program, suddenly making this level of measurement available and affordable for groups of all sizes.

"I am very excited about MeetingMetrics, and I have been since the first time I saw it, because I do believe it delivers strategy to our professional members," said Deborah Sexton, PCMA's president and CEO. "It allows them to be more strategic because they have the ability to capture information that hasn't been available before, both pre- and post-meeting. Now, a variety of people on the planning team - the meeting planner, CEO, marketing, and others - can get involved and create the types of questions that will help them get the results they need."

Widespread Appeal

Although MeetingMetrics was just unveiled to the meetings industry at PCMA's Annual Meeting last January, meeting professionals have quickly taken notice and begun utilizing it to the advantage of their groups. "I found out about MeetingMetrics at PCMA, and I was intrigued," said Krista Fleming, owner of KF Events in Shoreline, Wash. "As an independent meeting planner, I'm always looking for tools that already exist that I can tap into without inventing something on my own. I like to help my clients think strategically and look at the big picture, and this tool seemed like a way to build on that."

Fleming first used the tool with a labor union group in conjunction with its annual convention this summer. With three or four different surveys, the group was able to determine attendees' feelings about a number of important issues. For instance, planners wanted to gauge attendees' understanding of some legislative issues the group is involved with. Pre-meeting surveys conveyed a need for further understanding, so meeting content was altered. Group leaders felt that ideas were successfully communicated and understood at the event, but they couldn't be sure until results were actually measured.

"When we did the post-meeting surveys to evaluate the ROE [return on event], the attendees' increase in understanding was huge; it went up by 20 points," Fleming said. "It was great for them to see that and it was good to quantify that the sense we got was indeed accurate."

Not only did MeetingMetrics allow attendees the opportunity to gain more understanding, but it also offered the meeting location an opportunity to serve as host again. "Prior to the meeting, we knew there were some people who were unsure about the site and we weren't sure if attendees would want to come back to that city," Fleming said. "But post-meeting, [the level of interest in returning] went up because the city did a great job hosting us. MeetingMetrics helped the city too because they got another chance."

For groups like Fleming's, MeetingMetrics offers a strategic approach to measuring audience attitudes that may have never before been feasible or affordable. "My client was very impressed; for them, it was a big step forward in how they approach their meetings," Fleming said. "They loved how I could provide them with so many reports, and they're planning to use it again."

PCMA used MeetingMetrics on a limited basis prior to its 2007 meeting in Toronto, and has plans to fully utilize the tool in conjunction with its 2008 meeting in Seattle. "The information we got was quite good; we figured out there were some areas we might want to focus on more," Sexton said. "I love this tool; I ask these questions all the time, but [before MeetingMetrics], I didn't have a way to ask them of our members very efficiently."

An Added Level of Measurement

While MeetingMetrics has measured return on event (ROE, also known as return on objectives) since its inception, the latest addition to the tool is the capability to measure an event's return on investment (ROI). "Large association meetings are important revenue generators for the association; some associations earn as much as 40 percent of their annual revenue from their annual conference or convention program," said Ira Kerns, managing director of GuideStar. "They can easily do the math using known revenue and expenses to quickly determine their association event's ROI. So although event-level ROI can be calculated and reported in MeetingMetrics, most important is the ability to measure participant-level ROI. "What is new and unique is that MeetingMetrics now offers associations the ability to accurately determine participant-level ROI, something they have never been able to accurately capture before," Kerns continued. "Associations can, for the first time, confidently use participant meeting ROI to promote the value of the participant's investment in attending the meeting. For the first time, an association can prove the financial value of both their individual attendees and aggregated average attendee investment in the meeting."

The tool's ROI measurement component was developed with and is endorsed by the highly-regarded ROI Institute. Jack Phillips, chairman of the ROI Institute, says there are five levels of measurement for meetings: (1) reaction; (2) take away; (3) application, or what attendees did differently after the meeting, such as trying something new or implementing a new procedure; (4) impact, which is the consequence of that application, such as more sales, greater productivity, or greater efficiency; and (5) ROI, which is the monetary value of the impact compared to the cost of the event.

"MeetingMetrics has been evaluating meetings and events for some time and has a good track record, particularly in measuring pre-event perceptions and post-event reactions and learning," Phillips said. "What they didn't do was measure application and impact, and ultimately, ROI. Our focus is on levels three, four and five, which are the ones people struggle with most. [Meeting planners] are under more and more pressure to show that a particular event made a difference, so they have to go beyond what happened and show the impact."

With the addition of the ROI component, MeetingMetrics now provides the full spectrum of meeting measurement, including levels one through five. However, every level of measurement is not necessary for every meeting, Phillips said. "We have to caution people to make a conscious effort to decide what they want to measure at a particular meeting," he said. "The more high-profile, expensive meetings need more accountability. Some meetings are okay with measuring an application; they may want to make sure everyone implements a new strategy or follows a new policy. Some want to go to impact; maybe they want to make sure fraudulent activity is reported. And some want to go all the way to ROI, but you don't need to go to ROI for every meeting." (See sidebar on p. 88 for information about determining when to measure ROI.)

While it may not be worthwhile to measure ROI for every event, some level of measurement is almost always in order. "[MeetingMetrics] really aligns with a need that's been out there," said Max Suzenaar, president of Minding Your Business Inc. "Everyone's been looking at what is the value of producing an event, and this is a step forward in providing a solution. Within our industry, it is the only substantial tool out there that helps measure ROE as well as ROI."

A Strategic Approach

With the ability to capture attendee ROI at an event, a strategic meeting measurement tool like MeetingMetrics is poised to transform the way meetings are planned and the way planners are regarded within their organizations. "This tool is creating more awareness for planners that we've got to be more aligned with the overall strategy of the organization," Suzenaar said. "It's where the industry is heading."

Tools like MeetingMetrics are bound to "make the meetings industry more strategic and more client focused," Phillips said. "It requires them to think about content and specific objectives; it changes the approach to meetings. I'm not suggesting we take the fun or the warm fuzzies out of meetings, but some meetings need to drive business value and now we can measure that business value. Tools like this allow meeting planners to become more of a business partner, more of a strategic partner."

"Planners who embrace MeetingMetrics can become a central resource for key strategic information about meeting needs and meeting results," Kerns added. "Once their initial skills are developed, they will have expanded their roles by becoming more valuable, promotable, employable, and secure in their jobs. Because of the important strategic information they are now able to gather, interpret, and report from MeetingMetrics, they will be viewed as a more valued member of the team, someone who contributes beyond the meeting's success, to the organization's success. In a very real sense, MeetingMetrics is about transformation for a meeting planner who wants to take his or her skills and career to the next level, moving beyond meeting logistics to strategic meeting planning that boosts meeting effectiveness, and proves it."

"Many organizations think the meeting planning process is 100 percent about logistics; but while logistics are very important, our members are much beyond that," Sexton added. "In today's environment, it's all about the experience. It's about the content, and if you're not creating the 'wow,' you've missed an opportunity. To create that, our members have to be more involved in every step of the meeting. And MeetingMetrics allows them to do that."

For planners who have already been thinking and operating strategically, MeetingMetrics seems to offer an easy solution for automating that mindset. "I think it's a tool that accomplishes what I talk about with my clients," Fleming said. "I already ask those questions to help them think about the why and the how. This gives us a hands-on tool to accomplish that strategic approach."

An Independent Opportunity

Like Fleming's KF Events and Suzenaar's Minding Your Business, a number of independent meeting planning firms and other meeting service providers are interested in providing an extra level of service to their clients - and many of them are doing so by becoming MeetingMetrics Service Providers. These Service Providers purchase an annual license and a minimum number of meeting packages at a special professional discount price, in exchange for online training, sales presentation aides and online demonstrations, support and consulting. After successfully providing MeetingMetrics services through two meeting cycles with their clients, Service Providers can become "certified," which qualifies them for client referrals through MeetingMetrics.

"[MeetingMetrics Service Providers] can benefit in a number of ways," Kerns said. "They get a new specialized service offering for their business that can produce a new revenue stream; professional development of new, added-value consultant skills, which can deepen and broaden their client relationships as well as enrich their professional image; not to mention the competitive advantage they will enjoy in being able to offer the MeetingMetrics service." "There's only one way to really expand our business in a real way, and that's the delivery of meetings with targeted messages attached," Lorenz said. "MeetingMetrics allows us to do both with intelligence and end results that meet or surpass clients' expectations. [Planning meetings] in a void is not only ineffective, but it's also damaging because you can entirely miss major points that the audience wants to see addressed."

Contributing Editor Nancy Mann Jackson is a freelance writer in Birmingham, Ala.

When Should You Measure ROI?

Although MeetingMetrics makes it possible to measure the full extent of a meeting, including ROI, that in-depth measurement isn't necessary for every event, according to Jack Phillips, chairman of the ROI Institute. In fact, one component of thinking strategically about meetings and events involves knowing what should be measured at each event. In MeetingMetrics training sessions, the ROI Institute teaches planners how to make decisions about what to measure for each event. The Institute recommends considering the following:

• The meeting's lifecycle - Repeatable meetings are greater candidates for ROI; there is less value in measuring ROI for one-time-only events.
• The strategic focus of the meeting - The closer a meeting's focus is to the strategy of the organization, the greater its candidacy for measuring ROI.
• The meeting's role as a problem solver - If the meeting is important enough to correct a major problem/issue for the organization, it may be important enough to measure ROI.
• The cost of the meeting - The more expensive a meeting is, the more likely it is a candidate for measuring ROI. ‰ The size of the audience - A large annual conference would be a better candidate for measuring ROI than a small regional meeting.
• Meeting accountability - ROI measurement may be demanded by the ultimate owner of the meeting.

Use MeetingMetrics Effectively

While measuring attendees' attitudes and behaviors is certainly a valuable undertaking, there are a number of variables involved in effectively using MeetingMetrics and the data it generates. Keep the following in mind:

• Limit your use of the tool. Asking questions may be the only way to learn what your attendees are thinking, but asking questions all the time will drive your attendees crazy. For instance, while PCMA President Deborah Sexton is passionate about using MeetingMetrics, she is also acutely aware of respecting members' time. "I don't want to be constantly surveying our members," she said. "We have to be very, very careful in taking advantage of this tool without overdoing it for our members."
• Focus on attendees' needs. Max Suzenaar, president of Minding Your Business Inc., said he believes that getting people to respond to the pre-conference surveys could be a potential challenge in using MeetingMetrics. But in his first use of the tool, he overcame that challenge by positioning the survey as a relationship-building tool. "We just utilized it as an opportunity to reach out to attendees," he said. "We said, 'We want to customize your experience and focus on what's important to you, and that's why we need you to respond to the survey.'" Within one week, 68 percent of Suzenaar's audience of CEOs and senior vice presidents of large organizations responded to the pre-meeting survey.
• Respond to data immediately. If post-meeting survey results show that PCMA members want to know more about a particular topic, "they can't wait until next January," Sexton said. "If we haven't addressed something they needed at the meeting, we're going to create a Webinar, create a white paper, form a committee to study the issue, or something. We're going to develop a way to address that issue and let members know we're doing it. This is the way to constantly respond to members so they know you're listening."