Success Criteria

9 11 2009


Note for Workshop Discussion: Narratives and Success Criteria for NCCs
Michael Joroff and David Good 11.04.09

Two themes stood out at our May 2008 NCC workshop in Stockholm. The first was the need for every NCC project to have a directive narrative. A narrative is a story to which all participants can refer whenever they need to set or check their direction, and which weaves together the interests of the many stakeholders and enables the alignment of their collaborative actions. These directive narratives do not offer any simple designation of the projected outcome of any NCC project because that is to be developed and discovered, but they do provide a common orientation and, in the mutual development of the narrative over time, a way of ensuring that the shared understanding is maintained.

The second important theme was the challenge of keeping narrative alive and operative during the transitions that inevitably occur in multi-year projects. Transitions with great potential for disruption include: (1) leadership change within the project or within key stakeholder enterprises; (2) new stakeholders joining the project as it unfolds; (3) movement from one development phase to another, sometimes with team membership changes; and (4) regional or global economic change, either growth or retraction. These transitions are commonplace, but predicting their timing and consequences is difficult.

Discussion about narratives and transitions in Stockholm led us to begin exploring the success criteria used by NCC projects. Attendees acknowledged the link between a robust narrative and what success might or ought to look like. Agreement on success criteria among project stakeholders does more than signal whether aspects of the project are “on target” or not; agreement is also a key to obtaining buy-in to the vision and to harnessing energy around mutual interests.   A common interpretation of the narrative and agreement on success criteria also enhances the ability to turn operational feedback into useful project adjustments.

Our November 10th and 11th workshop in Seoul workshop will devote considerable time to NCC success criteria.  We will approach the discussion from two mutually supporting perspectives. The first perspective emanates from the field–from the people who run these projects. These are the people who must engage mayors, developers, city councils, and citizen groups.  The second perspective comes from people (including some of the first group) who look for frameworks that can usefully sort things out and show common threads across projects.

This note presents both perspectives and is intended to stimulate thinking and to encourage you to think of cases from your own work, and thus more perspectives from the field, through which you have understood success and failure. What follows is far from definitive, but the purpose of our workshop is to develop conceptions of success criteria and practical examples of the same that may be of use to one, some, or all of those present

Perspective from the field

One of our front line colleagues (anonymous) contributed this candid comment:

. . . the success of [a] project would be is a quite interesting topic, even when you are far from reaching your objectives. . . .Defining [goals . . . is extremely useful when working on a NCC project, where there is a staggering variety and number of potential objectives.  [This] often results in difficulty explaining what the project is and intends to be. You need half an hour!

According to our experience with our project, which is in an initial phase of implementation, the four fields included in your Balanced Scorecard [suggested in an earlier note sent by MIT] provide a correct framework for assessing a NCC project’s progress. If you had to list the four issues in order of importance for the project, you would need a dual list:

List of importance (long term view – professional, academic satisfaction)

1. NCC Project
2. Enterprises
3. Real Estate Portfolio

List of urgency (short term focus – primary need to survive)

1. Real Estate Portfolio
2. Enterprises
3. NCC Project

Yes, I deliberately missed the fourth topic: ‘Branding Host City as a Creative Place’. I really appreciate that one. It is one of the most frequent fields of work for us and promoting the project.  However, this should be a natural outcome of the process once you’ve been successful implementing the other objectives in the scorecard. If not, there is a risk of focusing too much on this one in the beginning of the project and cause at this point some distortion among the true needs to successfully build the project and the yearn for a too anticipated reputation.

Perspective of framework builders

We have touched on the topic of success criteria in conversation with many NCC participants. Their views fall into two broad categories: the first relates to process; the second concerns end states.

Process criteria are about actions. For example:

  • launching a training program
  • overcoming a regulatory hurdle
  • opening a building by a certain date
  • hooking-up to an ICT network

Or they may refer to organizational achievements:

  • the number of collaborative arrangements launched
  • the inclusion of area residents in the project planning process
  • collaborations among international researchers
  • university graduates who start new firms in the NCC

These are primarily indicators of efficiency–the ability to do something well, within agreed performance parameters.

End state criteria include measures related to the achievement of necessary or desired results.  These are indicators of efficacy — the ability to achieve the desired result. These include, for example:

  • the market penetration of broadcast programs produced
  • the number of product or service innovations
  • the market value of enterprises spawned within a cluster
  • the number of people employed by the cluster
  • the revenues generated in the local economy

Thus far, our examples of criteria have all related to success.  Indicators of failure or potential trouble could also be useful to participants in NCC projects.  Like our success criteria, they could be related to processes or organization behaviors:

  • a slip in supply chain logistics
  • attendance decline at coordinating meetings
  • failure to close on a long-pending deal
  • a reduction in venture capital investments
  • fewer people being hired on their completion of training programs
  • start-up firms not surviving five years

Alternatively, criteria (positive and negative) could be related to factors in the larger business or economic environment, such as an upturn or downturn in the commercial real estate market. Criteria, or metrics, such as these and those described above should be on the “dashboard” that project leaders use to monitor their projects.  Negative indicators should signal them to intervene with appropriate corrective actions.

Several general management tools can be used to sort out criteria and how they relate to narrative driven project visions. These include, for example, strategy and stakeholder game boards, balanced scorecards, and dashboards.  These and their applications will be discussed when we next meet. For basic definitions, consider the following.

A strategy map captures the development’s overarching strategic visions and expresses it as several layers of interrelated actions and supporting elements.

The balanced scorecard articulates specific objectives and the key quantitative and qualitative performance indicators that relate to the development’s objectives and the prime interests of the participating businesses, institutions, and impacted community groups. Relating these back to the strategy map makes transparent the relationship between vision, objectives, and their key performance indicators (KPIs).

A stakeholder game board notes the different people and organizations involved with the NCC and describes their interests and power.

A dashboard contains key performance indicators, like the fuel, speed, and engine temperature indicators in your automobile.  Because NCCs are developed through continuously evolving processes over time, the dashboard reflects the progress of process steps and organizational behaviors.

What does success mean to you?

While almost everyone will agree that projects need criteria, or metrics, of success, reaching agreement on what those criteria is not easy. Within an NCC project, different parties—developers, city planners, office holders, tenants, and so forth– measure success in different ways, depending on their interests.  For example, a developer views success as leasing every square foot at a high annual rate; a tenant, on the other hand, say a university research lab with a small budget, would view that high rate as a project failure. Self-interested views on success are to be expected, given how people seek to maximize their own contributions and benefits. Bringing these different perspectives into alignment is bound to be difficult and can only be accomplished when each project participant is clear and unambiguous in articulating its interests and in identifying its success criteria.

How those different criteria can be defined and aligned will be among the topics discussed when we soon meet in Seoul.