From Project Prioritization Process: Developing Criteria for Assessing and Evaluating Project Requests by Joan Knutson and Alan Gump, p. 301 in PMI 96 Proceedings Key Steps Used to Develop a Process to solicit, assess, and prioritize "corporate development" projects 1. Identify and Empower the Right Working Committee 2. Brainstorm and Categorize Evaluation Criteria "In a series of off-site meetings, the team...brainstormed relevant criteria it felt should be included in evaluating corporate level projects. Then the team grouped [the] criteria into five major areas: Strategy, Cost/Benefit, Risk, Relationships with Stakeholders, and Impact on the Corporation. Grouping the criteria helped clarify the intent of certain criteria, eliminated duplicates, amalgamated some criteria, and identified missing criteria that needed to be added to the list." 3. Develop Scoring Models for Criteria "The team developed four models for scoring criteria. In some cases, the team developed guideline questions to help the project sponsor think through the issue, and to help the review committees score the project." Model 1 - simple one-dimensional scoring grid (1-negative, 3-neutral, 5 positive). E.g. Category: Relationship with Stakeholders Criterion: Effect the project will have on relations with suppliers Sample Questions: Who are the suppliers? How does the project help the suppliers serve the customers? Does the project create win-win opportunities? Does it promote the quality of partnerships? 5. Strongly enhances relations with suppliers 4. Enhances relations with suppliers 3. Has no effect on relations with suppliers 2. Adversely affects relations with suppliers 1. Results in a strong adverse effect on relations with suppliers Model 2 - 3-4 point scoring method, with 1 neutral, 3/4 positive Category: Strategy Criterion: Synergy the proposed project has with other projects Sample Questions: How does this project affect other existing or planned projects? Are there economies to be gained by partnering this project with another planned or existing project? 4. Provides significant positive effect on multiple projects 3. Provides significant positive effect on one project 2. Provides some positive effect on one or more projects 1. Provides no synergy with any other projects Model 3 - two-dimensional matrix. Particularly useful in Risk. Category: Risk Criterion: Predictability of Market Demand Sample Question: What is the demand for this project's product? What is your evidence for this demand? Extent of Identified Demand for Project Deliverable High Demand 2 4 5 Medium 1 3 4 Low 1 1 2 Evidence: anecdotal moderate conclusive Model 4 - financial model to determine Annual Rate of Return costs, revenues, savings, etc. turn into ARR value, with range of ARR values turned into 1-5 rating 4. Develop a Short List of Watershed Criteria "The team had a strong desire to develop a system to prevent unworthy projects from ever reaching the executive committee for action. They developed an Assessment phases, in this proposals would be submitted ... and evaluated according to the new criteria scoring system, and a Prioritization phase in which only those projects passing ... would be submitted for final approval and prioritization..." "In the process, the team discovered three 'watershed' criteria that they would evaluate first. If a proposal failed any of the three tests, they would reject it: Does the project fit the corporate mission? (Yes = pass) Does the project conflict with any of the listed corporate values? (No = pass) What is the relative strategic importance of the project? 4. Promotes 3 or more identified corporate strategies 3. Promotes 2 identified corporate strategies 2. Promotes 1 identified corporate strategy 1. Promotes no identified corporate strategy = FAIL 5. Develop a Weighting Scheme for Criteria "The team first provided a total relative weight to the categories, then apportioned weights to the individual criteria...The team developed a 1,000 point scale. The team then developed a simple spreadsheet with formulas to convert raw scores to weighted scores, and to provide a total evaluation score. 6. Submit Two Proposals for A Trial Run Through the New Procedure "The team determined to put together a 'good enough' system rather than a 'perfect' system because it strongly felt that the system would require refinement over time. To test the validity of the system, the team submitted two proposed projects for evaluation by the team. The purpose was to find the ambiguities, contradictions, and mistakes in the new system, and not to provide a final evaluation of the submitted proposals. The trial unearthed several glitches and provided the team an opportunity to refine the process before putting it on line." 7. Develop a Project Solicitation Guidelines Document for Project Sponsors "The final step in the process was to develop a comprehensive set of guidelines to teach project sponsors how to present proposals. The guidelines provide: a standard format for all future project proposals a schedule of submission and review dates a detailed explanation of the assessment and prioritization process a map of the process a list of the criteria, along with scoring methods and weights The goal of the Solicitation Guidelines is to ensure that project sponsors know how to write a valid proposal and how their proposal will be evaluated." Six Month Evaluation 1. The Working Committee(s) They applied the assessment process and discovered several criteria that needed modification and clarification. "During evaluation, concern arose over the implications of releasing a prioritized list of projects. Specifically, would projects and other important work 'below the line' be abandoned? How would departments allocate resources if two projects emerged at the same level of priority? How could they determine whether sufficient resources would be available to run simultaneous projects?...The answer lay in development of a Master Resource Plan." 2. The Master Resource Plan "...The missing piece was the Master Plan that coordinates all project work, together with resource allocations, within the priority structure determined using the assessment process...The Project Office is able to retain the prioritization order at the _project_ level while adjusting priorities _at the task level across projects_ to facilitate scheduling... "The need to allocate resources to make the system work forced the development of effort and duration estimates _before_ the projects were actually approved. In other words, more up front planning was required before the project candidates were evaluated, which resulted in more accurate project budgets, more detailed deliverable descriptions, and a clearer understanding of the project 'drivers' - market issues (quality) or external deadlines (schedule)." "One huge win was the fact that the Master Plan demonstrated the value of measuring, planning, and developing an _integrated_ view of the projects." "Perhaps the most significant breakthrough associated with the Master Plan was that the Project Office determined that resource would be available 50% for corporate development projects and 50% for 'below the line' departmental projects and other work. This decision has given departments a clear expectation and understanding of their resource availability, and is forcing project managers not only to plan more carefully, but to track actuals. 4. Criteria Evaluation Of the 27 criteria used in the assessment process, only the Annual Rate of Return requires substantial modification. Basically, while the model worked reasonably well for Product-focused projects, it did not handle Process-focused projects well. There was a desire for more precision in the qualifying terms. However, the group "agreed that for the generic models, the language needs to remain imprecise, but that each project must present specific arguments that will help develop ad hoc understandings of the gradations." Some criteria were reworked to include "does not apply" or other limitations. Conclusion: "Implementing the assessment and prioritization system has produced some growing pains for the organization, but there is more confidence that the organization is now selecting the _right_ corporate development projects to implement. The system has forced improvements in the corporation's ability to allocate and manage resources, to resolve resource conflicts with the corporation's business needs in mind, and to monitor resource accountability. Project requesters now use a standardized proposal format which facilitates writing proposals as well as evaluating them. The organization is more confident that it is now hunting the fox instead of just running through the creeks."