DRAFT 

CAPACITY ASSESSMENT GUIDELINES

&

THE PROGRAMME APPROACH:

ASSESSMENT LEVELS AND METHODS 

Prepared for: United Nations Development Programme

Management Development and Governance Division

 

Dr. Thomas J. Hopkins

Consultant

 

13 January 1996

 

Table of Contents

 

I. Why Do We Need and How To Use The Guidelines 1

II. The Conceptual Framework 6

III. How To Operationalize The Framework 14

IV. Capacity Monitoring and Evaluation 30

Appendix

1. Organizational Change 1

2. Strrategic Level Questionnaire 17

3. Operational Level Questionnaire 29

 

CAPACITY ASSESSMENT GUIDELINES

AND THE PROGRAMME APPROACH: ASSESSMENT LEVELS AND METHODS

(Draft 13/1/96)

 

I. WHY DO WE NEED AND HOW TO USE THE GUIDELINES

 

Target Guidelines Users

 

This guide to capacity assessment within the programme approach is intended to be relevant and useful to:

- those who are engaged in programme/project design, planning and implementation at the country office and headquarters level;

 

- thoseofficials who are engaged in programme/project design, planning and implementation within relevant national ministries and departments; and

 

- those persons within the NGO and Private Sector who are engaged in national programme/project design, planning and implementation.

Organization of the Guideline

This Guide is organized into four successive sections:

Section I: Why do we need and How to Use the Guidelines

 

Section II: The Conceptual Framework

 

Section III: How to Operationalize the Framework

Section IV: Capacity Monitoring and Evaluation, which demonstrates that capacity assessment should be an on-going activity that generates a practical data base for future reference.

 

Appendix

Rationale

 

The basic concept behind this guide is that programme/project planning requires fairly clear knowledge regarding the state of the target action environment (i.e. economic, political and social factors and the current functioning of the institutional and organizational context), including pro-jections regarding their change ability).

Given the above challenge an important question requiring a focused answer is; Capacity Assessment for What and How? This question has seldom been addressed, or even asked. Most discussions and papers tend to focus on "the need for capacity building", "the need for sustainability", "the desire for self-reliance", and "the need for new approaches and models". These are issues that find a broad agreement. However, after thirty years of relative discouraging results there is a need to revisit and rethink technical cooperation and capacity building in general, methodological (how to) concerns in particular. Methodology is defined as -- a body of practices, procedures, and rules used by those who work in a discipline or engage in an inquiry; a set of working methods. It is the methodology aspects that adds substance to policy initiatives.

Therefore, capacity assessment guidelines should outline fundamental tasks such as: entry points strategies, data collection methods and capacity monitoring. The need for reliable contextual and institutional profiles based on fairly rigorous data has never been more self-evident. The most simple planning approach "...one must know what is before, he/she can determine what should be" is still appropriate. Capacity Assessment for what? For the design of better capacity development strategies and programmes. How? Through the use of fairly rigorous assessment strategies, data collection and dissemination methods.

How the Guidelines Should be Used

This Guide can be approached in a linear fashion, from cover to cover, or by locating the stage of your country programming efforts and focusing on relevant assessment strategies and data needs i.e. if your country is at the Administrative/Operational level this should be your take-off point, likewise, if your programming effort is more on the Strategic Level your take off point should be there. However, should your programming require a comprehensive effort and time and resources are available the total guidelines will be relevant.

The location of country programming efforts may or may not be a simple task. Every country programme is somewhere within a programme cycle. Through observation, review of documents and discussions with informed persons it is possible to plot your country on the programming cycle. Next simply ask yourself and informed others the question; "Where do we go from here, Strategic, Administrative/Operational or comprehensive?" The answer should help you focus on your country programming assessment needs and the guidelines should be used accordingly to design a capacity assessment strategy and plan.

The capacity assessment guidelines are by nature orderly, analytical processes, the logic of which sometimes runs counter to the immediacy of day-to-day organizational life. Merely attempting to do fairly rigorous assessments may require an organization to change some of its most time-honored traditions and practices. But it is becoming increasingly clear that organizations and individuals must change merely to cope with the many demands made on them. Capacity assessment approaches such as the one described here will do more than merely help organizations produce better programmes/projects. In the long run, organizations which incorporate a rational capacity assessment routine into their institutional lives will do a better job of accomplishing their organizational purpose.

This guide in not meant to be an exhaustive or in-depth treatment of the capacity assessment process. Rather, it has been developed as a basic primer alerting the user to potential data sources and strategic possibilities. Each country situation is complex and demands knowledge, sensitivity and adaptability, therefore, the user(s) must place the guidance and information offered in context and "cut and past" at will.

Among the components of the capacity development process, the capacity assessment is the least well developed or understood. Most technical assistance programmes and projects focus on meeting capacity needs, without undertaking the analysis that would normally be required to ensure that the solution is the most appropriate one in the circumstances. At the same time, the UNDP has come to recognize the need for sharpening the concept of and improving techniques for capacity building. Several initiatives have been carried out during the past few years.

In order to pursue, target and prioritize sustainable human development UNDP has established the following focus areas:

1. Poverty elimination;

2. Creation of employment and sustainable livelihoods;

3. Advancement of women and other disadvantaged groups;

4. Protection and regeneration of the environment; and

5. Sound Governance and management development.

 

Logically, as a next step, capacity assessment methodologies should be integrated into the same framework, priorities and context. This guide attempts to integrate capacity assessment methodologies into programming cycle and pay special attention to UNDP focus areas. The rational is that different programming levels require that different areas, functions and tasks be assessed from different perspectives. This, in turn, has implications for the monitoring and evaluation system.

 

Some Definitions

 

II. THE CONCEPTUAL FRAMEWORK

 

Programming Levels for Capacity Assessment

 

UNDP's programming strategy is a logical and natural framework for the design of a capacity assessment strategies. The programme approach, involving a country strategy, national programmes, programme support documents, and specific programme implementation arrangements provides the practical framework within which capacity assessment methodologies can be applied and practical results achieved.

 

However, the Programme Approach assumes that each country would have a capacity development policy framework and strategy. That the strategy would identify priority areas for capacity development in all sectors and at all levels, in government as well as in civil society and the private sector, relating directly to the implementation of the national development programmes. It also assumes the ability to identify areas in which capacity is unlikely to be sustainable and in which the country might continue to buy capacity from abroad, rather than making major capacity development investments that might be unsustainable, and so on. These are fundamental assumptions that in many, if not most situations, does not prove to be true. However, an analysis of the "current status" of national goals, priories and strategies is a necessary capacity assessment beginning.

 

Table 1 presents a modified UNDP programming cycle (excludes External Resources Mobilization and UN Country Strategy levels) and five (5) analytical dimensions. These assessment levels and information dimensions will provide important base line data. In essence, it is argued that once the Programme Approach (including capacity assessments) is fully implemented, capacity development would be an integral and central part of the UNDP's role in support of the country's national programme(s).

 

Capacity development should always starts with an assessment of the current situation. The nature of the assessment required will vary from situation to situation. In many cases the situation will be relatively well known in general, if not in specifics. In some cases an informal, rapid assessment might meet the requirements of a relatively minor investment in capacity development. For any significant investment in capacity development, however, a formal assessment that meet certain criteria would be indicated. The purposes of a capacity assessment are:

 

TABLE 1: Levels for Capacity Assessment and Development in the Programming Approach

 

National Goal(s), the Programme Approach and Beneficiaries

 

Figure 1 is a systematic view of national goal(s), the programme approach, and the importance of the beneficiaries. Briefly, national goals are goals that are inclusive not simply those of the government. Other sectors, civil society and the private, are included. The beneficiaries are viewed as the most important stakeholder in the system and the government, civil society and the private sector are providing necessary and desired services. Therefore, beneficiaries involvement is advocated and should be maximized.

 

All elements of the system function within an enabling environment comprising of the following:

 

The role of the various institutions and organizations, within the system, is to provide efficient and effective service to the beneficiaries and, therefore, achieve the desired national goal. Figure 1 identifies the various elements that may require a capacity assessment i.e. the statue of the enabling environment and the role of national goal(s), priorities and strategies and the operational and administrative capacity of the various institutions and organizations.

 

The Harvard Framework

 

Harvard University, under contract with UNDP, designed an analytical framework composed of five dimensions, and correspondingly, five levels of analysis, that affect capacity and capacity building interventions. It is argued that organizations do not exist in a vacuum, they are embedded in complex environments that affect their ability to carry out tasks effectively and efficiently. These Guidelines follow The Harvard Framework with approp-riate modifications.

 

FIGURE 1: An Institutional Systematic View of National Goal and the Programme Approach

 

The framework is designed to be a tool for decision-makers and managers. It can be used to assess constraints, capacity gaps, and opportunities and also as the basis for developing intervention strategies to build more effective capacity. The assessment can be focused on any level --contextual, institutional, inter-organizational, organizational, or individual. The Five Dimensions are described below and schematically presented in Figure 2:

 

1. The Action Environment -- The economic, social, and political milieu in which organizations attempt to carry out their activities and the extent to which conditions in the action environment facilitate or constrain performance. Within this dimension, a broad set of factors must be considered regarding their impact on the ability to carry out particular tasks at the organizational level such as the following:

 

2. The Public Sector Institutional Context -- A second dimension of capacity is the institutional environment within the public sector that facilitates or constrains organizational activities and affects their performance and the performance of other sectors (private, NGO and CO). This dimension of capacity includes the following:

3. The Task Network--The coordinated activities of several organizations that are required to accomplish tasks. Organization are not created and endowed equally. For example they have different bureaucratic support, influence, forecasting ability, input control, access to technologies, etc.. The interactions of organizations within this network can facilitate or constrain performance. This factor is not restricted to the Public Sector, an assessment of the Private Sector, NGO's and other Civil Organizations should be included.

 

How these networks function and the nature of formal and informal interactions among them are important aspects of performance for particular

tasks. Organizations within a task network can be public or private and can represent diverse levels of government, and belong to several task networks.

 

NOTE -- The remaining dimensions relate to both the Strategic and Operational/Administrative levels. Strategic in terms of policy initiatives and guidelines; Operational/Administrative in terms of policy interpretation and implementation.

 

4. Organizations -- The fourth and fifth dimensions of capacity are the organization and the human resource based the organization has to work with are closely intertwined. The fourth, organizations, focuses on organizational structures, processes, resources, and management styles that affect how individual talents and skills are used to accomplish particular tasks. This is an important dimension because organizations:

 

5. Human Resources -- This dimension relates to the training, recruitment, utilization, and retention of managerial, professional, and technical talent that contribute to task performance at the organizational level.

 

III. HOW TO OPERATIONALIZE THE FRAMEWORK

 

Section Two provided a conceptual framework for the integration of capacity assessment methodologies into the programming cycle, the importance of an systematic approach to identify national goals, priorities and strategies and the "capacity assessment framework" that provides realistic "baseline" data of the current state of affairs. This section will present and discuss ways of "How To" operationalize the framework.

 

Programme Cycle Levels

 

The programme cycle levels are Country Strategy, National Programmes, External Resource Mobilization Framework, UN Country Strategy Note or Programme Framework, Programme Support Document (PSD), Programme Support Implementation Arrangements, and National Execution (NEX). These levels are further delineated into two assessment categories (i) Country-Oriented Assessments and (ii) Donor-Oriented Assessments. Country-Oriented Assessments are -- Country Strategy, National Programme, Programme Support, Management and Implementation (NEX); and the two Donor-Oriented assessments are -- External Resource Mobilization and UN Country Strategy.

 

The country-oriented level is conducive to rational assessment methodologies, from "quick assessment" to fairly rigorous comprehensive assessments. The Donor-Oriented level is more difficult due to the elusiveness of conceptual clarity, donor motivation, donor leadership and personalities, resource leverage, and so forth. An assessment, at this level, would require political science methodologies premised on power/influence theory. While interesting it is felt that the utility of a fairly rigorous capacity assessment, at this level, is not practical. This is not to negate the importance of the donor-oriented level but to argue that the assessment methods, skills and resources would be difficult to acquire, justify and sustain.

 

The country-oriented level is further delineated into two broad categories; (i) Strategic, and (ii) Administrative/Operational. The scope of capacity assessment can vary from narrowly focused too comprehensive. The comprehensives of an assessment relates to many factors such as: assessment resources (time, funds, staff,) assessment rigor (primary and secondary data sources, data collection, data reliability and validity).

 

It is noteworthy that the more comprehensive, the more resources needed and the less rigorous; the more focused the fewer resources needed and the more rigorous. However, the more comprehensive the more likelihood causative factors will have been assessed. The more focused the more chance causative factors were not assessed. Likewise, the feasibility and relevancy of the "comprehensive" assessment is questionable. In determining a capacity assessment strategy one should be cognizant of the assessment focus and the potential "gaps" to make informed "trade-offs".

 

Sustainable Capacity and The "Ownership" Process

 

Ownership, Partnership, Programme Approach, and National Execution are interrelated concepts which promote sustainability. Each concept articulates a process not a specific product.

 

The best way to achieve sustainable systemic change is to involve individuals and/or organizations in the redesign of the system and their roles in it. An immediate benefit for this organizational learning process is the infusion of local understanding, experience and sensitivity. The ownership process means the involvement of those who understand the context and must live with the change, therefore, they should be responsible for its design, implementation and evaluation.

The UNDP document "Systemic Improvement of Public Sector Management: Process Consultation" presents an approach for achieving "ownership". In brief Process Consultation requires top management support for the suspension of bureaucratic command compliance behavior in favor of organizational learning. It is argued that:

 

The results of the Ownership Process are that Key Actors:

 

The ownership concept is at the heart of "process consultation". The central task of the process consultant is to secure the participant's ownership of the responsibility for doing what needs to be done.

 

Obviously, of equal importance, is "ownership" of the capacity assessment process since an assessment, becomes the baseline for a strategy, which becomes a target, project and/or programme. Therefore, the design, implementation and utilization of a capacity assessment, to be sustainable, must include and advocate an ownership process.

 

Country-Oriented Programming Level & Process Consultation

 

The country-oriented programming functions is the focus of these capacity assessment guidelines, first the Strategic and later the Operational level. Likewise, "Ownership" by the nation and/or those accountable to the nation is the process vehicle through which the capacity assessment is conducted. The following presentation format will be used: purpose, scope, applying the framework and data collection. The objective is to be informative not proscriptive, therefore, the capacity assessment methodology be viewed as an informed guide.

 STRATEGIC LEVEL ASSESSMENT -- Country Strategy and National Programme Levels

 

Purpose

 

The primary mandate of UNDP is to support the building of national capacity for self-reliant development. Specifically, this means to promote self-reliance in developing countries in terms of the managerial, technical,

administrative, and research capabilities required to formulate, implement and manage development plans and policies.

 

There is the need for an analytical framework to assess the status of national capacity which is clear and operational. An analytical framework for measuring institutional capacity to perform valued functions and to determine an institution's own policies, need, and relationships. The framework should describes how to undertake diagnostic work to acquire data and insight.

A diagnostic Strategic Level Capacity Assessment is proposed as the point of departure. The goal is to provide the basis for designing programmes/projects intended to achieve the required and feasible adjustments.

 

Scope

 

The Strategic Level Assessment focus is on decision-making and institutional relationships in the government and its effects on the bureaucracy and other sectors (private, NGO and COS's).

 

This approach, while institutionally based, focuses it's analytical attention on both external and internal factors. In essence it is concerned about public sector institutions, their systems resources, internal operations, external relations and impact, including impact on the private sector and NGO's. While the framework was designed to be generic; that is, to be applicable to institutions across the spectrums of size, locale, level, and function, institutional analysis always must be bound to context.

 

Applying the Framework

 

Using the Harvard Framework described in Section II as the guide the following are proposed ways and means to conduct the assessment.

The Harvard Framework is inadequate unless careful consideration is given to its application. There are various application modalities from the single consultant one shot approach, to broad base participation through focused workshops. Since "National Ownership" is an important goal, we recommend the Capacity Assessment Framework be applied through the Process Consultation (see page 15) and the strategic planning/management approach described below.

 

The strategic planning and management approach is critical for an effective strategic level capacity assessment. This approach assists developing country managers with the implementation of policies by looking at the changes in the internal and external environment that they must confront. The strategic approach consists of four main elements:

 

 

The Strategic Level Capacity Assessment should attempt to acquire commitment at the highest relevant level to ensure ownership by all concerned. Accordingly, a participatory process involving both senior and mid-level government officials and important private sector and NGO leaders for programme preparation creates opportunities for full understanding and shaping of the objectives and activities. The logical starting point for a participatory process bound for ownership is the engagement of mid-level actors in diagnostic or capacity assessment activities. This allows mid-level and rank and file actors to be mobilized for the collection and analysis of information that pertains to the environment to be targeted by a programme/project. In summary, clear "ownership" activities are critical at this stage -- in essence the stakeholders must be involved. Some fundamental steps follows (the questions presented are not ment to be exhaustive:

 

NOTE -- Step eight and nine relate more to Operational Assessment activities, particularly the PSD, PSIA and NEX. These steps are included here to demonstrate continuity.

 

Data Collection **NOTE -- Think about giving two data collection options (i) rapid and, (ii) comprehensive

The information gathering methods, strategies include the following:

 

The following are some specialized data collections techniques. See Appendix 1 "Organizational Change" for a more detailed review.

 

ADMINISTRATIVE/OPERATIONAL ASSESSMENT -- Programme Support (PSD/PSIA) & the Management--Implementation (National Execution) Level

 

Purpose

 

The purpose of the Operational/Administrative Capacity Assessment is to provide UNDP staff, government officials and relevant others (private sector & NGO's) with a transparent organizational development and management capacity assessment that identifies strengths and shortcomings (an organizational profile) and provides baseline data for a management capacity development strategy.

 

UNDP has taken the initiative to correct some of the faulty assumptions of earlier eras. Specifically, National Execution and the Programme Approach alerted the development partners to the importance of sustainable development planning, management and implementation issues.

 

Implementing the programme approach ultimately requires a clear and logical Programme Support Document (PSD) that targets UNDP's contribution to a National Programme. The main emphasis is on the development objective, how to attain it in a measurable way, and how to sustain it's impact. The PSD requires the identification of Capacity Building Targets in measurable terms and requires the preparation of annual Programme Support Implementation Arrangements (PSIA) documents which specify the details of implementation. Both the PSD and PSIA place considerable emphasis on measurability. This entails baseline surveys, benchmarks, progress and success indicators, without which it is not possible to judge either effectiveness or impact. The importance of establishing and tracking such measures cannot be overemphasized and is at the heart of any monitoring and evaluation system.

 

Programme implementation management is a key factor. Therefore, the assessment of management capacity is correspondingly critical at the PSD stage. Two fundamental issues must be addressed at the PSD/PSIA level: (i) capacity building targets, and (ii) management and implementation capacity. Capacity assessment tools (methodologies) are, obviously, necessary.

 

The fourth and fifth dimension of the Harvard Assessment Framework, Organizations and Human Resources (refer to Section II), are important in conducting an Operational/Administrative capacity assessment. The primary focus of a this assessment is the UNDP's programme support documentation (PSD/PSIA) and the management arrangements preferably under National Execution (NEX). These documents require specific programming details that can be monitored and evaluated. Obviously, there is an overlap between the Strategic and Operational assessments, however the Operational level will focus more on resources (human, physical and financial) in assessing the organizational development and management activities and tasks needed for successful programme implementation and monitoring.

 

Scope

 

By nature an Operational/Administrative Assessment should be feasible and highly focused. It is focused on the UNDP's policy and documentation needs. The output is a UNDP project/programme document.

 

Applying the Framework

 

Management capacity building is a fundamental strategy for sustain-able development. Without an acceptable managerial capacity base or a willingness to improve there cannot be efficient and effective execution or implementation. Therefore, an assessment of the current organizational development and management situation must be made. The assessment should be followed by : negotiations for and the design of a management capacity development strategy and plan for inclusion in the PSD/PSIA.

 

Once potential organizations, for execution and implementation, have been identified an assessment of their technical and administrative capabilities should take place considering the demands of the proposed programmes/projects and the requirements of UNDP. To make this assessment, formulators will have to visit the organizations that have been identified and acquire relevant organizational information related to the tasks and activities of executing and implementing a UNDP programme/project.

The major assessment categories for a Operational Capability Assess-ment follow:

These factors are the ones amenable to change through an institution building project. The efforts required to change these factors translate into the project's outputs, activities and inputs. It is therefore important to correctly establish the current state of these variables before designing the project's components. The components relate to the key management systems. This narrowly focused capacity assessment can easily identify strengths and shortcomings.

 

The Capacity Assessment should be divided into three phases:

 

The assessment will provide information about an organization's ability to carry out

common implementation activities such as:

 

These activities as well as the organizational structure and physical capacities should be systematically assessed in order to present a Capacity Profile of an organization.

 

Field and professional experience suggest that the organizations will fall into one of the following four categories:

 

Data Collection

 

A questionnaire, based on the assessment categories, is used (with contextual modifications) to collect information. This instrument may be used for public, private and NGO organizations. Interviewing of informants, observation, and review of documents are the major data gathering methods.

 

This questionnaire is used to collect information from organizations being considered to execute and implement UNDP assisted programmes. The data should be analyzed and used as the basis for an assessment of the organization's capability and appropriateness; and help identify training and other capacity building needs of the organization. The draft assessment report should elicit debate, discussions, further analysis, and eventually a Capacity Development Strategy and Plan to be included in the PSD/PSIA.

 

Besides establishing a basis for the recommendations on execution and implementation arrangements, the assessment serves a valuable "ownership" function. Participants appreciate the clarification of roles, responsibilities and procedures that occurred during the assessment. The assessment clarifies the advantages of National Execution (greater control over a project) and the disadvantage of additional administrative burdens.

 

The Assessment Profile: Data Analysis & Presentation

 

Figure 3 is a matrix that may be useful for presenting the results of the capacity assessment (Strategic and Operational/Administrative). This analysis should serve as a planning and discussion tool.

 

ADD MORE TEXT HERE

 

 

 

FIGURE 3: ANALYSIS OF INSTITUTIONAL PROFILES

IV. CAPACITY ASSESSMENT, MONITORING & EVALUATION

 

Introduction

 

Monitoring and evaluating capacity development programmes is a complex task due to the possibility of extraneous and intervening variables i.e. recessions, war or natural disasters. Such events are beyond the control of governments and aid agencies and are not readily anticipated in the design of capacity development strategies. M&E is further complicated by its various partners and funding. This makes it even more critical that UNDP have a transparent accountability system.

 

The M&E function is so critical that an M&E strategy should be integrated into the capacity assessment phase of programme design.

 

Monitoring, Evaluation and the Programme Approach Entry Levels

 

Similar to capacity assessment methodologies, the M&E system should relate to the capacity assessment entry levels of the programme approach. As stated, relevant entry levels are: the country strategy, national programming, the programme support, and the management and implementation levels. The Strategic Level should involve an assessment of the in-country stake-holders and their capacity to design and implement a sustainable M&E strategy. The downstream level would be concerned with the operational components and the clarification of capacity development targets.

Strategic Monitoring and Evaluation Assessment -- At this level the primary concerns are (i) who will do the monitoring and evaluating and, (ii) what is their capacity to implement and sustain this function. The Capacity Development Guidelines stated that:

 

Regarding "ownership" of the M&E system the Guidelines observed that:

 

Operational/Administrative Monitoring and Evaluation Assessment -- this level would assess (i) the operational components (management) of the M&E system and, (ii) the evaluability of the capacity development targets articulated in the PSD and PSIA. Clarification should be made of "process" vs. "product" activities and results. The Guidelines argued that:

 

Monitoring and Evaluating a "process" is different from a "product" this must be clear to the major actors and appropriate indicators and benchmarks agreed upon.

Elements to be considered in the assessment of the Programme Support Document and the Programme Support Implementation Arrangements:

 

Later the Guidelines continues that:

 

In summary, the M&E strategy and system should be assessed relative to the country-oriented assessment entry levels. Strategic factors and activities are more broadly focused (i.e. identification and involvement of critical stakeholders, and articulation of a M&E strategy). Operational func-tions and activities are more operationally focused (i.e. management of the system and the evaluability of the PSD/PSIA capacity development targets).

 

Monitoring and Evaluating What?

What does the organization produce? Implementation of the pro-gramme means the production of something. This something can be called a "deliverable", typical deliverables are:

 

The PSD and, more specifically, the PSIA should clearly specify the deliverables (Capacity Development Target) in qualitative and/or quant-itative terms. In turn, the M&E system should provide the information and the means to address issues relating to:

 

Assessment of Measuring Deliverables

 

Implementation assumes continual change therefore mechanisms must be developed for monitoring and analyzing the performance of the organiz-ation with respect to achieving desired goals and objectives. As the environ-ment changes, ministers change, elections occur, budgets go up or down, priorities also change. These elements can alter performance, priorities, and the desirability of certain policies. To maintain a good fit with the environ-ment an organization must be able to track these changes in order to adjust.

 

The monitoring process should be continuous, regular, and capable of feeding into decision-making. Managers should develop control mechanisms to gauge the efficiency of resources used and impact mechanisms to gauge the effectiveness of its actions. "What" and "how well" the implementation process is implementing should be known. Deliverables should have clearly specified targets in terms of quantity, frequency or quality, so that the impact can be measured. Therefore, thought should be given to the means by which these targets will be measured. Some capacity development targets are more easy to measure than others. Changes in quantity or in frequency are straightforward to measure, and can be represented either as net changes or as percentage changes. Changes in quality is quite difficult.

 

Guiding Principles for Developing a Monitoring and Evaluation System in the Context of the Programme Approach

 

The UNDP/OESP publication "UNDP Evaluation Findings in 1994" includes a section on "Emerging Directions for M&E Application". In the context of the national programme and the programme approach, the guiding principles articulate the following five major areas for M&E:

 

2. Management of M&E as a National Responsibility;

3. Incorporating M&E into Programme Formulation;

4. Establishing National M&E Capacity; and

5. Focusing M&E to UN System Contributions.

 

The above categories should be considered as the broad framework for an assessment of the national monitoring and evaluation strategy and/or system. There are a variety of subsets of information and data available for each category. The integration of the above into the programme approach entry points could systematically meet the requirements of the national programme ownership process and donor accountability needs.

 

APPENDIX 1: ORGANIZATIONAL CHANGE

 

(Excerpts taken from "Hip Pocket Guide to Planning & Evaluation" D. Craig, 1978; and "Management and the Environment for Implementation of Policy Change" MSI for USAID, 1992)

 

Initiating Organizational Change

 

Planning on paper is one thing. It's quite another to actually implement a plan within an organization. Introducing any new plan is likely to mean that something will have to change -- from the routines of an individual to the functioning of the entire organization -- and change, for most is not always easy to accept. As a planner trying to behave rationally the experience of implementing change is trying at best. Therefore, it is better to do rational implementation planning than to assume the "logic and rationality" of the plan is self-evident. This section concludes some ideas to help one think about the overall problem of introducing a carefully designed plan into the organization. Some steps one might use follow:

 

Analyze the Organization's Capacity for Change and the Change Agent(s) Attitude Toward Change

 

The change agents should assess their beliefs and typical responses to change. Following are two useful checklists;

 

Beliefs About Change

 

Do you believe the following statements are true, sometimes true, or false?

 

1. People tend to resist change.

2. Only large or momentous changes are worthwhile.

3. Nothing can be changed overnight.

4. Change means improvement.

5. Change brings hardships for some.

6. Change brings reward for the instigators.

7. No change is possible in a bureaucracy.

8. Technological change should be slowed.

9. Change usually comes by chance.

10. People can adapt to any change.

This exercise should get one thinking about change attitude and the change agents role. If you answered "true" to questions 2, 4, 6, and 10, it may indicate a need to be a little more realistic. If you answered "true" to question 7 and 8, this may indicate a pessimistic attitude that may make it hard for you to work energetically for change. If you answered "true" to questions 1,3, and 5, it indicates a recognition of the real problems involved in change.

 

Your Change Quotient

 

Yes No

excitement and challenge of finding a solution to problems in many areas, regardless of whether they are major or minor challenges?

Your quotient is high if you answered "Yes" to at least eight of the eleven questions. Innovators of change need great tenacity of purpose and stubborn resistance to discouragement. They need initiative, curiosity, and the ability to simplify the many reactions and events that occur during a change process.

 

Some organizations are more open and receptive to change than others. The organizational diagnosis in Figure 1 may help assess an organization's general operating style and what potential blocks to change one might run into. For each question circle the answer that best fits the organization. You may also want to get your supervisor, your co-workers, and your subordinates to try this

 

FIGURE 1: Characteristics of Management Systems

exercise so that a comparison of perceptions of the organization is possible. A good approach in one organization may be totally inappropriate in another. Here are example change strategies appropriate to each of the four types of organizations shown:

 

One way to visualize the process of change in an organization is to imagine an ice cube melting, turning into water, and then being frozen again into ice. In organizational terms, the process can be seen as three stages: "unfreezing" the existing status quo, "changing" through set of actions, and

"refreezing" into a new stable pattern. Figure 2 illustrates the change process.

 

There are a number of conditions that tend to indicate whether an organization will accept or reject specific changes.

 

Conditions for Accepting and Supporting Change

 

FIGURE 2: The Change Process

Conditions for Rejecting and Resisting Change

 

To review the preceding steps to Change an Organization:

 

Planned Change Some Assessment Tools

 

Personal attitudes toward change may produce a number of collective reactions to a given innovation. Group reactions may range along the following scale:

 

0 Sabotage 4 Indifference 8 Commitment

1 Slowdowns 5 Acceptance

2 Protests 6 Support

3 Apathy 7 Cooperation

Having gained a perspective of the change process, a discussion of three analytical tools that may help design a planned change strategy is presented. The following (i) Force-Field Analysis and, (ii) Political Mapping and (iii) Stakeholder Analysis will be reviewed. Knowing how an organization is likely to react to planned change is obviously important strategic information that may determine success or failure. The following analytical tools are designed to gain insight into this concern.

 

Force-Field Analysis

 

In some organizations, certain human forces resist the necessary changes by which the unit must adapt to its changing environment. An understanding of these forces can help to top management and/or change agents facilitate the changes they want to bring about.

 

One of the strongest blocks, unfortunately, can be the attitudes of one or more members of the top-management team toward the prospective change and toward the perceived status quo. The chief executive, for example, may have such a strong emotional stake in the organization's traditional product or service that he refuses to believe that the world no longer needs or wants it as strongly as before. He may not actively resist a movement toward expansion into other areas, but his lack of active support can often spell defeat for the prospective change.

 

Factionalism within the top management team can also dissipate their resources and weaken the effects of change processes. For example, a very financial comptroller can put the brakes on an aggressive decentralization strategy in many subtle ways.

 

Many healthy organizations can languish for months or even years while the grim drama of a palace war runs its course. The top management group may split into two warring factions and clients, waiting until they have sufficient strength to force the others out of the ring. A major government agency may stagnate in the face of a rapidly changing environment, while the chief executive and one or two "hatchet men" force a second-level executive out of the organization, usually by a long process of paper work, harassment, and increasing discomfort. Another significant source of emotional blockage is the general population of the organization -- the work force.

 

Kurt Lewin one of the first researchers to make a systematic study of human behavior in organizations, referred to the human arena as a "force field," wherein the interplay of various pressures either brought about change or kept the organization in the status quo. His notion of "force-field analysis" called for diagramming the major forces operating in an organization, as illustrated by Figure 3. In studying the various ways of bring about change. Lewin identified "helping forces," which act to facilitate the change in question, and "resisting forces," which act to impede the change. This kind of analysis clarifies the major change processes within an organization and suggests approaches managers can use to accelerate a desired change.

 

This practical view of the human nature of organizations suggests that top managers must do several things to bring about an adaptive change to meet a changing environment. First, they must have a very clear and specific idea of the change they want to bring about. They must clarify the benefits of the proposed change, and they must be prepared to minimize the undesirable impacts the change may have.

 

FIGURE 3: Force-Field Analysis

Second, they must combine their ideas and energies and concentrate their organizational resources to make the change come about. They must minimize factionalism and counterproductive personal clashes and develop a strong sense of group commitment to a clearly stated goal.

 

Third, they must gain the general commitment of the work force to the prospective change by whatever means they can. They must be prepared to compromise, to move at a controlled pace, to re-evaluate the objective from time to time, and to develop employee attitudes which will help the change come about.

 

And fourth, they must keep their collective attention focused on the organizational processes involved in the change and keep doing the necessary things until the change has become acceptable and permanent.

 

The forces over which the group is perceived as having some control are then ranked in terms of potency and acceptability. Strategies for overcoming the problem are devised and implementation procedures are tentatively designed. A similar analysis is then directed to those forces under someone else's control. An advantage is that it begins the action process at the very same time as the planner is engaged in assessing the problem and possible strategies to overcome it.

 

Having identified the helping and hindering forces, the next step is to write a short summary of how people will react to the plan. Indicate what problems are anticipated in getting the plan accepted and where these are most likely to come from such as:

 

1. Co-workers

2. Other staff

3. Managers

4. Policy makers

 

Political Mapping

 

Generally, managers and professionals in the public sector are poorly equipped to deal with either political analysis or the formulation of political strategies. When injected into the budget process for the first time many discover that their sector's needs are not automatically met. The rapidly declining levels of budget authority for the more vulnerable sectors such as education and health, attest to their inability to defend themselves against more able, though perhaps less need, competitors for budget resources.

 

Political Mapping is an analytical technique aimed at developing management skills in designing improved strategies for achieving goals and objectives. These techniques help in assessing the level of competition faced by the public manager, the channels of access to critical decisions, and the possibilities for coalition arrangements to help achieve objectives. The technique can be used on the macro or micro level and if used with force field analysis can provide very a powerful analysis.

Some premises about politics and politicians:

 

- No government can stand entirely on its own.

 

- To keep office a government must have the support of key actors.

 

- Without support, governments do not have authority.

 

- Without authority, governments cannot implement decisions.

 

- Support cannot be obtained with cost.

 

- The offer of support may be used to obtain benefits or increase influence in the government.

 

Political mapping need not be confined solely to the macro or national level as discussed above. Two other useful techniques are micro-mapping and network mapping. Micro mapping diagrams the relationships between actors at a micro political level, and is especially useful to illustrate relationships among actors in a particular sector. (e.g., health, education, agriculture).

 

Although a macro political map shows overall support for the government, it does not necessarily reveal support on specific issues. It is possible that though a government has solid overall support, on specific issues there may be massive or particularly intense opposition. A micro political map can clarify the distribution of support for specific issues, indicate how certain sectors will react to particular policies and clarify the positions of different organizations within the same sector.

 

In effect, politics may be viewed as a transaction in which support is traded for benefits or influence. But the important message here is that support is vital (decisions cannot be implemented without it), there is always a cost to obtain it, and there is generally competition for that support. Looking at politics this way helps one understand which actors are important and provides insight into the factors that affect the capacity of a government to implement decisions.

 

Two elements that complicate political analysis are the large number of actors present in any given political system and the vast quantity of information about politics available. The purpose of the political map is to organize and reduce the amount of information available regarding politics to a manageable quantity in order to focus on those aspects of the terrain most important to the decisions managers must make. The map organizes and identifies the most important political actors and spatially illustrates their relationships to one another.

 

The political map, (Figure 4) like the geographical map, has two dimensions: a horizontal (latitudinal) dimension and a vertical (longitudinal)

 

FIGURE 4: An Illustrative Political Map

dimension. At the center of the map is the government. The primary reason for locating the government at the center is simple because the government is the primary focus of decision making regarding how the benefits of society will be distributed. Political activity is centered on and directed toward influencing the government and its policy decisions.

 

Along the vertical axis, the different types of political actors are organize into four sectors: external actors, social groups, political parties, and pressure groups. The purpose of the horizontal axis is to assess the degree to which each group supports the government. Support for the government varies from core or central support to ideological or mild support while opposition is differentiated as either legal or anti-system opposition.

 

A criticism sometimes made regarding political mapping is its lack of dynamism. Unlike the geographical map, changes in the political terrain occur often and sometimes rapidly. Thus, a single political map may be likened to a snapshot -- it is a loyal interpretation of the political system at a particular point in time, but not at another. While it is certainly true that a particular map represents a particular point in time, by combining a series of maps over time, we can begin to appreciate the dynamics of politics. Actors begin to take on movement; we can see how support for the government wanes and wanes; and we can see coalitions take shape and later fall apart.

 

The major political actors follow:

 

The location of a group or actor on the map depends on a number of variables, and not simply the degree to which the group supports the government. In locating a group on the map there are two dimensions to be considered: first, the location of the group in terms of its support or oppositions to the government and second, the position of the group to the left or the right of the regime on the map. With respect to the first factor, a group will be located toward the core support area to the degree that it conforms to the following indicators:

 

- The group is in basic agreement with the fundamental rules of the political game.

 

- The group agrees with the objectives, goals, and policies of the regime.

 

- The group is important or critical to the government's permanence in power.

 

- The group is influential in the determination of important policies.

 

- The group is influential in the determination of important benefits.

 

The placement of a group to the left or the right of the government will depend on whether the analyst believes that the group is "more progressive" or more "conservative" than the government; whether the group is more "interventionist" or less "interventionist" than the state; whether the group is more "leftist" or more "rightist" than the government. These are informed subjective decisions.

 

Mapping can serve several purposes. First, it can provide a graphic representation of the health of a government. Second, it can tell us something about the vulnerability of the regime. Third, the map can detect the existence of opposing alliances and potential support coalitions. Finally, the map can detect new directions in policy. Although a political map can be an extremely useful instrument for clarification, it is neither a crystal ball nor a substitute for good analysis or judgment. The map is merely a tool, and like other tools, its usefulness will depend on the quality of data that goes into the construction of the map and the seriousness and quality of interpretation given to the data. If either are poor, the map loses utility and the decisions based on the map will suffer.

 

Stakeholder Analysis

 

The purpose of stakeholder analysis is to indicate whose interests should be taken into account when making a decision. At the same time, the analysis ought to indicate why those interests should be taken into account. How do we know when a group's or actor's interest must be given specific and serious consideration? First, if an actor or group is in a position to damage or weaken the authority or political support of the decision maker or the organization, then it should be taken into account.

Second, if the group's presence and/or support provides a net benefit or strengthens an organization and/or enhances the decision-maker's authority (and capacity to secure compliance to decisions), then it should be given close consideration. For example, it a group can bring new resources, provide entry into a new market or otherwise enhance the organization's strength, it should be taken into account.

 

Third, if a group is capable of influencing the direction or mix of an organization's activities, it needs to be counted as a stakeholder. Consumers are often viewed as stakeholders in organizations charged with the delivery of public services.

 

Generally, stakeholder analysis focuses on two key elements: groups or actors are analyzed in terms of; (a) the interest they take in a particular issue and b) the quantity and types of resources they can mobilize to affect outcomes regarding that issues. Obviously this could become a very large "group" and in most cases too inclusive. As a rule of thumb, to locate a relevant but manageable group, one might apply the following: only those groups or actors with real and mobilizable resources that can be applied for or against the organization and its interests to the issue at hand should be included. They are the ones that have the capacity to directly influence policy outcomes.

 

Lindenberg and Crosby (1981) designed the following stakeholder analysis approach. A matrix is developed in which information for each group is placed according to the group's interests, the level of resources it possesses, its capacity for mobilization of resources, and the group's position on the issue in question (see Figure 5).

 

FIGURE 5: Stakeholder Matrix

________________________________________________________________ __________

GROUP's RESOURCE POSITION

INTEREST IN MOBILIZ. ON

GROUP ISSUE RESOURCES CAPACITY ISSUE

________________________________________________________________ __________

 

In the first cell are listed those interests that will be affected by the policy or decision to be taken. What are the group's specific interests in the policy? The analyst should be careful to select only those two or three interests and/or expectations that are most important. In the second cell are noted those resources that the group possesses that could be brought to bear in the decision making or implementation of the policy. Can the group offer some special knowledge or information? Would the group's status and presence on one side of the issue be key to its implementation or blockage? If the group appears to have resources that can be brought to bear, it is important to know whether the group is capable of mobilizing those resources quickly or only slowly. Quickly mobilizable resources are advantageous if the issue has immediacy, but less so if the impact of the issue is further out into the future. If the group cannot mobilize or make effective use of its resources, then they are not really resources in any meaningful sense of the word. The analyst's judgment regarding mobilization capacity should be noted. Finally, the group's position regarding the issue should be examined and noted. Judgment should be more discrete than a simple for or against. If a group is barely in favor of an issue, a convincing argument could be enough to change its position.

 

While stakeholder analysis is certainly helpful to gain a better understanding of the interests and resources of the important players for policy decision-making and implementation, it is even more so when used in conjunction with other strategic management tools such as political mapping and force field analysis.

 

APPENDIX 2: STRATEGIC LEVEL QUESTIONNAIRE EXAMPLE

 

In February 1995 a UNDP/MDGD mission field tested a questionnaire (in Sierra Leone) based on the Harvard Framework. The questionnaire was used as a tool for identifying and measuring critical elements or bottlenecks in the overall enabling environment, and at the organizational and institutional levels for the building of national capacity for efficient and effective management, and which served as a guide during interviews and discussions with the various governmental and non-governmental officials. Below is the questionnaire (with slight modification) developed the mission.

 

Both public and private sector leadership and non-governmental organizations were involved in the identification of major gaps in national capacity for public sector management, including structures, systems, and procedures.

 

The questions formulated as a basis for the interviews related to the following five critical dimensions or levels of analysis for capacity in the public sector:

 

- Broad Action Environment

- Institutional Environment; and

- Network of Organizational Interactions;

- Organizational Structures, Processes, Resources, and Management Styles; and

- Human Resources Development

 

The questionnaire or "interview guide" was sent to relevant officials, six months before the mission's arrival, who circulated the document among several Ministries and private institutions for their review and comments.

 

Finally, the mission team engaged itself in detailed discussions at the end of each set of interviews during discussion meetings. The purpose of the discussion meetings was to further conceptualize the methodology, reach a common understanding of the issues at hand, and discuss eventual follow-up action. The questionnaire follows:

 

There are three possible outcomes or applications for the information gathered: (a) Broadening the intensifying the national debate on develop-ment objectives and strategies, (b) Stimulating the policy dialogue between UNDP and the Government, and (c) The design of Tailor-Made technical cooperation projects and programmes by UNDP and other donors.

 

 

APPENDIX 3: OPERATIONAL QUESTIONNAIRE EXAMPLE

 

There will be some overlap between the Strategic and Operational Levels of assessment, however, the primary focus of the Operational assessment is implementation through efficient and effective management. Therefore the Operational assessment must be cognizant of implementation capacity and donor's needs. Relevant implementation information should provide insight about an institution's ability to adequately carry out the common management activities. There are tasks and activities which are fundamental to the implementation of a UNDP funded programme/project. The common activities follow:

 

Project Management

 

Personnel

 

Contracts for Services and Works

Procurement of Equipment and Materials

 

Training

 

Financial Administration (maintained in US dollars)

 

Reporting

 

These activities as well as the organizational structure and physical capacities should be systematically assessed in order to present a fairly rigorous Implementation Capacity Profile of an institution.

 

The following questionnaire/interview guide has been used, with appropriate contextual modifications, to collect such information. This questionnaire should be used to collect information from organizations being considered to execute and implement UNDP-assisted projects. Data gathered here should be analyzed and used as the basis for an assessment of the organization's capability and appropriateness to execute and implement the specific project under consideration. This instrument may be used for public, private and NGO organizations with appropriate modifications.

 

The data may also be used to assess training and other institution-building needs of the organization related to project execution and implementation that could be met by UNDP with supplementary project funds.

 

Some of the items under Input Mobilization Capabilities and Project Implementation Capabilities may be irrelevant to the project being formulated and therefore may be omitted or other services added.

 

A. Prerequisites

 

 

B. General Information

 

 

 

 

 

 

 

C. Project Planning Capabilities

D. Input Mobilization Capabilities

 

Personnel

 

Procurement

Contracting

 

Fellowships and Study Tours

 

E. Project Implementation Capabilities

Project Management

 

Research

 

 

 

The data should be analyzed and distributed in a report format. Hopefully, the assessment will elicit debate, discussions, further analysis, and in time an Implementation Capacity Building Strategy/Plan.

 

Besides establishing a basis for the recommendation on execution and implementation arrangements, the assessment procedure serves a valuable educational function. The participants are usually appreciative of the clarification of roles, responsibilities and procedures that occurred during the assessment.