General Information

Document Type: Presolicitation Notice
Solicitation Number: Reference-Number-M-OP-REG-ALPS-267-02-04-358
Posted Date: Dec 24, 2003
Original Response Date:
Original Archive Date:
Current Archive Date:
Classification Code: R -- Professional, administrative, and management support services
Set Aside: Partial Small Business

Contracting Office Address

Agency for International Development, Washington D.C., USAID/Washington, 1300 Pennsylvania Avenue, NW Room 7.10-006, Washington, DC, 20523

Description

Monitoring and Evaluation (M&E) services needed for USAID/Iraq?s technical assistance portfolio. M&E Program Performance Phase 2 project will provide regular performance monitoring and periodic performance evaluations of a broad and complex range of activities. This notice introduces the Iraq Monitoring and Evaluation statement of work in draft form for general industry review and comment. Comments, best practices and lessons learned that are relevant to the statement of work are welcome. Replies are restricted to comments only. Interested parties will have an opportunity to submit questions and inquiries when the finalized Request for Proposals is posted. Please e-mail replies to the point of contact listed below not later than close of business January 9, 2004 NAICS: 541611. Competition Level (Set Aside: Choose One) Partial Small Business. Numbered Notes 2,12,13,23 are applicable.

Original Point of Contact

Charles Mosby, Contracting Officer, Phone 202-712-0832, Fax 202-216-3132, Email cmosby@usaid.gov - Anne Quinlan, Division Chief, Contracting Officer, Phone 202-712-4614, Fax 202-216-3396, Email aquinlan@usaid.gov

Place of Performance

Address: Iraq/Washington, D.C.


ARTICLE I - TITLE

Monitoring and Evaluation Program Performance Phase 2 (MEPP2)

ARTICLE 11 - OBJECTIVE

The objective of this activity is to provide data collection, performance monitoring and evaluation of USAID/Iraq’s technical assistance portfolio. The Monitoring and Evaluating Program Performance Phase 2 (MEPP2) project will provide regular performance monitoring and periodic performance evaluations of a broad and complex range of activities.

Monitoring: By implementing MEPP2, USAID/Iraq will continuously acquire updated information on current achievements and the overall performance of non-construction projects that contribute to the achievement of the Mission’s strategic objectives. Information gathered by this activity will provide quantitative and qualitative data that function as:

• Timely feedback on program management by USAID as well as related implementing partners, to improve achievement of the Mission’s strategic objectives and intermediate objectives.
• A continuous assessment of program progress toward specific benchmarks and targets.
• Information for USAID/Iraq reporting and dissemination, including short term requirements for sectoral or thematic data and longer term requirements such as the Mission’s Annual Report, and regular reporting to the U.S. Congress and OMB.
• Material for future evaluation.

Evaluation: The USAID/Iraq program has been conceptualized and realized on an unprecedented scale. An evaluation of this innovative program will be needed to inform any ongoing and/or next generation activities and to review completed activities.

Although evaluations generally focus on individual activities, the contractor will implement a single program-wide evaluation covering USAID’s non-construction activities in Iraq. The evaluation will be conducted at 7, 13, 19 and 24 months of programmatic activity, and is expected to assess the extent to which each contractor and grantee has accomplished the tasks laid out in its quarterly work plans with an analysis of implementation progress and achievement of work plan goals. It should also analyze the extent to which the activities conducted are actually enabling the accomplishment of intermediate results and strategic objectives. The contractor should work with each CTO/activity manager and activity implementing partner to agree on the details and scheduling of the evaluation process, including any additional criteria by which the program should be evaluated if deemed necessary and appropriate. Once the project evaluations are completed, the contractor will analyze the extent to which USAID/Iraq supported programs, individually and overall, are contributing to the accomplishment of the Mission’s strategic objectives.

The evaluation should result in and include recommendations on the programs that should be phased-out, extended, augmented, revised or combined, and suggestions as to any new programs that could enhance the ability of USAID/Iraq to achieve its reconstruction and development goals.

ARTICLE III - STATEMENT OF WORK

USAID requires the services of a contractor or consortium of contractors to provide a team of U.S., third country national, and Iraqi professionals to monitor and evaluate the performance and effectiveness of USAID/Iraq’s activities and their contribution to the achievement of the Mission’s strategic objectives. Work will be undertaken in the following areas:

• Monitoring: Technical assistance to inform, monitor, synthesize, and analyze implementation of USAID-funded non-construction projects. The contractor will be required to design a monitoring program, and identify baseline data with USAID’s implementing partners, develop workplans, and conduct the performance monitoring.

• Evaluation: Technical assistance to conduct an independent and rigorous program evaluation. The evaluation should suggest ways to improve program management and include recommendations as to programs that should be phased-out, extended, augmented, revised or combined, and new programs that could enhance the ability of the Mission to achieve the goals of its strategic objectives.

• Database Development: Technical assistance to create, populate and maintain an advanced database of program data that is capable of instant reporting along project, sectoral and thematic lines and which interacts with USAID implementing partner and related agency (CPA IMU) databases.

The contractor will monitor and evaluate activities in several functional areas. These areas may include, but are not limited to, the following:

• Logistical Support
• Infrastructure Development (evaluation only)
• Health System Strengthening
• Sub-National Governance and Civic Institution Support
• National Governance
• Basic Education
• Higher Education
• Agriculture Reconstruction and Development
• Economic Recovery, Reform and Sustained Growth
• Community Action
• Transition Initiatives

The assistance to be provided is further described in the tasks below.

Tasks and Activities

The contractor will provide the following monitoring and evaluation services for the functional areas described below:

Tasks and Activities - Monitoring

1. Program design: identify information needed to reveal indicators, sources, tools to acquire information (weeks 1-4)

At project start-up, the contractor will determine the scope and pattern of work for on-going monitoring over the following 24 months. Initially, the contractor will review USAID/Iraq and other documents related to ongoing projects in Iraq and to newly awarded projects to begin identifying key performance indicators for regular monitoring. The contractor will work with USAID/Iraq, the Coalition Provisional Authority (CPA) and USAID implementing partner staff in this process and will collaboratively develop Performance Monitoring Plans for all activities identified below. The contractor will develop precise definitions of indicators, units of measure, disaggregation, justification, and management utility.

To gather the information required above, the contractor will prepare the method for data collection (field visits, implementing partner data, beneficiary organizations, key informants) and the necessary tools for data/information gathering (reply forms/questionnaires, tabulation forms). This will be done in collaboration with the team developing the database to ensure the database is populated form the beginning with data accessible to USAID and other (CPA IMU) users.

The contractor will provide a comprehensive plan during this period for data verification and validation. Much of the data will come from implementing partners. Their data must be validated by site visits from verification teams and other methods proposed by the contractor.

The data validation plan may be part of the Start-Up and Program Design Report submitted to USAID to approve the methodology, indicators, and Performance Monitoring Plans proposed by the contractor. With USAID approval, the contractor will implement the data gathering plan in the field and will periodically reevaluate and optimize program design and implementation.

2. Assessment of baseline data and data quality (weeks 1-4)

In tandem with the preparation of the methodology, baseline data will be collected on all regions and sectors, while at the same time doing an assessment on the availability and quality of initial data, limitations and significance. In collaboration with functional area contractors and grantees, the MEPP2 contractor will gather data and verify its sufficiency to corroborate and describe the status of IRs.

Results of this activity will influence the monitoring method and indicators used. Together with the results of activity 1, the baseline data gathered will be presented in the Start-up and Program Design Report.

3. Monitoring Workplans (week 6)

The final methodology and tools prepared for data acquisition will be described in monitoring workplans – one for each functional area – that cover the following:

• Data collection method (data must be collected from direct and indirect beneficiaries of projects, as well as from implementing partners, USAID, the CPA, other USG agencies and Iraqi stakeholders)
• Method of acquisition, including information-gathering forms or tools
• Data sources
• Frequency/timing of data acquisition
• Data validation and verification
• Estimated cost of data acquisition

By the end of the week six, the contractor’s monitoring workplans will be submitted to the USAID CTO for review, modification, and approval. If necessary, additional modifications to the method/system may be based on quarterly reviews. Preparation of workplans will require joint selection of any target regions or activities.

4. Implementation of monitoring (month 2 - month 24)

By week 6, the contractor will begin monitoring the various project activities. During the 23 months of monitoring, the contractor will gather information from project teams and local counterparts, including government agencies, conduct interviews with USAID, government officials, implementing partners, other donors, and beneficiaries on a regular basis. The contractor will collaborate with this project’s database sub-contractor as well as other database holders in the US and Iraq (CPA IMU), and will input information into the system as it is received. This data will be available to USAID in Iraq and the United States 24/7. The contractor will provide summary reports on a schedule agreed to with the USAID CTO.


Tasks and Activities - Evaluation

To accomplish the tasks below, the contractor will develop a workplan for USAID approval. The work plan should provide target dates for initiation of project activities, provide reasonably detailed descriptions of the tasks and activities of each specialist and the project as a whole, and establish benchmarks for performance of each of the tasks. The workplan will be organized by USAID strategic objective and the functional areas outlined below.

The contractor shall conduct work twice annually starting on the seventh month of USAID program implementation (months 7, 13, 19 and 24). The contractor shall provide an outline of its draft evaluations by the end of the first week of each evaluation and shall periodically (no less than weekly during the evaluation period) brief Mission management, technical personnel and USAID staff in Washington on overall progress, findings and development of concepts/ideas, and preliminary results and recommendations.

1. Collect and review relevant program materials

Materials to be reviewed should include:

• Background materials on the sectors served by USAID in Iraq. USAID/Iraq can provide some of these materials but the contractor should update any information provided and collect additional material useful to this evaluation.
• The Mission Strategy Document – when available.
• Performance monitoring plans and performance monitoring reports.
• Scopes of Work, work plans, and interim reports particular to each activity as well as any additional materials deemed necessary by the project CTO/Activity Manager.
• Updates to any of these materials as projects develop and requirements change. Initial SOWs may be significantly modified by the work environment.

2. Develop criteria by which programs will be evaluated, individually and as a whole

The contractor should work with the CTOs and program and technical staff to determine the exact criteria by which each program and its contribution to the strategic objective will be evaluated. In general, evaluations should include the following measures:

• Impact: What are the benefits, who benefited and what is the measure or magnitude of those benefits? (e.g., How many people are in school, how many babies vaccinated, etc.)
• Effectiveness: Are benefits reaching the target beneficiaries? Have others captured the benefits? What can be done to better target benefits?
• Efficiency: Were results achieved at a reasonable cost? What does the benefit/cost analysis show?
• Replicability: What are the prospects that the new approaches and new ideas will be adopted throughout the country?
• Sustainability: What are the prospects for continuing benefits after USAID funding ends? What needs to be done to ensure financial, institutional, and management sustainability?

The CTO or designee for each project must approve the criteria before the evaluation begins. Source material for criteria should include the Performance Monitoring Plans, the work plans, and any other measure agreed upon as necessary and appropriate.

3. Conduct interviews with USAID staff and implementing partners, other donors, representatives of the civil/military authorities, appropriate Government of Iraq authorities, NGOs/civil society/community groups, and other stakeholders and beneficiaries in the reconstruction and development process.

Interviews should be designed to solicit information on project effectiveness and usefulness, priorities that need to be addressed to ensure the accomplishment of USAID’s strategic objectives, and environmental factors that might influence the short and long-term viability and/or desirability of USAID-funded projects. With USAID approval and coordination, the evaluation teams will conduct site visits as appropriate to accurately evaluate the scope and impact of USAID activities. The evaluation teams should coordinate the identification and selection of sites for field visits with their respective USAID CTO/Activity Managers.

4. Arrange meetings with all SO teams and donors involved with reconstruction assistance to identify and discuss the extent to which USAID activities support or complement their activities; the strengths and weaknesses of USAID activities and which ones might be extended, phased out or revised; and what other activities USAID might undertake that would be beneficial and productive.

Meetings should be held with a clear understanding of weaknesses and strengths in the USAID program and an understanding of unmet needs within the various functional areas. The contractor should be ready to provide an outline of the activities currently being carried out by the various SO teams and other donors in order to enable USAID to establish a clear sense of where it fits into the picture and how USAID can best maximize programmatic impact in Iraq.

5. Prepare draft evaluations with recommendations – one for each functional area. Present these to USAID staff, implementing partner staff, and counterparts for comment.

These individual evaluations will include but not be limited to the following:

• An examination of the extent to which each of the activity implementing partners has accomplished the tasks laid out in quarterly and annual workplans, with an analysis of progress and achievements, and any shortcomings.
• An analysis of the extent to which the activities conducted are actually enabling the accomplishment of Intermediate Results and Strategic Objectives.
• An assessment of the strengths and weaknesses of each activity and the extent to which the results envisioned for it in the relevant Performance Monitoring Plan have been accomplished.

6. Prepare a draft program evaluation, with recommendations for possible future program directions

Drawing upon the individual functional area evaluations, this “big picture” evaluation should examine the extent to which USAID activities as a whole are (or are not) contributing to the achievement of the goals stated in the Mission’s strategic objectives. This evaluation should include an assessment of the progress and impact (beneficial/harmful) in each functional area, and draw conclusions and prepare recommendations in respect to the value and impact of these activities. The evaluator should include consideration of how the various non-construction activities mesh with one another, with the capital construction projects, and with the projects of other donors.

7. Present complete evaluation and obtain feedback from USAID staff and selected counterparts.

A PowerPoint presentation should be made to USAID management and select counterparts. The presentation should complement the delivery of hard copies of the draft program evaluation.

8. Complete final evaluation drafts.

Final evaluation drafts – functional areas and program – will be made following receipt of verbal and written comments from USAID management.


Tasks and Activities – Database Development

• Process requirements

o The contractor will develop an information management plan for USAID approval. The contractor will coordinate plan with USAID contractors, USAID grantees and with the Coalition Provisional Authority.

o The contractor shall implement a formal and rigorous software development process, including formal design reviews, testing, implementation, maintenance and training.

o The contractor will develop and implement plans and procedures for user access, security, data backup and data recovery.

o The contractor shall provide a help desk and personnel to respond promptly to user requests.

o The contractor shall provide training to Iraqi staff from the Ministry of Planning and other government agencies to maintain, update and utilize this database.

• Product Requirements

o The database shall provide fields to track relevant information which may include, but is not limited to, the following:

o The number of direct and indirect beneficiaries
o The metrics or indicators defined for monitoring and evaluation
o Sector and sub-sector tracking
o Location, including governorate, districts, and other geographic indicators (e.g. GPS coordinates, military grid coordinates, UN coordinates, as may be required)
o Sectors and sub-sectors
o Financial information including funds obligated, budgeted, expended, and disbursed, as well as costs shared with other entities (e.g. CPA, UN, etc.)
o Project job orders
o Funding source
o Completion percentage by project duration or expense
o Schedule, including start date, milestone dates, and end date.
o Check boxes for technical review of project specific documents
o Other data, as required.

o The database shall store relevant project and program documentation, including contracts, schedules, plans, status reports, cost information, and other relevant documents.

o The database shall provide data search and sorting capabilities.

o The database shall provide a customizable web interface. Such an interface is intended to provide a “dashboard” that highlights key information and changes in status.

o The contractor shall supply a context-sensitive online help capability, including definitions of relevant terms.

o The contractor shall provide a standardized reporting tool. In addition, the contractor will provide and maintain standard report templates that will be defined by USAID. Furthermore, the reporting tool will permit the generation of user-defined custom reports and the modification of pre-existing reports. Such reports shall be exportable to Microsoft Excel or other standard desktop applications.

o The contractor shall provide the capability to generate geographic reports and maps showing the location of USAID projects and other related information.

• Technology requirements

o The contractor shall use off-the-shelf, commercially available technologies as much as possible. The database shall use open standards and shall be scalable to support the needs of USAID.

o The database shall provide capabilities for the automated exchange of data from other USAID contractors, subcontractors, and grantees; and to CPA or other U.S. Government entities. Such import and export may require real-time or near-real-time data exchange in XML or other standard formats.

o The contractor will incorporate appropriate security technologies, such as SSL, to prevent unauthorized access to the database.

• Ancillary requirements

o The contractor shall supply the associated computer equipment, computer software, communications services, facilities, including computer servers, storage, uninterruptible power supplies, and other related material.

o The contractor shall provide relevant user documentation, such as a Users Document.


Possible Functional Areas for Monitoring and Evaluation

Health System Strengthening

The activities to be carried out under this contract will contribute to the reestablishment and strengthening of the national and provincial public health system in post-conflict Iraq and its management by the MOH. Several organizations--such as the International Committee for the Red Cross (ICRC), the International Federation for the Red Cross and Red Crescent (IFRC), the Iraqi Red Crescent Society (IRCS), the World Health Organization (WHO), and UNICEF--have been supporting the health system (and in some cases the water/sanitation system) and MOH in the past and are expected to re-new their efforts as soon as possible following the conflict. The activities carried out this program will complement the efforts of these other organizations to achieve the overall post-conflict health objectives. To the degree possible, the program will be based on the needs of both the Iraqi people and MOH and will support and build upon the existing public health system rather than re-inventing, replacing, or duplicating existing mechanisms that are technically sound. Trained MOH health staff will be encouraged and supported in their efforts to continue operating the health system, although some additional training may be required. In addition, trained Iraqi health staff living outside the country will be recruited.

Local Institutions Support and Development Program

The purpose of this project is to procure and provide technical and other assistance to strengthen local administrations, civic institutions and processes in Iraq. Activities will respond to specific challenges in sub-national administration, democratic institutions and processes, civil society development, and citizen participation in local governance and access to services.

Education

USAID’s education reform project is designed to provide support at both the policy and classroom levels to improve the quality and availability of primary and secondary education. This activity is intended to induce altered attitudes and behaviors, improved practices and positive changes in basic education for Iraqi children and adults. While the specific program activities will need to adapt to the dynamic environment in Iraq, the program will support the Government’s education programs in six specific areas: education reform and accelerated learning; trauma alleviation, psychological counseling, and conflict resolution; teacher and administrator training; curriculum revision and textbook production; literacy and job skills; and, higher education for revitalization.

Agriculture Reconstruction and Development

The activities to be carried out under this contract will contribute to the reemergence and modernization of the agriculture system in a post-conflict Iraq. During Phase I (first three months), USAID will emphasize quick impact activities that stimulate agricultural production and contribute to food security. Through rapid rural appraisals, USAID will identify emergency needs and supply critical agricultural inputs for summer and fall planting season. Over the medium term (4-12 months) USAID will provide a wide range of technical assistance, financial services, training, and equipment to improve management of soil and water resources, reinvigorate local markets, stimulate agroenterprise development and increase agricultural trade and investment. The Agriculture Reconstruction and Development Program for Iraq consists of four components:

• Revitalization of agricultural production
• Income and employment generation through agro-enterprise and market development
• Rural finance
• Reclamation and improved management of soil and water resources.

Economic Recovery, Reform and Sustained Growth

The purpose of this project is to provide economic rehabilitation and reform for Iraq to stimulate the country’s international trade engagement, employment and broad-based prosperity. The tasks performed under this scope will facilitate a rapid and responsible economic integration of Iraq with its regional and international partners to create sustainable job generation, adopt international standards of production, harmonize economic policy, reinforce traditional trade linkages and develop new trade partnerships. The program is expected to provide two broad levels of interventions: transactional activities and economic governance activities.

Transactional activities will create a competitive private sector through: credit provision; development of local, regional and international business networks; support for business operations and strategy; workforce development and employment generation; entrepreneurship training and development; mobilization of domestic and foreign investment; and, privatization of state-owned enterprises, and trade capacity development.

Economic governance activities will include support to those public and private institutions that shape and implement economic and financial policy, regulatory, and legal reforms, including support to the Iraqi Central Bank, Ministry of Finance, Banking System, and Ministry of Trade.

Community Action

The Community Action Program will promote diverse and representative citizen participation in and among impoverished communities in Iraq, and will identify and address critical development needs. The program will facilitate the rapid dispersal of USAID resources to 250 communities, and engage citizens in processes vital for effective community/local government partnership.

The project will complement other USAID initiatives by implementing demand-side projects that promote civic participation, revitalize essential infrastructure, create employment and income generation opportunities, and address critical environmental problems. The activity will create community committees responsible for identifying and prioritizing community needs, mobilizing community and other resources, and monitoring project implementation. Broader cluster committees will be formed with representatives from various community committees to increase participation and cooperation on issues of regional concern, and to promote ethnic, religious, and tribal cooperation on the basis of shared interests.

University Partnership & Development

To be developed.


Coordination with Military and Civil Authorities: The contractor will be responsible for coordinating project monitoring and evaluation with appropriate national, local, and U.S. military and civilian authorities

Security: The contractor will be responsible for providing security for all aspects of monitoring and evaluation operation and for all project staff. The contractor will be responsible for the evacuation preparedness of project staff. This function is likely to be provided in liaison with and coordination with appropriate national, local, and U.S. military and police functions.

Transition: The contractor will be responsible for planning the transition of monitoring and evaluation functions as USAID-funded projects are completed and/or ramped down in scope and scale, including providing a continuing program of technical assistance as required and agreed by USAID.

USAID Inspector General: In collaboration with the USAID CTO or designee, the contractor will coordinate M&E activities with USAID/Iraq’s Office of the Inspector General. The contractor will share M&E reports with the IG, and, as necessary, will facilitate IG participation in M&E activities, including site visits and interviews.

Sub-Contracting Opportunities: The contractor shall identify a specific individual responsible for personally responding to inquiries in a timely manner related to sub-contracting opportunities.

Local Capacity Development: The contractor shall outline resources necessary to build the capacity of its local staff during the contract period to better conduct/manage similar projects in the future.