USAID Nigeria: Effective E-WASH final Evaluation Report

The United States Agency for International Development (USAID) awarded the Research Triangle Institute (RTI) International the five-year Nigeria Effective Water, Sanitation, and Hygiene (E-WASH) Activity in 2018. The final  report on its implementation was released in March 2022.

The primary purpose of the final performance evaluation was to determine whether the E-WASH Activity achieved its stated development objectives and to understand lessons learned. The evaluation team (ET) developed questions that assessed the Activity’s role in supporting state utilities’ commercial orientation, financial and operational viability, professional management, city-wide sanitation mapping, and use of environmental and construction guidelines. The ET collected qualitative data from November 29 – December 18, 2021, across five Activity states (Abia, Delta, Imo, Niger, and Taraba), and the Federal Capital Territory and triangulated it with secondary qualitative and quantitative data.

Download Full Report: EFFECTIVE EWASH FINAL EVALUATION REPORT_USAID

The five Evaluation Questions (EQ) used for the evaluation, the conclusions, and recommendations are reproduced below:

EQ1: To what extent did State Water Boards (SWBs) become commercially oriented and improve their financial viability?

E-WASH implemented several interventions to push the SWBs towards commercial orientation. Changes in the utilities’ commercial orientation occurred in six primary areas: Monitoring and evaluation (M&E), workforce capacity and attitude, customer orientation, water production efficiency, financial viability, and autonomy from government support.

The SWCs have made some shifts towards commercial orientation. This has led to increased awareness and attitudinal changes, as well as a shift towards improved revenue and overall capacity. However, many challenges remain, particularly regarding the utilities’ financial viability and overall service delivery.

In Taraba and Niger, improvements have been strong, but they have not met the anticipated targets. If states can improve capacity utilization and improve collection efficiency, then they are likely to be able to reduce NRW and improve OCCR, as demonstrated in Niger and Taraba.

All utilities except Abia saw improvements in water production. However, all five continue to grapple with high NRW, poor water quality, and low-cost recovery. This undermines the gains of EWASH.

A lack of water production is a significant hurdle, particularly in Abia. This is a specific challenge for the uptake of capacity-building efforts, as staff do not have the opportunity to implement what they have learned, and many trainings risk being forgotten. The Activity was heavily involved with the implementation of the ERP, which will improve the utilities’ commercial orientation. However, the utilities continue to struggle with the implementation of several modules due to a lack of capacity.

Recommendations:

For future activity design, USAID should:

1. Ensure that utility-based interventions directly target individual utility needs and be based on independent assessments or studies that take into consideration the basic requirements of the utility to accept such interventions, including the availability of water and infrastructure.
2. Ensure logical sequencing and prioritization of implementation and inputs. For example, key interventions such as the ERP should be prioritized early in activity implementation.
3. Ensure that advocacy to state government includes clear project deliverables and MOUs include verified commitments to implement identified activities (e.g., funding, human resources,).
4. The state government milestones within E-WASH were novel and well appreciated. However, future milestone efforts should consider including critical inputs, such as chemicals and electricity cost, to ensure smooth implementation and sustainable operations.

EQ1A: To what extent has the Utility Support Fund been effectively utilized to achieve its objectives?

The USF is expected to have some positive effect on utility operations, including on key indicators related to water production, customer confidence, revenue generation, and accountability. The limitations described above may inhibit progress, even for the more strongly performing utilities in Niger and Taraba, and the improvements may not be sufficient to place the SWBs on track towards overall financial and operational viability.

The ERP has the potential to be a major contribution to the sector. However, its long-term impact is contingent on sufficient political will and ownership to ensure that SWCs continue to use the ERP systems after embedded E-WASH staff leave. Success is also contingent on local governments providing funds for retraining, hosting the platform, and manning the help desk.

Recommendations:

1. For future activity design, USAID should ensure that the scope of interventions is designed in collaboration with the utility, capturing all components required to make the system fully functional and to secure ownership.
2. If future activity design includes a mechanism similar to the USF, the projects should be implemented early in the activity to maximize benefit and impact.

EQ1B: What impacts if any has COVID-19 had on SWB’s financial viability?

Overall, COVID-19 had negative impacts on the utilities’ financial viability. Niger and Delta were affected financially, and for the other utilities, the primary impact was in the form of delays in the delivery of the USF activities. COVID-19 helped create positive awareness of the importance of water supply among utility customers and the opportunity for the utilities to begin to leverage technology and innovative payment platforms.

Recommendations:

1. For future activity design, USAID should continue to support utilities improvements to billing and collection processes (e.g., prepaid meters and other e-method payment options).
2. Future activities should leverage available technology platforms for capacity development and meetings.

EQ2: To what extent did the Activity support operational viability and professional management of SWBs?

There were significant enhancements through capacity-building and training, which have led to a reported improvement in work ethic and culture. Many performance weaknesses will not be remedied without a functional structure, appropriate incentives, remuneration packages, staff retention plans, and autonomy.

Recommendations:

1. Future USAID projects should consider conducting and implementing organizational development early in the activity so that functional structures and human resources are in place. This should include proper staff recruitment and placement as well as policies for staff remuneration and incentives.
2. For future activity design, USAID v should develop a roadmap to assist the utilities in planning full implementation of anticipated activities (organizational development, strategic plans, PBC, strategic plan, etc.). This roadmap should include details regarding the timing, funding sources, and other pertinent details to ensure effective implementation.
3. USAID should consider future activities to support the states’ implementation of the PBCs and ensure they are cascaded across the utility staff. This should include realistic targets, proper incentives and penalties, and ensure that the regulatory agencies are empowered to monitor and track progress.
4. The Delta, Taraba, and Niger SWCs should intensify the implementation of the organizational development study and the states should support the utility with funding to realize full implementation.
5. Future projects in Abia and Imo should review the existing organizational development study jointly with the utility and support its implementation 6. The utilities should continue to prioritize capacity development of its staff, to sustain the various systems established by the activity

EQ3: To what extent did the Activity strengthen policy, institutional and regulatory frameworks for improved WASH SERVICES at the state and national level?

E-WASH successfully supported the development, implementation, and partial operationalization of WASH policies and laws across all five states. It also supported the strengthening of the institutional and regulatory framework in Imo and Taraba.

The policies and laws steered the utilities towards corporatization and a more commercial orientation, which granted some of the utilities a level of autonomy. However, there is no evidence to indicate that the policies, laws, and institutional and regulatory frameworks translated into improved service delivery in terms of provision of water for the citizens.

Of note, a lack of adequate funding also hindered the operationalization of some of the regulatory frameworks. There were other policies and guidelines (e.g., gender, sanitation) that are still currently undergoing development in some states and other aspects of the policies have not yet been fully implemented due to constrains in capacity and funding. The implementation of the MEL Plan in Delta was stalled by a lack of capacity.

Recommendations:

In an effort to ultimately improve service provision:

1. Through its implementing partners, USAID should advocate for the inclusion of a roadmap that identifies the sources of funding for future WASH policies and laws to ensure effective implementation.
2. Through its implementing partners, USAID should continue advocacy to the utilities to ensure that the government fully implements current WASH policies and laws, as well as complete the ongoing development of WASH policies and laws.

EQ3A: What accountability mechanisms related to implementation of policy and frameworks did the Activity put in place, how successful have they been, how could they be improved, and how might they impact sustainability?

The Activity has put in place several accountability structures across the five utilities to improve the quality of relationships (i.e., clear roles and responsibilities, transparency, and accountability) between the different stakeholders. Internal accountability structures like the BODs, procurement systems, and audit processes moved the needle to promote transparency and accountability within utilities.

External accountability structures like the WASH media forums have increased the transparency of utility operations. In some cases, the efforts have also contributed to changes in public perceptions of the utilities and boosted their image.

However, many challenges remain, including a lack of funding, water supply, and commitment by the management of the utilities; capacity gaps in the use of the ERP; insufficient ownership of some of the frameworks; and incomplete activity implementation due to the descoping. Because of these challenges, some of these structures may not be sustainable beyond E-WASH.

Recommendations:

1. For future activities, USAID should identify targeted accountability mechanisms within utilities and support future planning for capacity, knowledge transfers, and sustainable funding streams
2. The utilities, through the customer units, should sustain engagement with the customers and WASH media forums to ensure their sustainability.
3. The utilities should strengthen ERP execution through further training and retraining of staff

EQ4: To what extent did the Activity prepare target institutions to manage sanitation challenges through city-wide sanitation mapping?

The E-WASH Activity’s sanitation-related interventions did not achieve their objectives. While some sanitation progress was made and groundwork laid with laws, policies, and frameworks, the policies and laws did not address funding mechanisms, which have hampered implementation.

The utilities are still weak and struggling with capacity challenges, underfunding, and competing priorities. Though cited as best practice, the inclusion of sanitation in the mandate of the utilities was in its very early stages and the enabling environment for CWIS was not sufficient.

Utilities lacked familiarity with sanitation and the severity of the challenges in the sanitation sector in Nigeria. These challenges include the absence of required infrastructure, skills, and resources; inadequate water supply; and the uncertain policy and institutional environment. These barriers proved difficult to overcome.

Better timing relative to decision-making could have helped. For example, by the time the sanitation reports were available, the advocacy for the inclusion of sanitation into the mandate of the utilities had gained traction. The weak capacity and expertise in sanitation within the IP and the utilities, in conjunction with the challenges faced in the provision of water supply, contributed to an inability of the IP to appreciate the enormity of the situation, maximize opportunities, and implement the proffered solutions from the CWIS exercise.

“Easy wins” could have included working with the utilities to use the opportunities created by the intersectoral CSTF to involve the already existing community health workers and sanitary inspections in sanitation promotion.

Recommendations:

1. USAID should ensure RTI develops an exit plan as part of Activity closeout for each intervention state. The exit plan should incorporate the sanitation investment plans from the CWIS exercise to identify the key actions, opportunities, partnerships, and funding required to operationalize and strengthen sanitation units and task forces.
2. For future programming, USAID should consider the challenging sanitation environment clearly, as itemized in the sanitation mapping reports. This should be done prior to project design. USAID should focus interventions in preparing the mandated agencies to address these complex challenges.
3. USAID should provide oversight to ensure that institutional frameworks established by its IPs comply with best practices in Nigeria. For example, the composition of the CSTFs hampered instead of aided implementation.
4. USAID should make it a priority to support the State Water and Sewerage Corporations with high-level advocacy to state governors and external support agencies and identify donors who can continue to support this work. The objective would be to maximize the opportunities that the mapping identified.
5. USAID should ensure that IPs’ sanitation strategies assess the context early and often. This should begin before implementation and continue throughout. Additionally, USAID should require that IPs maintain staff with sufficient skills and abilities in sanitation. Penalties should be enabled if the requirements are not met.

EQ5: What lessons can be learned from the Activity’s implementation of USAID’s environmental and construction guidelines? How can they be used for adaptive management?

To a large extent, the IP implemented the recommendations from the MTE, which has assisted the Activity to focus on delivery of water supply. However, the implementation strategy adopted by the IP in descoping led to both positive and negative effects, as reported by most of the utilities. Similarly, the MTE reports were not adequately disseminated among stakeholders.

Recommendations:

1. USAID should ensure that learnings and adaptions, inclusive of descoping, based on evaluation findings are disseminated both internally and externally in a timely manner.
2. USAID should ensure that any future activity descoping processes include the assessment of the impact of non-completion on activity objectives and assess if other donors or partners could support de-scoped activities to avoid loss of investment.