Efficiency of international cooperation schemata in African countries : A comparative analysis using a data envelopment analysis approach

International cooperation has grown significantly in recent years, in fact, in 2009 the Organisation for Economic Co-operation and Development’s Development Assistance Committee (DAC) provided 85 874 million euros in official development assistance (ODA), of which 6% corresponded to Spanish ODA (MAEC 2009). The increase in the volume of aid and other resources towards achieving the United Nations’ eight millennium development goals by 2015 has been accompanied by a growing demand for effectiveness in development assistance (OCDE 2005:1). Countries that participated in the Second High Level Forum on Aid Development Effectiveness in Paris in 2005 recognised and committed to increasing aid effectiveness to achieve the millennium development goals.


Introduction
International cooperation has grown significantly in recent years, in fact, in 2009 the Organisation for Economic Co-operation and Development's Development Assistance Committee (DAC) provided 85 874 million euros in official development assistance (ODA), of which 6% corresponded to Spanish ODA (MAEC 2009).The increase in the volume of aid and other resources towards achieving the United Nations' eight millennium development goals by 2015 has been accompanied by a growing demand for effectiveness in development assistance (OCDE 2005:1).Countries that participated in the Second High Level Forum on Aid Development Effectiveness in Paris in 2005 recognised and committed to increasing aid effectiveness to achieve the millennium development goals.
(sub-Saharan Africa: Mozambique, Angola, Namibia, Senegal and Cape Verde).In the period [2002][2003][2004][2005][2006], countries belonging to North Africa received approximately 15% of bilateral ODA per year on average and those from sub-Saharan Africa received approximately 10% of bilateral ODA per year on average.Although different international cooperation schemata (ICS) are used to run these bilateral aid projects, all ICS are implemented either by non-governmental development organisations (NGDOs) or by the Spanish Agency for International Cooperation for Development (AECID).In each case, the primary focus is on effectiveness as the essential principle.
However, in a world of scarce resources-especially in the international cooperation sector-effectively achieving goals is not sufficient.What really matters is efficiency: that is, controlling the level of resources used to achieve a certain goal (Banerjee 2008;Duflo & Kremer 2003).Increasing efficiency implies that a project is able to carry out more activities using the same amount of resources, which allows the project to extend the effect of the assigned resources for development as well as to reduce the pressure on the founder organisations and entities.In addition, the trend of national governments towards outsourcing international development aid makes efficiency imperative for non-profit organisations, especially NGDOs.
Accounting measures of efficiency in the non-profit sector, such as programme spending, price, technical efficiency and allocative efficiency, are based on ratios.These ratio measures only show the average percentage of the available resources devoted to the mission of the non-profit organisation or to the goals of the projects implemented by a non-profit organisation, but they do not provide information about the level of the achievement of the objectives or the level of resources applied, that is to say, they indicate how much of the donors' contributions goes to the cause but do not take into account the impact of the non-profit's spending.
To address this limitation, DEA methodology is proposed to analyse the relative efficiency in the non-profit sector.This technique estimates the relative efficiency objectively and numerically, taking into account several inputs and outputs without assuming weights and functional form of the production function, which makes it especially suited for non-profit organisations and, consequently, for the evaluation of international cooperation projects.This methodology has already been used in earlier efficiency studies in the non-profit sector, but these studies have focused on the so-called Stage I of non-profit organisations, namely fundraising efforts (which are the most influential determinants of raising funds in order to increase the amount of contributions).However, for the so-called Stage II of nonprofit organisations, namely spending the achieved resources to programme services delivery, DEA studies are very scarce.In attempting to address this research gap, this article assesses how efficiently the aid funds are actually spent by different Spanish aid instruments for the intended purposes and to compare different instruments in this regard.Specifically, this study, applying DEA, evaluates and compares the efficiency of a set of Spanish international cooperation projects funded with different schemata by the AECID between 2002 and 2006 in two African countries that are top priority targets of Spanish international aid: Morocco and Mozambique.To the best of our knowledge, this investigation is the first study that applies DEA to the assessment of ICS.Consequently, we offer a significant contribution to the literature by overcoming the limitations of other techniques used to assess the efficiency and providing new insight into the efficiency of targeted ICS in international cooperation development projects.
The remainder of this article is organised as follows.'Implementation of Spanish ODA' section explains the various ICS applied to implement the ODA.'The efficiency issue' section presents the different measures that have been used to assess aid efficiency and the measurements resulting from applying DEA.The section also discusses the appropriateness of applying DEA methodology to this study.The 'Method' section describes the sample, information sources and variables used to measure efficiency.The 'Results' section provides the main results, and the 'Discussion and conclusions' section offers conclusions, discusses the main limitations of the study and provides some lines for further research.

Implementation of Spanish ODA
International cooperation projects are funded using different ICS from ODA.The Organisation for Economic Co-operation and Development (OECD 2008) defines ODA as resources transferred to recipient countries on the DAC 1 list as well as to multilateral institutions for development.This assistance is provided by international agencies to promote the development and welfare of recipient countries.In addition, to be characterised as ODA, transfers must be concessional 2 in character.Bilateral or multilateral ODA to recipient countries can be provided by different sorts of agencies (i.e.state or local governments or by their executive agencies).We focus on non-reimbursable and bilateral ODA schemata funded by the AECID.Permanent open-call subsidies (POC) and NGDO subsidies (NS) are the kinds of schemata most employed by the AECID.
POC subsidies fund international cooperation projects of a particular initiative of any non-profit organisation.Call resolutions of the current Spanish Cooperation Master Plan and the International Cooperation Annual Plan state the priorities for granting these subsidies.In the selection of applicants, the AECID considers the project's application to the sectorial and geographical priorities of Spanish cooperation; its complementarities with other projects related to cooperation for development; the content, relevance and 1.All low-and middle-income countries with the exception of G8 members, EU members and countries with an admission date for entry into the EU.
2.At least 25% of aid must be a contribution, and interest rates on loans must be below market levels.Spanish cooperation establishes geographical priorities for aid.These priorities are based on several factors: development indicators, the presence and articulation of Spanish cooperation in the country, the possible partnership framework in the country, the potential of the country as a development partner and the relative position of Spanish cooperation regarding other donors 3 .Given these priorities, beneficiaries of assistance can be broken down into three subgroups: (1) countries with a wide association, (2) countries with a focused association and (3) countries associated for consolidating development achievements.
Countries with a wide association include the least developed countries, with lower-middle income.In these countries, large volumes of ODA can be channelled through a longterm framework and the application of different kinds of ICS.
Countries with a focused association also include least developed countries with lower-middle income but in which the cooperation programme does not allow a long-term or established framework association.The development impact in countries in this group increases when the aid is focused on a particular aspect identified along with the partner country.Therefore, a more selective use of instruments is recommended.Finally, some countries associate to consolidate development achievements, that is, the aim of aid in these countries is to strengthen inclusive public policies, promote South-South cooperation, encourage triangular cooperation and provide global public goods.

The efficiency issue
To assess the efficiency of ICS, we must first define efficiency.
The concept of efficiency widely refers to the relation between the level of objectives achieved and the amount of resources applied (Martín, Martín & Hernangómez 2007).
The first problem relative to this definition is as follows: given the relation between objectives and resources, how can we determine whether it is efficient?Or, in other words, how do we establish an appropriate standard of efficiency?A second problem lies in the objective's multiple dimensions (outputs) and the multiple resources used (inputs), which requires that we assess all dimensions simultaneously and that we assign weights to all organisational factors (Hernangómez et al. 2009).
To evaluate the efficiency of ICS, we must determine the goal.
In this sense, the understanding of the framework within which they operate is required.That is, ICS are the means by which organisations act to improve the quality of life in developing countries.The schemata are tailored to address a specific need (or set of needs) in the receptor country.Consequently, if the objective of the ICS is related to the question of what they are trying to achieve, the answer is to improve the quality of life and welfare for people in developing countries, an objective that is shared by the NGDO subsector globally.Therefore, the objective of the ICS is to improve the welfare of as many people as possible by meeting the specific need for which they were created.
Extending this thinking to the institutions that manage ICS, the notion of efficiency suggests that if NGDOs were marketable, an investor would find more value in organisations with the ability to generate greater prosperity or bring benefit to a greater number of people using the least possible resources.That is, the buyer would choose the organisation that manages more efficiently the chosen instrument of cooperation.Therefore, to evaluate the efficiency of ICS, we extrapolate the arguments used to assess the efficiency with which NGDOs operate.

Assessing efficiency
Assessing the efficiency of non-profits organisations such as NGDOs and their instruments -in this case, cooperation for development projects -involves certain difficulties because the objective is often more difficult to identify and measure than the objective of a firm 4 .In addition, because projects have multiple dimensions (project quality, relevance to the needs they serve, sustainability, impact, etc.), the production function can be difficult to determine.
Proposed methods for evaluating the efficiency of NGDOs are commonly based on accounting measures, mainly ratio analysis, and include price, programme ratio, technical efficiency and allocative efficiency.Price is defined as the cost, for a donor, of purchasing a dollar of output for an organisation's beneficiaries.Without accounting for tax deduction, price is measured as the inverse of the percentage of expenditure on projects (Callen 1994;Posnett & Sandler 1989;Tinkelman 1998;Weisbrod & Dominguez 1986).Some studies use programme ratio represented as the percentage of project expenditures to total expenditures to measure efficiency (Baber, Roberts & Visvanathan 2001;Roberts, Smith & Taranto 2004).Baber et al. (2001) argue that this measure may indicate the fundraising strategy of the nonprofit organisation.Allocative efficiency, a very similar and also widely used measure (Andrés-Alonso, Azofra-Palenzuela & Romero-Merino 2010; Andrés-Alonso, Martín-Cruz & Romero-Merino 2006;Callen & Falk 1993;Hernangómez et al. 2009), calculates the percentage of project expenditures to total revenue of the entity.Technical efficiency or administrative expense ratio (Andrés et al. 2006(Andrés et al. , 2010;;Callen & Falk 1993;Greenlee & Brown 1999;Hernangómez et al. 2009) is defined as the percentage of administrative expenses on total expenditures.
However, these measures do not correspond to the concept of efficiency as previously defined -that is, the relation between the objectives achieved and the resources used -because they address neither the level of achievement of the objectives nor the level of resources applied.They only provide information on the extent to which a non-profit organisation devotes the available resources to its mission, which corresponds to the definition of efficiency established by Parsons (2003); that is, these definitions commonly show the average percentage of each donation that reaches the beneficiaries of the organisation.Despite their limitations, these indicatorsespecially technical and allocative efficiency -are widely used by both donors (Hyndman 1991;Khumawala & Gordon 1997) in their donation decisions as well as by the organisations themselves to show an image of transparency and fulfilment of their mission in hopes of generating greater fundraising.The use of these indicators is based mainly on its simplicity, easiness of obtaining information and the possibility to make uniform comparisons among entities.
In comparison with accounting methods, frontier analysis is becoming more and more important.Frontier analyses use the construction of efficient frontiers as a means to measure the relative efficiency of a group of organisations.The notion of the existence of a maximum achievable level underlies this methodology.Efficient frontiers can be calculated using either parametric or non-parametric techniques.Parametric techniques require the specification of a functional form of the production function to estimate (i.e.regress) the optimum relation between outputs and inputs.These techniques are very sensitive to the specification of the production function.
When the functional form is not correctly determined, results are significantly biased.Thus, the efficient frontier method is only recommended if a specification of the production function close to the underlying production technology can be obtained.
Non-parametric methods use DEA to determine the efficient frontier based on a global efficiency indicator that relates objectives (outputs) and resources used to obtain these objectives (inputs).Therefore, DEA allows us to measure the relative efficiency of homogeneous decision-making units (DMU) while including a large number of variables and relations (constraints).This method is particularly suitable for the non-profit sector for two reasons.Firstly, DEA is especially adaptable to the features of non-profit organisations, in which the pricing of products and services is difficult, production function is not easily defined and multiple inputs and outputs are at play.Secondly, DEA establishes an objective indicator of efficiency for groups of organisations that do not have aggregate indicators and considers the multiple simultaneous objectives that nonprofits try to achieve (Hernangómez et al. 2009).

Data envelopment analysis
Developed by Charnes, Cooper and Rhodes (1978), data envelopment analysis (DEA) consists of a linear programming technique that evaluates the relative efficiency of different DMU.The method identifies the efficient 5 units and builds a frontier shaped by the linear combinations between these units.The efficiency of other units is measured by their distance from the estimated efficient frontier.
The analysis is formulated as a mathematical optimisation problem in which the estimated variables are the efficiency indices -defined as the weighted sums of outputs and weighted sums of inputs ratio -and the corresponding weights.It is a non-parametric technique because it does not specify a functional form between the maximum achievable outputs and required inputs.Thus, the application of this methodology makes it possible to obtain an efficiency indicator for each DMU that varies between zero and one (with one as the indicator of those DMU located in the frontier).In addition, the weights are endogenously determined, with their values corresponding to the maximum achievable efficiency.Therefore, this methodology allows us to compare the efficiency level of different DMUs, assists in identifying best practice and objectively determines the productive improvement and overcomes the problems associated with accounting and parametric frontier methods.
Several DEA models exist according to different assumptions about production technology and the related restrictions incorporated.Figure 1 provides the mathematical formulation 5.Pareto efficiency criterion is used: A unit is efficient if no others obtain higher levels of one output without producing less of another output and without increasing the use of any inputs.Or, if no unit produces the same amount of outputs using less of some input without increasing the use of others (Charnes et al. 1978).

Output orientaƟon: I nput orientaƟon:
of the basic model, assuming constant returns to scale for output and input orientation.Where ϕ 0 and φ 0 represent the efficiency indicators for each orientation, y rj and x ij represent the amount of output r and input i of DMU j, ur 0 and vi 0 represent the weights of output r and input i.Finally, ε is a small enough-positive value which the weights cannot fall (Basso & Funari 2004).
As previously mentioned, multiple outputs and inputs can be included in the analysis.However, as the number of included variables increase, the discriminating power of the DEA 6 decreases.In addition, no tests can determine the statistical significance of the included variables or evaluate the goodness of fit.Because of these considerations, the general recommendation is that the number of inputs and outputs should not exceed one-third of the sample size (Banker, Charnes & Cooper 1984;Banker et al. 1989;McMillan & Datta 1998).Moreover, as DEA estimates relative efficiency, the DMU included in the analysis must be homogeneous because of the sensitiveness of this technique to extreme values.That means that they must apply the same production technology and operate under the similar institutional framework.
A final consideration is related to the choice of analysis orientation (output vs. input).This option depends on the nature of the problem and is related to the ability of organisations to control variables.If the analysed units are restricted in the management of inputs, output orientation is chosen, and vice versa.In the output-oriented approach, a unit is defined as efficient when no other unit can produce a higher level of output using a lower level of input.Likewise, for an input-oriented approach, a unit is defined as efficient when no other unit can reduce the input amount (keeping the proportion) and get the same amount of output.
In sum, we apply DEA to assess the efficiency of ICS.This analysis allows us to measure efficiency -defined by the relation between the objectives achieved and the resources used -and to include multiple inputs and outputs without defining the functional form of the production function.Furthermore, DEA makes possible valid comparisons among projects, and the available data meet the methodology requirements.

Applying DEA to evaluate efficiency of the non-profit sector
We are motivated to apply DEA to the non-profit sector to evaluate the efficiency of non-profits to produce the goods and services provided to beneficiaries with the donated resources simultaneously considering their multiple objectives.This methodology is particularly applicable because it allows us to identify the input reduction necessary to achieve efficiency (McMillan & Datta 1998) as well as the levels of inefficiency of different units (Charnes et al. 1994;Farrel 1957;Farrell & Fieldhouse 1962) and their causes.
6.A greater number of variables relative to sample size increase the likelihood to be efficient in any of the variables for the analysed units.
Although a functional form of the production function is not necessary, DEA does require that the relevant outputs and inputs of the evaluated activity be established.The most common approach to making this determination is the development of a list of outputs and inputs, identified either from observation of evaluated units or from a literature review.From this list, the most suitable outputs and inputs are selected, considering the preservation of DEA discriminating power (García & Marcuello 2007;Hernangómez et al. 2006Hernangómez et al. , 2009;;Marcuello 1999;Martín et al. 2005Martín et al. , 2007)).
Output determination requires the definition of the objectives to be achieved.Regarding the mission of the NGDOs, two main objectives have been considered, such as the eradication of poverty in the non-developed countries and education for development in Northern societies.In line with these objectives, prior empirical studies that have applied DEA to the field of international cooperation have used project expenditures (Golden et al. 2012;Marcuello 1999), total funds raised (García & Marcuello 2007;Golden et al. 2012) and the number of projects (Hernangómez et al. 2006(Hernangómez et al. , 2009;;Martín et al. 2005Martín et al. , 2007) ) as outputs.

Method Sample and information sources
The objective of this study is to analyse the efficiency of the ICS for Spanish ODA instruments applied in two African countries with a wide association (i.e. a long-term framework and the application of different kinds of ICS are possible), Morocco and Mozambique.These countries are considered with top priority for Spanish cooperation aid.The study period extends from 2002 to 2006.This term of 5 years includes the latest available data at the collection moment (June-December 2009) and is long enough to draw conclusions from the analysis.Because of the lack of a database, information was obtained at the headquarters of the AECID using information reports from NGDOs, AECID technicians and the audits of performed projects as well as in-depth interviews with technicians of the analysed countries.No database collects information about the ICS management.Therefore, we were required to build a new database with information provided by the AECID.In so doing, we needed to solve some problems.Firstly, no protocol exists for monitoring these instruments.As a result, quantitative information is not always available for all projects, and it is not possible to obtain all the items identified for all sample projects.In addition, the period of execution is commonly extended and delays occur.Consequently, the sample includes projects currently underway and projects for which information gaps exist.
Secondly, AECID has an established set of accounting items for budgeting ICS (see Appendix 1).In the database used to perform DEA, we set a correspondence between those items and the items associated with the National Public Accountancy Plan (NPAP).Thus, identified inputs correspond to the NPAP items.

Variables
In the section titled 'Applying DEA to evaluate efficiency of the non-profit sector', we presented the main variables used to measure the efficiency of NGDOs as well as their chief objectives and the main resources used in achieving them.However, we are not trying to assess the efficiency of the actual NGDO missions but rather the efficiency of the specific instruments used to achieve these missions.Therefore, we must conceptualise the objective of these instruments to define the outputs as well as the main resources to determine the inputs to be included in the analysis.Martín et al. (2012) argue that the objective of international cooperation projects performed by NGDOs is the maximisation of social welfare of the population or area where the project is being carried out.In this line, we define the objective of the ICS as meeting the specific need for which they were created for many people as possible.Then, we establish a list of dimensions that make this objective operative and for which data are available.Our list consists of the number of activities, beneficiaries, areas, families, and organisations and duration in months 7 .
When determining the dimensions to measure the output, we considered the availability of information and level of importance that AECID technicians gave to these dimensions.Our goal is to select dimensions with information available for as many instruments as possible so that we can make significant DEA comparisons.In addition, we asked 15 AECID technicians, who are responsible for 7.The time spent in a manufacturing process is taken as an input and, in the comparison of processes, efficiency is rated by the amount of time spent at the same cost: A more efficient process requires less time but equal cost.However, we consider real time by which duration is considered an output.That is, because a longer contract period extends the project's capacity to benefit the target population, more time at the same cost equals greater efficiency.Therefore, a realtime analysis assumes that between two projects with equal cost, a donor will prefer to finance the one that requires more time to perform.ex-ante evaluation, monitoring, supervision and ex-post evaluation of the different instruments, to rate the importance of the chosen dimensions on a scale from least important to most important.Table 2 shows the results.Unfortunately, the two dimensions most valued by the AECID technicians have the lowest level of information; therefore, data availability is the primary criterion for the selection of outputs, which dictate that we focus on duration, activities and areas served.
We select the following as the main inputs to measure the efficiency of ICS: purchases of raw materials, staffing costs (salaries), communications and other services, services from independent professionals, purchases of other supplies, repairs and maintenance, leasing charges and fees, and subsistence allowance.In selecting the inputs for the efficiency analysis, we assume that those inputs used by most instruments are the ones that endow a higher discriminating power to DEA because they allow for a more homogeneous comparison.Table 3 shows that most often used inputs are purchases of raw materials, staffing costs, and communications and other services -all of which are used by over 90% of the instruments included in the sample and together represent 91% of total executed expenditures.
In sum, the production function (the base scenario) with which we assess the efficiency of ICS includes purchases, wages and salaries, and communications and other services as inputs and activities performed, and areas served as outputs.We opt for an output orientation to perform DEA because we assume that, given a certain resource level (i.e.available budget), each instrument will attempt to maximise results (i.e. to reach the largest number of individuals or areas, perform the most activities, etc.).

Results
In this section, we present the results of the DEA analysis performed by means of the previously discussed outputs and inputs.The sample consists of 48 instruments, of which 18 are POC subsidies and 30 are NGDO subsidies.Firstly, we assess the efficiency of the two types of ICS.Then, we analyse the efficiency with which these instruments are implemented in each of the two priority African countries for Spanish ODA included in our sample (general model).These results are subjected to a sensitivity analysis, in the first place by removing the extreme cases (the most efficient and the least efficient instruments) and secondly by performing the DEA analyses with different inputs specification, to assess the effects from inclusion or exclusion of certain cost categories.Finally, we complement our analysis with two additional models (alternative models 1 and 2) to measure the consistency of results and to check the robustness of the analysis.Table 4 shows the inputs and outputs for the three models.Appendix 2 reports the results for the two alternative models.(Hernangómez et al. 2006(Hernangómez et al. , 2009;;Martín et al. 2005Martín et al. , 2007)).
Although these results allow us to compare the efficiency of different instruments, we continue our analysis by applying Families 4 2 †, From most important (1) to least important (6).Source: Martín, N., Martín, V. & Gámez, C., 2012, 'La eficiencia y el "misreporting" contable en las ONGD Españolas.Análisis de los proyectos de cooperación internacional para el desarrollo (Efficiency and accounting "misreporting" in Spanish NGDOs: An analysis of international development cooperation projects)', Academia, Revista Latinoamericana de Administración 51(4), 1-14   our DEA model to a narrower scope.Specifically, we consider independently each of the two countries included in the sample in Panel B of Table 5.This examination allows us to make comparisons within a single environment and thus avoid complications and differences caused by differing contexts.In addition, we can identify any pattern of significant differences across countries for particular instruments.Panel B shows the percentage of efficient instruments and the average efficiency in the performance of analysed instruments for each country.Morocco achieves results both in the percentage of efficient instruments as well as the average efficiency in the performance of analysed instruments that are clearly superior to Mozambique.That is, the results suggest that the different instruments of cooperation for development are implemented with greater efficiency in Morocco.
In Panel C, we further narrow our focus as we now analyse the ICS for Morocco alone, because, with a sample of 30 ICS, it is the only country that fulfils the requirements imposed by DEA and thus allows us to perform the analysis.As shown in Panel C of Table 5, three POC subsidies (27.4%) and five NGDO subsidies (26.3%) are efficient -in both cases, higher percentages than those obtained for the total sample, especially for POC.The average values are also higher than those achieved for the whole sample and show again that POC subsidies is the instrument that achieves the highest levels of efficiency (0.77).
When we estimate the frontier by removing the most efficient (12) and the least efficient (12) instruments, results (see Table 6) are quite similar regarding those obtained with the whole sample; again POC subsidies are more efficient than NGDO subsidies.Average efficiency and percentage of efficient instruments are a bit higher than initial results, but it must be considered that a small sample size reduces the discriminating power of DEA so that the number of efficient units and the average efficiency tend to increase.These results show no significant influence of extreme values on the efficient frontier.
We have also estimated the general model (outputs: activities performed and areas served) with different combinations of inputs, relative to the original model.Purchases, staffing costs, and communications and other services are the cost categories considered because they represent a substantial amount of the instruments' budget (at least 10% of the total expenditure) and are included in almost all instruments (over 90%).The results of the analyses performed with the abovementioned combinations of inputs (see Tables 7-9) are in line with the results of the initial analysis.
Table 7 shows the results of the analysis performed with salaries and purchases as inputs, the most important factors regarding their percentage over total expenses (45% and 34%, respectively).Average efficiency is almost identical relative to the base scenario (0.67 vs. 0.66), and the percentage of efficient instruments is quite similar (27.1% vs. 25%).POC subsidies efficiency is higher than NGDO subsidies efficiency both in the whole sample and in the subsample including only Morocco's instruments.In addition, the average efficiency of the instruments performed in Morocco is higher than the average efficiency of instruments executed in Mozambique.In general, these results give support to those obtained in the base scenario of the general model.Table 8 shows the results of the analysis performed with purchases and other services as inputs.We observe a similar tendency relative to the base scenario: POC subsidies are more efficient than NGDO subsidies both in the whole sample and in the subsample including only Morocco's instruments, and the average efficiency of the instruments performed in Morocco is higher than the average efficiency of the instruments developed in Mozambique.However, in this analysis, the average efficiency of the whole sample and the percentage of efficient instruments are lower than in the base scenario.These findings pose that salaries as input have a considerable influence on DEA results, which is understandable given its importance over total expenses (34%).
Table 9 shows the results of the analysis performed with salaries and other services as inputs.Again, we observe a similar tendency relative to the base scenario: POC subsidies are more efficient than NGDO subsidies both in the whole sample and in the subsample including only Morocco's instruments, and the average efficiency of the instruments developed in Morocco is higher than the average efficiency of instruments performed in Mozambique.Average efficiency of the whole sample and percentage of the efficient instruments are a bit lower than in the base scenario.Again, these findings show that purchases as input have a considerable influence on DEA results, which is understandable given its importance over total expenses (45%).
We have performed two additional analyses to check the consistency and robustness of our results, identified as alternative model 1 and alternative model 2 (see Appendix 2).
Alternative model 1 includes activities performed, areas served and institutions involved as outputs and purchases, wages and salaries, and communications and other services as inputs, whereas alternative model 2 includes activities performed, areas served and number of beneficiaries as outputs and the three aforementioned inputs.The results of these analyses show that among the instruments examined, again POC subsidies are more efficient than the NGDO subsidies and Morocco outperforms Mozambique regarding the level of efficiency achieved in the execution of the ICS.Average efficiency is higher in DEA alternative models than in the base scenario of DEA general model (0.74 and 0.76 vs. 0.66).
Likewise, the percentage of efficient instruments is higher in DEA alternative models than in the base scenario of DEA general model (29.8 and 36.4 vs. 25.0), which is understandable taking into account DEA properties: The discriminating power of the DEA decreases both when the number of included variables increases and when sample size decreases, resulting in better average efficiency and a higher percentage of efficient units.Therefore, these findings give support to the results obtained in the base case scenario of the general model.

Discussion and conclusion
As we pose throughout this study, measures for evaluating international aid projects with respect to efficiency are problematic because there is no evident bottom line, and the measurement of performance, unlike firms, cannot be done in terms of profit or profitability alone.Accounting measures, mainly ratio analysis, do not capture efficiency of output and are subject to measurement error through valid or invalid accounting manipulations (Golden et al. 2012).In a multiple input and output environment as is the case of international aid projects, DEA is an appropriate method because it provides a measurement that can simultaneously identify inefficient projects and also produces information regarding sources of inefficiencies, what inputs are being over-utilised and by how much to make the project efficiency and also identifies a set of efficient project against which to benchmark.Thus, DEA analysis allows for the comparison of international aid projects efficiency across different ICS responding the donors' concern of what types of instruments best allocate the donated resources to the dedicated cause.
Our results show that among the instruments examined within the sample countries, POC subsidies are more efficient than the NGDO subsidies.These findings are substantiated by the results from both alternative models, reported in Appendix 2. Furthermore, when we conduct the analysis exclusively for Morocco -the country in which the greatest number of ICS is put into effect among our sample countriesthe result holds.Therefore, we may infer that this result is considerably independent from the particular environment in which the ICS are performed.
With respect to the efficiency achieved in each country, assessed through the general model and all its different specifications, Morocco achieves a higher level of average efficiency in the execution of the ICS.This result is confirmed by both alternative model 1 and alternative model 2, which qualifies Morocco as the most efficient country.Considering the results of all models, Morocco appears to be the most efficient country, but it is required to take into account that efficiency can vary depending on the model specification (inputs and outputs employed to assess the efficiency).
Findings should be interpreted cautiously given that the relative position of countries in terms of efficiency could be variable and it is not usual that a country would be absolutely more efficient than the rest.
Several implications for the AECID (which is the founder of the analysed instruments) can be derived from these findings.The use of this ex-post measure of efficiency allows for the measurement in a specific and objective way of the results achieved by each project and to propose corrective actions for the future.Likewise, the comparison among ICS provides an opportunity to identify the conditions under which an ICS may achieve greater efficiency.However, DEA is not without drawbacks, for example, its sensitiveness to model specification, so a critical decision is the determination of inputs and outputs because the results could be substantially different if the model is estimated with a different combination of inputs and outputs.
Considering the pros and cons of DEA, the assessment of international aid projects needs to be complemented with other types of analyses to obtain consistent and reliable results.
A solid in-depth qualitative assessment of projects analysing dimensions such as their relevance, effectiveness, sustainability and impact (variables recommended by the DAC to assess project performance) is crucial to the success of the projects.
Although DEA can contribute towards shedding some light on assessing the efficiency of international aid projects, more research is needed to address this important issue.
This study has several limitations.Firstly, the main limitation is the sample size.A greater number of observations and even variety of ICS would be desirable; however, it is extremely difficult to obtain systematic information of inputs and outputs of the ICS because of the lack of a common procedure and computerised collection of information by agencies (in this case, the AECID).Secondly, DEA requires that units included in the analysis must be homogeneous because of the sensitiveness of this technique to extreme values.Although we consider our sample homogenous enough, we are aware that it is always advisable to gather a more homogeneous sample.Thirdly, instruments' data do not allow us to perform DEA analysis with the most relevant dimensions of the output, as they were ranked by the AECID experts.Fourthly, DEA results are sensitive to model specification, so it is very important to include relevant inputs and outputs.Future research may measure the impact of the ICS to improve the quality of inputs and outputs used to characterise the production function.Finally, the analysis period corresponds to a time horizon of 5 years.It would be interesting to extend this period and to perform an analysis by year to assess whether the composition of the selection committee may influence the choice of more or less efficient projects.Appendix 2

4.
Instead of efficiency, recent studies have focused on the outcomes or success of international cooperation projects (Ahsan & Gunawan 2010; Diallo & Thuillier 2004).

TABLE 1 :
Spanish bilateral official development assistance.

Table 5 ,
Panel A, provides a summary of results of the general model.The results show that 12 instruments are efficient, which represents 25% of the total observations included in the sample.In terms of efficiency by type of instrument, five POC subsidies (27.7%) and seven NGDO subsidies (23.3%) are efficient.The average efficiency for each type of instrument -0.70 for POC subsidies and 0.65 for NGDO subsidies -is in line with values obtained by prior empirical studies that apply DEA to the cooperation for development sector

TABLE 2 :
Data availability and importance of the outputs.

TABLE 3 :
Use and importance of inputs.
Note: Top three selected for data envelopment analysis.
POC, permanent open-call subsidies; NS, non-governmental development organisation subsidies.

TABLE 4 :
Models employed to perform data envelopment analysis.

TABLE 6 :
Instrument efficiency without extreme values: General model.

TABLE 8 :
Instrument efficiency: General model (Inputs: purchases and other services).

TABLE 9 :
Instrument efficiency: General model (Inputs: salaries and other services).