Evaluating thE MacroEconoMic iMpacts of iMf prograMMEs in latin aMErica , 1975-2004 : a gEE analysis

Using panel data for 21 Latin American countries over the period 1975–2004, this paper analyses macroeconomic effects of IMF programmes by using GEE methodology. In line with the results of previous studies, it is shown that these programmes have positive effects on the balance of payments, current account and investment. However, they negatively affect budget deficit, per capita GDP, FDI, inflation and consumption. JEL E63, F32, 34, 41, O19


Introduction
The basic conception of the IMF's role, as envisioned at Bretton Woods in 1944, was to promote exchange rate stability and provide short-term finance to deal with temporary current account deficits in advanced countries. Thus, the primary role of the IMF is to provide credits to member countries in balance-ofpayments difficulties. However, with the breakdown of the "par adjustable peg system" in 1973, the IMF lost its major role as the "guarantor of fixed exchange rates" among advanced countries. Nevertheless, the IMF did not disappear, and its role expanded instead into many new areas. The IMF has now evolved into the "crisis manager" and "development financier" for developing countries 1 .
In its 64 years of existence with 184 member countries (out of 191 independent countries in the world) in 2006, the IMF has been criticised because of its institutional structure and lending practices. Some argue that the IMF is a bureaucratic and nontransparent institution with no accountability for its actions. It has also been suggested that Fund-supported stabilisation programmes are ineffective and may create moral hazard.
Although IMF programmes have many different objectives, certain characteristics of Fund programmes are linked to its mandate to confront external payment problems. This is the so called "three-pronged approach" (Knight & Santaella, 1997;Mussa & Savastano, 2000). The first component of this approach involves securing sustainable external financing, since external financing typically evaporates at the onset of a crisis. The IMF assists countries suffering balance-of-payments problems by offering short-term loans through its Standby Arrangements (SBA) and medium-term loans through its Extended Fund Facility (EFF). Another common element of Fund programmes is the adoption of demand-restraining measures consistent with available financing. These first two measures comprise the macroeconomic policies intended to restore a sustainable balance between aggregate expenditure and income in programme countries. The last component of IMF programmes requires structural reforms intended to promote growth and adjustment in the medium and long terms. These policies aim to reduce government related distortions and structural problems which result in an inefficient allocation of resources in the economy concerned, thereby hindering growth.
The multilateral institutions, the IMF and the World Bank, play a significant role in determining the policy dynamics in many developing countries. As a result of the economic reform programmes supported by the IMF, many crisis-hit countries in the 1990s have temporarily succeeded in achieving macroeconomic stabilisation and the existing studies suggest that IMF programmes provide a short-run balance of payments relief to crisis-hit countries 2 . This effort has, however, been accompanied by temporary deceleration of real growth and prolonged recession in some countries.
There is considerable debate over the output and employment effects of IMF supported stabilisation programmes. This controversy seems especially heated for countries facing acute balance of payments problems and currency crises, as witnessed in 1997 in Korea, Indonesia and Thailand and elsewhere. Despite these strong statements about the value of recent IMF programmes, no consensus has emerged regarding the impact of these programmes on the real side of the economy 3 . Most empirical studies using panel data sets and regression techniques find that IMF-supported programmes improve the balance of payments and current account (e.g. Khan, 1990;Conway, 1994;Bagci & Perraudin, 1997;Bordo & Schwartz, 2000), but views on the ultimate output and employment effects are much more divergent (Haque & Khan, 1998).
As mentioned by Hutchison and Noy (2003), Latin America is a volatile region with a history of exceptionally high inflation rates, substantial macroeconomic instability, and a record of unsuccessful monetary and fiscal stabilisations. Latin American countries are also the most frequent users of IMF loans (prolonged users) and associated IMF-supported stabilisation programmes. In addition, IMF programme failures/cancellations are more frequent in Latin America and the output costs of completing a programme, when it does occur, are substantially higher than in other regions. The negative effects of the IMF programmes may be entirely explained by the poor performance of Latin American countries when they enter into an IMF-sponsored stabilisation programme. Further, there is little evidence of substantial improvement in the current account when entering into an IMF programme once the effects of sharp currency depreciations preceding programme approvals are taken into account.
As stated by Hausmann and Gavin (1996), the inflation in Latin America is more than 100 times as volatile (measured in standard deviations) than in the industrial countries, and is higher by far than in any other region of the world; real GDP growth in Latin America is about two to three times as volatile as industrial countries, and more volatile than in any region other than Africa and the Middle East; and while a "typical" recession lasts roughly two years both in Latin America and in other regions, it involves an output decline in real GDP of about eight per cent in Latin America compared to only about two per cent in industrial economies. Fernandez-Arias and Montiel (2001) also point to especially unfavourable external conditions facing Latin America, when explaining low growth in the region. Linked with macroeconomic instability, most Latin American experiences with stabilization reforms have been disappointing. Ivanova et al. (2003) investigate the implementation of IMF-supported programmes using measures of programme interruptions, compliance with conditionality, and the share of committed funds disbursed. They find that programme implementation depends primarily on the borrowing countries' political characteristics. In particular, strong special interests, political instability, inefficient bureaucracies, lack of political cohesion and ethno-linguistic divisions are correlated with low implementation rates.
To our knowledge, no study has focused on the detailed macroeconomic effects of IMFsupported programmes for Latin America over the past two decades. Owing to the reasons mentioned above, it will be interesting to investigate the effects of IMF programmes in this region.
This paper focuses on the effectiveness of fundsupported stabilisation programmes for 21 Latin American countries to establish whether the IMF achieves its own goals in these programmes. It also provides a discussion of alternative evaluation methods and their weaknesses before considering the selection of the evaluation method. The method of programme evaluation is based on the observation of relevant variables during pre-programme, programme, and postprogramme years. During the sample period, 21 Latin American countries which were provided with any of the four structural adjustment programmes: Standby Agreement (SBA), Extended Fund Facility (EFF), Structural Adjustment Facility (SAF), and Enhanced Structural Adjustment Facility (ESAF) were studied 4 .
The organization of the paper is as follows. Section 2 reviews the literature on the effects of IMF programmes and discusses the evaluation methods. Section 3 describes methodology, data and model, while Section 4 presents the empirical results. Finally, Section 5 concludes the paper.

Literature survey
In previous decades, one of the central queries has concerned whether the IMF stabilisation programmes have accurately achieved their broad objectives. For this purpose, a number of studies have been conducted to measure the effects of IMF stabilisation programmes on beneficiary countries' main macroeconomic indicators, such as the rate of inflation, the current account, overall balance of payments (BOP), and the rate of real economic growth, by means of using different approaches.
This section undertakes a review of the existing literature with regards to the effects of IMF programmes on macroeconomic indicators. Whether the IMF indeed influences macroeconomic performance has been subject to a huge number of studies 5 . In the literature, the most commonly employed approaches to measure the impact of the IMF supported macroeconomic stabilisation programmes have consisted of the "before-after approach", "the with-without approach", "the actual-versustarget approach", "comparison of simulations" and "generalised evaluation estimator (GEE) approach" 6 . Table 1 depicts a summary of the empirical studies relating to measurement of the effect of IMF programmes on macroeconomic variables by using different approaches. With regard to the growth effects of these programmes, the literature is far from reaching consensus. Haque and Khan (1998) find negative effects on growth over the short run, probably owing to the demand-restraining nature of IMF conditions, while over the long run, growth increases. Alternatively, Bird (2001), Goldstein and Montiel (1986), and Bird and Rowlands (2003) find that Fund programmes have a negative effect on growth. Conway (1994)  ). Yet, there is still no consensus about the growth effects of IMF lending. These conflicting results may arise from several sources, including differences in the types of IMF programmes that are investigated; differences in the groups of countries that are investigated (e.g., impoverished developing versus emerging market economies or transition economies); differences in the methodologies that are employed, and the way in which other factors influencing output growth are taken into account 7 .
The discussion of evaluation methods indicates that all types of programme evaluations are problematic, and a perfect solution to the problems of programme evaluation does not exist. However, the recognition of these problems is important with regard to the selection of the evaluation method and the interpretation of evaluation results.
Empirical studies analysing the effects of IMF programmes have employed a variety of methodologies. As discussed in many studies (Barro & Lee, 2003;Bird, 2001;Haque & Khan, 1998), some of these methodologies possess important shortcomings. In the literature, five types of approaches for the assessment of the macroeconomic outcomes of fund-supported programmes are found to be very common: • The before-after approach This compares the macroeconomic performance of an economy before and after the adoption of fund supported programmes 8 .
• The with-without approach The macroeconomic outcomes of fundsupported programmes are compared by differentiating the programme and nonprogramme countries 9 .

• The comparison of simulation approach
This relies on the simulations of econometric models to infer the hypothetical performance of the policies included in fund supported programmes and alternative policy packages 10 .

• Actual versus targeted approach
Actual outcomes for certain key macroeconomic variables are compared to their respective targets, for the variables specified by the authorities and the fund at the inception of the programme 11 .

• The generalised evaluation estimator approach
This compares the macroeconomic performance of the programme and nonprogramme countries, adjusting for the initial differences and condition among the countries and controlling for exogenous influences.
The recognition of the inherent biases in the "with-without" and "before-after" approaches led to the creation of the generalised evaluation estimator approach (GEE) which modifies the "with-without" approach in two ways. First, it accepts the non-random selection of the countries, and identifies the specific differences between the programme and non-programme countries in the pre-programme period. Second, it attempts to capture the effect of policy, exogenous shocks and other variables on the macroeconomic outcomes, taking into account the way in which policies would have evolved in the absence of the programme. The reaction function captures the effect of the policy. This function brings those policies into consideration that might have been adopted in the absence of the programme. The reasoning behind the counterfactual approaches is that either they compare the before and after situation in the economy or compare the sample of the programme countries with the sample of those countries that are not involved in the programme. At the same time the given country cannot be in both situations (programme and nonprogramme). Therefore, the reaction function is estimated by taking the difference between the vector of the desired values of target variables and the vector of actual values of the target variables in the last period. Then, by regarding the exogenous shock as an explanatory variable in the model, it provides a good measure for analysing their impact on the target variables. Its application to the case study also overcomes the two well known limitations, selection of random countries, and the problem of the degree of programme implementation, that is, the willingness of governments to implement certain programmes. Second, technically the methodology used in this study overcomes the problems associated with the "with-without", "before after" and the other above discussed techniques, so that the IMF estimator estimated with this technique does not provide biased results. Consequently it is known as the comparatively better technique to evaluate the fund-supported programmes. This technique is capable of serving the objectives set by this study. First, by taking economic growth, current account balance, inflation rate, budget deficit, balance of payments, consumption, foreign direct investment (FDI) and private investment as the target variables, we can estimate the impact of the fundsupported programmes on the macroeconomic performance of Latin American countries. Second, by including the reaction function in the model it also deals with the difference between the targeted and actual outcomes and furthermore concentrates on the effect of the other policy options that are not included in the programme design by taking the vector of the policies which might have been adopted in the absence of the programmes. Third, by including the vector for the exogenous variables, their effect on the target variable is also incorporated. The methodology that has been most widely applied was developed by Goldstein and Montiel (1986) by adapting techniques from the literature on labour training evaluation; it was employed for the evaluation of IMF programmes in this study. Essentially, this technique, referred to as the General Evaluation Estimator (GEE) or modified control group, involves using policy reaction functions estimated for countries that did not have support from a particular International Financial Institution (IFI) to approximate the counterfactual for countries that did enjoy IFI backing for their programme 13 . The GEE is a potentially powerful technique, although, as Goldstein and Montiel (1986) point out, it entails many restrictive assumptions; for example, it must be possible to characterise macroeconomic policy choices in a relatively simple reaction function based on quantifiable data, and it must be credible that the reaction functions estimated for countries that do not receive IFI support describe the counterfactual for countries that do receive such support.
Suppose that the target variable is determined according to: where Y ij is the jth target variable (e.g. current account balance, economic growth and inflation rate) in country i; x ik is a k-element vector of policy variables (e.g. the exchange rate, fiscal deficit, domestic credit, inflation rate) that would be observed in country i in the absence of IMF support; w ih is an h-element vector of foreign exogenous variables (e.g. the terms of trade, international interest rates) for each ih country i; d i is a dummy variable equal to 1 if an IMF programme is in effect during the period in question and zero otherwise; and ij f is a zero mean, fixed variance, serially uncorrelated error. For the jth target variable, jk b and jh a are kxl and hxl vectors, respectively, of fixed parameters. The parameter IMF b measures the effect of the programme during this period on variable yij.
It is important to note that the definition of b means that the x i refers to the policies that would have been adopted in the absence of a programme. The vector x i is therefore directly observable only if there is a fund-supported programme; for a non-programme country x i must be estimated. One way in which x i can be estimated is via the simple reaction function: Where y i is a vector of target variables, y i d is the vector of their desired values, c is an adjustment parameter, i h is a vector of random shocks, and D is first-difference operator. Equation (2) basically states that the change in the country's macroeconomic policy instruments between the current and previous period will be the function of the difference between the desired values of the target variables during this period and their actual values during the last period.
The model can be employed to examine the statistical properties of the before-after (BA) and with-without (WW) approaches to an estimation of the programme effects. The before-after estimator BA b is: Where P denotes the set of programme countries during the current period. The expected value of this estimator, conditional on observed values of the foreign exogenous variables, is: Which is equal to the true value of IMF b only if: The before-after estimator is unbiased if one expects that the non-programme determinants of y i would have behaved in such a way as to leave the y i unchanged, on average, between the programme and the non-programme periods. In other words any changes in the external market, innovation in policies, and other unobserved variables cancel each other out.
The WW estimator ww b is given by: Where ynj D is the average value of the yij D over some set N of non-programme countries. Since we can observe x D and ij Df for all i N ! , the information set, defined as X, now consists of: Taking expectations of ww b conditional on i P ! and X we have: The WW estimator will be unbiased if: In other words, if it can be expected that in the absence of the programme, the country would have behaved just like the average member of the non-programme reference group, then the estimator will be unbiased.
An alternative to the BA and WW approaches can be derived by using equation (2) to substitute out the unobservable policy changes that would occur in the absence of a fund programme (i.e. for x i ) from equation (1). The generalised evaluation estimator is: Equation (9) constitutes the basic GEE reduced form model as applied in earlier studies. Econometric estimation of equation (9) produces an estimate of the IMF b that is not subject to the criticism levelled at the BA and the WW estimators. This equation takes care of the estimation of the counterfactual by controlling for the factors that are systematically related to the policies that would have been followed in the country without the programme, which is to include the lag values of the target variables and the policy instruments in the specification (Haque and Khan, 1998: 21-3). The equation would be estimated by OLS estimation.

Empirical results and discussion
The model is nested so as to test the simultaneous effect of: a) the IMF programmes, b) policy shocks, and c) foreign exogenous variables. It serves our objective of diagnosing the impact of fund-supported programmes, while taking into account the effect of other policy options that might have been adopted in the absence of the programmes and foreign exogenous variables. The model takes the IMF programmes, other policy responses and the foreign exogenous variables as different explanatory variables, which makes it easier to gauge the net effect of the fund-supported programmes on the target variables. The targeted macroeconomic variables used in this study are the current account balance, balance of payments, budget deficit, GDP growth, FDI, inflation rate, private investment and consumption.
Equation (3) is estimated for all eight-target variables, discussed above. As we deal with panel data, the Panel Data Unit Root Test is used to check for stationarity. The results suggested that the data is stationary for all the target variables at their levels. The final results find no auto or partial autocorrelation, while the White-Heteroschedasticity test shows no evidence of heteroschedasticity.
The statistical parameters for the overall significance are represented in Table 2. The values of R 2 for the equations of the per capita GDP, balance of payments, budget deficits and current account balances are 0.99, 0.38, 0.36 and 0.28 respectively. However, the R 2 is quite low for foreign direct investment, consumption, investment and inflation.
The regression results for the current account balance, balance of payments and investment indicate that the effect of the IMF programmes is positive. Their coefficients have positive signs and the results are statistically significant. However, the results of the inflation, foreign direct investment, budget deficits, consumption and per capita GDP indicate that the IMF programmes have negative effects. Their coefficients display negative signs and the results are statistically significant.
The regression results indicate the positive impact of IMF programmes on the balance of payments, as expected. In other words, IMF programmes significantly improve the balance of payments during the programme years. The balance of payments estimator provides the result that IMF programmes have contributed to the balance of payments balance. The parameter of the IMF (0.60), with positive sign and statistically significant, shows that IMF programmes have brought about a considerable decrease in the balance of payments deficit.
The inflation estimator provides the result that IMF programmes have contributed to the decrease in the rate of inflation. The parameter of the IMF (-22.7) shows that IMF programmes have caused a considerable decrease in the rate of inflation, which is a chronic disease for the economy. The results of the investment and current account balance indicate that the IMF programmes lead to positive effects. Therefore, the IMF programmes have increased the investments and improved the current account balance of the Latin American countries during the programme years. On the other hand, Fund-supported programmes cause a deterioration in the growth and budget deficits, reduce foreign direct investment and the level of consumption in programme countries. The fall in consumption and inflation can be explained in terms of the programmes applied by the IMF because the IMF recommends tight fiscal policy, tight monetary policy and devaluation to the programme countries.  The regression estimates were obtained using an ordinary least squares (OLS) procedure. Standard errors and t-statistics of coefficients are computed using White's heteroschedasticityconsistent variance-covariance estimator.

Conclusion
This paper analyses the macroeconomic effects of IMF sponsored stabilisation programmes by using the GEE methodology. The data used in this study refer to 21 Latin American countries that were recipients of IMF programmes during the period 1975-2004. Four types of balance of payments programmes are considered: Standby Agreement (SBA), Extended Fund Facility (EFF), Structural Adjustment Facility (SAF), and Enhanced Structural Adjustment Facility (ESAF). Macroeconomic variables used in the empirical analysis include the current account balance, balance of payments, budget deficit, GDP growth, FDI, inflation rate, private investment and consumption.
The results of the GEE analysis indicate that IMF programmes significantly improve current account, balance of payments and investment during the programme years. Although stabilisation programmes seem to provide short-term relief in balance of payments, these improvements are not sustained during the post-programme period. However, these programmes exert a negative impact on budget deficit, per capita GDP, FDI, inflation and consumption in programme countries. The study suggests that Fund-supported programmes have worsened the budget deficits, per capita GDP, FDI and level of consumption in Latin American countries during the programme years. The findings of this study and previous studies in the literature surveyed suggests that IMF sponsored stabilisation programmes are unsuccessful, and these results indicate that Latin American countries have been implementing inconsistent macroeconomic policies.
The empirical results suggest that, when a country begins with a balance of payments crisis and IMF involvement enables it to overcome this, the macroeconomic situation deteriorates. The failure of stabilisation programmes and poor performance of Latin American countries can be explained by the following factors: the lack of policy credibility, inadequate financial structure, political instability, lack of political cohesion, external shocks combined with poor institutions, inefficient bureaucracies, a history of poor macroeconomic management, low completion of IMF programmes, and the nature of IMF sponsored stabilisation programmes. Considering the revolving nature of the Fund support, this result is inconsistent with the effectiveness of stabilisation programmes and may be interpreted as being a signal of moral hazard.  1978). It compares the macroeconomic performance before and after the adoption of a fund-supported programme, assuming all other factors remain constant. It has the advantage of ease of calculation but whenever the other factors that are assumed constant by this approach affect the macroeconomic position of the economy, it fails. This is because the fund-supported programme is one of those exogenous shocks that hit the economic variables. Hence if there are other factors such as terms of trade, industrial growth, movements in the interest rates etc., along with the fund-supported programme that affect the economic variables of the country, they produce biased results. 9 This is another counterfactual approach, which tried to overcome the drawback of the before-after approach as this takes the panel of the programme and non-programme countries and assumes that both these groups of countries have the same nonprogramme determinants. Though it overcomes the problem of the before-after approach, it also leads to a few inherent problems. The assumption of the same non-programme determinants existing between the programme and non-programme countries is quite unrealistic, because the countries selected are not taken randomly; in fact they are selected for having a poor economic performance prior to the programme. This implies that the programme countries had a weaker position prior to the programme than the non-programme countries. Thus the macroeconomic determinants between these two groups of countries would not be the same, and in this position the "withwithout" approach would produce biased results. 10 This approach differs from the other two in that it does not consider the actual outcomes of the programme but it relies on the econometric model to incorporate the impacts of the fundsupported programme. Khan and Knight (1981) and Khan and Knight (1985) have gauged the macroeconomic impacts of the fund-supported programme. But this approach attracted a famous Lucas critique that the actual effects can turn out quite different from the simulated ones. Second, due to credibility factors, the effect of the policy can be different when it is implemented inside and outside the fund-supported programme. 11 This is not one of those approaches that are frequently used in the literature of the fundsupported programsme. It compares the targets set by the programmes for certain variables of interest to the actual outcomes of the programme. The drawback of this is that the targets are mostly not available to the public. It is also deficient because targets may be overly ambitious, so that failure to achieve them does not necessarily imply that the adjustment programme was not effective. Conversely, the targets could be under ambitious so that exceeding them does not always mean that the programme was effective. 12 Country selection is driven by data availability.