No credit rating methodology currently exists for any of South Africa’s sub nationals.
To develop a generic, quantitative credit rating methodology for the Department of Health and the Department of Education
A comparison between generic and specific subnational credit rating methodologies to assess which fits the South African subnational environment best. Studies and results obtained from other nations were used to construct the approach.
In a typical credit rating methodology, both quantitative and qualitative information is considered. In South Africa (as a developing economy), the quantitative information equates to a smaller portion of the final credit rating. A generic quantitative credit rating methodology, as well as specific credit rating methodologies, was developed. The appropriateness of these generic and specific models was tested with regards to prediction accuracies using Red, Amber or Green (RAG) statuses on a traffic light series. An illustration of the predicted versus actual ranks is provided, as well as an example to illustrate how model-predicted RAG statuses, based on quantitative information, may be overlaid with more recent qualitative information to derive a final ranking.
A generic, quantitative credit rating methodology for the Departments of Health and the Department of Education combined was developed, as well as specific credit rating methodologies for each department separately. The specific subnational credit rating methodology outperformed the generic methodology considerably; more precisely, the generic models predicted a maximum of 50% of the new cases correctly as opposed to the specific Health and Education models’ 78%.
The primary contribution of this study was to develop and compare generic and specific subnational credit rating methodologies. A further contribution was to test the appropriateness of these models’ prediction accuracies using RAG statutes. The specific subnational credit rating methodology was found to outperform the generic methodology considerably.
Credit risk, defined by Scott (
Subnationals are defined as all tiers of government and public entities below the sovereign or national government, including states, provinces, counties, cities, towns, public utility companies, school districts and other special-purpose government entities that have the capacity to incur debt (Liu & Waibel
It is evident from the review of subnational credit rating methodologies documented by Fourie et al. (
Credit rating methodologies can be divided into generic and specific credit rating methodologies. The specific credit rating methodologies are developed to credit rate a specific population (e.g. a specific country’s subnationals) and the generic methodologies are developed to credit rate a wide range of different types of populations (e.g. a number of different countries’ subnationals).
In a typical credit rating methodology, both quantitative and qualitative information will be considered and, specifically in South Africa as a developing country, the quantitative information will equate to a smaller portion of the final credit rating.
The primary contribution of this study is to develop and compare a generic with a specific credit rating methodology for South Africa’s subnationals in order to predict payment behaviour, since no credit rating methodology currently exists for South Africa’s subnationals as indicated by Fourie, Verster and Van Vuuren (
The article is organised as follows: The ‘Literature review’ section provides a literature overview of existing subnational credit rating methodologies as well as South African laws and regulations applicable to subnational borrowing. The ‘Data and methodology used’ section describes the data and methodology used. The ‘Results’ section applies the methodology to develop a proposed generic quantitative credit rating methodology for the Departments of Health and Education combined, as well as specific credit rating methodology for each department separately. These two departments were selected because South Africa’s National Treasury (National Treasury) indicates that these two departments’ total expenditure equates to the biggest portion of the provinces’ total expenditure. In the 2015–2016 financial year, the Department of Health’s total expenditure equated to 32% of the provinces’ total expenditure and the Department of Education’s total expenditure equated to 41%, thus a combined proportion of 73% (National Treasury
The literature overview summarises the required background information and is documented in three sections. Firstly, examples of existing generic and specific subnational credit rating methodologies are provided in the ‘Existing subnational credit rating methodologies’ section. The ‘South African laws and regulations’ section documents South African laws and regulations applicable to subnational borrowing. This is followed in the ‘Data and methodology used’ section by a discussion on the data and the methodology used to develop credit rating methodologies for South African subnationals.
A credit rating is a grading of a borrower’s ability to meet its financial obligations in a timely manner (Scott
Credit rating methodologies are required to rate the creditworthiness of issuers and issues. As mentioned earlier, these methodologies can be divided into generic and specific credit rating methodologies. Some of the advantages of generic models in general are that they are freely available and less expensive. One of the advantages of specific models is that they are more accurate at predicting outcomes (Burazeri, Dhuci & Kufo
The three largest international credit rating agencies are Fitch Ratings (Fitch), Moody’s Investors Service (Moody’s) and Standard and Poor’s Financial Services (S&P). These credit rating agencies all generate generic subnational credit rating methodologies that could be used to determine the creditworthiness of any subnational government, regardless of the country within which the subnational government presides. Moody’s, for example, rated 306 regional and local governments in 35 countries outside the United States (Liu & Tan
Theoretically, these methodologies can be applied to South Africa’s provincial governments, departments and municipalities because of the generic definitions of subnationals (i.e. local and regional governments). However, no provincial government or department in South Africa is currently (2016) rated. On the other hand, all 278 of South Africa’s
In this study, proposed generic quantitative credit rating methodologies were developed for South African subnationals, more specifically, the Departments of Health and Education collectively as well as specific credit rating methodologies for each department separately.
The next section pays particular attention to some existing subnational credit rating methodologies, both generic and specific, because a thorough understanding of existing subnational credit rating methodologies is required in order to develop a credit rating methodology for South Africa’s subnationals. This is largely based on the work of Fourie et al. (
The three international credit rating agencies’ subnational credit rating methodologies are recapped in this section. These methodologies are considered to be generic because they could be used to determine the creditworthiness of a number of different subnational governments, regardless of which country the subnational governments reside in.
Fitch Ratings assesses the institutional framework within which the subnational government operates, as this provides the context within which the other factors interact. The four other factors are: debt and other long-term liabilities, finances, management and administration, and the economic status of the subnational government Fitch Ratings (
Moody’s (
S&P employs eight factors to determine the credit quality of a subnational government. Firstly, the institutional framework factor reviews the institutional and legal background of the subnational government. The other seven factors are: economy, financial management, budgetary flexibility, budgetary performance, liquidity, debt burden and contingent liabilities. These focus on the subnational government itself and are used to determine the stand-alone credit quality of the subnational (S&P
Fourie et al. (
A comparison of the subnational credit rating methodologies used by the three major international credit rating agencies provides insight needed to develop a credit rating methodology for South Africa’s subnationals. Liu and Tan (
The The
The five principal or broad factors found in the three international credit rating agencies’ subnational rating methodologies.
This section focuses on existing regional subnational credit rating methodologies. These are considered to be examples of specific credit rating methodologies since it is used to determine the creditworthiness of subnational governments within specific countries or regions, i.e. it applies to specific populations. The two examples of specific or regional subnational credit rating methodologies for
Dominion Bond Rating Service Limited assesses only two different types of factors to determine the credit rating of a Canadian provincial government, namely, province-specific operating risk and province-specific financial risk. This methodology does not apply to the three territories that also form part of Canada’s subnational governments (DBRS
The funds needed to fulfil the principal responsibilities of the Australian subnationals, that is, to provide the necessary public services and infrastructure, are provided by the national government. Thus, Australia Ratings (
Fitch adjusted its international rating methodology to focus specifically on India. Fitch employs the following five factors to determine the stand-alone creditworthiness of Indian subnational governments: institutional and administrative aspects, economic and social profile, fiscal and budgetary performance, debt, liquidity and indirect risk, and management.
Credit Rating Information Services of India Limited (
Note that regional subnational credit rating methodologies never review the
Although, the key factors used in the credit rating methodologies for developed and developing economies will generally be the same, the relative importance of these key factors will differ. The quantitative factors (financial fundamentals) and the qualitative factors (other key factors) included in the rating methodology for a developed country will be viewed as equally important. Thus, both will equate to 50% of the credit rating. In a developing country, the quantitative factors will only equate to 30% of the credit rating and the qualitative factors to 70% (Moody’s
In the absence of a South African regional subnational credit rating methodology, the Public Sector Risk Management Framework of South Africa (PSRMF) could be reviewed in order to gain background knowledge for the development of such a methodology. National Treasury developed the PSRMF in response to the PFMA (Public Finance Management Act No 1 of 1999) to aid provincial governments and departments, among others, in their implementation and maintenance of effective, efficient and transparent systems of risk management and control (National Treasury
Two key findings were drawn from the literature discussed above. Firstly, the variables used in subnational credit rating methodologies can be grouped into five broad factors, namely, the subnational’s economic conditions, the fiscal performance of the subnational, the financial and debt position of the subnational, the management quality and strength of subnational institutions, as well as the influence of the sovereign factors, intergovernmental relations and fiscal arrangements on the subnational.
The other key finding is the difference in the importance of quantitative and qualitative information for developed and developing countries. Typically, for a developed country the quantitative and qualitative information will contribute equally to a credit rating, whereas for developing countries the quantitative information usually equates to the smallest portion of a credit rating.
Both observations will be applied in the remainder of this article. Currently,
Although credit ratings are not assigned to the defined South African subnationals at present, the three largest international credit rating agencies do rate South Africa’s
The National Treasury developed the PSRMF to aid provincial (subnational) governments in the implementation and maintenance of effective, efficient and transparent systems of risk management and control. However, the framework is vague and impractical. National Treasury (
Two concerns when developing credit scoring models, including subnational credit rating methodologies, are data availability and data quality (Siddiqi
In addition to knowledge regarding existing subnational credit rating methodologies, an understanding of the regulatory framework within which subnational governments operate is also required when developing a country-specific subnational credit rating methodology. These laws and regulations should be investigated because these set the parameters within which subnational borrowing may take place.
The South African regulatory framework is summarised in the Borrowing Powers of Provincial Governments Act (No 48 of 1996) – that is, the Borrowing Act – and the PFMA, as well as the regulations that accompany this act, namely the Treasury Regulations for departments, constitutional institution and public entities. The Borrowing Act stipulates the borrowing powers of South Africa’s subnational or provincial governments (South Africa
Each of the acts has a different purpose. South Africa ( to regulate financial management in the national governments and provincial governments; to ensure that all revenue, expenditure, assets and liabilities of those governments are managed efficiently and effectively; to provide for the responsibilities of persons entrusted with financial management in those governments; and to provide for matters connected therewith. (South Africa
It is important to ensure that these laws and regulations are adhered to if it has been decided to extend credit to a subnational (based on a credit rating derived from applying the models to be developed), as non-adherence could lead to unenforceable loan arrangements. An example of such a specification is that any loan raised for the purpose of bridging finance must be redeemed within 30 days after the end of the financial year, making the maximum loan duration 13 months (South Africa,
This section encompasses the data used and the methodology applied in order to develop the proposed quantitative sections of a credit rating methodology. This is largely based on previous work by the authors – refer to Fourie et al. (
The data requirements were based on the information used in existing subnational credit rating methodologies and special focus was placed on the five broad factors identified in the ‘Comparison of the three international rating agencies’ rating methodologies’ section. The final data sets included variables representing the four relevant broad factors (the fifth factor was omitted as all the departments reside within the same country).
Siddiqi (
Firstly, separate data sets were compiled for each department: the Department of Health and the Department of Education. The one interactive data set and 70 different reports containing 5 years’ worth of data for the nine provinces resulted in 45 data points for each of the departments. Secondly, the department-level data were aggregated to construct provincial-level data, also referred to as the
Note that the Health and Education data sets are department-specific, and thus contains data from a specific population. Any credit rating methodology resulting from the department-level data will therefore be a specific credit rating methodology, but any credit rating methodology resulting from the combined data set will be a generic one.
Specific credit rating methodologies were considered because Thomas (
The constructed data sets contain only quantitative information. Therefore, the models developed from these will only relate to the quantitative part of the final credit rating. The qualitative part may be taken into account as an intuitive overlay, as will be illustrated.
Linear regression modelling was chosen to predict the quantitative part of the credit ratings for South African provincial departments. The principal advantage of linear regression is its simplicity, interpretability, scientific acceptance and widespread availability. Linear regression is also widely available in statistical software packages and business intelligence tools (Chambers & Dinsmore
The aim of the models developed in this study is to rank South Africa’s subnationals in terms of future payment behaviour. To forecast using linear regression models, the independent variables used to predict the dependent variable for the next year have to lag by 1 year. This also applies to this work as the aim is to predict the provincial departments’ credit ratings for the next financial year (e.g. 2015/2016) based on the current financial year’s data (e.g. 2014/2015).
The ultimate aim of a credit rating methodology is to predict default. Default can be viewed as a department’s capacity and willingness to repay debt obligations in full and on time. The cash-based accounting system used by the provincial departments (National Treasury
After various workshops and discussions with the National Treasury, no monthly or quarterly variables for the target variable could be found (National Treasury
Three data sets were constructed containing either 42 or 43 (one extra for the combined data set) independent variables. As mentioned previously, the data requirements were based on the information used in existing subnational credit rating methodologies – especially the broad factors identified by Liu and Tan (
The following methodology was applied to develop the 30% quantitative part of a credit rating methodology for the Combined, Health and Education data sets. Multicollinearity was dealt with first by using variable clustering – one variable was chosen from each cluster of variables as the cluster representative. Irrelevant independent variables were removed based on the variables’ correlations with the dependent variable. Lastly, the stepwise variable selection technique and a significance level of 10% were used to choose the final independent variables. The models developed via this process will henceforth be referred to as the
The
Note that Kutner et al. (
Furthermore, one can argue that the models taking into account interactions and excluding outliers will always be the better models when used for explanation purposes. However, the aim of this study was to predict and therefore the developed models need to be able to generalise too. A common pitfall in predictive modelling is to overfit to the data. An overly complex model (which may be the case for Advanced Models 1 and 2) might be too sensitive to peculiarities in the development sample data set and therefore may not be able to generalise well to new data. However, using too simple a model (possibly the Basic Models) can lead to underfitting, where true features are disregarded (SAS Institute Inc.
For these reasons, the results of all three models per data set are reported next and are also later on compared in terms of predictability.
The results of the basic and advanced linear regression models from all three data sets are documented below. The parameter estimate of an independent variable indicates the change in the dependent variable per one unit increase of the corresponding independent variable when the other independent variables are held constant (Kutner et al.
The methodology discussed in the previous section was applied to the combined data set to develop the generic linear regression models.
The
Details of the
Variable | Variable description | Parameter estimate | Description of effect |
---|---|---|---|
ln_x14 | ln (departmental capital expenditure to total provincial capital expenditure) | 1.31 | An increase in the ratio of departmental capital expenditure to total provincial expenditure, ln_x14, leads to a deterioration in payment behaviour. One possible reason for the increase in the ratio is an increase in the department’s capital expenditure. If this is the case, payment behaviour could be affected negatively because the capital expenditure is a long-term commitment and therefore may have an undesirable impact on future expenditure flexibility. |
Indicator_KZN | 1 = KwaZulu-Natal, 0 = all other provinces | −1.54 | The negative parameter estimate indicates that KwaZulu-Natal and Limpopo’s departments are better payers than the other provinces when this model is considered. |
Indicator_Lim | 1 = Limpopo, 0 = all other provinces | −1.49 | |
x27_grouped | Departmental annual appropriation of revenue to total departmental revenue (grouped) | −0.49 | As departmental annual appropriation of revenue to total departmental revenue (x27_grouped) increases the payment behaviour of the department improves. This is indicated by the negative parameter estimate of −0.49. One explanation is that revenue received directly from national government is unwavering, as opposed to some sources of own departmental revenue. Fitch ( |
x52 | Quality of department’s financial reports | 0.47 | The parameter estimate of 0.47 of x52 (quality of department’s financial reports) shows that as the quality of financial reporting worsens, the department’s payment behaviour also worsens (1 = clean audit, 2 = financially unqualified audit opinion, 3 = qualified audit opinion, or 4 = adverse audit opinion and disclaimer of audit opinion). Good financial reporting indicates good financial management. Three other rating agencies also examine the quality, transparency, level of detail, timeliness, frequency and audit opinion of a subnational’s financial reports (DBRS |
Time | Time | −0.15 | Payment behaviour improves as time passes. A possible explanation is that the departments learned from past mistakes and subsequently improved their financial management over time, including repayment of creditors. |
The resulting
Where
The
Details of the
Variable | Variable description | Parameter estimates | Description of effect |
---|---|---|---|
Indicator_Gau_ |
ln_x58 = ln (provincial GDP per capita growth) | 0.90 | An increase in provincial GDP per capita growth will lead to worsening payment behaviour for Gauteng. |
ln_x32_x52 | ln_x32 = ln (departmental capital expenditure to departmental total expenditure) |
0.23 | Payment behaviour worsens as the interaction term increases. Same applies for each of the separate independent variables. An increase in departmental capital expenditure to total departmental expenditure (ln_x32) will be detrimental to payment behaviour. As the quality of financial reporting worsens (x52) the department’s payment behaviour also worsens. |
Indicator_FS_ |
ln_x33 = ln (departmental surplus/deficit to departmental total revenue) | −30.26 | The payment behaviour of the Free State’s Departments of Health and Education will improve if its surplus/deficit to total revenue ratio increases. |
Indicator_Lim_ |
ln x50 = ln (provincial total receipts to provincial total payments) | −55.32 | Limpopo’s payment behaviour will improve if the province’s total receipts to provincial total payments ratio increases. |
Indicator_FS_ |
Indicator x28 Z = 0 if departmental own revenue = 0 |
1.55 | The Free State department’s payment behaviour is worse when the specific department generates own departmental revenue. |
time_ |
ln_x61 =ln (provincial dependent population to total population) | −0.19 | Payment behaviour improves as the interaction term increases. The same applies for each of the separate independent variables. Payment behaviour improves as time goes by, as well as when the ratio provincial total receipts to provincial total payments increases. |
Indicator_Lim_ |
x27_grouped = departmental annual appropriation of revenue to total departmental revenue (grouped) | −0.87 | Limpopo will have better payment behaviour when x27 contains high values (group = 1) compared with when x27 contains low values (group = −1). |
Indicator_Lim_ |
ln_x36_Zmean = ln (departmental net assets to departmental total expenditure) [missing mean] | 0.43 | An increase in ln_x36_Zmean will lead to a deterioration of payment behaviour for the Limpopo province. |
x27_grouped_ |
x27_grouped = departmental annual appropriation of revenue to total departmental revenue (grouped) ln_x50 = ln (provincial total receipts to provincial total payments) | −8.64 | Payment behaviour will improve as ln_x50 increases, provided that x27 is high (group = 1). In the case of low values of x27 (group = -−), an increase in ln_x50 will lead to a worsening payment behaviour. |
The equation of the
The results of the models specifically developed for the Departments of Health and Education are documented next.
The Basic Model developed specifically for the Department of Health (
Details of the
Variable | Variable description | Parameter estimate | Description of effect |
---|---|---|---|
Indicator_KZN | 1 = KZN, |
−2.74 | The negative parameter estimates of Indicator_KZN indicate that the Health Department of KwaZulu-Natal exhibits better payment behaviour than the other provinces’ Departments of Health. |
ln_x28_Zmean | ln(departmental own revenue to total departmental revenue) [missing mean] | 0.34 | As ln_x28_Zmean increases, the payment behaviour of the departments worsens. This is indicated by the parameter estimate of 0.34. A possible explanation is that departmental own revenue is a more volatile source of income than that received directly from national government – depending on the original source of own revenue. Fitch ( |
Indicator_Mpu | 1 = Mpu, |
−1.32 | The Health Departments of Mpumalanga and Limpopo are better payers than the other provinces’ Departments of Health. |
Indicator_Lim | 1 = Lim, |
−1.18 | |
ln_x58 | ln(provincial GDP per capita growth) | 0.90 | The positive parameter estimate of GDP per capita growth (ln_x58) indicates than an increase in GDP per capita growth will worsen payment behaviour when considering this model. This is contradicting Moody’s ( |
The Basic Model developed specifically for the Department of Education (
Details of the
Variable | Variable description | Parameter estimate | Description of effect |
---|---|---|---|
Indicator_Gau | 1 = Gauteng, |
1.76 | The positive parameter estimate of 1.76 indicates that Gauteng’s Department of Education payment behaviour is worse than the other provinces’ Departments of Education. |
ln_x36_Zmean | ln(departmental net assets to departmental total expenditure) [missing mean] | −0.72 | The negative parameter estimate of −0.72 for ln_x36_Zmean illustrates that an increase in the ratio of departmental net assets to departmental total expenditure leads to better credit repayment. Assuming that the increase in this ratio is caused by an increase in net assets, this phenomenon can be explained by the reasoning that good asset and liability management indicates good financial management. Australia Ratings ( |
ln_x57 | ln(provincial GDP per capita) | −1.34 | An increase in GDP per capita (ln_x57) will improve payment behaviour since this will lead to an increase in revenue, which could be used to repay creditors. This is confirmed by Moody’s ( |
Time | Time | −0.21 | Payment behaviour improves as time passes. A possible explanation is that the departments learned from past mistakes and subsequently improved their financial management over time, including repayment of creditors. |
For both the Department of Health and the Department of Education, two unique models resulted from the model advancement investigation. More details of these two models can be found in
Details of the Advanced Models (Health Advanced Model 1 [Model 2]), specifically developed for the Department of Health.
Variable | Variable description | Parameter estimate | Description of effect |
---|---|---|---|
Indicator_WC |
ln_x49 = ln(provincial surplus or deficit/provincial payments) | −19.85 |
Negative parameter estimate shows that increase in ln_x49 (provincial surplus or deficit/provincial payments) leads to an improvement in payment behaviour for the Western Cape. |
x27_grouped_ |
x27_grouped = departmental annual appropriation of revenue/total departmental revenue (grouped) |
−0.62 |
Increase in departmental annual appropriation of revenue/total departmental revenue leads to an improvement in payment behaviour, provided the department’s assets > liabilities. |
Indicator_Lim_ |
ln_x36_Zmean = ln(departmental net assets/departmental total expenditure) [missing mean] | −0.60 |
Increase in this variable leads to a deterioration in payment behaviour for Limpopo. |
Indicator_KZN_ |
ln_x48 = ln(provincial surplus or deficit/provincial receipts) | −32.18 |
KwaZulu-Natal’s payment behaviour improves if the surplus or deficit/provincial receipts ratio improves. |
Indicator_Mpu_ |
ln_x57 = ln(provincial GDP per capita) | −0.15 |
Increase in provincial GDP per capita is beneficial to Mpumalanga’s Department of Health’s payment behaviour. |
Indicator_Lim_ |
ln_x50 = ln(provincial total receipts/provincial total payments) | −65.09 |
Increase in ln_x50 leads to better payment behaviour; this only applies to the Limpopo province. |
Indicator_FS_ |
ln_x33 = ln(departmental surplus or deficit/departmental total revenue) | −22.70 |
The Free State’s Department of Health’s payment behaviour will improve if its surplus or deficit/total revenue improves. |
x27_grouped_ |
x27_grouped = departmental annual appropriation of revenue/total departmental revenue (grouped) ln_x34 = ln(departmental surplus or deficit/departmental total expenditure) | −13.93 |
Payment behaviour will improve as ln_x34 increases, provided that x27 has high values (group = 1). Where x27 has low values (group = −1), an increase in the departmental surplus or deficit/departmental total expenditure ratio leads to worsening payment behaviour. Remainder of values of x27 (group = 0) have no effect because of multiplication with 0. |
The
The equation of the
The equation of the
Both of the
Details of the Advanced Models (Education Advanced Model 1 [Model 2]), specifically developed for the Department of Education.
Variable | Variable description | Parameter estimate | Description of effect |
---|---|---|---|
ln_x57_ |
ln_x57 = ln (provincial GDP per capita) |
−1.38 |
Payment behaviour improves as ln_x60 increases. The interaction between these two variables makes intuitive sense since an increase in total population will ultimately lead to an increase in GDP. This in turn will lead to an increase in revenue, which could be used to support the dependent population within the province. Considering the two variables separately, an increase in GDP per capita (ln_x57) will improve payment behaviour since this will lead to an increase in revenue available to repay creditors. An increase in the proportional dependent population within a province (ln_x60) will lead to an increasing demand for public services, and thus less revenue available to repay creditors. |
time_ |
0.24 |
The payment behaviour of the Department of Education of the Northern Cape worsens as time goes by. | |
ln_x29_ |
ln_x29 = ln (departmental personnel expenditure to departmental total current expenditure) | −2.08 |
An increase in ln_x29 will lead to an improvement in payment behaviour. |
Indicator_Mpu |
ln_x50 = ln (provincial total receipts to provincial total payments) | −75.76 |
Mpumalanga’s payment behaviour will improve if the provincial total receipts to provincial total payments ratio increases. |
Indicator_EC_ |
Indicator_x36_Z = 0 if departmental net assets = 0. Indicator _36_Z = 1 if departmental net assets > 0 | −1.04 |
Eastern Cape’s Department of Education’s payment behaviour is better when assets exceed liabilities. |
Indicator_Lim_ |
Ln_x50 = ln (provincial total receipts to provincial total payments) | −42.72 |
An increase in ln_x50 will lead to better payment behaviour; this only applies to the Limpopo province. |
x27_grouped_ |
x27_grouped = departmental annual appropriation of revenue to total departmental revenue (grouped) ln_x50 = ln (provincial total receipts to provincial total payments) | −18.18 |
Payment behaviour will improve as ln_x50 increases, provided that x_27 is high (group = 1). In the case of low values of x27 (group = −1), an increase in ln_x50 will lead to a worsening payment behaviour. |
Time_ |
0.40 |
The payment behaviour of the Department of Education of Gauteng worsens as time goes by. |
The equation of the
The equation of the
In total, two generic models and six specific models were developed. In the ‘Testing the appropriateness of the proposed models’ section, these eight models will be compared and tested for appropriateness.
The ultimate aim of the developed models is not to predict a department’s numerical value of the dependent variable (i.e. the proxy of payment behaviour), but rather to rank the departments’ expected future payment behaviours in terms of RAG statuses. RAG statuses can be assigned based on absolute or relative rankings.
Absolute rankings make use of the absolute values of the dependent variable. For example, a red status is assigned when the ratio of accruals 30 days plus/total expenditure is higher than 150% and a green status is assigned if the ratio is lower than 40%. Literature suggests a number of different cut-offs that could be used to assign RAG statuses in this manner, see for example Moody’s (
However, as a proxy was used to determine payment behaviour, RAG statuses are assigned based on relative rankings for the purpose of this article. The following procedure was used to allocate the RAG statuses: the bottom third departments (bad payment behaviour based on high values of the payment behaviour proxy) are assigned red statuses, the middle third departments (average payment behaviour) are assigned amber statuses and the top third departments (good payment behaviour based on low values of the payment behaviour proxy) are assigned green statuses.
The aim of this section is to test the appropriateness of the developed models, that is, to decide on a model to be used in future to predict payment behaviour. With this as aim, this article includes a comparison of, among others, the number of correctly predicted RAG statuses of the Basic Models and the Advanced Models for both the generic and the specific developments (Sections ‘Validation’ and ‘Model comparison’). This comparison is used in the ‘Model recommendation’ section to provide recommendations with regard to which model should be used in future to rank the departments in terms of expected payment behaviour. The recommended model is then used to compare the RAG statuses predicted by the model with the actual RAG statuses of the financial year 2012/2013 in the ‘Predicted versus actual ranks and Red, Amber or Green statuses’ section. Lastly, an example is provided in the ‘Example: Deriving the final ranking of Gauteng’s Department of Health’ section. This illustrates how the RAG statuses predicted by the models based on quantitative information can be overlaid with some more recent qualitative information to derive a final ranking or RAG status. Dashboards were constructed to illustrate this.
Kutner et al. (
The RAG statuses predicted were compared with the actual RAG statuses and the number of correctly assigned RAG statuses was counted as a measure of the model’s predictive ability. The lowest number of RAG statuses correctly predicted by the models was two out of seven (22.2%) and the highest number was seven out of the nine (77.8%).
The model comparison is based on the
Summary of the linear regression models.
Model | Description | Data deleted | Terms | Independent variables | Data sources | Factors | Correct RAG status | Correct RAG status (%) | |
---|---|---|---|---|---|---|---|---|---|
Generic | Basic Model | 0.49 | - | 6 | 4 | 3 | 3 | 8 | 44.4 |
Advanced 1 | 0.55 | - | 9 | 10 | 5 | 3 | 9 | 50.0 | |
Specific: health | Basic | 0.51 | - | 5 | 2 | 3 | 2 | 2 | 22.2 |
Advanced 1 | 0.74 | - | 8 | 10 | 4 | 2 | 7 | 77.8 | |
Advanced 2 | 0.80 | 1 | 8 | 10 | 4 | 2 | 7 | 77.8 | |
Specific: education | Basic | 0.48 | - | 4 | 3 | 3 | 2 | 4 | 44.4 |
Advanced 1 | 0.70 | - | 8 | 7 | 4 | 2 | 5 | 55.6 | |
Advanced 2 | 0.74 | 1 | 8 | 7 | 4 | 2 | 7 | 77.8 |
RAG, Red, Amber or Green.
The model with the highest
When considering the generic models,
Thus, the specific models outperformed the generic models and it is therefore recommended that the specific models (
Predicted ranks and Red, Amber or Green statuses resulting from the example payment behaviour model (using the specific model:
Province | Predicted |
Actual |
||
---|---|---|---|---|
Ranking | RAG Status | Ranking | RAG Status | |
Limpopo | 1 | Green | 4 | Amber |
KwaZulu-Natal | 2 | Green | 2 | Green |
Western Cape | 3 | Green | 1 | Green |
Mpumalanga | 4 | Amber | 5 | Amber |
Northern Cape | 5 | Amber | 6 | Amber |
Eastern Cape | 6 | Amber | 3 | Green |
Free State | 7 | Red | 8 | Red |
North West | 8 | Red | 7 | Red |
Gauteng | 9 | Red | 9 | Red |
RAG, Red, Amber or Green.
An example (focusing on Gauteng’s Department of Health) is provided next to demonstrate how the developed models could be used in future. The predicted ranking with regard to the other provinces, based on quantitative information, was nine.
As mentioned previously, credit rating methodologies are normally based on a combination of quantitative and qualitative factors. Qualitative information will be assessed by a team of analysts and then used as input in the model. Typically, credit rating methodologies for developing countries comprise 30% quantitative factors and 70% qualitative factors (Moody’s
An example of qualitative information that should form part of the intuitive overlay is the news article issued by SABC News on 30 April 2014. This article states that the Zola-Jabulani Hospital building project was finished 5 years later than planned and the actual costs were three times higher than what was budgeted (SABC News
Typically, the above-mentioned information, as well as any other available qualitative information, will be reviewed by a team of analysts. The analysts will then rank Gauteng’s Department of Health’s expected payment behaviour with regard to the other provinces by taking into account their qualitative information as well. However, this example focuses only on Gauteng and therefore the other provinces’ qualitative information is not provided. For illustrative purposes, it is assumed that Gauteng has an average red status, thus a ranking of eight, based on provided qualitative information.
Taking both the quantitative and qualitative rankings into account, the final ranking of Gauteng’s Department of Health is about eighth. This is derived as follows: 30%×9+70%×8=8.3. Thus, the RAG statuses based on the quantitative (ranking = 9) and qualitative (ranking = 8) information, as well as the final RAG status (ranking = 8) were all red. In this case, the qualitative information reinforced the red status that was assigned based on the quantitative information (illustrated in
Combining a predicted ranking with a qualitative overlay to derive a final ranking.
The primary contribution of this study was to develop and compare both a generic and a specific subnational credit rating methodology (to date no such methodologies have been developed for South African provinces and departments). A further contribution was to test the appropriateness of these generic and specific models about prediction accuracies using RAG statuses, where the specific subnational credit rating methodology was shown to outperform the generic methodology by far.
Models resulting from the combined data sets (i.e. generic models) should not be used for future predictions. The generic models predicted a maximum of 50% of the new cases correctly as opposed to the specific Health and Education models’ 78%. The payment behaviour models developed specifically for the Department of Health or Education therefore performed better than the generic models in terms of predicting new data and should rather be used.
Thomas (
Lehmann (
The authors gratefully acknowledge the valuable input given by anonymous referees. This work is based on research supported in part by the Department of Science and Technology (DST) of South Africa. The grant holder acknowledges that opinions, findings and conclusions or recommendations expressed in any publication generated by DST-supported research are those of the author(s) and that the DST accepts no liability whatsoever in this regard.
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
This article originated from E.F.’s PhD. T.V. was the PhD promotor and G.W.v.V. was the PhD co-promotor.
Credit rating agencies summarise their opinions about the creditworthiness of an issuer or issue in ratings that are graded by symbols such as AA−, BBB+ and Caa1. Issuers or issues with the same rating symbols will portray the same credit characteristics and hence have the same relative creditworthiness. See for example Moody’s (