Original Research

Materiality in reporting integration in South Africa: A natural language processing analysis

Cornelis T. van der Lugt, Hans-Peter Bakker, Nadia Mans-Kemp

Received: 30 Apr. 2024; Accepted: 12 Nov. 2024; Published: 31 Jan. 2025

Copyright: © 2025. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background: The concept of materiality has evolved as companies grapple with non-financial reporting. One result of this is greater diversity in terminology used in reporting narrative. Natural language processing (NLP) analysis offers a promising alternative to traditional content analysis to assess how local reporters converse about sustainability performance, financial performance and integration between the two.

Aim: Integrated and sustainability reports of selected South African companies were assessed to detect which topics and themes reporters viewed as material. Connectivity in terminology was also explored to consider the level of integrated thinking.

Setting: This study was conducted in a market where reporting integration and a dynamic understanding of materiality are well advanced. South Africa is deemed an integrated reporting front-runner.

Method: Natural language processing was applied to 256 reports published by the highest and lowest quality reporters in South Africa as determined by the Ernst & Young Excellence in Integrated Reporting Awards. The NLP source dictionary was compiled with reference to international reporting standards.

Results: Top reporters displayed a more complete and targeted coverage of key sustainability terms and themes. Their reports provided evidence of integrated thinking as seen by their reference to sustainability and financial terms in close proximity.

Conclusion: The evolution of reporting standards and their understanding of materiality, including report target audience and ways of determining materiality, is shaping more strategic report content.

Contribution: New NLP capabilities can be used to assess divergent corporate reporting narratives for exploring their relevance and understanding of materiality.

Keywords: materiality; integrated reporting; sustainability performance; financial performance; natural language processing; reporting narrative.

Introduction

Natural language processing (NLP) experts admire the statement by linguist John Firth (1957) who said that ‘you shall know a word by the company it keeps’. In the business context, it could be said that an enterprise shall best be known by the company it keeps, and the material significance of its reporting content by the stakeholders that it targets. This points to the ability of NLP to process words and phrases, and highlight the linkages between them. Researchers can use NLP to assess what a reporting company views as its most material issues, which stakeholders it has in mind in defining reporting content and the level of integration in thinking as manifested in the company’s integrated report (IR).

Materiality relates to recognisable financial consequences within a 12 month timeframe. It also reflects value addition or destruction with longer-term strategic and societal perspectives in mind. The introduction of integrated reporting over a decade ago signalled a deepening re-interpretation of the reporting principle of materiality. This principle has evolved considerably since the 1990s from simply measuring the level of significance of omissions or errors in annual financial statements to being championed by the sustainability movement as a strategic concept.

In 2018, the International Accounting Standards Board (IASB) refined its definition of materiality, affirming, among others, that the considered users making ‘economic decisions’ are ‘the primary users of general purpose financial statements’, that is existing and potential investors, lenders and other creditors (International Financial Reporting Standards [IFRS] 2018a). This amended definition is effective since 2020, the year before the IFRS-associated International Sustainability Standards Board (ISSB) set out to develop standards for sustainability-related financial disclosures. The first general standard issued by the ISSB applied the same definition of materiality as the IASB, while broadening the agenda to ‘connected information’ such as ‘sustainability-related risks and opportunities’ (IFRS 2023).

Pursuing a midway between financial and sustainability reporting in the 2010s, the International Integrated Reporting Council (IIRC) linked materiality to value creation. It argued that the IR ‘should disclose information about matters that substantively affect the organisation’s ability to create value over the short, medium and long term’ (IIRC 2013). Materiality hence became a key reference in determining the strategic content and quality of company reports. Also, the IIRC defined financial stakeholders as the primary target audience of the IR. In response, several legitimacy and stakeholder theorists argued that non-financial reporting remains merely symbolic in seeking to improve reputation and adding content that powerful stakeholder groups deem material (Beske, Haustein & Lorson 2020).

Guidance on what should be deemed relevant content for IRs and sustainability reports (SRs) has signalled diverse views on target audience, notably shareholders versus other stakeholders (De Villiers & Dimes 2023; Eccles & Youmans 2016). Additionally, concepts such as financial and double materiality have highlighted considerations related to a narrow versus broad scope of coverage (the thematic dimension) (Lai, Melloni & Stacchezzini 2017), connectivity between financial and non-financial performance (the monetisation dimension) (Stroehle, Soonawalla & Metzner 2022), as well as short- versus longer-term developments (the temporal dimension) (Cooper & Michelon 2022). Double materiality extends beyond internal operations to incorporate the company’s impact on the environment and society. In turn, dynamic materiality acknowledges that the materiality of environmental, social and governance (ESG) concerns can change over time (De Cristofaro & Gulluscio 2023).

As the 2008 global financial crisis showed the shortcomings of complex financial statements, investors since then appeared to show renewed interest in qualitative reporting narrative. Integrated reporting was described as involving strategic storytelling, with a narrative nature (Beattie 2014) and displaying narrative accountability (Lai, Melloni & Stacchezzini 2018). The narrative of IRs has been criticised by some researchers as displaying the technocratic logic of a financial market, based on an accounting mindset with a preference for quantitative information (Brown & Dillard 2014; Cerbone & Maroun 2020). Yet, materiality involves judging both quantitative and qualitative information. As such, the IIRC’s guidance was criticised for lacking clarity on what integration really means (Dumay et al. 2017) and being vague in defining value creation (Feng, Cummings & Tweedie 2017).

Quality integrated reporting may provide an indication of how materiality is newly understood today. While materiality as a concept in financial reporting has received substantial scholarly interest, the materiality of non-financial information historically received less attention (Edgley 2014; Messier, Martinov-Bennie & Eilifsen 2005). With the expansion in the thematic scope of reporting, new approaches emerged to determine materiality. These approaches reflect diverse stakeholder interests while incorporating quantitative and qualitative information. Against this background, a comparison of the narratives of SRs and IRs lends itself to the application of software, notably NLP to assess the evolving understanding of materiality.

A prime market to apply NLP in assessing material reporting content is South Africa. The country has seen the greatest uptake of integrated reporting globally during the 2010s, and the Johannesburg Stock Exchange (JSE) was the first stock exchange to introduce the publication of IRs as a listing requirement (Appiagyei, Djajadikerta & Mat Roni 2023). Societal dynamics in South Africa also contribute to companies paying close attention to the interests of a multi-stakeholder audience and pursuing a stakeholder-inclusive process for materiality determination (Rensburg & Botha 2014).

The authors investigated materiality and sustainability considerations in reporting integration of the highest and lowest quality reporters in South Africa as determined by the Ernst & Young (EY) Excellence in Integrated Reporting Awards. The first objective was to employ NLP to assess the narrative content of IRs and SRs published by the sampled companies, tracking differences between top quality versus low-quality reporters in reporting material content as recommended by recognised reporting frameworks and standards. Aspects considered were material topics, themes and the primary stakeholders targeted by reports. The second objective was to determine whether top quality reports were more likely to display integration and connectivity between financial and non-financial performance information. Natural language processing was used to detect the level of integrated thinking, a management philosophy that plays a central role in defining material significance. The research therefore makes a contribution to the existing body of knowledge by applying NLP to consider whether the reporting narrative of selected South African corporate reports demonstrates a strategic and dynamic understanding of materiality.

Foundational theory, reporting standards and materiality redefined

Studies on non-financial reporting, its motivation, quality and impact, have tended to follow accountability and efficiency schools of thought (Van Der Lugt & Mans-Kemp 2022). These schools of thought reflect different perspectives on materiality, considering the ability of reporting to address the most relevant information needs of different stakeholders or to efficiently deliver the most decision-useful information to market players. The accountability school employed legitimacy, stakeholder and institutional theories, and focused on the social licence to operate (Beske et al. 2020; De Villiers, Hsiao & Maroun 2020; Farooq et al. 2018; Hoque 2018). The efficiency school centred on market efficiency. Prior authors have examined the association between IR quality and firm value, cost of capital and profitability (Barth et al. 2017; Horn, De Klerk & De Villiers 2018; Mans-Kemp & Van Der Lugt 2020).

Stakeholder theory has been the most popular theoretical foundation applied in academic research on materiality from a sustainability perspective (Fiandrino, Tonelli & Devalle 2022). The stakeholder accountability argument has also been a popular one in praxis. It has challenged corporate boards to prioritise and disclose their target reporting audiences (Eccles & Youmans 2016). A stakeholder focus implies that a broader range of issues is considered than when narrowly concentrating on shareholder interests. Broader focus brings renewed questions of reporting complexity and readability. Corporate leaders also have to consider conciseness as a guiding principle when prioritising material issues, in other words the level of coverage devoted in reports to material topics (Rivera-Arrubla, Zorio-Grima & García-Benau 2017).

The more inclusive approach of double materiality (Baumüller & Sopp 2022; Fiandrino et al. 2022) dictates that corporate leaders must consider how their companies’ actions impact society and the environment where they operate, as well as the related financial implications for the reporting entity. Debates on double materiality highlight competing institutional logics pertaining to the internal versus the external impacts on society and the environment. Reporting on external impact implies greater attention to the views of diverse external stakeholders. Reporting on internal impact is more aligned with a traditional financial accounting focus and reference to investors’ information needs. The material relevance of a consideration can furthermore change over time, as highlighted by the term dynamic materiality (WEF 2020).

Central to the current debate on the meaning of materiality are therefore references to financial materiality, double materiality and dynamic materiality. Convergence between standards promoted by financial and non-financial reporting communities in defining materiality is accompanied by interaction between experts from different disciplines. This brings the possibility of developing hybrid logics around the motivation, quality and impact of reporting. With regard to quality, a key feature that the IR was proposed to reflect is integrated thinking, also in its presentation of material content.

Since its foundational years, sustainability reporting was expected to result in greater transparency and accountability towards diverse stakeholders. The use of the Global Reporting Initiative (GRI) guidelines was intended to reduce the reporting–performance gap (Bouten et al. 2011). The GRI had to account for the tension between institutional logics considering sustainability reporting as a performance management tool (efficiency) versus reporting as an accountability tool (linked to legitimacy) (Brown, De Jong & Levy 2009; Farooq & De Villiers 2019).

The GRI (2016) stated that an SR ‘shall cover topics that reflect the organisation’s significant economic, environmental and social impacts; or substantively influence the assessments and decisions of stakeholders’. Importantly, the GRI argued that the impacts referred to here are external impacts and that its understanding is similar to double materiality. The G4 version of the GRI guidelines put this understanding of materiality centre stage (Farooq & De Villiers 2019; Jones, Comfort & Hillier 2016). In how far double materiality results in more substantive reporting on impacts remains to be seen. Despite the focus that is currently placed on double materiality in the European Union, initial evidence suggests the prevalence of a symbolic legitimacy reporting approach (Correa-Mejía, Correa-García & García-Benau 2024).

The arrival of the IR advanced the examination of interlinkages between divergent dimensions of sustainability, including different types of financial and non-financial capital (Hahn & Kühnen 2013). The IIRC recommended targeting financial stakeholders through integrated reporting (IIRC 2013). This recommendation has important implications for examination of materiality, including terminology and combinations of terms assessed when conducting analyses on corporate reports. While past investigations of the volume of reporting relied on keywords, sentence and page counts (Du Toit 2017; Liu, Jubb & Abhayawansa 2019), more focus should be placed on assessing the quality of IRs. The associations between sustainability and financial indicators and terms also warrant further research. The evaluation of report quality should hence involve assessments of both quantitative and qualitative information.

The IIRC published its initial framework when some seasoned reporters arguably developed stakeholder engagement fatigue. The Sustainability Accounting Standards Board (SASB 2017) reminded the reporting community that stakeholder engagement is only one of a number of activities required to determine materiality. Furthermore, materiality should firstly be defined in terms of relevance to the reporting entity. In turn, reference to and interest in the reporting entity is the common denominator that connects its shareholders and other stakeholders. Research has highlighted how auditors consider company- and industry-specific features when applying materiality judgements (Messier et al. 2005). Contextual information helps to augment quantitative materiality assessments.

Recognised non-financial disclosure standards over the last two decades defined key tests for determining materiality. A pioneer in advancing principles-based stakeholder engagement was the AA1000 standard for sustainability-related assurance. The recommendations by the AA1000, GRI, IIRC and SASB are compared in Table 1. Key tests recommended by these standards are outlined, including stakeholder views, industry trends (linked to institutional theory) as well as regulatory and societal expectations (related to legitimacy theory). They display significant overlap, pointing to dominant references by leading companies to define key material topics for the content of their sustainability and integrated reporting.

Internationally influential initiatives for non-financial reporting were recently released that build on the work of the early pioneers. These initiatives recommend materiality approaches that echo the above criteria for determining materiality. The General Requirements for Disclosure of Sustainability-related Financial Information issued by the ISSB in 2023 confirmed its prioritisation of financial impacts as well as financial stakeholders or ‘primary users of general purpose financial reports’ as its target audience (IFRS 2023). Furthermore, the European Sustainability Reporting Standards (ESRS) of the European Union’s Corporate Sustainability Reporting Directive (CSRD 2022) is accompanied by Implementation Guidance for Materiality Determination published by the European Financial Reporting Advisory Group (EFRAG 2023). Its recommended steps start with context and refer to criteria such as business relationships, peer analysis, legal and regulatory developments, scientific studies and the views of affected stakeholders (EFRAG 2023). The ESRS will have impact beyond the European Union, as South African and other multinational corporations with significant operations in Europe will have to apply these standards.

Evidently, non-financial reporting has come a long way from covering a brief list of topics such as environmental and employee disclosures to consideration of multiple topics and related indicators. The GRI has substantially contributed to the formalisation of sustainability reporting (Tschopp & Huefner 2015). While the GRI recommended indicators related to topics from a comprehensive sustainability agenda, the more principled IIRC framework focused on the logic behind the use of different capitals and value creation processes. Under the IIRC’s guidance, materiality is therefore closely associated with value drivers, business model and strategy (IIRC & International Federation of Accountants 2015). Since the 2008 global financial crisis, capital providers showed greater interest in narrative description of these aspects (Lewis & Young 2019). While the narrative might have expanded, greater reporting volume did not per se advance transparency (Unerman & Zappettini 2014).

The IIRC foresaw that integrated reporting would have particular value if it succeeds in addressing different financial and non-financial capitals in an interconnected and integrative manner (Okwuosa & Atkins 2023; Siegrist et al. 2020). Yet, research that used the IIRC guidance to define IR quality evaluation criteria illustrated the shortcomings of the framework in being principles-based. Its recommended content elements could furthermore result in tick-box compliance (Liu et al. 2019).

Related to conciseness or depth and aggregation of information, another important consideration in defining material content is the level of analysis (Whitehead 2016). Materiality can be defined at the topic level or at an indicator level, for example climate change, climate mitigation, greenhouse gas emissions or scope 3 emissions. The SASB emphasised that materiality needs to account for sector-specific context. This approach is pursued by the ISSB, whose IFRS sustainability disclosure standards development is incorporating the SASB standards. In its entity-specific approach, focus is placed on what is seen as practically most relevant through the eyes of the management. As highlighted in Table 1, one of the key tests recommended by the IIRC is whether a topic ‘forms the basis of boardroom discussions’.

Overview of prior research on non-financial reporting narrative

Several authors explored non-financial corporate reporting before the release of the IIRC Framework. They found that a growing number of companies report on their environmental and social performance to satisfy the diverse information needs of internal and external stakeholders. They predominantly relied on the GRI guidelines and centred on assurance in the triple bottom line context (Ballou, Heitger & Landes 2006; Kolk 2003; Milne & Gray 2013). Often explored were factors influencing the adoption, extent and quality of reporting (Hahn & Kühnen 2013), including the influence of corporate reputation on reporting quality (De Villiers & Van Staden 2011). Higher quality of reporting was assumed to imply the reporting of more relevant or material information, and drivers behind this included both stakeholder pressure and a desire to comply with recognised reporting frameworks and standards (Malola & Maroun 2019).

Prior research on the complexity of decision-making on materiality included the use of the GRI guidelines as reference in applying analytic hierarchy processes (Calabrese et al. 2016). The recognised GRI taxonomy was used to assess the relative importance that decision-makers and other stakeholders attach to its indicators. The extent of coverage of related ESG themes and the level of detail provided on each enable analysts to explore materiality considerations. The GRI is a useful reference since its institutionalisation has involved the ‘emergence of new language and concepts’ as well as a ‘shared understanding of the meaning of new terms’ (Brown et al. 2009:578).

When dealing with non-financial performance information, auditors find it difficult to set materiality thresholds as the qualitative information involved is often expressed in ‘unfamiliar’ measurement units. Confusion about the target audience further complicates materiality judgements (Green & Cheng 2019). Calabrese et al. (2019) found materiality matrices to be facilitating a structured, quantitative process to manage the level of subjectivity involved with multi-stakeholder involvement. Materiality matrices in company reports often present stakeholder versus management views on the relative significance of material topics.

Studies on the unstructured information found in reporting narrative have applied discourse analysis, content analysis and NLP. Discourse analysis has been used to assess language as a reflection of social context, content analysis to assess emphasis on different topics, while NLP offers the ability to detect specified words, combinations of terms, text similarity, topic modelling, thematic analysis and sentiment analysis (Kang & Kim 2022; Lewis & Young 2019). Enabling the processing of large volumes of information and identifying latent features, NLP can be used to interpret text by detecting certain linguistic strategies, including framing issues and using common business terminology (Ferguson, Sales De Aguiar & Fearfull 2016). Prior scholars used NLP to count and classify words and sentences in SRs (Bouten et al. 2011) and also to detect the tone, language and placement of integrated reporting narrative (Williams & Adams 2013). Historically, manual text analysis and NLP have found that poor readability of annual reports is a good predictor of poor performance (Fisher, Garnsey & Hughes 2016).

Over the last decade, several researchers conducted content analysis on IRs to assess compliance with the IIRC Framework. The criteria involved centred on the related content elements and guiding principles, including materiality, stakeholder relationships, conciseness and completeness. Manual content analysis of IRs has included reading and coding to determine adherence to the materiality determination process recommended by the IIRC (Jebe 2017). Assessment of the level of material disclosure in IRs included word counting and scoring reports on the extent to which the recommended process and material topics have been covered (Fasan & Mio 2017). Manual analysis of the IRs of large financial institutions in South Africa confirmed that greater quantity of ESG information in reports does not, by definition, imply greater integrated reporting quality (Naynar, Ram & Maroun 2018).

The guiding principle of stakeholder relationships raises the possibility of symbolic disclosure that legitimacy theory argues is used to secure reputation and a social licence to operate. Some scholars have conducted interpretative analysis of reporting language by examining how reporters interpret certain events and assessing the function of the IR narrative as a ‘mode of cognition’ (Lai et al. 2018:1382). Because symbolic disclosure might involve impression management and ‘management obfuscation’ (Li 2008:221), researchers must be alert to the role of consultants who work with report preparers in polishing the narrative. Natural language processing analysis and its findings on readability have shown how complexity, including complex words and jargon, is used to obfuscate poor performance.

Content analyses of reports from the GRI’s and IIRC’s databases have revealed inconsistent application of the materiality determination process recommended by both frameworks and a lack of clarity on stakeholder engagement (Steenkamp 2018). Ahmed Haji and Hossain (2016) furthermore noted poor materiality determination and a lack of completeness when analysing IRs. Yet, in comparison to sustainability reporters, pilot IR producers displayed better quality disclosure on materiality (Fasan & Mio 2017). The textual and network text analysis of pioneer IRs of the 2010s by Quarchioni, Ruggiero and Damiano (2021) focused on vocabulary, word frequency patterns and word-to-word relationships to assess how IRs reflect integrated thinking and convey meaning. Prior authors also noted that the scope and nature of disclosures differ for high- and low-quality reports by conducting content analysis on reports published by South African companies in 2015 and 2016. Companies with better quality IRs tended to complement their IRs with SRs and valued external assurance of disclosures (Malola & Maroun 2019).

Analysis of initial IRs published by JSE-listed companies pointed to weaknesses in the way that the <IR> Framework requirements have been met, including vagueness in reporting on material topics (Ahmed Haji & Anifowose 2016; Setia et al. 2015). Furthermore, Du Toit (2017) reported that the readability of IRs published by the JSE Top 100 companies could improve, based on an assessment of readability criteria including the length of sentences, complex words and wordy expressions. Likewise, international analysis of IRs published by companies from 14 countries between 2013 and 2016 showed that most IRs require a university level of education to understand them at first reading (Gerwanski, Kordsachia & Velte 2019).

Similar to efficiency approaches highlighted earlier, research on IR narrative has also considered value relevance and the link with financial performance. Caglio, Melloni and Perego (2020) conducted software analysis on the textual attributes of IRs published by selected JSE-listed companies between 2011 and 2016. By accounting for report length, bias and tone, they found that IR readability is associated with higher market valuation and conciseness with higher share liquidity. Noting sector differences among IIRC pilot companies by applying software analysis, Melloni, Caglio and Perego (2017) found that IIRC pilot companies with highly volatile shares provide more concise reporting. By conducting computer-aided text analysis of the reporting narrative of leading South African and European integrated reporters, Dimes, De Villiers and Chen (2023) found that the companies that more effectively displayed integrated thinking reap economic benefits in the form of return on assets.

A common but faulty assumption is that IRs should be readable by all stakeholders. Targeting all stakeholders with the IR tends to result in greater report length and confusion. It creates an unnecessary perceptions gap (Naynar et al. 2018) due to the differences in the level of sophistication (level of investment knowledge) between financial versus non-financial report users. Integrated reports as foreseen by the IIRC (2013) are meant to target a specific user group, namely the providers of financial capital who are familiar with a certain business vocabulary. The current study considered the possibility that quality IRs include complex terms such as ‘value creation’ which may be described as jargon by some, yet are common business terms known by investors. More substantive reporting may thus involve more technical language, and it should not be assumed that more quantitative as opposed to more qualitative information by definition implies greater reporting quality. The next section outlines how NLP was used to assess non-financial reporting narrative, detecting key material topics and themes recommended by international frameworks and standards.

Research design and methodology

The introduction of mandatory integrated reporting for JSE-listed companies at the beginning of the previous decade sets the scene for this investigation. The number of mentions of key material topics in selected IRs were analysed by conducting NLP analysis. Care was taken not to merely use NLP to detect references to generic terms (i.e. compliance with official IIRC terminology) but rather to consider how the reporting narrative addresses material topics. An example of climate change illustrates the approach adopted. Instead of only determining whether or not an analysed report referred to ‘climate change’, the researchers accounted for the extent to which the related text was company-specific, strategic and implementation-focused by using terms such as ‘physical risk’ and ‘transitional risk’. Although the number of references to a generic term in a report could indicate prioritisation, the authors paid attention to specific terms at the indicator level and the so-called narrative ecosystem (sentences, paragraphs and pages) where these terms were used.

The disclosure of material content was compared for ‘top’ (high) and ‘bottom’ (low) quality integrated reporters over the period 2013–2018. The well-established EY Excellence in Integrated Reporting Awards was used to determine high versus lower quality IRs. This award scheme has applied a consistent methodology with independent judges during the research period. The sample included the top 15 companies that were included in the EY ‘excellent’ category for most years during the research period. The bottom 15 companies formed part of the ‘progress to be made’ or ‘average’ categories for most study years.

Subjectivity of the researchers was mitigated by (1) relying on the results of a recognised award scheme with independent judges, and (2) conducting software analysis by using key terminology taken from internationally recognised frameworks and standards (see Table 2). The sustainability lexicon employed in the NLP analysis was predominantly based on the GRI’s universal and topic-specific standards, by using key terms from its aspects and indicators (see Table 2). A selection of core concepts of the IIRC (30 terms), SASB (35 terms) and the Task Force on Climate-related Financial Disclosures (TCFD, 14 terms) was added. Key terms related to these standards and frameworks included the ‘six capitals’ (IIRC), ‘business model resilience’ (SASB) and ‘climate scenarios’ (TCFD).

Furthermore, to enable comparison with reporting usage of conventional business and finance terminology, core financial value driver terms that are commonly used by managers and investors, as identified in literature, were also added to the lexicon. The financial terms included revenue, sales growth, profit, capital expenditure, cost of capital and tax. While the application of NLP to the narrative content in this study was not used to assess biases in the tone or language used, the tracking of reference to key material topics as recommended by the recognised reporting standards enabled a testing of completeness, integration (interconnectedness) and transparency.

The list of terms taken from the selected standards was used to form a data frame. Some terms were single words, such as ‘employment’ while others combined two or three words, such as ‘climate change’. Each of the included terms was linked to standards, codes, themes, aspects and so-called ‘umbrella’ or headline terms. The data frame was then transformed to a ‘long format’ in which the terms used by each company formed a single variable called ‘organisation’. Pluralised versions of applicable terms were also added. Theme relative frequency and organisation relative frequency columns were then added to allow for weighting based on these frequencies. For example, 15.54% of all the considered terms related to the economic theme and 76.80% of the terms originated from the GRI standards. This process resulted in the data frame listing 849 terms and 9 variables.

The IRs and SRs of the top 15 companies were considered for the NLP analysis. The bottom 15 companies tended to publish only IRs. Table 3 outlines the type of reports that were analysed per annum. In total, NLP was conducted on 256 reports.

The uniform resource locators (URLs) of the portable document format (PDF) versions of the reports, as shown in Table 3 were used to analyse the narrative content thereof. Each of the reports was analysed by using the R PDF reader function. Natural language processing with R or any other software deals with real language and as such is subject to the vagaries of the written form. To mitigate these limitations, pluralised forms were singularised and fuzzy logic was considered to deal with variations of spelling. The text was captured by page, and the total number of pages of each of the reports was recorded. This process resulted in a dataset containing 28 567 pages of text. The text was then tokenised into words (6 488 848 words); into adjoining words (12 806 929 bigrams) and into three adjoining words (12 778 667 trigrams). The dataset was analysed using R Core Team and R Studio.

The dataset comprising the terms was used to filter out unlisted terms. This process resulted in a dataset in which each row, inter alia, identified the listed terms per page and per report, the standard associated with the terms, the associated headline terms and the relative frequencies per identified theme. To assess interlinkages between sustainability and financial terms, the number of business and financial terms per page and per report were tallied, as well as the counts of three prominent sustainability terms (corruption, employment and climate change as identified in literature). The aim was to detect reference to a select financial term and a select sustainability term on the same page, signalling a connection being made in the relevant narrative.

The total number of pages per report containing key terms was also determined. The total number of words, bigrams and trigrams listed in the terms file were counted per report and the relevant proportions were computed. For example, if a word such as ‘stakeholder’ appeared 78 times in a report of 261 pages, the proportion was determined as 0.30 (78/261). These proportions were used to create the figures that will be discussed in the next section.

To assure reliability and validity, the lexicon used to conduct the NLP analysis was based on renowned international standards and frameworks. This serves to address critique of NLP application in accounting research based on a lack of transparency in dictionary development and problems with replicability (Dimes et al. 2023). The outlined approach was consistently applied for the sampled companies.

Ethical considerations

This project (number SBS-2023-29628) was exempted from ethics review and clearance by the Research Ethics Committee (REC), Stellenbosch University, on 17 November 2023.

Results and discussion

The majority of the top 15 sampled companies published IRs alongside SRs. This tendency suggests that they have viewed providers of financial capital as the primary target audience of their IRs, in line with the IIRC (2013). These companies have also published SRs to provide additional information on sustainability topics to a broader range of stakeholders. Yet, the majority of the considered bottom 15 companies only published IRs, thereby seeking to address all stakeholders in a single report.

The first objective was to account for material topics and themes in the analysed reports of the sampled companies. About 77% of the adopted lexicon consisted of GRI terms. Based on the weighted term frequencies, it was found that the top 15 companies used the GRI and IIRC terms more frequently than the bottom 15 companies. Figure 1a and 1b show the results for the top 35 terms that were used by the top 15 versus bottom 15 companies in their analysed IRs, respectively. In the case of the top 15 companies (Figure 1a), 20 terms were above the indicated median (dotted vertical line), versus 14 terms for the bottom 15 companies (Figure 1b). While the latter figure shows a strong dominance of generic business and financial terms, the IRs of the top 15 companies included more sustainability terms found in the GRI standards, such as governance, employee, stakeholder, opportunity and skills.

FIGURE 1: Top 35 terms* (a, b, c and d) used in the integrated reports published by the top and bottom companies as well as the sustainability reports published by the top companies.

In terms of sustainability terminology, the analysed SRs (1d) had 21 terms above the median line relative to the comparative IRs (1c) that only had 12 terms above this line. The terms employee, development and water, energy, emissions, waste, education, socioeconomic compliance and human rights were frequently used specifically in their SRs. The GRI terms used in the analysed reports were furthermore evidently clustered around headline themes.

The material terms that were identified in Figure 1 were then categorised in headline themes. From this analysis, it was evident that the bottom 15 reporters aimed to cover some key sustainability themes within their IRs. Seven of the main themes found in the IRs of the bottom 15 reporters were widely known GRI-based sustainability themes, including governance, customer health, social impact, waste, stakeholder engagement, staff composition and water. The related terms that were used in the analysed reports signalled a compliance-driven approach. The sustainability terms that received greater coverage in the IRs of the top 15 reporters related to more opportunity-related themes, such as strategy, energy and skills development.

Prior authors mentioned that SRs could lack strategic focus (Zúñiga et al. 2020). The headline themes were thus also compared for the top 15 reporters’ IRs and SRs. The colours in Figure 2 indicate prominent similarities and differences between their reporting on material themes, as outlined in the footnote. In the case of the SRs of the top 15, there was greater coverage of the headline terms compared with their IRs. While their IRs centred on organisational profile, taxes and governance (Figure 2a), the four material themes in the top reporters’ SRs (Figure 2b) were water resources (GRI 303), staff composition (GRI 405), energy (GRI 302) and emissions (GRI 305). Issues such as water, energy, employment and social impact are common headline themes across sectors in South Africa.

FIGURE 2: (a and b) Top 25 headline terms* found in integrated reports versus sustainability reports of the top 15 reporters.

The governance theme was more extensively addressed in quality IRs rather than in SRs. Governance disclosures, inter alia, covered board structure, composition, remuneration, oversight and risk management. The analysed IRs also covered economic performance and wealth-related content (GRI 201–1) by incorporating terms such as sales growth, capital expenditure and economic empowerment. The high coverage of strategy (GRI 102–15) was largely based on key IIRC terms, such as future orientation, business model, resilience and TCFD terms, such as scenario analysis and climate mitigation that were linked with the headline theme named strategy.

The preceding discussion covered indicator level terms and headline themes, thereby addressing the article’s first objective to assess key material topics discussed by the sampled companies. The identified main themes were furthermore clustered in terms of their topic areas (similar to the GRI 200, 300 and 400 series). Where applicable, EESG (GRI 100 series) categories were also considered. The weighted sums for each topic area are shown in Figure 3, reflecting the weighting based on the proportion of disclosed terms that were identified per topic area (by accounting for the GRI series).

FIGURE 3: Spread of terms per economic, environmental, social and governance category in the analysed integrated reports.

Perusal of Figure 3 shows that both the top 15 and the bottom 15 companies gave the least coverage to the environmental category in their IRs. The top 15 displayed considerably greater coverage of the other three categories than the bottom 15. Furthermore, the top 15 focused more on governance and organisational matters in their IRs by reporting on strategy, governance, ethics and stakeholder engagement. South Africa has a well-established corporate governance framework advocating a stakeholder-inclusive approach. The first King Report was published in 1994, before the considered frameworks and standards existed, contributing to the extensive focus on the governance category by the reporters. By focusing on EESG, corporate leaders can effectively manage EESG risks and caution against sacrificing long-term gains for short-term profits (Rezaee 2017). Furthermore, in a prior South African study, Zúñiga et al. (2020) argued that Thomson Reuters’ EESG score can be used to measure integrated reporting quality, as the score shows how a company’s financial and non-financial disclosures can be equally weighted. Likewise, the better reporters in this study had higher disclosure on all EESG pillars.

The second research objective, namely to assess the level of integration in the narrative of the analysed IRs was achieved by using NLP to account for references to key financial terms and three headline sustainability terms (climate change, employment and corruption) in close proximity (same page) in the narrative. In doing this, possible connectivity between these terms could be detected.

The following core financial value driver terms were considered as identified in literature: revenue/sales growth, profit margin, capital expenditure, liquidity, cost of capital and leverage. Figure 4 indicates the sum of the proportion of pages per analysed IR that contained both the corporate financial terms and reference to any of the three indicated sustainability terms for the top 15 and bottom 15 companies.

FIGURE 4: Connection between key financial and sustainability terms.

Eleven of the top 15 companies were above the median shown in Figure 4. This result suggests that the reporting content of their IRs displayed a more integrated narrative. Their coverage of climate change, employment and/or corruption appeared on the same pages where they addressed key financial value drivers in several instances. The narrative of the majority of the top 15 reporters thus addressed these terms in the same narrative ecosystem and arguably showcased ‘connectivity’ and ‘integrated thinking’, as promoted by the IIRC (2013). Employment was the most commonly addressed theme when core financial terms were mentioned in close proximity. Such integration indicates a more holistic understanding of value creation (Dimes & De Villiers 2024). Dimes et al. (2023) noted that companies that displayed integrated thinking reaped economic benefits. Reference to the select sustainability terms on the same pages as the financial value driver terms is not infallible proof of the issuers making a direct link between the two groups of terms. Yet, the reference to these themes in the same narrative ecosystem (page or section) is telling, signalling awareness of the connectivity between sustainability and financial performance.

The company that showed the highest level of connectivity between the considered sustainability and financial terms has global operations and receives substantial pressure from stakeholders on fossil fuels use and climate change considerations. With regard to the theme of corruption, it should be noted that greater mentioning of corruption linked with financial narrative does not per se imply instances of corruption. Such disclosures often reflected actions to mitigate corruption and enhance transparency in combatting corruption.

Prior authors largely used NLP to assess the sentiment (Cherry, Mohamed & Brahmbhatt 2023), language used (Bouten et al. 2011; Fisher et al. 2016; Smeuninx, De Clerck & Aerts 2020) and disclosure on specific sustainability considerations (Guo et al. 2021). By comparison, the tracking of reference to key material topics in IRs and SRs in this study, based on the recognised reporting standards, enabled the researchers to account for key terms, themes and interconnectedness.

Conclusion and recommendations

The principle of materiality has evolved considerably since the 2000s to become a strategic concept with a dynamic nature. Double materiality confirms agreement that both the impact of an entity on society and the environment as well as the related impacts on the entity itself should be considered. Tests for determining materiality have pointed to the need for assessing stakeholders’ views and regulatory developments. Consideration of different stakeholders’ views encourages a more wide-ranging and diverse agenda of key material topics, resulting in an expanded reporting narrative that presents scope for NLP application.

The report(s) in which companies opt to report their performance related to material topics influence their prioritisation and framing of key topics. In the 2010s, the arrival of integrated reporting displayed an attempt to define a synthesis report based on financial and sustainability performance considerations. The IR was meant to be a strategic report with a more integrated understanding of materiality that speaks to financial capital providers. Its institutional approach sought to connect external and internal impacts, and connect sustainability and financial performance with a longer-term view. One of the reporting principles of the IIRC Framework (2013) is ‘connectivity of information’, including interdependence between diverse capitals.

The NLP analysis of the IRs and SRs of the sampled companies indicated prominent differences in material topics and themes covered in their reporting narratives. In terms of integrated thinking, key sustainability and financial terms were in close proximity in several of the analysed IRs. The low-quality reporters addressed a diverse audience (investors and other stakeholders) in one report, namely their IRs. Their reporting content hence displayed diverse business and sustainability terms. The latter were typically associated with regulation and compliance disclosure. In contrast, the high-quality reporters had greater coverage of sustainability terms, as well as high coverage of strategy (GRI 102–15) that involves complex terms such as future orientation, business model and climate scenario analysis. The IRs of the top 15 reporters devoted most attention to governance, a trend that could in part be ascribed to the country’s advanced corporate governance code.

The NLP analysis illustrated that the evolution of reporting standards and their understanding of materiality, including report target audience and ways of determining materiality, is shaping report content that can be positioned as per Figure 5 based on the reporting standards considered in this study. The figure positions the types of information content advanced by leading international standards, based on their inclusivity in approach (narrow to multi-stakeholder) and level of integration of content (fragmented to connected). While most analysts recognise that the ‘notion of organisations producing separate reports for financial and other information seems outdated’ (De Villiers & Dimes 2023:288), the reality is that separate content continues to be tailored to the specific information needs of different target stakeholder groups.

FIGURE 5: Mapping reporting content based on inclusiveness and integration levels.

Figure 5 reflects two semantic oppositions found to be common in IRs published during the 2010s analysed by Quarchioni et al. (2021) in a textual and network text analysis, namely between internally-oriented versus externally oriented perspectives and financial versus organisational perspectives. Considering the clustering of terms around six capitals, they found financial capital to be the most discussed theme. This is due to the focus on financial stakeholders as user audience, even though the content is integrated.

In turn, the application of NLP to IRs and SRs in this study signalled the implications of different reports targeting different users, showing divergent levels of integration and how differentiated content can also enable greater quality of integration. The impact reporting promoted by the GRI is highly inclusive, but fails to address the related economic consequences for the reporting entity. This is beyond the mandate of the GRI as standard setter. Global Reporting Initiative content can therefore be described as holistic, but from a societal perspective. The ideal type of IR, as per cell A1 in Figure 5 would thus be entity-holistic (focusing on the health of the entity within society and its natural environment). However, it is debatable whether this ideal IR has been achieved by the IIRC.

The top 15 reporters displayed greater sophistication in defining integrated content that was accompanied by sustainability content in a separate SR targeting additional stakeholder groups. The publication of both an IR and an SR does not by definition imply fragmentation (Malola & Maroun 2019; Quarcchioni et al. 2021). Separate reports can reflect a holistic view on different, yet aligned types of information for specified stakeholder information needs. A large corporation may publish a suite of different reports that should convey a consistent story, covering headline material topics.

Financial reporting standards have always described materiality as an aspect of ‘relevance’: ‘materiality is an entity-specific aspect of relevance based on the nature or magnitude’ of an issue (IFRS 2018b). Whether nature and magnitude signal greater level of significance is often determined by connectivity, signalling cause-and-effect or ripple effect relations. Materiality is therefore indicative of connectivity. Greater connectivity between different capital developments signals greater significance. Greater level of impact materiality or financial materiality points to relevance based on magnitude and nature, which again highlights connection between non-financial and financial impact. More frequent mentions of key sustainability terms, on their own and connected, thus reflect greater material significance in reporting narrative. It displays the attempt by a narrative to communicate the relative importance of different subjects. Furthermore, the link between select terms in the present study conveyed one meaning of integrated thinking, namely connectivity between key non-financial and financial issues.

The greatest test for an IR is arguably its ability to showcase integrated thinking. Proximity in the analysed narrative of three prominent sustainability terms and key financial terms suggested that the top 15 reporters displayed greater connectivity addressing the issues involved than the bottom 15 reporters. The top 15 reporters displayed key hallmarks of integrated thinking, in particular integrated strategy and integrated business intelligence, as also outlined by Dimes and De Villiers (2024). High-quality reporters likely have more integrated performance management systems and integrated reporting processes, including regular discussions internally between the finance and non-finance functions. Quality reporters also have the ability to incorporate conclusions from multi-stakeholder engagements into their SRs, as well as IRs.

Natural language processing application is based on the assessment of ‘natural’ that is human language as found in a specified narrative, such as a report. This raises critical questions about the reference vocabulary employed as well as about the significance of a report or reporting as such. Firstly, critical questions are asked about objectivity in the creation of dictionaries for NLP assessment of integrated thinking (Dimes et al. 2023). To limit subjectivity, the present study relied on a recognised independent award scheme for identifying quality IRs and key international standards for defining a dictionary to assess material content, including connectivity between select terms.

Secondly, IR quality should not by definition be seen as a proxy for quality of integrated thinking. Ecim and Maroun (2023) recommended that other proxies, such as aspects of risk management or management control systems could be examined. The researchers recognise the value of the IR as a strategic document, the narrative of which represents an effort by executive management to communicate to providers of financial capital their firm’s impacts and dependencies on diverse capitals. The quality of that communication is indicative of internal integrated decision-making and organisational integration (Busco et al. 2019). In future, NLP analysis can be used to obtain a more holistic understanding of materiality and integrated thinking as reflective of internal organisational integration and integrated decision-making. Such analysis can cover differences in levels of integrated thinking for companies of various sizes operating in diverse industries.

Future researchers could also conduct a comparison of material content included in IRs and SRs published in developed and emerging markets, employing the international standards-based dictionary developed for this study. Connectivity between material sustainability and financial terms in the same narrative sections, paragraphs and sentences warrants further investigation in light of the recently released IFRS sustainability disclosure standards. While the dictionary employed in this study is relatively limited, more sophisticated analytic approaches and advanced NLP techniques can be employed to examine combinations of a wider range of key sustainability terms linked with core financial terms, providing expanded evidence of integrated narrative. Manual content analysis can follow up on this research to examine whether and how report issuers directly connect the sustainability and financial performance concepts involved. Future research can also apply network text analysis to measure semantic connectivity as found in terms of density and centrality to determine the most influential nodes in the networked text of IRs (Quarchioni et al. 2021). The growth of non-financial reporting worldwide on the basis of international standards creates vast opportunities for applying NLP in assessing the company that material terms keep.

Acknowledgements

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.

C.T.v.d.L. conceptualised the article and led the literature review, overall analysis and writing. H.P.B. developed and applied the NLP analysis, and wrote the NLP methodology and results section. N.M-K. contributed to the conceptualisation of the article, literature review and finalisation of the article.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

The analysed corporate reports are available in the public domain. A copy of the coding used for the NLP analysis can be obtained upon request from the corresponding author, C.T.v.d.L.

The views and opinions expressed in this article are those of the authors and are the product of professional research. It does not necessarily reflect the official policy or position of any affiliated institution, funder, agency or that of the publisher. The authors are responsible for this article’s results, findings and content.

References