4  Plots and tables

4.1 Summary

4.1.2 Comparative Annual Distribution of Impact Evaluations and Systematic Reviews

The second plot separates the two study types, impact evaluations and systematic reviews, side by side for each year. This representation allows for a clear visual comparison of the output volume between the two over the same timeline, indicating any shifts in research focus or intensity.

4.1.4 Global Distribution of Impact Evaluations by Country

The map visualizes the geographic distribution of impact evaluations by country. Countries with a higher concentration of studies are represented by darker shades on the map, indicating regions with more intense research activity. The provided count offers a snapshot of the total number of studies and countries involved, reflecting the global scope of impact evaluation.

For a more in-depth analysis or visualization of this map, users can utilize Geographic Info, which allows for the highlighting of specific study types or statuses, enabling a more focused examination of the data.

4.1.5 Evaluation type

The chart displays the proportion of impact evaluations to systematic reviews, visually comparing the frequency of each study type in the dataset.

4.1.6 Evaluation design

The pie chart categorizes the various methodological approaches used in the impact evaluations.

4.1.7 SR Critical Appraisals

This chart illustrates the distribution of critical appraisal scores for systematic reviews.

4.1.8 Ethical Approval

The chart indicates the percentage of studies that have received ethical approval against those that haven’t or where such information is missing.

4.2 EGM summary

4.2.1 Cumulative distribution of included studies by publication year

This graph illustrates the cumulative number of studies over time, segmented by publication year and differentiated by intervention types or thematic areas, with each colored segment indicating a distinct category. Each colored segment represents a different category, illustrating how the composition of studies has evolved over time. The layered bars indicate the growing body of research within each category, providing a visual representation of research trends and focus areas in a longitudinal perspective.

4.2.2 Cumulative distribution of included studies by publication year (Aggregated into five years)

The chart likely exhibits the aggregate distribution of studies across various publication years, possibly separated by study type or methodological approach. The data appears to be grouped into time intervals, which could help in identifying trends and patterns in research activities over time.

📝 Note - Color coding

Each intervention type is assigned a unique color based on its frequency in the dataset, ensuring a consistent color scheme across all plots for easy trend identification and comparison. Color for each intervention will be assigned based on the frequency. And this color code will be consistent across other plots

4.2.3 EGM custom variable

The provided bar chart and table offer a breakdown of the number of studies conducted at varying intervention scopes, classified as Local/community, Subnational, National, Multi-level, and Transnational. The bar chart displays a descending trend in the number of studies from local to transnational scopes. Additionally, the table complements the chart by detailing the methodological approach of the studies, further categorized into Quantitative, Qualitative, and Systematic Review. The table includes numerical data and percentages, highlighting the distribution and methodological preferences within each intervention scope. Users can select one specific variable of interest and the plot and table will be updated automatically.

4.2.4 Equity count

Three bar charts detail equity considerations in research methodologies: the first shows the frequency of equity codings, the second details the number of impact evaluations with multiple equity codings, and the third categorizes systematic reviews by equity-sensitive approaches.

  • The first chart is a horizontal blue bar chart showing the frequency of various equity codings

  • The second is a vertical maroon bar chart detailing the number of impact evaluation studies with multiple equity codings

  • The third chart, a vertical blue bar chart, categorizes the number of studies from a systematic review that included equity-sensitive approaches.

4.3 EGM detail

4.3.1 Intervention Overview

The pie chart illustrates the proportional distribution of studies across various intervention categories within a specific research field. Different colors in the pie chart correspond to different intervention types, which are quantified and explained in the legend. The number of intervention code will be the same when selecting study arm level as unit of analysis.

The pie chart provides a visual representation of the distribution of studies by intervention category within a particular research domain. Each segment’s color uniquely identifies an intervention type, with the corresponding quantities detailed in the legend. It is important to note that the total count of intervention codes remains consistent when “study level” is selected as the unit of analysis.

4.3.2 Distribution of Included Studies by Intervention Domain and Subdomain (intervention level)

The second chart showcases a grouped bar chart that compares the number of studies across different intervention domains, segmented into two types of research methods: Impact Evaluation and Systematic Review. The interventions are listed on the vertical axis, and the horizontal axis displays the number of studies. Color coding is used to differentiate between the types of interventions as shown in the chart’s key.

4.3.3 Outcome Overview

The final chart provides a visual comparison of the number of studies according to the outcomes they measure within the same research field. This bar chart is organized to display various outcomes on the vertical axis, with the corresponding number of studies on the horizontal axis.

4.3.4 Distribution of Included Studies by Intervention Domain and Subdomain (outcome level)

In a similar format to the previous chart, this graph further refines the data by showcasing the number of studies that correspond to specific intervention subdomains at the outcome level. It retains the use of grouped bars to compare the frequencies between different outcome categories, offering a detailed account of where research efforts are concentrated in terms of measured effects.

4.3.5 Intervention x Outcome Domain

A scatter plot, “Intervention vs. Outcome Domain,” cross-references intervention and outcome domains, with dot sizes reflecting the frequency of each intervention-outcome pairing. The size of each dot corresponds to the number of studies, providing a visual representation of the frequency at which each intervention-outcome pair has been studied. The intervention domains are listed on the vertical axis, while the outcome domains are spread across the horizontal axis. Two types of studies are differentiated by colors: ‘Impact Evaluation’ and ‘Systematic Review’. The bottom right corner includes a key indicating the total number of unique studies and total observations.

The accompanying text provides a tabular breakdown of the counts for each intervention-outcome combination, offering a detailed numeric representation to complement the visual data in the scatter plot. This table is beneficial for readers who prefer or require exact figures to understand the data distribution patterns.

4.3.6 Multi-Domain Interventions

The table is organized to show how often each combination of interventions occurs across different outcome domains, which are listed as columns. The “Number of Intervention Domains” column indicates whether the interventions are dual, triple, or quadruple combinations. The “total_count” reflects the total frequency of each intervention combination across all outcome domains.

Each row represents a different combination of intervention domains and displays the count of studies addressing various outcomes such as service availability, knowledge attitudes, public behaviors, health outcomes, practices of gender-based violence, and enabling environments. The table identifies the complex interplay of multiple interventions and their associated research focus areas, allowing for an understanding of which intervention combinations are most frequently studied in relation to specific undefined

4.3.7 Detailed Multi-Domain interventions

The scatter plot displays a multi-domain intervention analysis, showing the relationship between different intervention domains and a variety of outcome domains. The vertical axis lists intervention domains while the horizontal axis details outcome domains. The size of the dots indicates the volume of studies pertaining to each intervention-outcome pair within the multi-domain analysis.

The table provides a quantitative summary of the same multi-domain intervention analysis, detailing the count of studies for each intervention-outcome combination. It presents a structured count across a matrix of intervention domains against outcome domains, offering a detailed enumeration of the data points illustrated in the scatter plot.

4.4 Methods

4.4.1 Evaluation Design

The chart shows the percentage and the number of studies in each category.

4.4.2 Evaluation Method

  • 📈The pie chart breaks down into more specific research methodologies, including Randomised Controlled Trials, Fixed Effects, and others, indicating the number of studies for each method with corresponding percentages.

  • 𝄜 The accompanying table titled “Detailed study method” provides a breakdown of the research methodologies used within the experimental and quasi-experimental categories. It lists the specific methods such as Randomised Controlled Trials, Fixed Effects, and others, along with the count (n) of studies employing each method. The table further details the quasi-experimental methods into sub-categories, each with its respective count and implied percentage of the total quasi-experimental studies. Additionally, it includes a section for qualitative methodologies and their respective usage frequency.

4.4.3 Mixed Method

This pie chart demonstrates the division of studies that either incorporate or do not incorporate mixed methods, presenting the data in a simple ‘yes’ or ‘no’ comparative format with the number of studies and their corresponding percentages displayed.

4.4.4 Pre-registration

The pie chart categorizes studies based on whether they were pre-registered. It delineates the data into three categories: ‘Yes’, ‘No’, and ‘Missing’, each with their associated count and percentage of the total number of studies.

4.4.5 Ethical approval

This pie chart shows the ethical approval status across the studies, segmented into ‘Yes’, ‘No’, and ‘Missing’. It provides a visual representation of the proportion and actual number of studies within each ethical approval category.

4.4.6 Intervention by study design

  • 📈This horizontal bar chart displays the number of studies categorized by intervention type and study method. Interventions are listed on the vertical axis and include categories like ‘Multi-domain’ and ‘Vouchers, cash, or in-kind transfers’. The horizontal axis shows the count of studies, with separate bars representing the number of ‘Impact Evaluation’ and ‘Systematic Review’ studies for each intervention type.

  • 𝄜 The table summarizes data across three research methodologies: Quantitative, Systematic Review, and Qualitative. Each row represents a different intervention category, with the corresponding number and percentage of studies employing that methodology. The bottom row provides the total count of studies for Quantitative and Systematic Review methods, while the Qualitative column indicates the number of studies without specifying a total count.

4.4.7 Unit of Analysis

The chart is a horizontal bar chart indicating the distribution of total studies across various units of analysis, such as individual, household, and community, showing the number of studies for each category.

📝 Note - Plot customization

Customize plots via the “Plot handler” feature, allowing adjustments to titles and axes for tailored data presentation.

4.5 SR Critical Appraisals

4.5.1 Intervention by SR Critical Appraisals

  • 📈This horizontal bar chart provides insights into the distribution of studies based on intervention domains and categories from systematic review critical appraisals. The horizontal axis reflects the study count.

  • 𝄜 The accompanying table compiles the study count data across different intervention domains. The vertical axis represents the outcomes of systematic review critical appraisals, while the horizontal axis lists the individual interventions.

4.5.2 Outcome by SR Critical Appraisals

  • 📈This horizontal bar chart offers a comprehensive view of studies categorized by outcome domains and systematic review critical appraisal categories. Similar to the previous chart, the horizontal axis denotes the study count.

  • 𝄜 The table, in parallel, summarizes the study count within distinct outcome domains. The vertical axis indicates the results of systematic review critical appraisals, and the horizontal axis identifies each intervention being studied.”

4.6 Geographic info

4.6.1 Global Distribution of Impact Evaluations by Country (More control)

The map serves as a visual representation illustrating the geographical spread of impact evaluations across different countries. This particular tab offers users several options for control:

  1. Selection of Study Type and Its Distribution: Users have the ability to choose the type of studies they wish to view and analyze in terms of their distribution.

  2. Study Status Selection: This tab allows users to filter studies based on their status, including ‘Completed,’ ‘Ongoing,’ ‘Protocol,’ and ‘Protocol/Ongoing.’

Additionally, users have the option to export raw country-level data by utilizing the ‘Country Coding Download’ button for further analysis and research purposes.”

4.6.2 Proportional distribution of intervention by continent

This chart provides an overview of the intervention distribution of studies based on World Bank regional categorization. It offers insights into how studies are distributed across different regions.

4.6.3 Proportional distribution of outcome by continent

This chart provides an overview of the outcome domain distribution of studies based on World Bank regional categorization. It offers insights into how studies are distributed across different regions.

4.7 Agency

The ‘Agency’ tab contains fields for three main types of agencies: Research Funding Agency, Implementation Agency, and Program Funding Agency. Due to multiple entries for some studies, the number of observations can exceed the total number of studies.

4.7.1 Agency Overview

The first set of bar charts categorizes studies based on the type of funding or implementing agency, such as Government agencies, Academic institutions, and others. Each chart corresponds to a different agency type: Research Funding, Implementation, and Program Funding. The bars indicate the count of studies associated with each agency, excluding instances with missing information. The charts aim to provide a visual distribution of the studies’ affiliations, with the recognition that some agencies may appear under multiple entries. For example, if DEP ID: 15505 contains 10 mentions of various funding agencies but only two unique names, “Government Agency” and “International Financial Institution,” the chart will reflect these two distinct categories.

📝 Note - Display control panel in agency tab

A control panel is available within the ‘Agency’ tab for customized data viewing. By deselecting an agency type, such as “Implementation Agency,” the displayed plots and tables will adjust to show only the information related to the remaining selected agency types.

4.7.2 Agency detail

The table offers a detailed account of the distribution of studies among various types of agencies for both Impact Evaluation and Systematic Review. It shows the count of studies per agency type, providing an overview of the involvement of different agency categories in research funding and implementation. The table is instrumental for understanding the landscape of agency participation in research studies, excluding missing or unspecified data to maintain clarity.

4.7.3 Top 10 research funders

This horizontal bar chart displays the most frequently mentioned agencies across studies, ranking the top 10 for each category: Research Funding, Implementation, and Program Funding. The purpose of this chart is to identify and highlight the agencies most commonly involved in funding research studies, distinguishing the primary contributors in the field.

4.7.4 Agency data cleaning

The agency dataset contains common inconsistencies that necessitate cleaning for accurate analysis. These issues include:

  1. Case Sensitivity Errors: Variations in capitalization may lead to the same agency being counted multiple times (e.g., ‘USAID’ vs ‘usaid’).

  2. Naming Conventions: Different but synonymous names for the same agency are not automatically recognized as identical (e.g., ‘USAID’ vs ‘United States Agency for International Development’).

  3. Categorization Mistakes: Agencies may be misclassified (e.g., ‘USAID’ sometimes erroneously coded as a ‘Government agency’ rather than an ‘International aid agency’).

The provided Excel file is equipped with functionalities to rectify these discrepancies, consolidating entries and ensuring each agency is counted only once under a standardized name and correct category.

📝 Note - Agency data cleaning excel file

The Excel file designated for agency data cleaning comprises two pivotal worksheets:

  1. Common Agencies Sheet: This lists frequently mentioned agencies and assigns a uniform code for each, streamlining disparate entries into a single recognized identifier.

  2. Cleaning Data Sheet: This worksheet delves into detailed naming variations, offering a mapping function to amalgamate different iterations of agency names (like ‘Oxfam’, ‘Oxfam America’, etc.) into a single, consolidated entry.

By utilizing this Excel tool, users can automate the data cleaning process, significantly enhancing the accuracy of the agency-related datasets.

4.8 Authors

The ‘Authors’ tab is structured to visualize authorship data within the research. It provides an array of plots that depict the quantity of authors per study and examines the geographical data regarding the authors’ locations and the countries they study.

4.8.1 Number of Authors Histogram

The histogram illustrates the distribution of the number of authors per study, categorized by the type of study. This visual representation offers a broad view of authorship patterns across studies.

4.8.2 Map of Impact Evaluation by Author`s Country

This map depicts the geographic distribution of impact evaluations, showing where the authors of the studies are based. The map excludes entries with unspecified or unreported data, ensuring clarity in the visualization of the authorship locations.

4.8.3 Relationship Between the Author`s Country and the Country Being Studied

The horizontal bar chart reveals instances where the authors’ country of residence aligns with the country being studied. It highlights the top twenty countries where this concordance is most common, offering insights into local versus international research efforts.

4.8.4 Network Plot of Author`s Country and Studies Country

The network plot displays the connections between authors’ countries and the countries in which their studies are conducted. This plot delineates the flow of research from the origin to the target, illustrating the global landscape of evidence production.

4.8.5 Control box

  • The first authors vs authors : A control panel allows users to filter the data according to whether they want to consider only the first-listed author of a study or all listed authors.

  • Single country vs Multiple country: Another filter in the control panel lets users choose between visualizing studies with a single country focus or those encompassing multiple countries.

4.9 Takeaways

  • Data Cleaning and Accuracy: The plots are constructed after applying specific data cleaning rules to ensure accuracy, with captions included for transparency about the number of studies and any missing values.

  • Export Functionality: Many of the tabs provide functionality for data export, such as EGM coding and country coding, allowing for detailed external analysis.

  • Customization of Visuals: The “Plot handler” tab offers options to customize the titles and labels of the axes on the plots, enabling personalized data representation.