Methods

Data Sources

One of the main challenges in developing the UESI’s data is achieving the right balance between quality, spatial detail, and geographical and temporal coverage. This balance is crucial for creating indicators and indices that accurately reflect environmental performance and the fair distribution of its benefits across cities in both hemispheres. Ultimately, the datasets selected are reviewed and verified for these criteria as part of each iteration, and the sources are updated as needed to ensure the most recent information is included for each city. Datasets are also periodically updated to include additional cities, particularly from developing countries.

Compared to nationally-derived datasets, spatially explicit information at the sub-urban level is generated with much less consistency. To overcome this limitation, the UESI uses primary and secondary data primarily from large-scale remote sensing datasets and open-source geospatial data, such as OpenStreetMap, which is integrated with official administrative data on population and income collected at the neighborhood and block-level. The UESI applies a set of criteria to determine which datasets to select for inclusion (see Box 1: Guiding Principles for Indicator Selection in the UESI).

All sources of data are publicly available and include:

• Satellite data, such as from LANDSAT and MODIS products;
• Geospatial datasets, such as gridded global population, annual ground-level NO₂,  and Gross Domestic Product;
• Open source data, including transportation and geographical boundaries from Open Street Maps;
• Official statistics on population and income from national and local governments collected through census or other initiatives.

Data Quality Tiers

For the socioeconomic variables, we prioritized the use of official population and income data from national, regional and local authorities, but for many cities, we were unable to obtain this data, either because it is not readily available from official websites, or because the data does not exist (i.e., many countries and cities don’t measure or directly report income as a dollar amount but as part of other variables such as socioeconomic status or poverty status). To overcome this problem, primarily for the Equity calculations, we use a proxy for neighborhood-level income data: Gross Domestic Product (GDP) in 2017 USD international dollars data from a globally generated dataset (Kummu et. al. 2024 In Review). While not equivalent to income per capita data, we consider it to be the best available proxy. To ensure transparency, the UESI categorizes cities into Tier I (cities with available income per capita data) and Tier II (cities where GDP per capita is used). This distinction highlights potential differences when comparing equity indicators, as Tier II cities might show different results if the distribution of GDP does not accurately reflect income distribution. We encourage Tier II cities to contact us if their neighborhood-level income per capita data is available.

Constructing the UESI

The UESI framework was developed through a multi-step stakeholder engagement process with input from academic researchers, urban planners, practitioners, and city representatives. The starting point for the environmental issues we selected was drawn from the 2016 Environmental Performance Index (EPI), a biennial ranking that evaluated national environmental performance in 9 high-priority issue areas (e.g., Air and Water Quality, Biodiversity and Habitat Protection, Forests, Fisheries, Agriculture, Climate Change and Energy, Environmental Health). We then conducted a comprehensive literature review of existing urban environmental sustainability metrics to evaluate prior work and where we could add value (Thomas et al., 2020). This process involved reviewing the environmental justice and social equity literature from a multi-disciplinary angle (see Equity and Social Inclusion issue profile) in order to design indicators that incorporate an equity lens in environmental performance. Ultimately, the UESI framework and indicators strike a balance between scientific rigor and data availability. We rely on neighborhood-scale data for most indicators, except for the Water Indicators (Water Stress and Wastewater Treatment) and Climate Change indicators (Climate Policy and GHG Emissions), which are city-wide and not included in the equity calculations.

Filtering ‘non-urban’ neighborhoods

To integrate both environmental and socioeconomic data in the analysis, the UESI primarily used the officially-defined administrative divisions of a city as the spatial units for calculating the environmental indicators, along with corresponding population and income data. However, we identified that in many cases, a city’s own defined boundaries did not necessarily correspond exclusively to an urban area and often included non-urban areas, such as sparsely populated areas, rural sections and even wildlife conservation areas. 

As a result of this preliminary analysis, and due to the need to maintain a certain level of homogeneity between the districts analyzed within a city, we identified and filtered urban areas from non-urban areas, since the inclusion of non-urban areas in the UESI calculation would create unintended results in the equity analysis and the performance scores, by either rewarding or penalizing some cities, depending on the indicator. 

Drawing from the criteria used by organizations like the US Census Bureau and UN-Habitat, we implemented a statistical clustering process. This process combined factors such as land cover data, population density, the neighborhood’s distance from the city center, the neighborhood’s share of the city’s total population, and its proportion of the total city area. Using this method, we were able to classify all neighborhoods and differentiate entirely or partially urbanized areas, rural zones, as well as certain large non-impervious areas that are functionally part of the city, such as large parks near the boundaries of a city’s urban core. 

We identified and filtered around 1,000 neighborhoods that were entirely rural or mostly rural when compared to the rest of our UESI neighborhoods, representing around 6% of the initial set of city neighborhoods. The identified rural or mostly rural areas include the Western Water Catchment in Singapore, Pachacamac in Lima, El Esparragal in Murcia, Chunan County in Hangzhou, Mentougou in Beijing, and some Census Tracts in cities like Albuquerque, Las Vegas, Charlotte and others. Some of these examples can be seen in Figure 2, and further details on the approach can be explored in Box 2: Identifying Non-Urban areas for the UESI.

Leveraging Cloud Computing and Global Data

To scale up our analysis from both the original 32 pilot cities to nearly 300 cities in 2024, we leverage the capabilities of cloud computing resources to gain access to the ever-increasing catalog or remote sensing products, as well as to create replicable methods that would allow us to scale the analysis to additional cities in the future. We used Google Earth Engine, a platform for global scale geospatial analysis to process global data, and extract information from raster datasets to the neighborhood or city level for the UESI cities, as well as Google Collab platform to perform spatial operations between vector datasets. Leveraging the tools and information available through them we were able to incorporate 116 new cities for a total of 278 cities.

UESI Environmental Performance Score and Environmental Equity Indicators Calculation

We present the UESI data in multiple formats, recognizing that not all users experience and process information in the same way. The data available for the UESI includes the raw values in native units, which provide a largely unadulterated view of how neighborhoods and cities perform, as well as environmental performance scores for each category and an aggregated performance score for each city.

The environmental performance scores are transformed values using a “proximity to target” (Box 3: Measuring Environmental Performance: Proximity-to-Target) method to assess how close or far neighborhoods and cities are to achieving an identified policy target. The targets are high-performance benchmarks or established scientific thresholds, such as the benchmark for exposure to fine particulate pollution is 10 μg/m3, a threshold set by the World Health Organization as safe exposure. A high-performance benchmark can also be determined through analyzing the best-performing cities in the UESI. Some of our indicators set benchmarks, for example, at the 95th percentile of the range of data. Scores are then converted to a scale of 0 to 100 by simple arithmetic calculation, with 0 being the farthest from the target and 100 being the closest (See Box 3 for further details). In this way, scores convey analogous meaning across indicators, policy issues, and throughout the UESI.

A key contribution of the UESI lies in its equity indicators, which are developed using a Concentration Curve (see Equity Issue Profile). This approach assesses equity by examining how environmental benefits or hazards are distributed in relation to income distribution within each city. Four categories included in the UESI (Air Quality, Urban Heat, Urban Ecosystem, and Transportation) have an associated Concentration Index that allows the user to evaluate if less affluent citizens might be exposed to a higher proportion of an environmental hazard, or have less access to an environmental benefit.

Finally, the UESI considers two approaches for communicating its results to the larger audience. The first approach is centered around and aggregated environmental performance and inequality indexes, which represent a weighted average of how a city performs on the underlying indicators. All issue categories (e.g., Air Quality, Climate Change, and Urban Ecosystem) are weighted equally (1), with one exception – the Public Transportation Category – which has a lower weight than the other three categories (0.5) due to the limitations and potential issues related to data quality of the source dataset of this category when compared to the rest, given that unlike other categories that are sourced from peer-reviewed datasets, the Public Transportation source is fundamentally crowdsourced from OpenStreetMap. Within each issue category, individual indicators are weighted equally. Figure 4 presents a graphical summary of the weighting schemes for both the aggregated performance and concentration indices.

The city-wide indicators (Water Stress, Wastewater Treatment, and Climate Policy) are not included as part of our aggregated evaluation of each city’s equity indicators, since they are not available for all cities at this time and don’t provide disaggregated information to assess their within-city variability.

In addition to the aggregated results, we also place particular importance in presenting specific results for each of the UESI components, including their scores, relative performance, and equity evaluation. Thus, the 2024 report also includes Issue Profiles that explore each component in detail, as well as a dedicated website and city explorer which provides all the disaggregated results at the neighborhood level for citizens, policy makers and researchers to openly explore. 

Tracking Performance and Equity over time

This report also includes an analysis on the evolution over time of Performance and Equity leveraging the results from the 2019 UESI and this report to give insights on whether cities have become more equitable or increased their performance between the two periods. To perform the analysis in a consistent way, we used the UESI 2019 weighting structure to compose an additional performance and concentration indicator using the 2023 UESI indicators. This process allowed for an evaluation of the UESI performance scores and the concentration performance over time.