A Framework for Digitization of Asphalt Mix EPD Data using Python-Based Web Data Extraction

Abstract: Systematic extraction of massive datasets from web sources poses substantial challenges in a variety of disciplines. This is especially true when data is scattered across multiple files, each of which must be downloaded and processed separately. This issue affects a wide range of stakeholders, including public agencies, research organizations, and academic institutions. On ongoing research study at Oklahoma State University is addressing this challenge in the context of Environmental Product Declarations (EPDs) for asphalt mixtures. A central repository developed by the National Asphalt Pavement Association (NAPA) stores publicly accessible EPDs for > 3,300 (as of October 2024) asphalt mixes in the United States. Each EPD file contains several tabular data sets, including Environmental Impact Summary tables and Life Cycle Impact Indicators. As states get ready to implement specifications targeting green public procurement regulations, analysis of the EPD data for benchmark developments is critical. Given the large number of files, manual extraction of these tables would be extremely time-consuming. To address this, the authors have developed an automated data extraction framework using Python. This approach enables effective parsing and structuring of tabular data from HTML files. This study represents the first ever comprehensive comparative analysis of all publicly available EPDs for asphalt mixtures in the US, a task previously hindered by data extraction challenges. This paper will present details about the extraction methodology, demonstrating how open-source Python programming can be used by transportation agencies as well as by researchers for large-scale web data extraction. The algorithm will be explained along with visualization of the extracted data. The proposed framework will be of great value, considering the current emphasis global warming potential (GWP) benchmarking and threshold developments to define low-carbon transportation materials.