Template-Type: ReDIF-Paper 1.0 Author-Name: Eric S. M. Protzer Author-Email: eprotzer@hks.harvard.edu Author-Workplace-Name: Center for Global Development Author-Name: Sultan Orazbayev Author-Name: Andres Gomez-Lievano Author-Email: Andres_Gomez@hks.harvard.edu Author-Workplace-Name: Center for International Development at Harvard University Author-Name: Matte Hartog Author-Email: matte_hartog@hks.harvard.edu Author-Workplace-Name: Center for International Development at Harvard University Author-Name: Frank Neffke Author-Email: frank_neffke@hks.harvard.edu Author-Workplace-Name: Center for International Development at Harvard University Author-Person: pne139 Title: A New Algorithm to Efficiently Match U.S. Census Records and Balance Representativity with Match Quality Abstract: We introduce a record linkage algorithm that allows one to (1) efficiently match hundreds of millions of records based not just on demographic characteristics but also name similarity, (2) make statistical choices regarding the trade-off between match quality and representativity and (3) automatically generate a ground truth of true and false matches, suitable for training purposes, based on networked family relationships. Given the recent availability of hundreds of millions of digitized census records, this algorithm significantly reduces computational costs to researchers while allowing them to tailor their matching design towards their research question at hand (e.g. prioritizing external validity over match quality). Applied to U.S Census Records from 1850 to 1940, the algorithm produces two sets of matches, one designed for representativity and one designed to maximize the number of matched individuals. At the same level of accuracy as commonly used methods, the algorithm tends to have a higher level of representativity and a larger pool of matches. The algorithm also allows one to match harder-to-match groups with less bias (e.g. women whose names tend to change over time due to marriage). Creation-Date: 2024-12 Keywords: U.S. Census, Machine Learning, Network Science Number: 238 Handle: RePEc:glh:wpfacu:238 File-Url: https://growthlab.hks.harvard.edu/sites/projects.iq.harvard.edu/files/2024-12-glwp-238-ipums_matching.pdf File-Format: application/pdf