AI Insight
This study uses simulation models based on mobility data from four U.S. metropolitan areas to examine how location-based recommendation systems create feedback loops that affect urban segregation and business survival. The research finds that recommender algorithms cause at least 20% of food venues to lose all visits, with more than half of this loss due to the system retraining on its own recommendations. Critically, the most accurate algorithms increase experienced income segregation by 15-21% compared to actual visitation patterns, while a re-ranking intervention that promotes less popular venues can reduce both segregation and venue disappearance.
Why it matters
The findings reveal that optimizing recommendation systems for accuracy alone may have harmful societal consequences, increasing economic segregation and accelerating small business closures. This suggests platforms should evaluate their algorithms using metrics beyond prediction accuracy, including social equity measures and economic sustainability indicators.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Location-based recommender systems shape where people go in cities and learn from the visits they steer, forming a human-AI feedback loop. This feedback loop shapes whom people share places with across income groups and which venues keep attracting customers. Yet platforms judge these systems by their offline accuracy on past visits, a test in which the suggestions never change where people go and which therefore cannot capture long-term effects. Studying these effects in cities is challenging because, unlike online resources, travel constraints and limited venue capacity shape where people can go, while each recommended visit also changes whom other visitors encounter. To capture such features, we build a long-term simulation framework based on fine-grained human mobility data from four U.S. metropolitan areas, with distance and capacity constraints. Focusing on food venues, we study two urban consequences: how unevenly people’s exposure is distributed across income groups (experienced income segregation) and the share of venues receiving no visits from simulated residents (venue disappearance). We find that, under every algorithm and in every metropolitan area, at least a fifth of venues stop receiving visits, and that more than half of this loss arises from retraining. More importantly, more accurate algorithms leave people more segregated, and the most accurate algorithm leaves people 15-21% more segregated than they actually are. We then test a re-ranking intervention that favours less popular venues and runs on top of a platform’s existing algorithm. It lowers both venue disappearance and segregation across algorithms, largely without directing people to venues where they may feel less comfortable. Our results suggest that impact assessments of recommender systems should account for the feedback loop and report segregation and venue disappearance beyond accuracy.
Source: Recommendation Feedback Loops Shape Urban Income Segregation and Venue Visitation