Leveraging Sentiment and Topic Analysis in Social Media Messages to Understand Quality of Life in Urban Areas

Abstract

Understanding public sentiment, societal trends and the quality of life (QoL) in urban environ- ments is of special interest for urban policy making, planning, and services. The widespread use of social media offers a valuable source of data on public opinion, popular topics, and indicators of QoL in a city. This thesis explores the potential of social media data to reveal insights about urban life by analyzing Twitter content from two major metropolitan areas: New York City (NYC) and São Paulo (SP). QoL assessments often rely on surveys that can be costly, influenced by subjective factors, and limited in scope. The primary goal is to investigate how topic modeling, sentiment analysis and binary classification can be applied to extract meaningful QoL indicators and to identify similarities and differences between the two cities. The research draws on recent advances in natural language processing (NLP) and large lan- guage models (LLMs) to construct a robust data analysis pipeline. The methodology includes collecting and preprocessing large volumes of geo-located Twitter data, followed by the applica- tion of a single LLM to perform multiple tasks: topic analysis, sentiment analysis, and QoL-related binary classification. The pipeline transforms raw textual input into interpretable visualizations. The use of a single LLM model across tasks shows the versatility of LLMs in handling diverse NLP challenges within urban analytics. Validation steps are included to assess model performance and to evaluate the reliability of the results. The comparative study reveals notable differences and commonalities in public discourse be- tween NYC and SP. Topics related to economic conditions emerged more positively in NYC, whereas tweets in SP reflected greater concern towards these. SP showed stronger associations between family themes and perceived QoL, while this link appeared less pronounced in NYC. Both cities showed indications for dissatisfaction towards the political and governmental land- scape. The main contributions of this thesis include (1) demonstrating the adaptability of a single LLM across multiple text analysis tasks, (2) proposing a scalable pipeline for urban sentiment and topic analysis, and (3) offering a comparative framework for assessing QoL indicators via social media. The study concludes with a discussion of the implications for urban studies and suggestions for future research in this interdisciplinary field.