The Community Archive: An Ethically Sourced X (Twitter) Dataset

Overview & Synopsis

We present The Community Archive, a novel dataset of over 5 million posts from the social media site X (previously known as Twitter). In contrast with other large scale social media datasets, which are generally assembled without user consent via web scrapers, The Community Archive uses an opt-in model to obtain author consent for every post in the dataset. This enables us to release both the metadata and the content of users' posts at a previously unprecedented scale. Our dataset supports work in a wide variety of domains, including social network analysis, narrative tracking, and diachronic language modeling. We demonstrate the utility of our dataset by conducting a sample analysis of content-based homophily in our network.