pipette
ESEspañol

The Community Archive: An Ethically Sourced X (Twitter) Dataset

Brandon Duderstadt, Omar Shehata, Ivan Vendrov, Francisco Carvalho

PreprintBold claims, read critically

In the authors' words

We present The Community Archive, a novel dataset of over 5 million posts from the social media site X (previously known as Twitter). In contrast with other large scale social media datasets, which are generally assembled without user consent via web scrapers, The Community Archive uses an opt-in model to obtain author consent for every post in the dataset. This enables us to release both the metadata and the content of users' posts at a previously unprecedented scale. Our dataset supports work in a wide variety of domains, including social network analysis, narrative tracking, and diachronic language modeling. We demonstrate the utility of our dataset by conducting a sample analysis of content-based homophily in our network.

Main resultThe abstract does not state a limitation.

Appeared: Friday, October 23. arXiv. Preprint, not yet peer-reviewed.