Skip to content

Guest article: Using Twitter as a data source: an overview of social media research tools

Twitter and other social media platforms represent a large and largely untapped resource for social data and evidence. In this post, Wasim Ahmed updates his recurring series on the Impact Blog, to bring you the latest developments in digital methods and methodologies for researching Twitter and other social media platforms.

This post builds upon the 2015, and 2017 editions of this post, captures key trends and events which are shaping social media research for social scientists and provides a collection of research methods and tools for the analysis of social media data.

Since the 2017 edition of this blog post, I have seen even more unique and interesting uses of social media data across a wide variety of research disciplines, such as sociology, computer science, media and communication, political science, and engineering  to name only a few. Social media platforms generate a vast amount of data on a daily basis on a variety of topics and consequently represent a key source of information for anyone seeking to study 21st century society.

Twitter remains the most popular platform for academic research, as it still provides its data via a number of Application Programming Interfaces (API). In contrast, the aftermath of the Cambridge analytica ‘data breach’ has led to certain social media platforms to limit data provided through their Application Programming Interfaces. However, although, it may not be possible to get data from all social media platforms, it is still possible to conduct qualitative and quantitative research such as interviews and surveys, with members of online communities.

Studies in social media can be framed by drawing on a wide-variety of theories, constructs and conceptual frameworks from a wide-variety of disciplines and I would recommend taking a look at this paper: Social media research: Theories, constructs, and conceptual frameworkswhich nicely summarises a number of these approaches.

There are also a number of research approaches that can be drawn upon such as Netnography and Digital Ethnography, which provide frameworks for conducting research in the online world. Netnography, for instance, can be based on downloading data directly from a social media platform, noting personal observations of an online community, and interviewing social media users. Furthermore, there are also a number of specific methods for the analysis of social media data summarised in Table 1 below.


 Overview of research methods


Table 2 below provides an overview of tools for retrieving social media data

An overview of tools for 2019


Recently, it has also become increasingly difficult for academics to access historical Twitter data with a number of services for academics coming to an end. This has given rise to services such as those provided by ScrapeHero which allow users to pull in historical Twitter data for free using web-scraping. However, this form of retrieving Twitter is not recommended.

For researching other platforms on the Internet, such as web forums, blogs and other social media platforms there are tools such as Scrape Storm which is an AI-powered visual web scraper and claims to be able to retrieve data from almost any platform.

There are also a number of advanced data analysis and statistical applications which can be used to analyse social media data, such as:

These packages should be researched when deciding which application is to be used for a project. I’d also like to mention The Digital Methods Initiatives list of tools, and Ryerson University’s list of tools from its Social Media Lab. For retrieving Twitter data it is also worth checking out the DMI-TCAT (free). A further review of 100 social media tools was recently published by SAGE Ocean.

For image analysis I would recommend checking out the Google Cloud vision AI and there are also tools such as Instaloader which allow you to download Instagram photos of public accounts. A really interesting study was conducted on Instagram and analysed the hashtag #CheatMeal using thematic content analysis and it can be accessed here.

Another rapidly developing field of social media research looks at ethics. It is important to conduct ethical social media research and I recently published an open access book chapter, which examines the use of Twitter as a source of data and provides an overview of ethical, legal and methodological challenges. The chapter can be accessed here.

Due to a number of requests I have also started to run regular training events (see a list here) with virtual attendance possible. The first of these events took place at the London School of Economics and Political Sciences on May 17th 2019 and our hashtag #SMRM19 contains a host of informative material as the event was live tweeted.


About the author

 Dr. Wasim Ahmed is a Lecturer in Digital Business and a social media researcher with many years of experience working within academia with government and industry. Wasim hosts his own research blog, which links to a wide-variety of resources related to social media research. Wasim is a keen Twitter user (@was3210), and will be happy to answer any technical (or non-technical!) questions you may have.

This article originally appeared on LSE

Scroll To Top