Social Media Observatory

Learn to build a web app that can monitor millions of posts across all social media platforms to chart public opinion in real time.

Social media has become our modern-day agora, the place where we get our news and engage in political discourse. With disinformation, hate speech, foreign interference, authoritarianism, market monopolization, and political polarization all on the rise, monitoring social media in the public interest has never been so difficult nor so urgent. Social Media Observatory is an in-depth beginner’s guide to coding full-stack web applications in Python to track and analyze social media activity at scale, giving you unparalleled insights into the cultural and political conversations of our time, then empowering you to reach your own conclusions.

You’ll start by coding a web app to monitor Russian disinformation channels discussing Ukraine on Telegram, then learn more adversarial methods to scrape the Wayback Machine, Instagram, and TikTok. You’ll detect Twitter bots following Saudi influencers, extract text-embedded URLs to reveal a white supremacist blogosphere, parse archived tweets to reveal the discourse leading up to the January 6 insurrection, and learn to circumvent Meta''s and ByteDance''s defenses to scrape Instagram and TikTok, collecting evidence of war crimes in the Middle East. Finally, you’ll write an API and deploy your observatory to a cloud server to collaborate with other investigators. By the end of the book, you’ll have a working website that allows collaborators to log in, request data collection, and perform analysis of collected data, via a point-and-click interface.

Along the way, you’ll learn to:

• Use Python and RabbitMQ to retrieve Telegram data from a remote API
• Store data with PostgreSQL and query it with SQLAlchemy
• Analyze the data you’ve retrieved with Pandas, NetworkX, Gephi, and other tools
• Use state-of-the-art AI tools to transcribe audio and classify text
• Deploy a website with an interactive dashboard using FastAPI, Dash, Cytoscape, Uvicorn, Gunicorn, Nginx, AWS, and Let’s Encrypt
• Containerize your apps with Docker and provision infrastructure with Pulumi