weibospider

Weibo Spider is a powerful Sina Weibo crawler program designed to help users efficiently obtain public data from the Weibo platform. It supports continuous data scraping for single or multiple Weibo users, covering user information (such as nickname, gender, location, education/work experience, follower count, etc.) and detailed Weibo content (such as post text, publish time, like/retweet/comment counts, publishing tool, location information, etc.). Key features include: - Multi-format export: Supports saving scraped results as TXT, CSV, JSON files, or directly into MySQL, MongoDB, SQLite databases. - Multimedia download: Supports automatic downloading of original images and videos from Weibo posts. - Flexible configuration: Users can set scraping time ranges, user ID lists, and storage paths via configuration files. - Extensibility: Provides clear API interfaces, facilitating developers to customize or integrate into automated tasks. - Compatibility: Supports Python 3 environment and runs on Windows, Linux, and macOS. The project obtains access permissions by setting cookies and provides detailed documentation guiding users on how to acquire cookies and user IDs. Additionally, it includes citation instructions for academic research and maintains a cookie-free version (weibo-crawler) for user choice. This tool is widely used in social media data analysis, public opinion monitoring, and academic paper research.