Crawl Baidu migration data, divided into 3 types: city, province, country
Use python to crawl the population flow data in the Baidu migration platform: http://qianxi.baidu.com/, and uncomment the corresponding code according to the required type. The crawled content will be saved as an Excel file in the move_in, move_out folders in the root directory of the code
-
Path: Please modify the absolute path before use
-
Date: Please modify the date in the [migration_all_date] function (note that the left interval includes, and the right interval does not include)
-
City interval: Since there is no asynchronous operation set during crawling, it will result in a timeout (time_out) if too many cities are crawled at one time. Therefore, it is recommended to crawl between partitions, you can copy multiple source program files, and start crawling multiple times in different intervals at the same time
-
Run: Run the main.py file directly after making sure the above information is correct
- requests
- xlwt