Commit Graph

9 Commits (82bc922ccac4de215bdea110cce3a4976536ad00)

Author SHA1 Message Date
voussoir 82bc922cca Transform comment parent_id from pushshift api.
Re #19
2023-02-22 17:50:44 -08:00
voussoir 2c0ae7291a Hardcode pushshift ratelimit due to removal of /meta. 2023-02-22 17:17:45 -08:00
voussoir 6867d32798
Use separate loggers per module instead of common.log.
The separate names assist in debugging.
2021-11-14 13:48:51 -08:00
Ethan Dalool e2d1f2b9ef Some small whitespace and log tweaks. 2020-10-08 14:17:19 -07:00
Ethan Dalool 55d9c064bd Move thank you message into the pushshift module itself. 2020-10-08 13:47:02 -07:00
Ethan Dalool 5adc0943aa Use pushshift's /meta endpoint to discover ratelimit. 2020-09-06 21:42:30 -07:00
Ethan Dalool e136aeea4e Log the size of received batches from pushshift. 2020-09-06 21:41:44 -07:00
Ethan Dalool 83d5a8e70e Fix retry loop not breaking after successful try! 2020-09-06 21:31:46 -07:00
Ethan Dalool b3db322a78 Rename timesearch directory -> timesearch_modules.
This marks the first in a long series of rename commits.
Hold on to your databases!
2020-01-27 18:32:05 -08:00