Skip to content

Instantly share code, notes, and snippets.

@plu5
Created April 10, 2022 09:49
Show Gist options
  • Select an option

  • Save plu5/6c2247d6046bb827bdd2cb578dfca67d to your computer and use it in GitHub Desktop.

Select an option

Save plu5/6c2247d6046bb827bdd2cb578dfca67d to your computer and use it in GitHub Desktop.
Reddit tally number of comments on a list of threads per user using PRAW [yet yet yet yet again]
import praw
from collections import defaultdict
discussion_threads = \
["https://www.reddit.com/r/thehemingwaylist/comments/rthfhy/buddenbrooks_chapter_1/",
"https://www.reddit.com/r/thehemingwaylist/comments/ru7p9c/buddenbrooks_chapter_2/",
"https://www.reddit.com/r/thehemingwaylist/comments/rv0o67/buddenbrooks_thomas_mann_chapter_3_discussion/",
"https://www.reddit.com/r/thehemingwaylist/comments/rvsc05/buddenbrooks_thomas_mann_chapter_4_discussion/",
"https://www.reddit.com/r/thehemingwaylist/comments/rwluwr/buddenbrooks_thomas_mann_chapter_5_discussion/",
"https://www.reddit.com/r/thehemingwaylist/comments/rxdycb/buddenbrooks_thomas_mann_chapter_6/",
"https://www.reddit.com/r/thehemingwaylist/comments/ry6g8o/buddenbrooks_thomas_mann_chapter_7/",
"https://www.reddit.com/r/thehemingwaylist/comments/ryyhrb/buddenbrooks_thomas_mann_chapter_8/",
"https://www.reddit.com/r/thehemingwaylist/comments/rzot5g/buddenbrooks_thomas_mann_chapter_9/",
"https://www.reddit.com/r/thehemingwaylist/comments/s0hi8g/buddenbrooks_thomas_mann_chapter_10/",
"https://www.reddit.com/r/thehemingwaylist/comments/s1aafj/buddenbrooks_thomas_mann_part_2_chapter_1/",
"https://www.reddit.com/r/thehemingwaylist/comments/s24twk/buddenbrooks_thomas_mann_part_2_chapter_2/",
"https://www.reddit.com/r/thehemingwaylist/comments/s2wmss/buddenbrooks_thomas_mann_part_2_chapter_3/",
"https://www.reddit.com/r/thehemingwaylist/comments/s3qcul/buddenbrooks_thomas_mann_part_2_chapter_4/",
"https://www.reddit.com/r/thehemingwaylist/comments/s4jb4p/buddenbrooks_thomas_mann_part_2_chapter_5/",
"https://www.reddit.com/r/thehemingwaylist/comments/s5axz0/buddenbrooks_thomas_mann_book_2_chapter_6/",
"https://www.reddit.com/r/thehemingwaylist/comments/s63mjf/buddenbrooks_part_2_chapter_7/",
"https://www.reddit.com/r/thehemingwaylist/comments/s70ki8/buddenbrooks_part_3_chapter_1/",
"https://www.reddit.com/r/thehemingwaylist/comments/s7pu28/buddenbrooks_book_3_chapter_2_discussion/",
"https://www.reddit.com/r/thehemingwaylist/comments/s8i3sr/buddenbrooks_book_3_chapter_3/",
"https://www.reddit.com/r/thehemingwaylist/comments/s98p21/buddenbrooks_book_3_chapter_4/",
"https://www.reddit.com/r/thehemingwaylist/comments/sa5upo/buddenbrooks_book_3_chapter_5/",
"https://www.reddit.com/r/thehemingwaylist/comments/saqxdh/buddenbrooks_book_3_chapter_6/",
"https://www.reddit.com/r/thehemingwaylist/comments/sbjjw3/buddenbrooks_book_3_chapter_7/",
"https://www.reddit.com/r/thehemingwaylist/comments/scbfsk/buddenbrooks_book_3_chapter_8/",
"https://www.reddit.com/r/thehemingwaylist/comments/sd3cy2/buddenbrooks_book_3_chapter_9/",
"https://www.reddit.com/r/thehemingwaylist/comments/sdwkfv/buddenbrooks_book_3_chapter_10/",
"https://www.reddit.com/r/thehemingwaylist/comments/seo1pj/buddenbrooks_book_3_chapter_11/",
"https://www.reddit.com/r/thehemingwaylist/comments/sfhrpf/buddenbrooks_book_3_chapter_12/",
"https://www.reddit.com/r/thehemingwaylist/comments/sg7wc5/buddenbrooks_book_3_chapter_13/",
"https://www.reddit.com/r/thehemingwaylist/comments/sgyp0q/buddenbrooks_book_3_chapter_14/",
"https://www.reddit.com/r/thehemingwaylist/comments/shs1v9/buddenbrooks_book_3_chapter_15/",
"https://www.reddit.com/r/thehemingwaylist/comments/silyna/buddenbrooks_book_3_chapter_16/",
"https://www.reddit.com/r/thehemingwaylist/comments/sjgbfw/buddenbrooks_book_4_chapter_2/",
"https://www.reddit.com/r/thehemingwaylist/comments/skasx7/buddenbrooks_book_4_chapter_3/",
"https://www.reddit.com/r/thehemingwaylist/comments/sl6gbn/buddenbrooks_book_4_chapter_4/",
"https://www.reddit.com/r/thehemingwaylist/comments/sltgvt/buddenbrooks_book_4_chapter_5/",
"https://www.reddit.com/r/thehemingwaylist/comments/smmpe0/buddenbrooks_book_4_chapter_6/",
"https://www.reddit.com/r/thehemingwaylist/comments/snhmxg/buddenbrooks_book_4_chapter_7/",
"https://www.reddit.com/r/thehemingwaylist/comments/so9zcp/buddenbrooks_book_4_chapter_8/",
"https://www.reddit.com/r/thehemingwaylist/comments/sp48m5/buddenbrooks_book_4_chapter_9/",
"https://www.reddit.com/r/thehemingwaylist/comments/spxnqh/buddenbrooks_book_4_chapter_10/",
"https://www.reddit.com/r/thehemingwaylist/comments/sqproh/buddenbrooks_book_4_chapter_11/",
"https://www.reddit.com/r/thehemingwaylist/comments/srgndy/buddenbrooks_book_5_chapter_1/",
"https://www.reddit.com/r/thehemingwaylist/comments/ss7mxj/buddenbrooks_book_5_chapter_2/",
"https://www.reddit.com/r/thehemingwaylist/comments/st08u8/buddenbrooks_book_5_chapter_3/",
"https://www.reddit.com/r/thehemingwaylist/comments/stsdv4/buddenbrooks_book_5_chapter_4/",
"https://www.reddit.com/r/thehemingwaylist/comments/sunn4f/buddenbrooks_book_5_chapter_5/",
"https://www.reddit.com/r/thehemingwaylist/comments/svh50a/buddenbrooks_book_5_chapter_6/",
"https://www.reddit.com/r/thehemingwaylist/comments/swa0o5/buddenbrooks_book_5_chapter_7/",
"https://www.reddit.com/r/thehemingwaylist/comments/swxzbh/buddenbrooks_book_5_chapter_8/",
"https://www.reddit.com/r/thehemingwaylist/comments/sxqkim/buddenbrooks_book_5_chapter_9/",
"https://www.reddit.com/r/thehemingwaylist/comments/syl2me/buddenbrooks_book_6_chapter_1/",
"https://www.reddit.com/r/thehemingwaylist/comments/sze80g/buddenbrooks_book_6_chapter_2/",
"https://www.reddit.com/r/thehemingwaylist/comments/t085jd/buddenbrooks_book_6_chapter_3/",
"https://www.reddit.com/r/thehemingwaylist/comments/t12cqn/buddenbrooks_book_6_chapter_4/",
"https://www.reddit.com/r/thehemingwaylist/comments/t1v947/buddenbrooks_book_6_chapter_5/",
"https://www.reddit.com/r/thehemingwaylist/comments/t2l5cc/buddenbrooks_book_6_chapter_6/",
"https://www.reddit.com/r/thehemingwaylist/comments/t3cpni/buddenbrooks_book_6_chapter_7/",
"https://www.reddit.com/r/thehemingwaylist/comments/t452fg/buddenbrooks_book_6_chapter_8/",
"https://www.reddit.com/r/thehemingwaylist/comments/t4xh6f/buddenbrooks_book_6_chapter_9/",
"https://www.reddit.com/r/thehemingwaylist/comments/t5pe85/buddenbrooks_book_6_chapter_10/",
"https://www.reddit.com/r/thehemingwaylist/comments/t6ggmx/buddenbrooks_book_6_chapter_11/",
"https://www.reddit.com/r/thehemingwaylist/comments/t76u9y/buddenbrooks_book_7_chapter_1/",
"https://www.reddit.com/r/thehemingwaylist/comments/t7w259/buddenbrooks_book_7_chapter_2/",
"https://www.reddit.com/r/thehemingwaylist/comments/t8m8m3/buddenbrooks_book_7_chapter_3/",
"https://www.reddit.com/r/thehemingwaylist/comments/t9d9ai/buddenbrooks_book_7_chapter_4/",
"https://www.reddit.com/r/thehemingwaylist/comments/ta58nl/buddenbrooks_book_7_chapter_5/",
"https://www.reddit.com/r/thehemingwaylist/comments/tawqxx/buddenbrooks_book_7_chapter_6/",
"https://www.reddit.com/r/thehemingwaylist/comments/tbo20i/buddenbrooks_book_7_chapter_7/",
"https://www.reddit.com/r/thehemingwaylist/comments/tcgkl0/buddenbrooks_book_7_chapter_8/",
"https://www.reddit.com/r/thehemingwaylist/comments/td4kpa/buddenbrooks_part_8_chapter_1/",
"https://www.reddit.com/r/thehemingwaylist/comments/tduonj/buddenbrooks_part_8_chapter_2/",
"https://www.reddit.com/r/thehemingwaylist/comments/telmg2/buddenbrooks_part_8_chapter_3/",
"https://www.reddit.com/r/thehemingwaylist/comments/tfe9la/buddenbrooks_part_8_chapter_4/",
"https://www.reddit.com/r/thehemingwaylist/comments/tg9r2v/buddenbrooks_part_8_chapter_5/",
"https://www.reddit.com/r/thehemingwaylist/comments/th01i5/buddenbrooks_part_8_chapter_6/",
"https://www.reddit.com/r/thehemingwaylist/comments/thtvj7/buddenbrooks_part_8_chapter_7/",
"https://www.reddit.com/r/thehemingwaylist/comments/tihe24/buddenbrooks_part_8_chapter_8/",
"https://www.reddit.com/r/thehemingwaylist/comments/tj919k/buddenbrooks_part_8_chapter_9/",
"https://www.reddit.com/r/thehemingwaylist/comments/tk0lkg/buddenbrooks_part_9_chapter_1/",
"https://www.reddit.com/r/thehemingwaylist/comments/tktvcn/buddenbrooks_part_9_chapter_2/",
"https://www.reddit.com/r/thehemingwaylist/comments/tm3faw/buddenbrooks_part_9_chapter_3/",
"https://www.reddit.com/r/thehemingwaylist/comments/tnjwlz/buddenbrooks_part_9_chapter_4/",
"https://www.reddit.com/r/thehemingwaylist/comments/tolist/buddenbrooks_part_10_chapter_1/",
"https://www.reddit.com/r/thehemingwaylist/comments/tphi0a/buddenbrooks_part_10_chapter_2/",
"https://www.reddit.com/r/thehemingwaylist/comments/tq5vna/buddenbrooks_part_10_chapter_3/",
"https://www.reddit.com/r/thehemingwaylist/comments/tqvqyh/buddenbrooks_part_10_chapter_4/",
"https://www.reddit.com/r/thehemingwaylist/comments/ts4qc1/buddenbrooks_part_10_chapter_5/",
"https://www.reddit.com/r/thehemingwaylist/comments/tswtk1/buddenbrooks_part_10_chapter_6/",
"https://www.reddit.com/r/thehemingwaylist/comments/ttr702/buddenbrooks_part_10_chapter_7/",
"https://www.reddit.com/r/thehemingwaylist/comments/tuh2cs/buddenbrooks_part_10_chapter_8/",
"https://www.reddit.com/r/thehemingwaylist/comments/tv6x4x/buddenbrooks_part_10_chapter_9/",
"https://www.reddit.com/r/thehemingwaylist/comments/tvzrtz/buddenbrooks_part_11_chapter_1/",
"https://www.reddit.com/r/thehemingwaylist/comments/twtwy0/buddenbrooks_part_11_chapter_2_12/",
"https://www.reddit.com/r/thehemingwaylist/comments/txl1y5/buddenbrooks_part_11_chapter_2_23/",
"https://www.reddit.com/r/thehemingwaylist/comments/tydprm/buddenbrooks_part_11_chapter_2_33/",
"https://www.reddit.com/r/thehemingwaylist/comments/tyzx9h/buddenbrooks_part_11_chapter_3/",
"https://www.reddit.com/r/thehemingwaylist/comments/tzt8e2/buddenbrooks_part_11_chapter_4_final_chapter/"]
reddit = praw.Reddit(client_id='PLACEHOLDER CLIENT ID',
client_secret='PLACEHOLDER CLIENT SECRET',
user_agent='PLACEHOLDER USER AGENT')
tally = defaultdict(int)
for url in discussion_threads:
submission = reddit.submission(url=url)
# do away with MoreComments objects
submission.comments.replace_more(limit=None)
comments = submission.comments.list()
for comment in comments:
author = comment.author.name if comment.author else "[deleted]"
tally[author] += 1
# output
print("Total participants: ", len(tally))
for index, k in enumerate(sorted(tally, key=tally.get, reverse=True)):
print(index + 1, ':', k, tally[k])
@plu5
Copy link
Author

plu5 commented Jan 26, 2023

The Google Apps Script script on the Oxford Book of English Verse spreadsheet to populate the column with total number of comments per thread:

function getNumberOfCommentsPerThread() {
  const outputSheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Data-noblanks");
  const dataRange = outputSheet.getRange("C2:C");
  const numRows = dataRange.getNumRows();
  const outputArray = [];

  for (let i = 1; i <= numRows; i++) {
    const url = dataRange.getCell(i,1).getValue();
    if (!url) break;
    const response = UrlFetchApp.fetch(url + '.json');
    const result = JSON.parse(response)[0].data.children[0];
    const data = result.data;
    outputArray.push([[data.num_comments]]);
  }

  console.log(outputArray);

  const range = "D2:D" + (outputArray.length + 1);
  outputSheet.getRange(range).setValues(outputArray);

  // set cell with last update date
  const now = new Date();
  outputSheet.getRange("G2").setValue('last update: ' + now.toISOString());
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment