Extracting YouTube Comments at Scale for Sentiment Analysis
Extracting YouTube Comments at Scale for Sentiment Analysis
YouTube comment sections are an underrated data source. Unlike a lot of social platforms where reactions are limited to likes or short replies, YouTube comments tend to be longer-form and more specific — people explain why they liked or disliked something, which is exactly the kind of text that makes sentiment analysis actually useful instead of just a like/dislike count with extra steps.
Getting comments out in bulk
Pulling comments manually or via the official YouTube Data API works fine at small scale but gets rate-limited and quota-capped quickly if you're analyzing more than a handful of videos regularly. For bulk collection across many videos or an entire channel's back catalog, you want an endpoint that returns paginated comment threads without you managing OAuth scopes for read-only public data.
import requests
def get_video_comments(video_id, token, cursor=None):
params = {"video_id": video_id, "token": token}
if cursor:
params["cursor"] = cursor
res = requests.get(
"https://ensembledata.com/apis/youtube/video/comments",
params=params
)
return res.json()
Paginate through with the cursor value returned in each response until you've collected the full set (or hit whatever depth is reasonable for your analysis — for a video with 50,000 comments, you rarely need all of them for a representative sentiment read). Full pagination behavior and response fields are documented in EnsembleData's API reference.
Running the actual sentiment pass
Once you've got comments in hand, you don't need anything exotic for a first pass. A pretrained sentiment model (VADER for a quick and dirty baseline, or a transformer-based classifier like a fine-tuned DistilBERT if you want better nuance) applied to each comment gets you a distribution you can plot and track over time.
from nltk.sentiment import SentimentIntensityAnalyzer
sia = SentimentIntensityAnalyzer()
def score_comments(comments):
return [
{"text": c["text"], "score": sia.polarity_scores(c["text"])["compound"]}
for c in comments
]
Watch out for these gotchas
Sarcasm and inside-jokes specific to a channel's community will trip up general-purpose sentiment models — a comment like "this aged well" can be praise or brutal criticism depending entirely on context the model doesn't have. For channels with a strong recurring community, it's worth spot-checking a sample manually before trusting the aggregate numbers, especially if you're using sentiment trends to make actual decisions rather than just satisfying curiosity.
Also worth tracking separately: comment volume over time, independent of sentiment. A video that generates a lot of negative comments is still generating conversation, which isn't automatically bad depending on what you're optimizing for.