When I first stepped into my role, I wanted to quickly get a clear picture of the competitive landscape in my domain. Traditional methods — digging through competitors’ websites, reading product pages, or scanning YouTube reviews — felt too polished and often biased. I was looking for something more raw and authentic: real discussions from real users.
That’s when I turned to Reddit.
Why Reddit?
Unlike polished marketing materials, Reddit is filled with unfiltered conversations. It’s where users openly share their frustrations, wish lists, and honest takes on products. Tapping into that collective voice felt like the right way to uncover themes, pain points, and opportunities that wouldn’t surface elsewhere.
Automating the Research
Manually scrolling through endless Reddit threads wasn’t realistic. So I turned to AI to automate the heavy lifting. Using Anthropic’s Claude, I asked it to guide me step-by-step in building a Reddit scraper.
Claude walked me through everything:
- Setting up credentials in Reddit’s developer portal.
- Writing a Python script that connected to Reddit’s API.
- Automating inputs, so I could type in a competitor’s name (say, Notion or Asana) and instantly gather every relevant discussion.
What I got back was a treasure trove: tens of thousands of conversations about each competitor.
Turning Raw Data Into Insights
Scraping data was just the start. I then fed these files back into Claude, asking it to analyze:
- What themes dominated conversations?
- Which pain points kept recurring?
- What feature requests surfaced most often?
- How did users talk about AI features — their expectations vs. their reality?
Suddenly, I had a clear, unbiased snapshot of the market. Instead of reading polished press releases, I was seeing products through the eyes of real users.
Why This Matters
This process gave me a powerful starting point. I could form hypotheses on competitor strengths and weaknesses, identify gaps to address, and better understand user expectations around AI in particular. It wasn’t just competitive intelligence — it was a roadmap for where to focus.
Takeaway
By pairing AI with a community like Reddit, you can shortcut months of manual research and uncover insights that truly reflect the voice of the customer.
קריאת הבלוג 
האזנה לפרק 
