X Search Scraper: Monitor Keywords & Hashtags Without Enterprise API
X's official Search API costs $5,000/month for the Enterprise tier. The Basic tier ($200/month) gives you 10,000 tweets per month ā roughly 333 per day. If you're monitoring multiple keywords across multiple brands, that runs out fast.
Most teams don't need the full firehose. They need targeted keyword monitoring: what are people saying about their brand, their competitors, their industry? They need search results, not real-time streaming.
This guide shows you how to search X, monitor keywords over time, track hashtags, and build an alert system ā at a fraction of the Enterprise API cost.
Basic Keyword Search
Start with a simple keyword or phrase search:
const axios = require('axios');
const API_KEY = process.env.SOCIAVAULT_API_KEY;
const BASE_URL = 'https://api.sociavault.com';
async function searchTwitter(query) {
const response = await axios.get(`${BASE_URL}/v1/scrape/twitter/search`, {
params: { query },
headers: { 'X-API-Key': API_KEY }
});
const tweets = response.data.data || [];
console.log(`\n=== X SEARCH: "${query}" ===`);
console.log(`Results: ${tweets.length} tweets\n`);
tweets.slice(0, 10).forEach((tweet, i) => {
const text = tweet.full_text || tweet.text || '';
const stats = tweet.legacy || tweet.statistics || {};
const author = tweet.author?.screen_name || tweet.user?.screen_name || 'unknown';
const likes = stats.favorite_count || stats.like_count || 0;
const retweets = stats.retweet_count || 0;
const replies = stats.reply_count || 0;
console.log(`${i + 1}. @${author} (${likes} likes, ${retweets} RTs, ${replies} replies)`);
console.log(` "${text.substring(0, 150)}${text.length > 150 ? '...' : ''}"`);
console.log('');
});
return tweets;
}
// Search for anything
await searchTwitter('social media analytics');
await searchTwitter('#AIagents');
await searchTwitter('SociaVault');
Cost: 1 credit per search.
Build a Keyword Monitoring System
Track multiple keywords daily and log results for trend analysis:
import requests
import json
import os
from datetime import datetime
API_KEY = os.getenv('SOCIAVAULT_API_KEY')
BASE_URL = 'https://api.sociavault.com'
headers = {'X-API-Key': API_KEY}
def monitor_keywords(keywords, output_file='x-keyword-monitoring.json'):
"""Monitor multiple keywords on X and log results"""
timestamp = datetime.now().isoformat()
results = []
for keyword in keywords:
response = requests.get(
f'{BASE_URL}/v1/scrape/twitter/search',
params={'query': keyword},
headers=headers
)
tweets = response.json().get('data', [])
# Calculate metrics
total_likes = 0
total_retweets = 0
total_replies = 0
authors = set()
for tweet in tweets:
stats = tweet.get('legacy', tweet.get('statistics', {}))
total_likes += stats.get('favorite_count', stats.get('like_count', 0))
total_retweets += stats.get('retweet_count', 0)
total_replies += stats.get('reply_count', 0)
author = (tweet.get('author', {}).get('screen_name') or
tweet.get('user', {}).get('screen_name', 'unknown'))
authors.add(author)
results.append({
'timestamp': timestamp,
'keyword': keyword,
'tweet_count': len(tweets),
'total_likes': total_likes,
'total_retweets': total_retweets,
'total_replies': total_replies,
'unique_authors': len(authors),
'avg_likes': round(total_likes / max(len(tweets), 1), 1)
})
print(f' "{keyword}": {len(tweets)} tweets, {total_likes} likes, {len(authors)} authors')
# Save to log
history = []
if os.path.exists(output_file):
with open(output_file, 'r') as f:
history = json.load(f)
history.extend(results)
with open(output_file, 'w') as f:
json.dump(history, f, indent=2)
print(f'\nLogged {len(results)} keywords at {timestamp}')
return results
# Run daily
monitor_keywords([
'your brand name',
'competitor 1',
'competitor 2',
'industry keyword 1',
'industry keyword 2',
'#YourHashtag'
])
After a week of daily runs, you can compare:
- Which keywords are growing vs declining
- How your brand mentions compare to competitors
- When spikes happen (and correlate with events)
Hashtag Impact Tracking
Want to know if your hashtag campaign is working? Track it:
async function trackHashtag(hashtag, label) {
const response = await axios.get(`${BASE_URL}/v1/scrape/twitter/search`, {
params: { query: hashtag },
headers: { 'X-API-Key': API_KEY }
});
const tweets = response.data.data || [];
// Analyze tweet quality
const metrics = tweets.map(t => {
const stats = t.legacy || t.statistics || {};
return {
author: t.author?.screen_name || t.user?.screen_name,
followers: t.author?.legacy?.followers_count || t.user?.followers_count || 0,
likes: stats.favorite_count || stats.like_count || 0,
retweets: stats.retweet_count || 0,
replies: stats.reply_count || 0,
views: stats.view_count || t.views?.count || 0,
text: (t.full_text || t.text || '').substring(0, 100)
};
});
// Calculate reach
const totalReach = metrics.reduce((s, m) => s + m.followers, 0);
const totalEngagement = metrics.reduce((s, m) => s + m.likes + m.retweets + m.replies, 0);
console.log(`\n=== HASHTAG REPORT: ${hashtag} ===`);
console.log(`Label: ${label}`);
console.log(`Tweets found: ${metrics.length}`);
console.log(`Potential reach: ${totalReach.toLocaleString()} (combined followers)`);
console.log(`Total engagement: ${totalEngagement.toLocaleString()}`);
console.log(`Unique authors: ${new Set(metrics.map(m => m.author)).size}`);
// Top tweets by engagement
metrics.sort((a, b) => (b.likes + b.retweets) - (a.likes + a.retweets));
console.log(`\nTop tweets:`);
metrics.slice(0, 5).forEach((m, i) => {
console.log(` ${i + 1}. @${m.author} (${m.followers.toLocaleString()} followers)`);
console.log(` ${m.likes} likes, ${m.retweets} RTs: "${m.text}..."`);
});
return { metrics, totalReach, totalEngagement };
}
// Track your campaign hashtags
await trackHashtag('#YourCampaign2026', 'Spring Campaign');
await trackHashtag('#CompetitorCampaign', 'Competitor Campaign');
Building an Alert System
Get notified when someone influential mentions your brand or keyword:
def check_for_high_impact_mentions(keywords, follower_threshold=10000):
"""Alert when high-follower accounts mention your keywords"""
alerts = []
for keyword in keywords:
response = requests.get(
f'{BASE_URL}/v1/scrape/twitter/search',
params={'query': keyword},
headers=headers
)
tweets = response.json().get('data', [])
for tweet in tweets:
author = tweet.get('author', tweet.get('user', {}))
author_legacy = author.get('legacy', author)
followers = author_legacy.get('followers_count', 0)
screen_name = author.get('screen_name', author_legacy.get('screen_name', 'unknown'))
if followers >= follower_threshold:
stats = tweet.get('legacy', tweet.get('statistics', {}))
text = tweet.get('full_text', tweet.get('text', ''))
alerts.append({
'keyword': keyword,
'author': screen_name,
'followers': followers,
'likes': stats.get('favorite_count', stats.get('like_count', 0)),
'text': text[:200],
'verified': author.get('is_blue_verified', False)
})
if alerts:
print(f'\nšØ {len(alerts)} HIGH-IMPACT MENTIONS FOUND:\n')
for alert in sorted(alerts, key=lambda a: a['followers'], reverse=True):
badge = 'ā' if alert['verified'] else ''
print(f' @{alert["author"]}{badge} ({alert["followers"]:,} followers)')
print(f' Keyword: "{alert["keyword"]}"')
print(f' "{alert["text"]}..."')
print(f' {alert["likes"]} likes')
print('')
else:
print('No high-impact mentions found.')
return alerts
# Check every few hours
check_for_high_impact_mentions(
['SociaVault', 'sociavault.com', '@sociavault'],
follower_threshold=5000
)
Pipe these alerts into Slack, Discord, or email for real-time notification. More on that in our Slack integration guide.
Competitive Keyword Comparison
Track how often people talk about you vs competitors:
async function shareOfVoice(brands) {
const results = {};
let totalMentions = 0;
for (const brand of brands) {
const response = await axios.get(`${BASE_URL}/v1/scrape/twitter/search`, {
params: { query: brand },
headers: { 'X-API-Key': API_KEY }
});
const tweets = response.data.data || [];
const totalEng = tweets.reduce((s, t) => {
const stats = t.legacy || t.statistics || {};
return s + (stats.favorite_count || stats.like_count || 0) + (stats.retweet_count || 0);
}, 0);
results[brand] = {
mentions: tweets.length,
engagement: totalEng,
avgEngagement: Math.round(totalEng / Math.max(tweets.length, 1))
};
totalMentions += tweets.length;
}
console.log('\n=== SHARE OF VOICE ON X ===\n');
Object.entries(results)
.sort((a, b) => b[1].mentions - a[1].mentions)
.forEach(([brand, data]) => {
const share = ((data.mentions / Math.max(totalMentions, 1)) * 100).toFixed(1);
const bar = 'ā'.repeat(Math.round(parseFloat(share) / 2));
console.log(`${brand}: ${data.mentions} mentions (${share}%) ${bar}`);
console.log(` Total engagement: ${data.engagement.toLocaleString()} | Avg: ${data.avgEngagement}`);
});
return results;
}
await shareOfVoice(['Notion', 'Obsidian', 'Roam Research', 'Logseq']);
Cost Comparison: X Enterprise API vs SociaVault
| Feature | X Enterprise ($5K/mo) | X Basic ($200/mo) | SociaVault API |
|---|---|---|---|
| Search tweets | ā 1M tweets/mo | ā 10K tweets/mo | ā Pay per search |
| Full-archive search | ā | ā | ā (recent only) |
| Real-time streaming | ā | ā | ā |
| Profile data | ā | ā | ā |
| Follower lists | ā | Limited | ā |
| Quote tweets | ā | ā | ā |
| Cost for 100 searches/mo | $5,000 | $200 | ~$5 |
| Best for | Enterprise monitoring | App development | Targeted research |
If you need real-time streaming or full tweet archives, X's Enterprise API is worth it. If you need targeted keyword monitoring, competitor tracking, and brand mention alerts ā you can do that for 99% less.
Get Started
Sign up free ā 100 credits lets you run 100 keyword searches on X.
Full Twitter/X API docs: docs.sociavault.com/api-reference/twitter
Related Reading
Found this helpful?
Share it with others who might benefit
Ready to Try SociaVault?
Start extracting social media data with our powerful API. No credit card required.