Using cURL

How to Scrape Twitter User Tweets with cURL

Get recent tweets from a Twitter/X user using cURL. This comprehensive guide will walk you through the entire process, from setup to implementation.

Overview

What You'll Learn

  • Setting up your cURL environment
  • Installing the required HTTP client
  • Authenticating with SociaVault API
  • Making requests to Twitter
  • Handling responses and errors

What You'll Get

  • Access to user tweets data
  • JSON formatted responses
  • Real-time data access
  • Scalable solution
  • Error handling patterns

Prerequisites

1. API Key

First, you'll need a SociaVault API key to authenticate your requests.

Get your free API key

2. Development Environment

Make sure you have the following installed:

  • cURL installed
  • A code editor (VS Code, Sublime, etc.)
  • Command line interface access

Implementation

Step 1: Install HTTP Client

We'll use curl to make HTTP requests.

bash
# No installation required on most systems

Step 2: API Implementation

Now let's make a request to the Twitter API using cURL. Replace YOUR_API_KEY with your actual API key.

curl
curl -X GET "https://api.sociavault.com/twitter/user-tweets?handle=elonmusk" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json"

Testing Your Code

API Parameters

ParameterTypeRequiredDescription
handlestringYesExample: elonmusk

Expected Response

You should receive a structured JSON response containing the user tweets data.

json
{
  "tweets": [
    {
      "__typename": "Tweet",
      "rest_id": "1828402665845322123",
      "core": {
        "user_results": {
          "result": {
            "__typename": "User",
            "id": "VXNlcjo0NTIwMjQxMjA5",
            "rest_id": "4520241209",
            "is_blue_verified": true,
            "profile_image_shape": "Circle",
            "legacy": {
              "created_at": "Fri Dec 18 02:48:59 +0000 2015",
              "default_profile": false,
              "default_profile_image": false,
              "description": "Social Media Scraping API's: https://t.co/eSvJcfOZwF\n\nWeb Scraping Course: https://t.co/aTGlgWnvD7",
              "entities": {
                "description": {
                  "urls": [
                    {
                      "display_url": "scrapecreators.com",
                      "expanded_url": "https://scrapecreators.com/",
                      "url": "https://t.co/eSvJcfOZwF",
                      "indices": [
                        29
                      ]
                    }
                  ]
                },
                "url": {
                  "urls": [
                    {
                      "display_url": "thewebscrapingguy.com",
                      "expanded_url": "https://thewebscrapingguy.com/",
                      "url": "https://t.co/gNUelkV9LA",
                      "indices": [
                        0
                      ]
                    }
                  ]
                }
              },
              "fast_followers_count": 0,
              "favourites_count": 87301,
              "followers_count": 17297,
              "friends_count": 1139,
              "has_custom_timelines": true,
              "is_translator": false,
              "listed_count": 162,
              "location": "Austin, TX",
              "media_count": 1152,
              "name": "Adrian | The Web Scraping Guy",
              "normal_followers_count": 17297,
              "pinned_tweet_ids_str": [
                "1628769691547074562"
              ],
              "possibly_sensitive": false,
              "profile_banner_url": "https://pbs.twimg.com/profile_banners/4520241209/1710267319",
              "profile_image_url_https": "https://pbs.twimg.com/profile_images/1413647704161275904/1tTdl4v9_normal.jpg",
              "profile_interstitial_type": "",
              "screen_name": "adrian_horning_",
              "statuses_count": 17751,
              "translator_type": "none",
              "url": "https://t.co/gNUelkV9LA",
              "verified": false
            },
            "professional": {
              "rest_id": "1554172330263339015",
              "professional_type": "Business"
            },
            "tipjar_settings": {
              "is_enabled": false,
              "bandcamp_handle": "",
              "bitcoin_handle": "",
              "cash_app_handle": "",
              "ethereum_handle": "",
              "gofundme_handle": "",
              "patreon_handle": "",
              "pay_pal_handle": "",
              "venmo_handle": ""
            }
          }
        }
      },
      "edit_control": {
        "edit_tweet_ids": [
          "1828402665845322123"
        ],
        "editable_until_msecs": "1724763740000",
        "is_edit_eligible": false,
        "edits_remaining": "5"
      },
      "is_translatable": false,
      "views": {
        "count": "493762",
        "state": "EnabledWithCount"
      },
      "source": "<a href=\"https://tweethunter.io\" rel=\"nofollow\">Tweet Hunter Pro</a>",
      "legacy": {
        "bookmark_count": 1030,
        "bookmarked": false,
        "created_at": "Tue Aug 27 12:02:20 +0000 2024",
        "conversation_id_str": "1828402665845322123",
        "display_text_range": [
          0
        ],
        "entities": {
          "media": [
            {
              "display_url": "pic.x.com/fh8xBzkrKI",
              "expanded_url": "https://x.com/adrian_horning_/status/1828402665845322123/photo/1",
              "id_str": "1828402662682841089",
              "indices": [
                217
              ],
              "media_key": "3_1828402662682841089",
              "media_url_https": "https://pbs.twimg.com/media/GV_KiM-W0AEa31c.png",
              "type": "photo",
              "url": "https://t.co/fh8xBzkrKI",
              "ext_media_availability": {
                "status": "Available"
              },
              "sizes": {
                "large": {
                  "h": 117,
                  "w": 182,
                  "resize": "fit"
                },
                "medium": {
                  "h": 117,
                  "w": 182,
                  "resize": "fit"
                },
                "small": {
                  "h": 117,
                  "w": 182,
                  "resize": "fit"
                },
                "thumb": {
                  "h": 117,
                  "w": 117,
                  "resize": "crop"
                }
              },
              "original_info": {
                "height": 117,
                "width": 182,
                "focus_rects": [
                  {
                    "x": 0,
                    "y": 0,
                    "w": 182,
                    "h": 102
                  }
                ]
              },
              "media_results": {
                "result": {
                  "media_key": "3_1828402662682841089"
                }
              }
            }
          ]
        },
        "extended_entities": {
          "media": [
            {
              "display_url": "pic.x.com/fh8xBzkrKI",
              "expanded_url": "https://x.com/adrian_horning_/status/1828402665845322123/photo/1",
              "id_str": "1828402662682841089",
              "indices": [
                217
              ],
              "media_key": "3_1828402662682841089",
              "media_url_https": "https://pbs.twimg.com/media/GV_KiM-W0AEa31c.png",
              "type": "photo",
              "url": "https://t.co/fh8xBzkrKI",
              "ext_media_availability": {
                "status": "Available"
              },
              "sizes": {
                "large": {
                  "h": 117,
                  "w": 182,
                  "resize": "fit"
                },
                "medium": {
                  "h": 117,
                  "w": 182,
                  "resize": "fit"
                },
                "small": {
                  "h": 117,
                  "w": 182,
                  "resize": "fit"
                },
                "thumb": {
                  "h": 117,
                  "w": 117,
                  "resize": "crop"
                }
              },
              "original_info": {
                "height": 117,
                "width": 182,
                "focus_rects": [
                  {
                    "x": 0,
                    "y": 0,
                    "w": 182,
                    "h": 102
                  }
                ]
              },
              "media_results": {
                "result": {
                  "media_key": "3_1828402662682841089"
                }
              }
            }
          ]
        },
        "favorite_count": 1722,
        "favorited": false,
        "full_text": "I just scraped 2.8 million companies from crunchbase 🤯\n\nName, website, semrush stats, etc.\n\nI'm giving the entire thing away in the next 24 hours\n\nComment \"crunchbase\" and I'll send it to you. Make sure DM's are open https://t.co/fh8xBzkrKI",
        "is_quote_status": false,
        "lang": "en",
        "possibly_sensitive": false,
        "possibly_sensitive_editable": true,
        "quote_count": 12,
        "reply_count": 3186,
        "retweet_count": 64,
        "retweeted": false,
        "user_id_str": "4520241209",
        "id_str": "1828402665845322123"
      },
      "quick_promote_eligibility": {
        "eligibility": "IneligibleUserUnauthorized"
      },
      "url": "https://x.com/Austen/status/1935730646267158797"
    }
  ]
}

Best Practices

Error Handling

Implement comprehensive error handling and retry logic for failed requests. Log errors properly for debugging.

Caching

Cache responses when possible to reduce API calls and improve performance. Consider data freshness requirements.

Security

Never expose your API key in client-side code. Use environment variables and secure key management practices.

Troubleshooting

401

Unauthorized

Check your API key is correct and properly formatted in the x-api-key header.

402

Payment Required

You ran out of credits and need to buy more.

404

Not Found

The resource (user, video, etc.) might not exist or be private.

429

Too Many Requests

You have exceeded your rate limit. Slow down your requests.

Frequently Asked Questions

How much does it cost to scrape Twitter?

SociaVault offers 50 free API calls to get started. After that, pricing starts at $10 for 5k requests with volume discounts available.

Is it legal to scrape Twitter data?

Scraping publicly available data is generally considered legal. We only collect public data that is accessible without logging in.

How fast can I scrape Twitter?

Our API handles the rate limiting for you. You can make requests as fast as your plan allows.

What data format does the API return?

All API responses are returned in JSON format, making it easy to integrate with any programming language or application.

Related Tutorials

User Tweets in Other Languages

User Tweets with Node.jsUser Tweets with JavaScriptUser Tweets with PythonUser Tweets with PHP

More cURL Tutorials

TikTok Profile DataTikTok Profile DemographicsTikTok Profile VideosTikTok Video Details

Ready to Start Scraping?

Get started with 50 free API calls. No credit card required. Stop worrying about proxies and captchas.

Start Scraping FreeView Documentation