Python integration guide

This guide shows you how to use the ChatGPT Scraper API with Python to programmatically collect AI-generated responses and structured metadata.

Installation

Before you begin, install the requests library if you haven’t already:

pip install requests

Quick start

Import required libraries

You’ll need the requests library for making HTTP requests and json for handling the response data.

import json
import requests

Configure your API request

Set up your payload with the prompt and optional parameters. Replace YOUR_API_KEY with your actual API key from the cloro dashboard.

# API parameters
payload = {
    'prompt': 'Compare the top 3 programming languages for web development in 2025',
    'country': 'US',
    'include': {
        'markdown': True,
        'rawResponse': True,
        'searchQueries': True
    }
}

# Get a response
response = requests.post(
    'https://api.cloro.dev/v1/monitor/chatgpt',
    headers={'Authorization': 'Bearer YOUR_API_KEY'},
    json=payload
)

Handle the response

Process the API response by printing it to stdout or saving it to a file.

# Print response to stdout
print(response.json())

# Save response to a JSON file
with open('response.json', 'w') as file:
    json.dump(response.json(), file, indent=2)

Complete example

import json
import requests

# API parameters
payload = {
    'prompt': 'Compare the top 3 programming languages for web development in 2025',
    'country': 'US',
    'include': {
        'markdown': True,
        'rawResponse': True,
        'searchQueries': True
    }
}

# Get a response
response = requests.post(
    'https://api.cloro.dev/v1/monitor/chatgpt',
    headers={'Authorization': 'Bearer YOUR_API_KEY'},
    json=payload
)

# Print response to stdout
print(response.json())

# Save response to a JSON file
with open('response.json', 'w') as file:
    json.dump(response.json(), file, indent=2)

import json
import requests

try:
    # API parameters
    payload = {
        'prompt': 'Compare the top 3 programming languages for web development in 2025',
        'country': 'US',
        'include': {
            'markdown': True,
            'rawResponse': True,
            'searchQueries': True
        }
    }
    
    # Get a response
    response = requests.post(
        'https://api.cloro.dev/v1/monitor/chatgpt',
        headers={'Authorization': 'Bearer YOUR_API_KEY'},
        json=payload,
        timeout=180  # 3 minutes timeout
    )
    
    # Raise an exception for bad status codes
    response.raise_for_status()
    
    # Process successful response
    data = response.json()
    print(data)
    
    # Save response to a JSON file
    with open('response.json', 'w') as file:
        json.dump(data, file, indent=2)
        
except requests.exceptions.Timeout:
    print("Request timed out. ChatGPT responses may take longer for complex prompts.")
except requests.exceptions.HTTPError as e:
    print(f"HTTP error occurred: {e}")
except requests.exceptions.RequestException as e:
    print(f"Error occurred: {e}")

Request parameters

You can customize your API request using these parameters:

Parameter	Description	Default value
`prompt`*	The prompt or question to send to ChatGPT (1-10,000 characters)	–
`country`	Optional country/region code for localized monitoring (e.g., `US`, `GB`, `DE`)	–
`include.markdown`	Include response in Markdown format when set to true	`false`
`include.rawResponse`	Include raw streaming response events for debugging (+2 credits)	`false`
`include.searchQueries`	Include query fan-out ChatGPT used to generate response (+2 credits)	`false`

The prompt parameter is mandatory. All other parameters are optional.

Handling the response

The API returns a JSON object with the following structure:

{
  "status": "success",
  "result": {
    "model": "gpt-5-mini",
    "text": "When comparing programming languages for web development in 2025, three languages stand out...",
    "markdown": "**When comparing programming languages for web development in 2025**, three languages stand out...",
    "shoppingCards": [...],
    "searchQueries": ["web development languages 2025"],
    "rawResponse": [...]
  }
}

You can access specific fields like this:

data = response.json()
model = data['result']['model']
text_response = data['result']['text']
markdown_response = data['result']['markdown']

Saving responses to files

To save the ChatGPT response to a JSON file:

import json

# Save the complete response
with open('response.json', 'w') as file:
    json.dump(response.json(), file, indent=2)

# Or save just the text response
data = response.json()
with open('chatgpt_response.txt', 'w') as file:
    file.write(data['result']['text'])

Timeout configuration

Set an appropriate timeout (120-180 seconds) when making requests, as ChatGPT responses may take time to generate depending on complexity.

response = requests.post(
    'https://api.cloro.dev/v1/monitor/chatgpt',
    headers={'Authorization': 'Bearer YOUR_API_KEY'},
    json=payload,
    timeout=180  # 3 minutes
)

For production use, consider implementing a retry mechanism instead of relying solely on timeouts, as the cloro system retries automatically.

Error handling best practices

Implement comprehensive error handling to manage different failure scenarios:

import requests
import json
from time import sleep

def make_chatgpt_request(payload, max_retries=3):
    """Make a request to ChatGPT API with retry logic."""
    
    for attempt in range(max_retries):
        try:
            response = requests.post(
                'https://api.cloro.dev/v1/monitor/chatgpt',
                headers={'Authorization': 'Bearer YOUR_API_KEY'},
                json=payload,
                timeout=180
            )
            
            # Check for successful status code
            response.raise_for_status()
            
            return response.json()
            
        except requests.exceptions.Timeout:
            print(f"Attempt {attempt + 1} timed out")
            if attempt < max_retries - 1:
                sleep(5)  # Wait before retrying
                continue
            raise
            
        except requests.exceptions.HTTPError as e:
            if response.status_code == 429:  # Rate limit
                print("Rate limit reached, waiting before retry...")
                sleep(10)
                continue
            elif response.status_code >= 500:  # Server error
                print(f"Server error on attempt {attempt + 1}")
                if attempt < max_retries - 1:
                    sleep(5)
                    continue
            raise
            
        except requests.exceptions.RequestException as e:
            print(f"Request failed: {e}")
            if attempt < max_retries - 1:
                sleep(5)
                continue
            raise
    
    raise Exception("Max retries reached")

# Use the function
payload = {
    'prompt': 'Your prompt here',
    'country': 'US'
}

try:
    result = make_chatgpt_request(payload)
    print(result['result']['text'])
except Exception as e:
    print(f"Failed to get response: {e}")

Always check the status field in the response to verify successful execution before accessing the result data.

Next steps

Explore the API reference for detailed endpoint documentation
Learn about shopping card extraction for e-commerce data
Check out advanced features like query fan-out insights

Get Started

Core Concepts

Guides

Use Cases

Installation

Quick start

Complete example

Request parameters

Handling the response

Saving responses to files

Timeout configuration

Error handling best practices

Next steps

​Installation

​Quick start

​Complete example

​Request parameters

​Handling the response

​Saving responses to files

​Timeout configuration

​Error handling best practices

​Next steps

Installation

Quick start

Complete example

Request parameters

Handling the response

Saving responses to files

Timeout configuration

Error handling best practices

Next steps