Complete Guide to Building Tunnel Proxy Pool with ProxyCat

⏱️ Estimated Reading Time: 15 minutes

Overview

ProxyCat is an innovative middleware solution that transforms short-lived, cost-effective IPs into permanent tunnel proxies. It can convert IPs available for 1-60 minutes into a stable proxy pool, providing a highly cost-effective IP management solution.

While traditional tunnel proxies cost $3-6 per day, short-term IPs cost only a few cents each, averaging $0.03-0.50 per day for operation.

Core Features of ProxyCat

1. Multi-Protocol Support

• HTTP Protocol: Web browser and HTTP client support
• SOCKS5 Protocol: Compatibility with diverse applications

2. Flexible Proxy Management

• Sequential Mode: Use proxies in order
• Random Mode: Random proxy selection to prevent patterns
• Custom Mode: Proxy selection tailored to specific requirements

3. Intelligent Proxy Management

• Dynamic Proxy Acquisition: Real-time proxy collection via API calls
• Automatic Validation: Auto-removal of inactive proxies at startup
• Auto-Failover: Immediate switch to backup proxy on failure

4. Security & Authentication

• User Authentication: Username/password-based authentication
• IP Whitelist/Blacklist: Access control management
• Real-time Monitoring: Track proxy status and switch timing

5. User-Friendly Management Interface

• Web UI: Intuitive web management interface
• Dynamic Configuration: Configuration changes without service restart
• Multi-language Support: Korean and English interface

Installation and Configuration

1. System Requirements

# Minimum Requirements
- Python 3.7 or higher
- Memory: 512MB RAM
- Storage: 100MB
- Network: Internet connection required

# Recommended Requirements
- Python 3.9 or higher
- Memory: 1GB RAM
- Storage: 1GB
- CPU: 2 cores or more

2. Repository Clone and Basic Setup

# Clone repository
git clone https://github.com/honmashironeko/ProxyCat.git
cd ProxyCat

# Create Python virtual environment (recommended)
python3 -m venv proxycat_env
source proxycat_env/bin/activate  # macOS/Linux
# proxycat_env\Scripts\activate    # Windows

# Install required packages
pip install -r requirements.txt

3. Configuration File Setup

ProxyCat’s core settings are managed in the config/config.ini file:

[GLOBAL]
# Default language setting (ko/en)
language = en

[PROXY_POOL]
# Proxy selection mode (sequential/random/custom)
mode = random

# Proxy list (format: protocol://ip:port or protocol://username:password@ip:port)
proxies = http://proxy1.example.com:8080
         http://user:pass@proxy2.example.com:8080
         socks5://proxy3.example.com:1080

[SERVER]
# Local server settings
listen_host = 0.0.0.0
listen_port = 8888
protocol = http  # http or socks5

# Authentication settings (optional)
auth_enabled = false
username = admin
password = password

[WHITELIST]
# IP whitelist (optional)
enabled = false
ips = 127.0.0.1,192.168.1.0/24

[API]
# Dynamic proxy acquisition API (optional)
enabled = false
endpoint = https://api.proxyservice.com/get
headers = Authorization: Bearer YOUR_TOKEN

4. Docker Deployment

ProxyCat supports easy deployment via Docker:

# Deploy using Docker Compose
docker-compose up -d

# Or run Docker directly
docker build -t proxycat .
docker run -d \
  --name proxycat \
  -p 8888:8888 \
  -p 5000:5000 \
  -v $(pwd)/config:/app/config \
  -v $(pwd)/logs:/app/logs \
  proxycat

5. Service Execution

# Run in CLI mode
python ProxyCat.py

# Run with Web UI
python app.py

# Run in background
nohup python app.py > proxycat.log 2>&1 &

Practical Use Cases

1. Web Scraping and Data Collection

ProxyCat is highly effective for bypassing IP blocking in large-scale web scraping operations:

import requests
import time
from itertools import cycle

# ProxyCat proxy configuration
proxycat_proxy = {
    'http': 'http://localhost:8888',
    'https': 'http://localhost:8888'
}

# Large-scale data collection example
def scrape_with_proxycat(urls):
    session = requests.Session()
    session.proxies = proxycat_proxy
    
    results = []
    for url in urls:
        try:
            response = session.get(url, timeout=10)
            if response.status_code == 200:
                results.append(response.text)
                print(f"✅ Success: {url}")
            else:
                print(f"❌ Failed: {url} (Status: {response.status_code})")
        except Exception as e:
            print(f"🔄 Proxy switching: {e}")
            time.sleep(1)  # ProxyCat automatically switches proxy
    
    return results

# Practical usage example
target_urls = [
    'https://example1.com/api/data',
    'https://example2.com/products',
    'https://example3.com/listings'
]

scraped_data = scrape_with_proxycat(target_urls)

2. API Rate Limit Bypass

Many API services impose per-IP request limits. ProxyCat can effectively bypass these limitations:

import requests
import json
import time

class APIClient:
    def __init__(self, proxycat_host="localhost", proxycat_port=8888):
        self.session = requests.Session()
        self.session.proxies = {
            'http': f'http://{proxycat_host}:{proxycat_port}',
            'https': f'http://{proxycat_host}:{proxycat_port}'
        }
    
    def bulk_api_requests(self, endpoints):
        results = []
        for endpoint in endpoints:
            try:
                response = self.session.get(endpoint, timeout=15)
                if response.status_code == 200:
                    results.append(response.json())
                elif response.status_code == 429:  # Rate limit
                    print("🔄 Rate limit detected, waiting for proxy switch...")
                    time.sleep(2)
                    # ProxyCat automatically switches to new proxy
                    response = self.session.get(endpoint, timeout=15)
                    results.append(response.json())
            except Exception as e:
                print(f"❌ Request failed: {endpoint}, Error: {e}")
                time.sleep(1)
        
        return results

# Usage example
api_client = APIClient()
api_endpoints = [
    'https://api.service.com/v1/users',
    'https://api.service.com/v1/products',
    'https://api.service.com/v1/orders'
]
data = api_client.bulk_api_requests(api_endpoints)

3. Security Testing and Penetration Testing

ProxyCat is used by security professionals to hide IPs and simulate access from various regions during testing:

# Network scanning with Nmap
proxychains4 -f /etc/proxychains4.conf nmap -sT target.example.com

# Integration with proxychains
echo "socks5 127.0.0.1 8888" >> /etc/proxychains4.conf

# Burp Suite integration
# Burp Suite → Proxy → Options → Upstream Proxy Servers
# Host: localhost, Port: 8888, Type: HTTP

4. Social Media Automation

IP diversification is necessary for multi-account management or content automation on social media platforms:

import selenium
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

def setup_browser_with_proxy():
    chrome_options = Options()
    chrome_options.add_argument('--proxy-server=http://localhost:8888')
    chrome_options.add_argument('--disable-dev-shm-usage')
    chrome_options.add_argument('--no-sandbox')
    
    driver = webdriver.Chrome(options=chrome_options)
    return driver

# Multi-account management example
def manage_multiple_accounts(accounts):
    for account in accounts:
        driver = setup_browser_with_proxy()
        try:
            # Access with different IP for each account
            driver.get('https://platform.example.com/login')
            # Perform login and tasks
            print(f"✅ {account['username']} account tasks completed")
        finally:
            driver.quit()
            time.sleep(5)  # Wait for proxy switch

accounts = [
    {'username': 'account1', 'password': 'pass1'},
    {'username': 'account2', 'password': 'pass2'}
]
manage_multiple_accounts(accounts)

5. Geo-Restricted Content Access

Useful for accessing geo-restricted content or analyzing regional search results:

import requests
import json

class GeoContentAnalyzer:
    def __init__(self):
        self.proxycat_proxy = {
            'http': 'http://localhost:8888',
            'https': 'http://localhost:8888'
        }
    
    def check_geo_content(self, url):
        """Check regional content variations"""
        session = requests.Session()
        session.proxies = self.proxycat_proxy
        
        try:
            response = session.get(url, timeout=10)
            # Check IP information
            ip_info = session.get('https://ipapi.co/json/', timeout=5).json()
            
            return {
                'country': ip_info.get('country_name'),
                'city': ip_info.get('city'),
                'content_length': len(response.text),
                'status_code': response.status_code
            }
        except Exception as e:
            return {'error': str(e)}
    
    def analyze_multiple_regions(self, url, samples=10):
        """Analyze content from multiple regions"""
        results = []
        for i in range(samples):
            result = self.check_geo_content(url)
            results.append(result)
            print(f"Sample {i+1}: {result}")
            time.sleep(2)  # Wait for proxy switch
        
        return results

# Usage example
analyzer = GeoContentAnalyzer()
geo_results = analyzer.analyze_multiple_regions('https://news.example.com')

Web UI Management Interface

ProxyCat’s Web UI is accessible at http://localhost:5000 and provides the following features:

1. Dashboard Features

• Real-time Proxy Status: Active/inactive proxy overview
• Traffic Statistics: Request count, success rate, response time
• Proxy Switch History: Track when and why proxies were switched

2. Proxy Management

• Add/Remove Proxies: Dynamically manage proxy pool
• Manual Proxy Switch: Force switch to specific proxy
• Proxy Testing: Check individual proxy connection status

3. Configuration Management

• Real-time Config Changes: Update settings without service restart
• Authentication Settings: User authentication and IP whitelist management
• Log Level Adjustment: Adjust log levels for debugging

Monitoring and Log Management

1. Log Analysis

ProxyCat provides detailed logs for troubleshooting and performance analysis:

# Real-time log monitoring
tail -f logs/proxycat.log

# Filter error logs only
grep "ERROR" logs/proxycat.log

# Track proxy switch events
grep "PROXY_SWITCH" logs/proxycat.log

2. Performance Metrics

# Analyze request success rate
grep -c "SUCCESS" logs/proxycat.log
grep -c "FAILED" logs/proxycat.log

# Calculate average response time
grep "RESPONSE_TIME" logs/proxycat.log | awk '{sum+=$NF} END {print sum/NR}'

Advanced Configuration and Optimization

1. Proxy Pool Optimization

[PROXY_POOL]
# Proxy health check interval (seconds)
health_check_interval = 300

# Failed proxy retry count
max_retry_count = 3

# Proxy response timeout (seconds)
proxy_timeout = 10

# Maximum concurrent connections
max_concurrent_connections = 100

2. Load Balancing Strategy

[LOAD_BALANCING]
# Load balancing method (round_robin/least_connections/weighted)
strategy = round_robin

# Weighted proxy selection (IP:weight)
weighted_proxies = proxy1.com:10,proxy2.com:5,proxy3.com:1

3. Caching and Performance Tuning

[PERFORMANCE]
# Connection pool size
connection_pool_size = 50

# Connection keep-alive time (seconds)
connection_keep_alive = 60

# DNS cache TTL (seconds)
dns_cache_ttl = 3600

Troubleshooting Guide

1. Common Issues

Issue: Proxy Connection Failure

# Solutions:
1. Check proxy server status
2. Verify network connectivity
3. Check firewall settings
4. Review config.ini settings

Issue: Web UI Inaccessible

# Solutions:
1. Check port 5000 usage: netstat -tulpn | grep 5000
2. Open firewall port: sudo ufw allow 5000
3. Restart service: python app.py

Issue: High Memory Usage

# Solutions:
1. Adjust connection pool size
2. Lower log level
3. Clean up inactive proxies
4. Monitor system resources

2. Debug Mode

# Run in debug mode
DEBUG=true python ProxyCat.py

# Enable detailed logging
export PROXYCAT_LOG_LEVEL=DEBUG
python ProxyCat.py

Security Considerations

1. Network Security

• SSL/TLS Encryption: HTTPS proxy usage recommended
• Enhanced Authentication: Strong username/password settings
• Access Control: IP whitelist utilization

2. Log Security

• Sensitive Data Masking: Exclude passwords, API keys from logs
• Log Rotation: Regular log file cleanup
• Access Permissions: Restrict log file read permissions

Performance Benchmarks

ProxyCat performance metrics in actual test environment:

Environment: Ubuntu 20.04, 4GB RAM, 2CPU
Proxy Count: 50
Concurrent Connections: 100
Test Duration: 24 hours

Results:
- Average Response Time: 1.2 seconds
- Success Rate: 98.5%
- Memory Usage: Average 256MB
- CPU Usage: Average 15%
- Proxy Switches: Average 150/hour

Conclusion

ProxyCat is a cost-effective proxy pool management solution that serves as a powerful tool across various domains. It’s applicable for web scraping, API rate limit bypass, security testing, geo-restricted content access, and more, with the added benefits of convenient Web UI management and simple Docker deployment.

Particularly notable is the 90%+ cost reduction compared to traditional tunnel proxies while maintaining stable service, making it highly valuable for projects requiring large-scale data collection or security testing.

Next Steps

1. Check GitHub Repository: ProxyCat Repository
2. Community Participation: Issue reporting and feature requests
3. Explore Extensions: BabyCat modules, machine protocol support roadmap

Experience more efficient and economical proxy management with ProxyCat!