ScrapeHubAI

🚀 Powered by ScrapeGraphAI - The most advanced AI-powered web scraping API

An open-source LangGraph-based agent that analyzes GitHub stargazers, traces them to their companies, and evaluates these companies as potential sales targets for AI scraping infrastructure.

🌟 Features

• GitHub Stargazer Analysis: Fetches and analyzes users who starred a repository
• Company Identification: Traces GitHub users to their affiliated companies
• Intelligent Evaluation: Scores companies based on size, industry, and technology fit
• Web Scraping: Uses ScrapeGraphAI API to gather additional company information
• Beautiful UI: Streamlit-based interface for easy interaction
• Export Results: Download analysis results as CSV for further processing

🚀 Quick Start

Prerequisites

• Python 3.10 or higher
• API keys for GitHub, OpenRouter, and ScrapeGraphAI

Installation

1. Clone the repository:

git clone https://github.com/yourusername/ScrapeHubAI.git
cd ScrapeHubAI

2. Install dependencies:

pip install -r requirements.txt

3. Configure API keys:

Getting a GitHub Personal Access Token

1. Go to GitHub → Settings → Developer settings → Personal access tokens → Tokens (classic)
2. Click "Generate new token" → "Generate new token (classic)"
3. Give your token a descriptive name (e.g., "ScrapeHub")
4. Select the following scopes:
   • public_repo (required for reading public repositories)
   • read:user (optional, for better user information)
   • read:org (optional, for organization information)
5. Click "Generate token" at the bottom
6. Important: Copy your token immediately - you won't be able to see it again!

Setting up your .env file

Create a .env file in the project root:

GITHUB_TOKEN=your_github_personal_access_token
OPENROUTER_API_KEY=your_openrouter_api_key
OPENROUTER_API_BASE=https://openrouter.ai/api/v1
SGAI_API_KEY=your_scrapegraphai_api_key

Or copy from the example:

cp .env.example .env
# Then edit .env with your actual API keys

Getting Other API Keys

• OpenRouter: Sign up at OpenRouter.ai and get your API key from the dashboard
• ScrapeGraphAI: Register at dashboard.scrapegraphai.com and obtain your API key

4. Run the application:

streamlit run src/app.py

📊 How It Works

1. Fetch Stargazers: The agent retrieves users who starred the specified GitHub repository
2. Trace to Companies: Identifies companies through user profiles and organization memberships
3. Gather Intelligence: Uses ScrapeGraphAI to scrape additional company information
4. Evaluate & Rank: Scores companies based on multiple criteria:
   • Technology relevance (AI, ML, data analytics, scraping)
   • Industry fit (e-commerce, SaaS, fintech)
   • Company size and growth indicators
   • Explicit data processing needs

🎯 Use Cases

• Sales Intelligence: Identify potential customers for AI/data infrastructure products
• Market Research: Understand which companies are interested in specific technologies
• Partnership Discovery: Find companies with complementary technology needs
• Competitive Analysis: See which companies are following competitor repositories

🛠️ Architecture

• LangGraph: Orchestrates the multi-step analysis workflow
• OpenRouter: Provides LLM capabilities for intelligent evaluation
• ScrapeGraphAI: Powers web scraping for company information
• Streamlit: Creates an intuitive user interface

📁 Project Structure

github-star-to-company-agent/
├── src/
│   ├── agent.py          # LangGraph workflow definition
│   ├── tools.py          # GitHub and scraping tools
│   ├── evaluator.py      # Company scoring logic
│   └── app.py            # Streamlit UI
├── tests/
│   └── test_agent.py     # Unit tests
├── docs/
│   └── usage.md          # Detailed usage guide
├── .env                  # API keys (create this)
├── requirements.txt      # Python dependencies
└── README.md            # This file

🧪 Testing

Run the test suite:

python -m pytest tests/

🔧 Configuration

Environment Variables

• GITHUB_TOKEN: Personal access token for GitHub API
• OPENROUTER_API_KEY: API key for OpenRouter LLM service
• OPENROUTER_API_BASE: OpenRouter API endpoint (default: https://openrouter.ai/api/v1)
• SGAI_API_KEY: API key for ScrapeGraphAI service

Advanced Settings

Customize analysis parameters through the Streamlit UI:

• Maximum number of companies to display
• Minimum score threshold for filtering
• Analysis depth and timeout settings

⚖️ Ethical Considerations

• Respect Rate Limits: The agent implements automatic rate limiting
• Privacy: Only analyzes publicly available information
• Terms of Service: Ensure compliance with all platform ToS
• Responsible Use: Designed for legitimate business intelligence only

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Built with LangGraph for agent orchestration
• Powered by ScrapeGraphAI for web scraping
• UI created with Streamlit
• LLM capabilities via OpenRouter

📞 Support

For issues, questions, or suggestions:

1. Check the usage guide
2. Open an issue on GitHub
3. Review existing issues for solutions

---

Made with ❤️ from ScrapeGraphAI team