What is Web Bench?
Web Bench is a comprehensive dataset and benchmark designed to evaluate AI web browsing agents. It features 5,750 tasks across 452 different websites, providing a robust framework for assessing the performance of autonomous and copilot AI models in real-world web browsing scenarios.
The benchmark includes two main categories: the Autonomous Dataset, which focuses on navigation and data extraction tasks, and the Copilot Dataset, which involves logging in, form filling, and file downloading. This structured approach allows for detailed performance comparisons across various AI models and organizations.
Features
- Autonomous Dataset: Focuses on navigation and data extraction tasks
- Copilot Dataset: Involves logging in, form filling, and file downloading
- Leaderboard: Ranks AI models based on performance scores
- Verified Results: Ensures accuracy and reliability of benchmark data
- GitHub Integration: Allows for community contributions and access to technical details
Use Cases
- Evaluating the performance of AI web browsing agents
- Comparing different AI models in real-world web browsing tasks
- Research and development of autonomous AI browsing capabilities
- Benchmarking copilot AI assistants for web-based interactions
- Assessing AI navigation and data extraction accuracy on websites
FAQs
-
What types of tasks are included in the Web Bench dataset?
The dataset includes 5,750 tasks across 452 websites, categorized into Autonomous tasks (navigation and data extraction) and Copilot tasks (logging in, form filling, file downloading). -
How are AI models ranked on the Web Bench leaderboard?
AI models are ranked based on their performance scores (percentage) in the benchmark, with verified results to ensure accuracy and reliability. -
Can I contribute to or access the Web Bench dataset?
Yes, contributions and access are available through the GitHub repository linked on the website.
