The landscape of artificial intelligence evaluation is undergoing a significant shift as developers move away from testing simple question-and-answer capabilities toward assessing the performance of autonomous agents in complex, multi-step professional workflows. Perplexity, a leader in the AI-assisted search and research space, has announced the release of WANDR (Wide ANd Deep Research), a new open-source benchmark and evaluation harness specifically designed to measure how effectively AI agents can handle large-scale data collection and evidence-backed research tasks. Unlike traditional benchmarks that often focus on a single "ground truth" answer, WANDR challenges agents to build massive, structured collections of information where every individual claim must be supported by verifiable source material.
The Shift from Narrative Reports to Structured Data Collection
The introduction of WANDR marks a strategic expansion of Perplexity’s evaluation ecosystem. It serves as a companion to the company’s previously released DRACO benchmark. While DRACO was designed to evaluate "deep research"—the ability of an agent to produce a coherent, objective, and accurate long-form narrative report—WANDR focuses on the "wide" aspect of professional knowledge work. In a professional setting, research often takes the form of identifying a vast number of qualifying entities and then performing a deep dive into each one. This includes tasks such as competitive mapping, due diligence, and talent sourcing, where the goal is not just a summary, but a comprehensive database of facts.
Perplexity identifies a critical gap in current AI testing: most existing benchmarks do not account for the scale and evidentiary requirements of real-world knowledge work. In a professional environment, providing a few compelling examples of a market trend is insufficient; an analyst is often required to find every relevant competitor, executive appointment, or financial filing within a specific set of parameters. WANDR is built to simulate these demands, requiring agents to discover a large set of entities and investigate each one thoroughly enough to provide specific, cited evidence for every data point.
The Technical Framework of WANDR
At the heart of the WANDR benchmark is a composable qualification key hierarchy. This structural approach allows the benchmark to represent a variety of complex research patterns, such as flat lists, nested searches, or matrix-style data collection. For example, a task might be structured as a path: company(n) -> employee(m) -> url(k). In this scenario, an AI agent is tasked with finding ‘n’ companies that meet certain criteria, then identifying ‘m’ specific employees within each of those companies, and finally providing ‘k’ supporting source URLs for each employee record.
This hierarchical approach ensures that the benchmark evaluates the agent’s ability to maintain accuracy across multiple levels of discovery and enrichment. Every complete path through the task tree is validated independently by an automated grader. This grader does not rely on a static answer key, which would quickly become outdated in the fast-moving world of web-based information. Instead, the grader re-fetches the URLs provided by the agent during the evaluation process to verify that the page is currently accessible, fits the scope of the task, and contains the specific excerpts cited by the agent.
To illustrate the scale of these tasks, Perplexity highlighted a sample requirement involving "CEO and CFO appointments." This task requires the agent to identify at least 70 U.S.-based companies that announced a new executive appointment between March 1 and April 30, 2026. For each company, the agent must provide an authoritative appointment page and a separate listing-authority page, resulting in a total of 140 source-backed records for a single task.
Methodology and Task Generation at Scale
WANDR consists of 500 realistic and challenging data-collection tasks. Rather than relying on synthetic prompts generated by other AI models—which can often lead to "hallucination loops"—Perplexity built these tasks using de-identified patterns observed in real-world production usage. This ensures that the benchmark reflects the actual problems that professionals are trying to solve with AI agents today.
The tasks were developed through a semi-automated pipeline involving four distinct stages: seeding, authoring, admission, and curation. This pipeline utilized an interleaved author-critic loop, where one system proposed tasks and another critiqued them for clarity, feasibility, and structural integrity, followed by mechanical "linting" to ensure technical compliance.
The resulting dataset is massive. Across the 500 tasks, WANDR calls for a staggering 170,495 source-backed records. The tasks are evenly distributed across three levels of difficulty—lower, middle, and higher—based on the complexity of the per-record work required rather than just the sheer volume of data. The median task in the benchmark requires the agent to find 50 members and generate 245 total records.
Evaluating Performance: Precision, Recall, and the "Soft/Hard" Metric
Because WANDR tasks are open-ended and based on live web data, the grading system must be more sophisticated than a simple binary "correct or incorrect." Perplexity utilizes a multi-dimensional scoring system:
- Precision: Measures the quality of the records the system actually submitted. If an agent claims to have found 50 executives but only 25 are backed by valid evidence, its precision is penalized.
- Recall: Measures quality-adjusted completion. This evaluates how much of the requested volume the agent actually delivered.
- Soft F1 Scores: These provide partial credit for incomplete members within the hierarchy. If an agent finds a company but fails to find the required supporting URL for its CEO, it still receives some credit for the initial discovery.
- Hard F1 Scores: These are much more stringent, counting only those members whose entire subtree (all nested requirements) is perfectly correct.
This granular grading allows developers to isolate exactly where an agent is failing. Is it failing at the "discovery" phase (finding the companies)? Or is it failing at the "extraction" phase (finding the specific evidence on a webpage)?
Benchmarking Production Systems: A Wide Gap in Capability
Perplexity ran six leading production systems through the full 500-task WANDR benchmark. The results revealed that while current AI technology is making strides, no system is currently close to "solving" the challenges of wide and deep research at scale.
Perplexity’s own "Search as Code" (SaC) system led the field with a Soft F1 score of 0.363 and a Hard F1 score of 0.133. Anthropic’s models followed as the closest competitors in terms of quality, achieving a Soft F1 of 0.249, though they required more time and higher token usage to reach those results. Other systems, including those from OpenAI and Exa, were generally faster and less expensive but produced significantly lower accuracy scores, with the best among them hitting a Soft F1 of only 0.121.
The data also highlighted the immense computational and financial costs associated with high-level autonomous research. Costs across different settings and systems spanned four orders of magnitude, ranging from a mere $0.03 per task for basic configurations to as much as $324.83 per task for high-intensity research settings. Perplexity’s leading SaC system averaged a cost of $5.20 per task, requiring a median of 14.9 minutes and 3.82 million tokens per task.
Key Findings and Industry Implications
The release of WANDR has provided several critical insights into the current state of AI research agents:
- Discovery is the Primary Bottleneck: The first structural hurdle for any agent is simply finding the entities that meet the criteria. Top-level discovery completion ranged from 61% to 95% across systems. Most missing data resulted from an agent’s failure to find enough candidates rather than errors in merging duplicates.
- Evidence Extraction is Harder than Retrieval: While finding a usable webpage is relatively easy for modern search engines, turning that page into verified evidence remains difficult. Perplexity found that 41.4% of pages retrieved by its system missed a substantive requirement, and 57.5% of the excerpts provided failed to fully support the specific claim being made.
- Scale Compounds Failure: The benchmark proved that as hierarchies become deeper, the likelihood of total success drops sharply. Every additional requirement in a task (e.g., adding a requirement for a specific URL after finding a name) acts as a potential failure point that can invalidate the entire record under "Hard" scoring.
- The Advantage of "Search as Code": Perplexity’s Search as Code approach performed best because it allows the agent to express research logic—such as filtering, fan-out, and deduplication—as a program. This allows deterministic compute to handle repetitive operations, freeing the language model to focus on higher-level reasoning.
Conclusion: A New Standard for the Agentic Era
As AI moves from being a conversational tool to a functional workplace assistant, the industry requires more rigorous ways to measure utility. WANDR provides a framework for evaluating the "blue-collar" work of the information age: the systematic collection and verification of data.
By open-sourcing WANDR, Perplexity is inviting the broader AI research community to move beyond simple accuracy tests and toward a more nuanced understanding of how agents handle the scale and complexity of professional research. The benchmark suggests that the future of AI research lies not just in better language models, but in more sophisticated "agentic" architectures that can navigate the web, verify facts in real-time, and manage the high costs of large-scale data acquisition. For now, the low scores across all tested systems indicate that there is still a significant journey ahead before AI agents can fully automate the complex research tasks currently performed by human analysts.
