The Tradewinds Solutions Marketplace is the premier offering of Tradewinds, the Department of War’s (DoW’s) suite of tools and services designed to accelerate the procurement and adoption of artificial intelligence, machine learning, digital, and data analytics capabilities.

Every day, DoW teams make mission-critical decisions under uncertainty. Preseen helps teams test the assumptions behind their plans and use forecasts in research, risk assessments, and decision-making. We work with diverse clients including hedge funds, media and business-information companies, and strategic advisory firms.

“Government teams need more than a prediction. They need to understand how likely an outcome is, what evidence supports that estimate, and what would change it,” said Veniamin Veselovsky, co-founder and CEO of Preseen. “We’re working to make rigorous forecasting a practical part of government analysis and planning.”

Our solution video, “Preseen: AI forecasting platform for decision makers,” is available on the Tradewinds Solutions Marketplace. To view it, log in or create a government account at tradewindai.com and search for the title.

To discuss your team’s forecasting needs, contact government@preseen.com.

About Preseen

Preseen builds AI systems that research questions and produce probabilistic forecasts. Each forecast includes the evidence and reasoning behind the estimate. Its systems participate in public forecasting competitions on Metaculus, which scores predictions against actual outcomes. Learn more at preseen.com.

Media inquiries: mohamed@preseen.com.

About the Tradewinds Solutions Marketplace

The Tradewinds Solutions Marketplace is a digital repository of post-competition, readily Awardable solution videos that address the Department of War’s most significant challenges in artificial intelligence, machine learning, digital, and data analytics. Awardable solution videos have been assessed through competitive procedures and are available to government customers with a marketplace account. Tradewinds is housed in the DoW’s Chief Digital and Artificial Intelligence Office.

For more information or media requests, contact: success@tradewindai.com.

The Summer 2026 Metaculus Cup was a breakthrough for bots, with Laertes becoming the first bot to win the tournament. Preseen had two entries: the baseline Preseen bot and Symbiosis, a hybrid team consisting of two human forecasters, Scott Eastman and Robert de Neufville, who worked with the bot and had final authority over forecasts. Symbiosis finished tenth and the baseline bot eleventh among all human, bot, and hybrid competitors.

Symbiosis outperformed the default bot, but the difference wasn’t great enough to draw definitive conclusions about their relative abilities. Although their total scores were close, their forecasts differed substantially on a number of questions. Working with the Preseen bot was a learning process for Scott and Robert. The main challenge was knowing when and how to intervene. Their experience also raises questions for experts using AI in other fields, including how to judge when an intervention is likely to help.

How Symbiosis worked

In Metaculus Cup tournaments, competitors forecast 50-60 questions over the course of four months. Competitors can’t see what other individual forecasters are forecasting but can generally see the Metaculus Community Prediction, which is a recency-weighted median of current forecasts on each question.

The baseline Preseen bot generated forecasts for each question at regular intervals. Initially, it was set up to produce forecasts every day, although later it was set to forecast less often on slower-moving questions. By default, Symbiosis initially submitted the same forecasts as the baseline bot. Once the human forecasters intervened, their version of the forecast developed separately from the baseline bot’s.

Scott and Robert had four main ways to affect Symbiosis’ forecasts:

• They could initiate new forecasts.

• They could give the bot additional context to use in making its forecasts.

• They could manually override the bot’s forecasts.

• They could prevent the bot from automatically submitting new forecasts.

They could also chat with the bot about a forecast, asking what evidence it had considered and how it had reached its conclusion.

Giving the bot additional context let them supply information it hadn’t found or ask it to consider something it had missed. Manually overriding the bot’s forecasts would substitute their own judgment for the bot’s judgment, even if they were still drawing heavily on the research it had done.

Manual overrides usually went hand-in-hand with preventing the bot from automatically submitting new forecasts. Scott and Robert would then have to keep those forecasts up to date themselves.

Learning to use a light touch

Scott and Robert initially decided to generally defer to the bot’s forecasts, to avoid “fussing” with it, unless its forecasts seemed clearly wrong. Preseen had a good record against human forecasters in previous tournaments and its analysis was generally clear and convincing. It wasn’t obvious their judgment would necessarily be better than the bot’s judgment.

Preseen could find things they would normally miss. On one question about the Pope’s travel schedule, the bot immediately noticed when the Holy See Press Office issued a press release announcing that the Pope would be visiting San Marino at the end of August. The Pope’s planned visit wasn’t otherwise widely reported. They outperformed the Metaculus Community on that question simply by allowing the bot to forecast without intervening.

Supplying missing context

But the Preseen bot was also sometimes unable to get information it needed. On a question about the Tony Awards, the bot was unable to read a Gold Derby page in JavaScript showing that most critics expected Liberation to win the Tony Award for Best Play. It was able, to an extent, to infer the contents of the page from discussions of the critics’ views on websites it could read more easily, but it significantly underestimated the strength of the critical consensus.

Scott and Robert addressed that issue by giving their version of the bot the latest contents of the Gold Derby page as additional context. The baseline bot gave Liberation a 52% chance of winning, but with the new context Symbiosis’ version of the bot gave it a 74.2% chance of winning. When Liberation ultimately did win, Symbiosis beat the baseline bot on the question by almost 19 points.

In a May 18 chat about the forecast:

Robert de Neufville: “Did you consider the best play predictions at Gold Derby or Kalshi?”

Preseen: “No. I did not incorporate Gold Derby or Kalshi into the final forecast.”

Additional context supplied to Preseen: “Consider that as of May 18, 2026, 14 out of the 14 experts and editors surveyed predict Liberation will win the Best Play Tony…”

On other questions, including one about data center regulation, Preseen missed updates to the resolution conditions that were posted in the comments sections. Metaculus admins and question writers occasionally issued authoritative comments on how the questions would be resolved without updating the resolution conditions section of the question. In these cases, Scott and Robert gave their version of the bot the admin comments as additional context.

During the tournament, we updated Preseen so it could read Metaculus administrators’ comments. This access did not include comments from other forecasters.

When context wasn’t enough

By late May, Scott and Robert were concerned that the bot’s forecast for Iran’s 2026 Global Peace Index score might be too high. Higher index scores represent less peaceful conditions, and many components of the index significantly lag events, so the 2026 score might not fully reflect the recent conflict.

Scott and Robert treated this like another missing-data problem and gave their bot additional data about lags in previous versions of the index. The bot appeared to take the issue into consideration in the write-ups of its forecast, but instead of lowering the forecast, the bot actually raised it somewhat in response to the new context they provided. It’s still not entirely clear why the bot’s analysis differed from their analysis.

The additional context asked Preseen to compare Ukraine’s 2022 and 2023 Global Peace Index scores to assess how much of a recent conflict might appear in that year’s report.

In their May 29 exchange, Robert questioned why the additional context had raised the forecast:

Robert de Neufville: “Your forecast with the context is meaningfully higher than your forecast without it. … My intuition was that it would suggest less of the war would be captured by the report.”

Preseen: “I’m not saying the context should unambiguously raise the forecast. It is a two-sided analogue. The completed forecast interpreted it more as evidence for some meaningful same-year capture, while your reading emphasizes limited same-year capture.”

Iran’s Global Peace Index score ultimately turned out to be lower than almost any forecaster expected. The baseline bot and Symbiosis had both confidently forecast higher numbers than the community, and this was their worst-scoring question of the tournament.

In hindsight, it seemed like it would have been better if Scott and Robert had used their own judgment and manually overridden the bot. They had strongly suspected its forecast was too high, but the additional context did not bring it down as they had expected. At that point, they became more skeptical of the bot’s analysis and began to override its forecasts more often.

Over the following weeks, Scott and Robert manually overrode forecasts on more questions while continuing to supply new context. The size of those overrides varied considerably.

_{Figure 1. Questions receiving new context or manual overrides each week. The lower panel shows the median size of a manual adjustment on binary questions, in percentage points.}

When overrides were costly

One question where Scott and Robert disagreed with the bot’s forecast was about Shakira’s World Cup song “Dai Dai”: would it outpeak “Waka Waka” on the Billboard Hot 100?

The bot reasoned that “Dai Dai” could surpass Shakira’s previous World Cup anthem on the US charts after her live performance at the World Cup Final. The song was already a global hit and would be boosted by the fact that the US was one of the hosts of the World Cup.

That seemed like a plausible view, but the song started slowly on the US charts. Robert didn’t think the song was that great, saying it wasn’t “a certified banger,” despite its success on the global charts. That on its own wasn’t compelling evidence it wouldn’t pass “Waka Waka,” but the Metaculus Community was extremely skeptical too, at one point giving it just a 0.1% chance.

Scott and Robert didn’t think the odds were that long, but it reinforced their view that it was a long shot and at one point they went as low as 3%. The baseline forecast below put the probability at 18% on June 22.

Preseen baseline forecast, June 22, 2026:

“I estimate an 18% chance that Dai Dai reaches #37 or higher on the U.S. Billboard Hot 100 by July 31, 2026. The song is a clear global World Cup hit, but its U.S. Hot 100 inputs are still too thin: weak U.S. Spotify, no verified U.S. top-37 chart result, and YouTube no longer counting for U.S. Billboard charts. The live path to Yes is a large July 19 World Cup final halftime spike, possibly helped by sales, remixes, or a focused U.S. push.”

_{Figure 2. Baseline and Symbiosis forecasts for Dai Dai. Community values are observations quoted in Slack, with a dashed line connecting them.}

As “Dai Dai” began to climb the charts, Scott and Robert concluded they should have trusted the bot’s forecast on the question more, and they stopped manually overriding it. The song ultimately outpeaked “Waka Waka.” In the end, the baseline bot outscored Symbiosis by about 91 points on the question.

Later, Scott reported back in Slack: “In the background I’m hearing a neighbor play Dai Dai!”

Becoming more selective

Scott and Robert became more cautious about replacing the bot’s forecasts after Dai Dai. In the analysis shown below, 74% of manual overrides in June and July scored better than the model forecasts they replaced, but their estimated net effect was a loss of about 64 points. In August, 94% scored better, with an estimated net gain of 19 points. These later results were encouraging, although the questions and opportunities to intervene also changed over time.

_{Figure 3. Estimated score effect of manual overrides, grouped by the month submitted. Counts refer to individual overrides, including repeated interventions on the same question.}

Robert reflected on the approach on July 29:

“We should certainly step in where it clearly doesn’t understand something or is behaving erratically… but our threshold for intervening should be higher and we should intervene less dramatically.”

What we learned from Symbiosis

The bot’s comparative advantage was its ability to rapidly conduct an enormous amount of research and analysis. It didn’t directly access the Metaculus Community Prediction or read the comments of other forecasters. Scott and Robert could bring information it had missed into its analysis, as they did on the Tony Awards question.

Symbiosis outscored the baseline bot on the questions where Scott and Robert just added context without manually overriding it at any point. Most manual overrides also improved Symbiosis’ score. But a handful of manual overrides were extremely costly, leaving a net loss from overrides. In addition, manual overrides could result in stale, out-of-date forecasts when Scott and Robert failed to update them as the situation changed.

It wasn’t always easy to distinguish between a forecasting error and a difference in subjective judgment. On the three questions where overrides cost Symbiosis the most points (Dai Dai, the SDNY appointment, and Lebanese Armed Forces deployments), Scott and Robert replaced the bot’s forecast with something closer to the Metaculus Community Prediction. They also moved the forecast in the direction of a “no” resolution in each case. Those decisions raise a question about how much their own judgments were being influenced by the community’s view.

Because they chose which questions to intervene on and how, this comparison doesn’t establish that supplying context generally works better than overriding a forecast.

What this suggests for experts working with AI

Symbiosis suggests that working effectively with a strong AI system requires an additional skill: knowing when and how your own expertise can improve its output. Scott and Robert could identify missing information and question assumptions in the bot’s analysis. Deciding when those concerns justified replacing its forecast proved harder.

A useful starting point is to identify the reason for an intervention. Is information missing, has the system misunderstood something, or do you disagree with how it has weighed the evidence? After supplying context, check how the analysis and forecast changed. If you replace the forecast yourself, record why and keep it up to date.

Evaluations should track these decisions alongside the final results. Performance may change as people gain experience with a system, but the questions and circumstances can change too. A short trial gives us a limited view of that process. We suspect similar challenges arise in fields such as finance, insurance, and law, where experts must decide when to rely on an AI system and when to intervene.

What this means for Preseen

We’re no longer entering Symbiosis as a separate competitor. We’re using what Scott and Robert learned to improve Preseen itself.

Analysts already use Preseen every day to inform their decisions. We’re turning what we learned from Symbiosis into a practical guide to working with Preseen: what to watch for, where human expertise can help, and how to decide when to step in.

This is one of the things we’re most excited about building. Preseen can research questions at a scale that would be difficult for an analyst working alone, while an analyst can bring context and judgment the bot may miss. We want to make it easier for people to put both to use.

There is more to test, including which human interventions help across a wider range of questions. We’re using the successes and mistakes of Symbiosis to shape that work and the guidance we give analysts using Preseen.

Note on the comparison

Symbiosis joined the tournament 12 days late. Its official score was 931.6, compared with 918.2 for the baseline bot. The comparison isn’t robust across different ways of cutting the data, and the difference in score largely hinged on a small number of questions. That is why we can’t draw a firm conclusion about their relative performance from this tournament.

James is the Max Palevsky Professor of Sociology and Data Science at the University of Chicago and founding director of Knowledge Lab and the Chicago Center for Computational Social Science. His papers have been widely published in Nature, Science, PNAS, and top computer and social science outlets.

His connection to Preseen goes back to its origins. James helped inspire our co-founder and CEO, Venia Veselovsky, to pursue forecasting important world events in the wild. Over the past year, he has helped shape our research agenda and will now lead that work.

James studies how people (and machines) generate, share, and evaluate knowledge about the world, and how AI can help us anticipate discoveries and identify promising ideas that researchers might otherwise overlook. He leads a University of Chicago research project awarded $20 million by the National Science Foundation to anticipate scientific and technological advances. His recent work includes PreScience, a dataset and benchmark for testing scientific forecasting in AI research, several works showing how AI can unintentionally narrow future science, and how to counterbalance these tendencies for more impactful discoveries and forecasts.

These questions are central to the research we want to pursue at Preseen: how to make better forecasts, broaden the factors they consider, evaluate them at scale, and help AI systems incorporate forecast results to drive critical decisions.

A sneak peek at what we’ll be working on:

• A common layer for knowledge sharing across AI agents.

• Evaluations for forecasting scientific and technological progress.

• Predicting key human decision-makers’ decisions

• Categorizing and counteracting informational bubbles.

• Improving human decision-making through complex forecasts

If you’re interested in working with James and the Preseen team on forecasting research, reach out to careers@preseen.com.

Technologies that remake the world rarely announce themselves. When Bell Labs unveiled the transistor in 1948, The New York Times gave it a throwaway item on page 46. mRNA vaccines spent years as an unfundable side project before they helped to end the COVID pandemic. This pattern holds across fields: the best-funded programs and most-cited papers are seldom the ones that matter most, and the advances that do are often ones no one saw coming. As geopolitical competition sharpens and capital floods into critical technology areas, the cost of backing the wrong bets rises with it. Today we announce a partnership between Preseen and Knowledge Lab (KLab) at the University of Chicago to help decision makers spot what matters before it becomes obvious, by pairing KLab’s models of how discoveries emerge with Preseen’s forecasting agents.

Preseen builds AI systems that research questions and produce probabilistic forecasts. Each forecast includes the evidence and reasoning behind the estimate. Preseen’s forecasting systems compete publicly on Metaculus, where predictions are scored against subsequent outcomes. In the Spring 2026 Metaculus Cup, Preseen placed third overall among 1,283 participants and was the highest-ranked bot. Preseen’s work has been covered by Scott Alexander and The Economist. Its co-founder and CEO, Veniamin Veselovsky, also discussed AI forecasting on The New York Times’ Hard Fork Podcast.

James Evans, who directs KLab and leads one of the National Science Foundation’s multi-year projects on Assessing and Predicting Technology Outcomes (APTO), was part of Preseen’s founding. Among the questions that drew him in was one that sits at the heart of APTO’s mission: how do you identify and invest in the improbable discoveries most likely to reshape critical technology areas before they do? Through the extended partnership, APTO-funded data and models will be incorporated with Preseen’s research and forecasting agents, bringing that research to a deployed product with clients and impact at scale.

A forecasting question specifies an outcome and a time frame. For a funder deciding where to direct resources, that might mean asking: “If NSF invests $100M in quantum fabrication infrastructure in 2026, how likely is it to yield commercially viable devices within five years?” Preseen’s agents then break the question into parts, research the relevant evidence, and combine their findings into an estimate of how likely an outcome is. The accompanying explanation shows how they arrived at that estimate.

In a prediction market like Kalshi or Polymarket, participants trade contracts on future events, and prices reflect their collective expectations. Preseen produces forecasts by researching the question through parallel agentic research, so it can also address questions for which no active market yet exists. The aim is to produce well-calibrated probabilities: among events assigned a 70% chance of happening, roughly 70% should happen over time.

The research will bring KLab’s methods together with Preseen’s AI forecasting systems. Current AI forecasting systems struggle with two classes of problems that matter most for scientific progress: conditional forecasts and open-ended questions.

Conditional forecasts ask how the likelihood of one event changes if another occurs. A forecast of a technology’s commercial adoption, for example, may depend on whether a technical bottleneck is resolved. We plan to examine those dependencies and connect the forecasts of individual milestones. The challenge is to keep forecasts consistent with one another as assumptions change and that would help researchers and funders understand which developments matter most to the overall outlook.

The second challenge is surprise. AI forecasting systems tend to underperform on questions whose answers are genuinely unexpected, which is where the most important scientific progress tends to happen. KLab has developed models that treat surprise as a signal, identifying anomalous results and overlooked findings most likely to matter downstream.

More tangibly, the relationship will consist of (1) identifying possible scientific and technological advances (2) mapping their dependencies (3) converting them into dated, testable questions, (4) forecasting those questions, and (5) monitoring the probabilities as the world changes.

One of APTO’s projects at KLab is being built for this. A multi-agent system led by postdoc Robbie Ward approaches prediction from the opposite direction to Preseen. Where Preseen starts with a proposed outcome and works backward to estimate its probability, KLab’s starts with a surprising actual or possible technological advance and generates future developments in science, technology, markets, and security most likely to follow, along with their probabilities. It does this recursively: It ingests data from sources like scientific papers, patents, policy documents, and clinical trials, then models how ideas from different domains are likely to build on and transform following the advance.

The integration of open-ended simulations of technology outcomes with Preseen will leverage each system’s strengths. Preseen can act as an AI forecaster on top of KLab’s generated prediction graphs, re-evaluating conditional probabilities and updating the network as new information arrives. KLab’s simulations, in turn, can take Preseen’s outputs as seeds and expand them into branching maps of downstream consequences to surface antecedents and edge cases that a one-shot forecast might otherwise miss. The result is a system that can both answer a specific question about the future impact of technologies like AI and quantum, and map the web of likely developments upon which they depend.

In the future, we believe that AI will be at the core of scientific and technological progress. AI will help determine many of the most promising initiatives to explore, track how science and technologies are evolving, and inform the methodology to lead in the most powerful directions. The KLab-Preseen partnership tackles the first two parts of that problem. The goal, ultimately, is a world where the best science and technology gets funded before its importance is appreciated.

The question we get most often asked at Preseen is if we’re good at forecasting the future why don’t we open up a hedge fund.

We believe that this question is provincial.

AI forecasting will be far larger than what any fund can manage. When people make a decision, they select an action that maximizes a certain outcome they care about. This conditional – expected outcome given an action – is a forecast. All decisions are forecasts.

Until now, decisions have been made on vibes, expensive consultants, analysts, and forecasters whose job is to decide on how to convert this forecast into an actionable decision.

We are now reaching a point where AI systems are achieving parity with the best humans at forecasting the future (checkout some of Preseen’s past track record). This means that soon AI systems will be better than humans at making decisions.

The problems with forecasting so far is that it’s been tough to productize (good article on this). We believe that this is because forecasts have been too focused on predicting one off events, without being personalized to the action space of the organization that cares about the one off event.

There are two sides to these forecasts that Preseen is building towards.

• What action should we take to maximize the chance of a certain outcome? If the USA wants to limit Iran’s enrichment what action should they take?

• What action should we take in response to a certain outcome in the world? If the Strait of Hormuz stays closed, how should we adjust our supply chain?

Forecasting is going to be the decision-making layer of AGI.