Manifold vs Polymarket Volatility Comparison

Comparing LK-99 Prediction Markets

[Data and code]

[posting on August 3, 2023. The markets I’m comparing here will probably be active for awhile, so if people are interested I’ll keep collecting data and updating the results.]

Intro

Like everyone else, I’ve been following the LK-99 room temperature superconductor news out of South Korea, wondering if it’s legit and if the results will replicate. I don’t know enough about physics to actually read the pre-prints coming out, so I’ve mostly just been watching the up and down swings of the prediction markets on Manifold and Polymarket.

While we all wait for more information to come out, I figured this could be a good time to test out a hypothesis I had about Manifold and Polymarket, and see if there’s any evidence to support or challenge it.

For those who don’t already know, Polymarket is a real-money prediction market site (illegal to bet on in the US, but legal in other countries). Manifold is a fake-money prediction market site, where users bet a fake currency called “Mana,” with the hope that their reputation will still serve as an incentive to make accurate predictions, even with no real money involved.

Hypothesis / Results

For awhile, I’ve been wondering if Manifold is more volatile than Polymarket, since there’s no real money involved and people might be willing to take bigger risks. The past couple days there have been some pretty big swings in the LK-99 prediction markets as new information comes out. So, I thought it would be interesting to compare the volatility of the Manifold and Polymarket prices for LK-99 being legit. I was basically wondering if the swings look something like this:

Fake plot where 2 assets respond in the same direction to news, but one (blue) is more volatile and has more extreme swings than the other (red).

Like maybe both markets have similar long run averages, but maybe Manifold has more intense swings in response to news, since there’s no real money involved and people are less scrupulous and more eager to take risks. This isn’t really a 100% apples-to-apples comparison looking at real-money / fake-money, because there could also differences in trading volume, number of traders, etc between the markets, but I thought it would at least be interesting to look and see if the prices look like this.

Also, I was apparently not the only one to think this. I saw a lot of comments on the internet like:

suggesting that the fake-money nature of Manifold could be leading to people making wild bets they wouldn’t otherwise make, potentially leading to high market volatility (in fact — I myself made a reckless bet of YES on Manifold that I probably wouldn’t have made with real money! ).

Anyway, this is NOT what I found when I looked at the actual data. Here’s a plot of hourly prices on these markets over the past couple days:

At least looking at it visually, it doesn’t appear that Manifold has wilder, more extreme swings than Polymarket.

Next I looked at an actual measure of volatility: rolling coefficient of variation. Coefficient of variation (CV) is just a fancy term for the standard deviation divided by the mean. “Rolling” means that for each datapoint, I looked at that datapoint, the datapoints from the 5 previous hours, and the datapoints from the next 5 hours, and calculated the CV for this 11-hour span. So basically just think of this as a plot of volatility over time. Here’s the plot:

Again, nothing too exciting, and no obvious differences, other than some anomalous bumps in volatility for Polymarket between 7-29 and 7-31 (also visible on the price vs time plot).

Lastly, I wanted to get a single numerical measure of overall volatility for each market, so I calculated the hour-to-hour percent changes for the data, and looked at the variation in these distributions of percent changes:

Polymarket SD of Percent Changes: 13.084

Manifold SD of Percent Changes: 9.019

Here’s also the CV (again, just the above standard deviations divided by the means of the hour-to-hour percent changes):

Polymarket CV of Percent Changes: 12.947

Manifold CV of Percent Changes: 10.217

So yeah, this does NOT support my earlier hypothesis that Manifold would be more volatile, due to people being more risk-seeking when there’s no real money involved. In fact, it looks like Polymarket is slightly more volatile in this case, although there’s no enough evidence yet to say if this is a general trend that will replicate across markets. (Actually I might start making my own prediction markets about whether results I publish on this blog will replicate or not lol)

Feature Request

Nevermind about this part! Turns out I was wrong and this data IS available on the API (at least for Manifold, will need to look again for Polymarket). If anyone is having the same problem as me and can’t find it, it’s in the /v0/bets part of the API, not /v0/market.

I’m going to end this post with a feature request from Manifold and Polymarket (I think some of their employees actually read this blog, so maybe it’ll get some traction). First of all, I just want to say that I’m a huge fan of both sites. I think the work they’re doing is extremely valuable, and that prediction markets are very useful as a serious epistemological tool (as we’re seeing now with the LK-99 news).

I would like to keep doing these types of comparative analysis projects about different prediction markets. However, one of the biggest limiting factors I’m facing is that it’s actually pretty difficult and time-consuming to collect the price vs time data. Even though it’s all available to look at on the specific market pages, I can’t find any easy way to download it, and as far as I can tell there isn’t any way to get this data from the APIs (at least for an active market). (If I’m wrong about this and someone knows how to get it from the API, please let me know!)

So to write this post, I actually had to go through and manually collect the hourly price data, by tracing my mouse across the plot and typing the numbers into a spreadsheet.

This is my request to any Manifold / Polymarket employees reading this — is it possible to add time-series price data to the API for active markets, so that users can easy download the data that’s being plotted on the market information pages? That would be awesome, and would make it easier for hobbyists like me to do these kinds of informal research projects. Thanks!