Embarking on a Quest

Once upon a time, noble knights embarked on a legendary journey, known as Wikispeedia . Their quest? Armed only with their wit, they ougth to navigate through a labyrinth of articles to reach their destination.

However, not all of the adventurers emerged victorous; many of them succumbed in the face of adversity and their own frustation. These poors souls became known as quitters .

Was their lack of skill and perseverance that sealed their fate? Or was it the sheer difficulty of their quest? This project The Quitter's Quest seeks to not only find answer to these questions, but uncover the tricks of the Bias Enchantress.

The Dataset

A Chronicle of Victors and Vanquished

The objective of the quest is clear; given a source article reach the specified target article , using as few clicks as possible. Every article is linked to others through hyperlinks.

In the archives of the kingdom lies a record of every adventurer’s journey:

• Finished quest/games: players reached the target article from a source article successfully.
• Unfinished quest/games: players did not reach the target article.

To understand why some fail and others succeed, we must determine whether success is dependent on the player's choices, or on the difficulty of the game he faces?

However, given a specific quest (a source and a target), how can one estimate its difficulty? To answer this, we need a metric to evaluate the true challenge of a path.

The Difficulty Conundrum

We began our quest by exploring player-estimated difficulty. Whenever players completed a game, they rated how challenging they found it, but we still lack a metric that can extend to all games, whether completed or abandoned.

To address this, we aim to test potential metrics that capture the difficulty of a path. Our approach is to group all games—both finished and unfinished—according to the metric being tested. For each group, we calculate:

• Estimated difficulty: The average difficulty rating given by players who completed games in the group.
• Failure ratio: The proportion of unfinished games relative to all games in the group.

By plotting these groups, comparing the estimated difficulty of finished games against the failure ratio, we can evaluate the effectiveness of the metric. A strong correlation between player-estimated difficulty and the failure ratio would indicate that the metric successfully captures the inherent challenge of a path.

This approach allows us to explore not only the nature of difficulty but also how it varies across groups of games.

Level 1: Adventuring into the Kingdom

Intuitively, one might resort to the target to explain the difficulty of a game; specially its category, the area of knowledge it belongs to.

Key Insights

• The categories People and Everyday Life rank high in both difficulty and the proportion of unfinished games.
• People includes modern, popular figures. Without specific knowledge about them, players face challenges finding paths.
• Everyday Life consists of specific concepts that might not have obvious or direct connections (e.g., Book → Bean)
• Countries and Geography rank low in in both difficulty and unfinished game rates.
  • These categories are easier to navigate, with hierarchical connections (e.g., France → Europe → Paris) and general cultural knowledge aiding players.

Level 2: Traversing the Dark Forest

Although a relation has been found between target's category and dificulty, we can one step further and analyze its subcategory in an attempt to find a better correlation.

Key Insights

Similarly to before:

• More comparmentalized subcategories (e.g; Novels , Railway Transport) rank high in both difficulty and the proportion of unfinished games.
• More 'general-knowledge' subcategories (e.g; Languages, European Geography, USA Presidents) rank low in in both difficulty and unfinished game rates,

While variability remains, we've uncovered an important insight: concepts with more connections-easier to link- provide a less challenging path to navigate through.

Level 3: Delving into the Labyrinth

To evaluate the linkability of articles:

1. We model a graph, where articles are represented as nodes, and links between them form the edges.
2. We then calculate the shortest path length between all source and target combinations played in the game.

Unfinished games have a higher median and maximum shortest path length compared to finished games!

Indeed, there is almost a perfect correlation between the ratio of unfinished games and estimated difficulty for games according to shortest path length.

At last, we have uncovered a metric that allows us to compare games based on their 'objective' difficulty. With this newfound knowledge, the maze begins to take shape, and we quickly arrive at the oracle's lair.

Level 4: The Oracle's Counsel

The oracle's riddle inspires us to study how the properties of the nodes between the source and target vary across these shortest paths, according to two metrics: PageRank and closeness centrality.

Unlocking the Labyrinth's Secrets

Key Insights

• Both metrics start low, peak at the second or third node, and then decrease till the target.
• PageRank: peaks early as path goes through globally influential hub articles that bridge regions of the graph between source and target. After these hubs, it drops sharply.
• Closeness centrality: declines more gradually, reflecting that intermediate nodes, while less globally influential, remain well-connected locally.

Into the Depths of the Labyrinth

The differences between the metrics highlight how finding the target becomes increasingly challenging as the shortest path lengthens.

While PageRank remains consistently low for nodes along longer paths, indicating a lack of influential connections; closeness centrality shows a progressive decline.

This reveals that each successive node in the path becomes less connected to the rest of the graph, forcing players onto a more constrained and isolated route.