New York Times' Connections: A Case Study on NLP in Word Games

How can a solver find the intended relationships in a Connections puzzle rather than settle for associations that merely sound plausible? That question brings together game development, natural language processing, and search. Shafik Quoraishee approaches it as a New York Times games developer with a background in machine learning, mobile development, and data science.

The solver explored here is his independent experiment, separate from The Times’ internal research. Inspired by third-party work on AI and games, it is an investigation with preliminary results, not an official account of how The Times builds or evaluates its puzzles.

Connections entered beta in June 2023 and officially launched in August, according to Quoraishee. Edited by Wyna Liu, it rapidly became a major Times game: he reports hundreds of millions of plays in its first year and popularity second only to Wordle among Times games. He also emphasizes that the puzzles, mechanics, and programming are human-made, expressing that as an ongoing commitment. The AI experiment concerns solving the game, not generating its daily content.

Each daily board contains 16 words. The player must partition them into four groups of four, with every word assigned to exactly one group. A fourth incorrect guess ends the game. A plausible quartet is not enough: it must fit a complete, nonoverlapping solution.

The progression changes what a solver must recognize. Matching familiar meanings may handle an obvious group; a harder group can require reconsidering what a word means or how it participates in a phrase.

These decoys make Connections a useful test of abstraction and resistance to superficial similarity. The fixed board and intended solution allow repeated trials, while the category labels give a solver something concrete to explain. In particular, solving a difficult purple group requires identifying a relationship that accounts for all four members. A convincing explanation still has to match the intended grouping.

Quoraishee demonstrates the failure mode by entering a puzzle into ChatGPT and receiving an incorrect solution. The demonstration uses GPT-4o; he qualifies the comparison because stronger models reason better. The example establishes that this model response fails on this puzzle, rather than measuring the performance of every model available.

Human players face a related tension between System 1, fast intuitive recognition, and System 2, slow deliberate reasoning. An obvious association can be recognized almost immediately. An ambiguous word may require searching background knowledge, considering another meaning, and checking which category can accommodate it. Effective play combines both modes.

Either mode can fail. Fast intuition accepts a group that looks obvious even though one of its words belongs elsewhere; deliberate reasoning can overcomplicate a group whose connection was straightforward. Quoraishee points to third-party Connections studies showing improving but still imperfect LLM performance. Those studies are separate from Times research, and their results depend on how solving is evaluated.

Random guessing provides a baseline for the size of the problem. Quoraishee describes the chance of completing the untouched board randomly as approximately zero. With one correct category removed, he estimates roughly one chance in five or six thousand; with two removed, he gives one in 35. The recording does not specify a retry protocol, so these figures are best understood through the combinatorics of choosing a complete partition, rather than as win rates under the game’s four-mistake rule.

For 4k remaining words, count unordered partitions into k quartets by permuting the words, then dividing out the order within each quartet and the order of the quartets themselves:

The calculation can be expressed directly in Python:

from fractions import Fraction
from math import factorial


def partition_count(groups: int) -> int:
    return factorial(4 * groups) // (
        factorial(4) ** groups * factorial(groups)
    )


for groups in (4, 3, 2):
    count = partition_count(groups)
    probability = Fraction(1, count)
    print(f"{4 * groups} words: {count:,} partitions; P = {probability}")

This derivation assumes uniform sampling of one complete partition and one intended answer. The last probability is approximately 2.86%, rather than the spoken approximation of about two percent. Even after solving half the board, relationships must do substantial work to narrow the search.

Graph coloring supplies the next starting point. In ordinary graph coloring, colors are assigned to vertices so that adjacent vertices receive different colors; the chromatic number is the minimum number needed. Backtracking and greedy coloring are familiar approaches, with applications including scheduling and frequency assignment in wireless networks.

Quoraishee calls his Connections representation augmented graph coloring. Each word becomes a vertex, each hidden category corresponds to a color, and the goal is to give the four words in a category the same color. Edges carry estimated relationship strength. His prototype visualizes candidate groupings using this representation.

The augmentation matters: a strong relationship edge here encourages words to belong together, whereas an edge in ordinary proper coloring prohibits a shared color. The Connections formulation is therefore closer to constrained grouping than to applying an off-the-shelf coloring rule unchanged. Its purpose is to make relationships guide the search while preserving the four-quartet structure. Without useful relationship information, the solver falls back toward arbitrary sorting.

A single notion of similarity cannot describe every valid Connections category. Quoraishee makes the point with a transformer-title joke: semantic similarity is not all you need. A solver must distinguish relationships involving spelling, word structure, world knowledge, and association.

• Orthography: Anagrams relate through their letters, regardless of whether their meanings are close.
• Morphology: Shared suffixes can provide a connection. kingdom and fiefdom share -dom; Quoraishee also mentions connectedness while discussing word structure.
• Encyclopedic knowledge: Globe, Mirror, Post, and Sun form a newspaper category, despite having other familiar meanings.
• Association: strawberry, rose, and Mars can connect through redness rather than a shared object type.

These examples require different evidence. A spelling comparison can expose an anagram, but recognizing newspaper names requires knowledge beyond the letters or a word’s most common meaning.

Polysemy makes the problem harder still. mole can denote an animal, a birthmark, a spy, or a unit. Choosing the wrong sense can produce a coherent-looking but incorrect group. Quoraishee identifies this multiple-meaning problem as a major source of difficulty for both people and AI: the solver must retain alternative interpretations long enough to see which one fits the board.

Quoraishee uses relational alignment for a score associating two words under a chosen relationship metric. His heat-map simulation lets him switch metrics and inspect the resulting pairwise scores. The proposed use extends beyond selecting groups: if the intended categories are strongly coherent under a metric, perhaps the puzzle will be easier to solve.

For the illustrated comparison, Quoraishee cites reported solve rates of 19% for a hard puzzle and seventy-something percent for an easier one. He observes a rough difference in their coherence or alignment scores. The puzzle identities, player population, and measurement protocol are not specified, so this is a motivating comparison rather than a calibrated predictor of difficulty.

Plotting these scores across puzzles and categories introduces a time dimension. Trends could reveal changes in relational coherence, but anomalies need interpretation: Quoraishee tentatively attributes the zero plotted on December 12 to a broken puzzle. That point should not automatically be read as evidence of an exceptionally difficult puzzle.

There is also no reason to expect one score to capture every relationship. The same pair can align strongly under morphology and weakly under an encyclopedic category, or vice versa. A multidimensional alignment profile retains those differences instead of collapsing them immediately into one number.

The radar-chart demonstration makes those profiles visible. Charts labeled FIRE words, Cooking Verbs, Homophones, and Chess Pieces show distinct shapes across relationship dimensions. The shape gives a solver—or a human inspecting it—a way to compare which kinds of evidence support a group. The talk introduces this representation without specifying the underlying scoring formulas.

The semantic distribution evaluation framework then moves from individual profiles to historical counts. It assigns categories to relationship buckets such as hypernymy, morphology, and orthography, displays their occurrence across days, and aggregates them into counts or histograms. Those distributions offer another source of trends for later analysis: not just how strongly words align, but which kinds of relationships the puzzles use over time.

The coloring approximation reduces the search space; adding multiple relationship types turns the problem toward graph clustering. Quoraishee describes the richer representation as multidimensional graphs or hypergraphs and demonstrates convergence in a three-dimensional view. The display is an illustration of the representation, not a specification of its hyperedges.

Two kinds of strength become useful to inspect. Intercluster strength describes relationships crossing candidate groups; intracluster strength describes relationships within a group formed by the algorithm. Both matter in Connections. A quartet can be internally coherent yet compete with another quartet for the same word, so evaluating only its internal similarity misses the board-wide conflict.

To construct these graphs, Quoraishee names WordNet, ConceptNet, and word embeddings. They contribute complementary evidence: WordNet organizes sense-specific lexical relationships, ConceptNet supplies broader commonsense relationships, and embeddings provide numerical representations from which relatedness can be estimated. These are inputs to the representation, not interchangeable definitions of a single similarity score.

His flat, two-dimensional example shows co-occurrence and conceptual links, including dog–cat relatedness, a cat’s capacity for play, and its role as a pet. He also describes a mouse hunting a cat; that apparent reversal illustrates why a directed relationship needs its subject and object checked, not merely a connection between two words. Explicit links expose such details to inspection.

This is the motivation for building the representation procedurally. Instead of asking an LLM for a final grouping and accepting its explanation, the experiment makes more of the relationship structure available to track. The intended benefit is a more explainable search process: one can inspect which relations were represented and which candidate groups they supported.

The learned solver combines a graph convolutional neural network with reinforcement learning. The graph network takes the constructed graph as input and produces candidate solution subgraphs. Quoraishee describes optimizing node and edge weights, identifying which graphs fit candidate solutions, and tracking cluster morphology—the structure or shape of the resulting clusters. These are candidate groupings, not automatically accepted answers.

Learning agents then work with the graph-based candidate system. At the level explained in the talk, the division of labor is straightforward: the graph network proposes structure, while reinforcement learning helps navigate the semantic subclusters. The system diagram receives no detailed walkthrough, so the architecture remains conceptual here rather than a reproducible training recipe.

The final three-dimensional visualization illustrates traversal through the semantic space after graph construction. Quoraishee notes that its cluster points are difficult to see. Its intended role is to show which traversals the system permits as it moves among candidate subclusters, connecting the relationship representation to a search process.

Quoraishee reports a qualitative improvement in solvability after the changes on a small subset of hard puzzles. He tested only a few puzzles before and after and gives no numerical improvement or detailed evaluation protocol. The next proposed step is to expand the evaluation to more puzzles and explore applications in game development.

Broader evaluation also has to address what a correct answer demonstrates. Connections solutions are published online, and LLMs are trained on internet data. That creates a possible contamination problem: an answer could reflect exposure to a published solution rather than successful reasoning through a new board. Quoraishee raises this as a concern, not a demonstrated explanation of any particular model result.

The remaining motivation is transparency. A correct LLM answer can still leave its actual route to the solution opaque; an explicit graph offers relationships, candidate clusters, and traversals to inspect. Quoraishee closes by proposing a connection to the ARC-AGI benchmark, without presenting an integration or score. The research direction is to test whether a solver can build and navigate the relationships a new puzzle requires—and make that process visible enough to examine.