Navigating the New Frontier: Testing Code When You Can't Predict the Output

The Changing Landscape of Software Testing

For decades, software testing has relied on a fundamental assumption: that code behaves deterministically. Given the same inputs and environment, a well-written program should produce predictable outputs. But this assumption is rapidly crumbling as large language models (LLMs) and AI-driven agents become integral to modern development. In a recent discussion, Ryan SmartBear hosted Fitz Nowlan, Vice President of AI and Architecture at SmartBear, to dissect how these shifts are forcing developers to rethink not only how they write code but also how they test it. The conversation explored a world where non-determinism—the very thing traditional testing seeks to eliminate—is becoming a feature, not a bug.

Leaving Old Assumptions Behind

Conventional software testing frameworks assume that a unit of code will always produce the same result for a given input. This assumption underpins unit tests, integration tests, and even end-to-end suites. However, when code is generated or influenced by an LLM, outputs can vary with each run due to randomness in sampling, temperature settings, or model updates. As Nowlan notes, "We are moving away from the idea that code is static and predictable. Instead, we have to treat each execution as a unique event."

Embracing Non-Determinism

The rise of LLM-driven agents brings a new category of software: agents that make decisions during runtime. These agents may call external APIs, synthesize responses, or even rewrite parts of their own code. Testing such systems requires moving beyond assert-equals logic. Nowlan suggests that testers should focus on behavioral outcomes and constraint satisfaction rather than exact output matching. For example, instead of checking that an agent returns the string "Hello, world!", a test might verify that the output is a polite greeting under 200 characters.

Challenges in Testing MCP Servers

One of the emerging paradigms is the Model Context Protocol (MCP) server, which mediates between LLMs and external tools. Testing MCP servers introduces unique difficulties because the server must handle unpredictable requests from the LLM while ensuring reliability and safety. Nowlan explains, "An MCP server doesn't know in advance what the LLM will ask. It has to be robust to any valid query, which means your tests must simulate a wide range of possible interactions."

Data Locality and Construction as Core Strategies

When source code becomes trivial to generate—thanks to LLMs—the bottleneck shifts from writing code to curating the data that trains and tests these models. Data locality—the practice of keeping data close to where it is consumed—gains new importance. Nowlan emphasizes that constructing high-quality, representative datasets for testing is now more valuable than writing lines of code. "If you can generate code on demand, the real value lies in knowing which data will make your system behave correctly," he says.

Practical Strategies for Testing Non-Deterministic Systems

So how can teams adapt? The discussion highlighted several actionable approaches:

• Property-based testing: Instead of checking specific outputs, define properties that must always hold true (e.g., "the response must be valid JSON").
• Statistical profiling: Run the same query multiple times and assess whether the distribution of outputs meets expected patterns.
• Mock external dependencies: For MCP servers, simulate the LLM's behavior to create repeatable test scenarios.
• Continuous data curation: Treat test datasets as living artifacts that evolve alongside the model and its use cases.

Using Anchor Links to Navigate

This article itself uses internal anchor links to help readers jump between sections. For instance, you can revisit the discussion on new assumptions or data construction strategies. In technical documentation and testing guides, similar anchors can link from a test case description to the relevant section of the system architecture.

The Future of Testing Is Adaptive

As AI continues to infiltrate every layer of software, testing methodologies must evolve. The conversation with Fitz Nowlan makes one thing clear: the era of predictable code is giving way to an era of probabilistic systems. Testing in this new world requires a shift in mindset—from verifying exact outputs to ensuring trustworthy behaviors. By investing in data construction, embracing non-determinism, and building flexible test frameworks, development teams can stay ahead of the curve.

Ultimately, the value of a test lies not in its ability to predict the future but in its capacity to detect when the system deviates from safe bounds. As Nowlan puts it, "We're not testing code anymore; we're testing behavior." And that requires a whole new toolbox.