Which AI startups actually land enterprise contracts? — Brian Lewis, Millennium

Enterprise AI already leaves much of the value available at current model intelligence unused. That is the opening hypothesis of Brian Lewis, who works on product at Millennium, a hedge fund, and speaks here personally rather than for his employer. His examples concern evaluating outside products, rather than describing proprietary systems.

The room contains both sellers and buyers, and both have work to do. Sellers supply models and products. Buyers supply the systems those products must enter. The talk follows those two sides in order: first the requirements that eliminate vendors, then the organizational foundations that buyers need to improve.

Why teach vendors how to sell to him? Lewis’s mother asked essentially that question when he rehearsed the talk. His answer is an interest in seeing things come together and work well. The purchasing advice that follows is unusually concrete: enterprise readiness means making a product usable inside the customer’s organization, through the awkward details that a demo can skip.

The buying process begins with an internal pain point and a build-or-buy question. For a given problem, Lewis might find 10 to 15 interesting startups. Initial due diligence reduces those to two or three demo calls, which lead to zero or one pilots. Roughly one in four pilots eventually becomes a contract. His overall estimate is that about 5% of demo calls end in a signature. These are approximate observations; the resemblance to an industry benchmark is his reported comparison.

What eliminates the others? The failures fall into efficacy and commercial value, security, reliability and legal issues. Lewis assigns 40% to efficacy and commercial issues, without developing a measurement method. Walking through the requirements exposes the distance between a startup’s idea of enterprise readiness and a purchase the customer can actually approve.

At 4:19, efficacy becomes four practical conditions:

• Solve the problem: The product must address the pain point that started the evaluation.
• Price the value: The commercial model must make sense for what the buyer receives.
• Show working integrations: Integration needs to be demonstrable on day one. A future possibility cannot establish that the product works in the customer’s environment.
• Use buyer-defined success: The customer writes the success criteria, keeping the evaluation tied to its need.

The internal alternative remains live. Some pitches describe something the platform team could rebuild in about six weeks. That does not settle the decision: buying can still be preferable even when rebuilding is possible. It does require the vendor to offer a persuasive reason to purchase beyond the existence of an implementable idea.

One pricing proposal makes the mismatch tangible. A vendor’s wrapper used LLM traffic passing through the buyer’s own gateway. None of that traffic ran through vendor infrastructure. The vendor nevertheless asked the buyer to report its gateway telemetry so the vendor could charge a large margin on the traffic. The proposal failed. The traffic path supplied a usage number, but the buyer did not accept that number as justification for the proposed markup.

Other failures consume the evaluation window without moving the product forward: promises still lack delivery estimates after two months, or salespeople keep pitching features the customer already declined. Meanwhile, pilot expectations have shortened over Lewis’s little more than two years at Millennium—from six months, to three months, to potentially two weeks. A shorter pilot gives future promises less room to compensate for missing functionality; the pilot window describes evaluation, rather than the time needed for a full deployment.

Security evaluation asks whether the product works under the buyer’s required configuration. Lewis describes himself as a frontline participant in these conversations rather than a security expert. His preferred arrangement is zero data retention, or ZDR. Where retention cannot be avoided, he wants customer-managed encryption keys—with the crucial qualification that enabling them must not break the product. A promised security capability is little help if the application stops functioning when the customer uses it.

The deployment and access requirements address different kinds of control:

• Traffic and hosting: Route LLM traffic through the buyer’s gateway and support deployment in the buyer’s own cloud infrastructure.
• Group-based permissions: Tie role-based access control, or RBAC, to existing AD groups or other entitlement groups through the SCIM-tied arrangement Lewis requests. The practical requirement is that the product use the organization’s existing group structure to give different groups different access at different times.
• API configuration: Make access and feature decisions configurable through an API, rather than offering only an all-users switch.
• Security staffing: Even a smaller vendor should have at least one real security hire who can understand and discuss the system’s behavior.

A one-click integration can conceal a serious restriction: it may only work if granted read and write access to everything. The connection succeeds technically, while the access model makes it unsuitable for the enterprise. New or beta features enabled automatically with each release create a related problem. The buyer needs to decide which capabilities become available, to whom and when.

Other pilot failures concern the data path itself: vendors send data to their providers or cloud servers despite the buyer’s requirements. Lewis says the pilots use non-production data. Repeatedly postponing the security architecture diagram leaves the buyer unable to understand that path. And answering a breach-response question with the fact that no breach has happened yet supplies no response plan. Prior good fortune cannot explain what happens next.

Reliability begins with a working control plane: the administrative part of the product that lets the customer configure and manage it. Every admin setting should be available through an API, configuration changes should produce audit logs, and the customer should control rollout. If five administrators have access and one changes something late at night, the organization needs to reconstruct what happened. Meaningful service-level agreements, or SLAs, and a reachable support engineer complete the operating arrangement.

Consider the update example at 10:02. An application ships updates multiple times a day and offers 3,000 users an attractive “Relaunch to update” button. Each user can move independently, so the deployment becomes a population running different versions. If a new release breaks SSL certificates, the customer struggles to track the failure because users no longer share a consistent version. Frequent shipping, independent adoption and poor version visibility combine to make deployment at scale difficult.

How does a convenient update button become an operational problem? The diagram follows the change from rapid releases to mixed versions to difficult diagnosis. The branch matters: a faulty release reaches a deployment whose users have updated at different times. Rollout control would let the customer manage that transition; audit logs would help reconstruct configuration changes alongside it.

Documentation also needs a history. A support page can introduce terms or risks missing from the contract, then display only “Updated yesterday.” Without earlier versions, the customer cannot compare what changed. Operational visibility matters just as much during outages: Lewis recounts a core API being unavailable for multiple hours during a busy trading day in an environment trading billions of dollars, without attributing a financial loss to that outage. A status page, an SLA roadmap and access to support give the buyer ways to understand and manage disruption.

The legal requirements cover how data is used, who handles it and who bears risk. No training on customer data should apply regardless of the feature or product. Subprocessors—the vendor’s own providers—need to be transparent because their risks become risks the customer must manage. Lewis also wants intellectual-property indemnification with reasonable liability caps, giving the buyer contractual protection against infringing model output when it does not control the models. These are purchasing requirements, rather than a statement of how the law automatically assigns liability.

A ZDR promise can exist in the agreement and still fail in the application. During one pilot, a vendor mentioned something it had noticed in customer data. That observation exposed possession of information the vendor was not supposed to retain. The conversation changed the buyer’s understanding of the product: a legal commitment had not produced the promised data behavior. Lewis’s instruction is appropriately short: “If you say ZDR, do ZDR.”

Feature terms can change the arrangement too. Lewis describes vendors releasing new features only as beta, with beta clauses permitting data retention. Using the new functionality then introduces retention permission that the customer may not expect from the general product assurance. Fourth-party risk tucked away on a webpage rather than listed in the contract creates another visibility gap. The customer needs to understand the data arrangement across implementation, individual features and outside providers.

The credible vendor package now comes into focus: working security architecture, responsive support engineers, an admin API from the beginning, a 90-day plan for deployment into the buyer’s infrastructure and cloud, and buyer-written success criteria. The deployment plan addresses a different task from the potentially two-week pilot discussed earlier. Both require concrete work and dates behind the pitch.

The talk turns to buyers at 13:15. Lewis contrasts his stated average of a new frontier model every 11 days with architecture that may be a decade or more old. Companies can still be working through ERP migrations begun five years earlier. The release cadence is his comparison, rather than an independently established statistic here; its purpose is to expose the mismatch between rapidly arriving capabilities and slowly changing enterprise systems.

His allocation of adoption work is explicitly unscientific: 40% models and products, 60% data hygiene, clean architecture, integration, enablement and change management. This is a separate estimate from the earlier 40% assigned to purchasing failures. It expresses where he thinks effort belongs. Model capability does not itself repair legacy architecture or carry an organization through a change in working practices.

The flashlight metaphor explains the mechanism. AI exposes the systems and practices already helping or hindering work. It can accelerate a process whose foundations work well, while quickly encountering weaknesses in one that does not. Technology leaders therefore need to address those weaknesses before expecting an AI layer to make everything work. That is the reason to start with Lewis’s “boring sixty percent.”

Entitlements—the access a person or system is allowed to exercise—are the first foundation to revisit. People in large organizations may already have too much access or too little. The way those permissions are managed becomes more difficult when agents act quickly, work on a wider set of tasks and exercise judgment. Permission design has to account for both what an agent can reach and what it can do there.

Cross-platform integration comes next. AI’s usefulness depends on the systems it reaches, so connections across platforms become more important when the intended workflow spans them. A capable agent confined to one disconnected application cannot draw on the rest of the organization’s systems.

Centralized knowledge supplies another foundation. Drawing on Emil’s earlier keynote, Lewis calls for “thinner agents and a smarter substrate.” He makes that framing concrete: documentation, support articles and the knowledge that keeps a company operating should be centralized and easy to consume. The agent can then draw on shared organizational knowledge. A possible next step is for AI to help write that knowledge in real time, feeding new information back into the shared base. Lewis describes vendor discussions about this feedback loop, rather than a completed implementation.

Some organizations may also need a separate ecosystem for experimentation. When the gap between legacy architecture and the intended destination is too large, that environment could help teams discover what works before deciding how to proceed. Lewis presents it as an option they have considered, without prescribing a migration design.

The closing warning returns to entitlements: agents inherit the foundations already in place. Integration supplies reach, knowledge supplies usable context, and permissions determine which actions that reach allows. Existing weaknesses come along too. Lewis’s warning that agents could multiply rogue behavior by 100 times is rhetorical rather than a measured multiplier; his practical recommendation is to fix entitlements now, before faster execution magnifies the problem.

Lewis closes with a demanding customer in view: Millennium has about 8,000 people and tight security, compliance and regulatory requirements. He expects startups designed for customers with this level of control to be well prepared for many other buyers, though that is an expectation rather than a guarantee of acceptance. The immediate opportunity is simpler: build products that such customers can actually use.

Buyers must make that use possible too. Authentication, entitlements, governance and audit logging deserve sustained work alongside product selection. Vendors need to supply controllable, supportable software; enterprises need systems that can apply those controls. Getting enterprise-grade AI to work requires both sides of the table to finish their part.