# AI Writes Code Faster Than Uber Can Merge It - Preetam Dwivedi & Manjari Akella, Uber Preetam Dwivedi, Uber · Manjari Akella, Uber [Talk page](https://ai.engineer/talks/hX140a5klW0-ai-writes-code-faster-than-uber-merge) · [Video](https://www.youtube.com/watch?v=hX140a5klW0) ## AI Writes Code Faster Than Uber Can Merge It A 44% rise in hourly merge requests pushed Uber’s 95th-percentile land time from 30 minutes to three or four hours. To bring it back down, the developer platform team changed which requests could run together and which checks could wait until after merge. ### How two green pull requests break trunk [0:04](https://www.youtube.com/watch?v=hX140a5klW0&t=4s) Uber’s merge queue has to answer a question that a passing pull request leaves open. Will this change still build after someone else’s change lands? Preetam Dwivedi and Manjari Akella work on that problem across six language-based monorepos. More than 5,000 developers contribute over 36,000 commits and about 27,000 pull requests a week across those repositories. The Go monorepo is the largest and the focus of their account. Take two developers who open PRs A and B against the same commit on trunk. Both builds pass. A merges, and the build for its merge commit passes too. What did B’s green check establish? CI tested B on the original base. When B merges, trunk contains A and B together for the first time, and that build fails. Both developers had good reasons to expect their changes to work. Neither had a test result for the state they actually produced. SubmitQueue, Uber’s merge queue, brings that combined state into CI before it reaches trunk. With two requests, there are three candidate futures worth considering: A alone, B alone, and A plus B. A speculative build checks one of those futures while the changes are still queued. If the combined build passes, the queue can land the pair with evidence for the state it will create. If it fails, the two individual passes are insufficient to land that combination safely. ### How SubmitQueue kept a green trunk moving [3:51](https://www.youtube.com/watch?v=hX140a5klW0&t=231s) Testing possible futures creates its own scheduling problem. As more requests arrive, which ones need a combined build? Which speculative build should get resources first? The original queue used three optimizations. - **Target analysis.** The queue identifies the build and test targets a change affects. Two developers working on different services can have disjoint sets of targets. The queue can build and test those changes independently and merge them without making one wait for the other. - **Probabilistic speculation.** Build history and characteristics of the incoming change provide a score for how likely it is to succeed. That score helps prioritize the speculative builds most likely to be needed to land the change. The build itself still provides the pass or fail result. - **Bypassing large changes.** A slow build can already be in progress when a smaller change arrives. If the smaller change can build and land first, the queue lets it pass. Arrival order alone needn’t hold a ready change behind a long build. ### At 130 requests an hour, the long wait was measured in hours [5:33](https://www.youtube.com/watch?v=hX140a5klW0&t=333s) How much more traffic did it take to turn a half-hour wait into an afternoon? In January 2026, the queue handled roughly 90 requests an hour, with a 95th-percentile land time of 30 minutes. By March it was handling 130 requests an hour, a 44% increase that the team attributed primarily to AI-generated changes. At that rate, P95 land time reached three to four hours. The queue still kept trunk green. The cost showed up in how long developers at the slow end of the distribution waited to merge. ### Admitting cheap changes without making library owners schedule their own merges [8:04](https://www.youtube.com/watch?v=hX140a5klW0&t=484s) The first change put a gate at the entrance to the active queue. One way to estimate the cost of a request was already available from target analysis. A change affecting a few targets leaves room to build unrelated changes alongside it. A change to a widely used library can affect much of the build graph. It can hold resources while the queue validates it against many other requests. A small edit to that library can therefore be an expensive merge. During peak hours, the gate admitted cheap changes for the usual builds and tests. It rejected expensive ones and left their authors to retry outside peak hours. The business-hours land rate rose from 130 to about 160 an hour, a gain of over 20%. That policy worked particularly well for developers changing individual services. For a library maintainer, routine work could repeatedly trigger a rejection. The gate had effectively given that maintainer a second job as the scheduler. A secondary queue took over the retry. When an expensive request arrived during peak hours, the gate persisted it in the deferred queue. Once capacity permitted, for example after the peak, the system replayed that same request into active SubmitQueue. The developer could submit once and go home. The expensive change still had to wait for an opening, but nobody had to wake up early or stay late to press the button. ### Two weak links tie six requests to one queue [12:19](https://www.youtube.com/watch?v=hX140a5klW0&t=739s) Admission gating controlled what entered the queue. A separate constraint governed what the admitted requests could do. Imagine six requests, A through F, each touching a set of targets. Draw a link between two requests when their sets overlap. A shares a target with B, B shares one with C, and further overlaps connect the rest. A and F can end up in the same connected component without sharing a target themselves. Under the strict conflict rule, the queue schedules the whole component as one serial group. Conflict relaxation gives each link a strength. SubmitQueue uses Jaccard scoring for dependency overlap, which measures the fraction of distinct targets across two sets that they share. It also accounts for new dependency edges introduced by the changes. The combined score ranges from zero to one, with higher values indicating stronger overlap. The scheduler keeps the stronger links and can drop weaker ones to meet a configured parallelism target. The score measures overlap. An actual build is still needed to find out whether the combined changes work. Which links would free enough work to run in parallel? In the six-request example, a target of at least 50% parallelism calls for at least three independently working queues. The two weakest edges are B–C and B–E. Dropping those two scheduling constraints splits the single component into three components of two requests each. Each pair still has an ordering constraint inside its own queue, while the three queues can make progress at the same time. The shared targets still exist in the code. ### Assign conflicts on arrival to preserve speculative builds [16:19](https://www.youtube.com/watch?v=hX140a5klW0&t=979s) When should the scheduler choose which edges to relax? A global calculation could reconsider the entire conflict graph whenever work arrived or finished. But suppose a speculative build was already running for a combination chosen under the previous graph. Reassigning the requests could make a different combination the one that matters. Frequent updates would make it harder to use work the queue had already scheduled. Uber chose a local decision at entry. On arrival, a request is compared with everything already in the queue. The scheduler decides which of the arriving request’s edges to keep, then reuses that assignment. The next arrival gets its own comparison. Existing requests keep the relationships that their speculative builds assumed, so those builds can still help the requests land. Conflict relaxation delivered the largest throughput gain of the three changes. Peak throughput rose from about 160 lands an hour to nearly 250, sometimes higher, an increase of over 50%. It also changed the promise SubmitQueue could make. Two requests separated by a dropped edge could each pass pre-merge validation and land before anything tested their combination. Uber had deliberately reopened the possibility of the A-plus-B failure. ### Stovepipe tracks the last trunk state that passed [19:15](https://www.youtube.com/watch?v=hX140a5klW0&t=1155s) Return to A and B. SubmitQueue relaxes their conflict, tests each independently, then lands A followed by B. The combined build fails on trunk. Stovepipe, Uber’s post-merge validation system, checks the code as it actually landed. That gives consumers two references on the same history. `main` follows the latest landed commit. Verified main follows the latest trunk state that Stovepipe has validated successfully. How far apart can the two references move? In a fresh example, both start at known-good commit A. Then B, C and D land, moving `main` to D. Stovepipe checks the state produced by B through D on top of A. The build fails, so verified main stays at A. When E, F and G land, the next check covers B through G on top of A, including the changes from the failed check. This time the build passes. Verified main can advance directly from A to G even though the earlier state at D failed. Keeping a usable pointer takes one action on failure: leave it at the last passing commit. Repairing trunk takes human work. Stovepipe pages the on-call engineer, who prepares a fix or a revert and sends it through SubmitQueue. Once the repair lands, Stovepipe validates the resulting state. A passing check lets verified main move forward again. Where do the two references point after each build? Follow the top row for what has landed and the bottom row for what consumers can use as verified. At D the references diverge. The passing check at G brings them back together. - main at A — Known-good starting commit. - main at D — B, C and D have landed on A. - main at G — E, F and G have since landed. - Verified main at A — Both references start together. - Still at A — The integrated state at D fails. - Verified main at G — The integrated state at G passes. - main at A → main at D (Land B through D) - main at D → main at G (Land E through G) - main at D → Still at A (Check B–D on A · fail) - Verified main at A → Still at A (Hold pointer) - main at G → Verified main at G (Check B–G on A · pass) - Still at A → Verified main at G (Advance pointer) ### Back to a 30-minute P95, with recovery work still to do [23:42](https://www.youtube.com/watch?v=hX140a5klW0&t=1422s) The team’s final peak figure was roughly 300 lands an hour while maintaining a P95 land time of 30 minutes. That is about 233% more lands per hour than January’s baseline of 90, at the same reported P95. SubmitQueue and Stovepipe were both being developed in the open. The next engineering decisions concerned build cost, the scope of a green result, and the time spent recovering from a red one. - **Batch related changes.** SubmitQueue could group related requests so that fewer builds cover the same work. The aim is to land more changes with the available build resources. - **Track green states by service or project.** A single verified-main pointer describes the whole repository. A service-level result could tell a team that its service still passes at a commit where another part of the repository fails. Rollouts of healthy services could then proceed while the affected project recovers. - **Automate fixes and reverts.** The on-call engineer still owns the repair after a failed post-merge check. The team wanted to automate preparing a fix or revert and landing it through SubmitQueue to shorten the time trunk spends broken. Uber’s hybrid approach gives a merge and a verified commit different meanings. Before merge, SubmitQueue still checks individual changes and the combinations linked by stronger conflicts. After merge, Stovepipe runs exhaustive validation on the state those changes actually produced. A developer can see a change land while consumers who need a passing build continue to use the last verified commit. ## Key takeaways - An edit’s merge cost depends on how much of the build graph it affects. Deferring broad changes protected peak-hour capacity, and persisting the request relieved its author of the retry while leaving the wait for capacity in place. - Weak overlaps can tie many requests into one serial group. Relaxing two edges in the six-request example yielded three independent queues. Choosing edges at entry let existing speculative builds remain useful. - A verified-main pointer gives consumers a passing commit during a trunk failure. The pointer can advance as soon as a later integrated state passes, while repairing the failure remains a separate operational job. ## Slides ### Merging at AI Scale Green branches merge into a shared trunk. Just past the last merge, a circled X marks where the trunk turns red. Preetam Dwivedi and Manjari Akella trace how Uber’s once-green trunk began to fail. [0:00](https://www.youtube.com/watch?v=hX140a5klW0&t=0s) [The subtitle reads “From Green to Broken Trunk.” Green branches converge on a trunk that turns red at a circled X.](https://assets.ai.engineer/sites/aiecode2025/sha256/41dc8944660e412fe306171e1813d35d25a9ae075d3fb1e37b923ae5c4740496.webp) Merging at AI Scale From Green to Broken Trunk Preetam Dwivedi Manjari Akella 01 / 30 Interface text: Cascades | SEA ### About us Preetam Dwivedi (left), a staff software engineer, and Manjari Akella (right), a senior software engineer, work together on Uber’s Developer Platform. [0:19](https://www.youtube.com/watch?v=hX140a5klW0&t=19s) [Side-by-side portraits introduce Preetam Dwivedi, Staff Software Engineer, and Manjari Akella, Senior Software Engineer. Both work on Uber’s Developer Platform.](https://assets.ai.engineer/sites/aiecode2025/sha256/ec785c0380068eed9af71103709dacc10a3aa0dd4c4c473a92afd0b032802dc8.webp) SPEAKERS About us Preetam Dwivedi Staff Software Engineer Developer Platform, Uber Manjari Akella Senior Software Engineer Developer Platform, Uber 02 / 30 Interface text: Cascades | SEA ### Outline The speakers start with how Uber merges code, then explain how the team adapted as AI sped up development. They close with the results and what they plan to do next. [0:32](https://www.youtube.com/watch?v=hX140a5klW0&t=32s) [The agenda covers merging at Uber, how AI changed the pace, how the team adapted, and results and next steps.](https://assets.ai.engineer/sites/aiecode2025/sha256/9f5def0e7840d06966b97ad0c771c897e5a7bf71b9a314a38ac2339d8921b6b6.webp) Outline • Merging at Uber • AI changed the pace • How we adapted • Results and what’s next 03 / 30 Interface text: Cascades | SEA ### Two green PRs. One untested combination. Pull requests A and B branch from the same commit, H. Each passes CI on its own. If A lands first, B’s passing result still describes H + B. CI has yet to test H + A + B. [0:52](https://www.youtube.com/watch?v=hX140a5klW0&t=52s) [Pull requests A and B branch from base H. H+A and H+B each pass CI. Neither pull request has landed, and their combination remains untested.](https://assets.ai.engineer/sites/aiecode2025/sha256/fd0914c9c73cede9bc032ec7ead64c6ca23e1927913859da4dfd027aae059fd7.webp) THE INTEGRATION PROBLEM Two green PRs. One untested combination. Both PRs pass CI against the same base. Neither has landed. main H Base H A PR A User A B PR B User B ISOLATED CI Both PRs pass H + A ✓ H + B ✓ 05 / 30 Interface text: Cascades | SEA ### SubmitQueue speculates over futures Before either pull request lands, SubmitQueue builds and tests three possibilities from the same base H: A alone, B alone, and B on top of A. In this example the combined build passes, so the queue merges A then B into main. [1:55](https://www.youtube.com/watch?v=hX140a5klW0&t=115s) [Ready pull requests A and B enter SubmitQueue, which builds and tests H+A, H+B and H+A+B. In the passing example, A then B land on main after H.](https://assets.ai.engineer/sites/aiecode2025/sha256/3b7fde7c461cd9b10bc17fe1fdc60b4a33be4c62be9f093dabd31ed38d3d1427.webp) UBER’S MERGE QUEUE SubmitQueue speculates over futures Coordinate the order. Speculate futures. Land on green. READY PRs A PR A B PR B SUBMITQUEUE Speculate futures H — A H — B H — A — B Build + test possible states MAIN H — A — B Land on green Example: both pass 07 / 30 Interface text: Cascades | SEA [3:01](https://www.youtube.com/watch?v=hX140a5klW0&t=181s) [Starting from head H, SubmitQueue tests pull request A alone, B alone, and A followed by B. In the illustrated passing case, main advances from H to A to B.](https://assets.ai.engineer/sites/aiecode2025/sha256/de700f82c6fce9b2a5717b596bcc29a2eb9e87a1f597d04a5df82e291e11af0f.webp) UBER’S MERGE QUEUE SubmitQueue speculates over futures Coordinate the order. Speculate futures. Land on green. READY PRs A PR A B PR B SUBMITQUEUE Speculate futures H — A H — B H — A — B Build + test possible states MAIN H — A — B Land on green Example: both pass 07 / 30 Interface text: Cascades | SEA ### How SubmitQueue scales SubmitQueue uses build targets to find changes it can test independently. It runs the builds most likely to be needed first. A ready change can merge ahead of a large one when all required speculative outcomes agree. [3:49](https://www.youtube.com/watch?v=hX140a5klW0&t=229s) [Target analysis splits independent work into virtual queues. Probabilistic speculation tests the most likely states first. Ready work can bypass a large change when all required speculative outcomes agree.](https://assets.ai.engineer/sites/aiecode2025/sha256/adffbe546c641060c3e99e4d260f0038ca5ced06d777e15c3026e695feb1a1fa.webp) SUBMITQUEUE OPTIMIZATIONS How SubmitQueue scales 01 Target analysis Separate independent work Build-target relationships split work into virtual queues. 02 Probabilistic speculation Build likely futures first Prioritize speculative states most likely to be needed. 03 Bypass large changes Let ready work move ahead Finalize later work when all required speculative outcomes agree. 08 / 30 Interface text: Cascades | SEA ### Then came AI Uber’s merge queue had kept pace when developers wrote the code themselves. With AI, pull requests started arriving faster than Uber could merge them. [5:32](https://www.youtube.com/watch?v=hX140a5klW0&t=332s) [Large green letters emphasize AI on a dark background. The subtitle reads, “Changes arrived faster than we could land them.”](https://assets.ai.engineer/sites/aiecode2025/sha256/e2b13923d19bec60738e766bc436e781030309353d4c3453aba93af8dd558eab.webp) Then came AI Changes arrived faster than we could land them. 09 / 30 Interface text: Cascades | SEA ### AI changed the pace of integration At peak, Uber merged about 90 code changes per hour in January 2026 and about 130 per hour in March. The speaker attributes the roughly 44% increase over those two months primarily to AI-generated changes. [5:44](https://www.youtube.com/watch?v=hX140a5klW0&t=344s) [Peak lands per hour rose from about 90 in January 2026 to about 130 in March 2026, an increase of about 44% in two months.](https://assets.ai.engineer/sites/aiecode2025/sha256/0b7362a8ac36a899d304d2c2457a933ba994ca52b8509bc50be728cd67aa5759.webp) JANUARY TO MARCH 2026 AI changed the pace of integration January 2026 ~90 Peak lands / hour → March 2026 ~130 Peak lands / hour ~44% increase in two months March was the tipping point. 10 / 30 Interface text: Cascades | SEA ### Green trunk. Hours to land Uber kept its main branch green by making developers wait. As traffic rose from about 90 to 130 requests per hour, the 95th-percentile time to land a change rose from about 30 minutes to 3 to 4 hours at peak. [6:17](https://www.youtube.com/watch?v=hX140a5klW0&t=377s) [The 95th-percentile time to land a change rose from about 30 minutes to 3–4 hours at peak. Developers waited longer to keep main green.](https://assets.ai.engineer/sites/aiecode2025/sha256/1bafa93282b269c117b8968365f4b7a8497657216228da258d04d76bf4efc1db.webp) THE DEVELOPER IMPACT Green trunk. Hours to land Land-time p95 ~30 minutes → 3 to 4 hours at peak We were protecting main by making developers wait. 11 / 30 Interface text: Cascades | SEA ### What we changed Uber first controlled which changes entered the merge queue and used a second queue to hold deferred changes. It then relaxed conflict rules so more changes could proceed in parallel. [6:47](https://www.youtube.com/watch?v=hX140a5klW0&t=407s) [Admission gating shapes traffic, deferrals automate the wait, and conflict relaxation allows more work to proceed in parallel. The slide places demand management before changes to the merge model.](https://assets.ai.engineer/sites/aiecode2025/sha256/7376ef5a14d9ccee18397158a8047a6930a7c83ae7e3dde67cd110a3e5c50c1e.webp) THE RESPONSE What we changed 01 Shape the traffic Admission gating 02 Automate the wait Deferrals 03 Unlock parallelism Conflict relaxation Manage demand first. Then change the merge model. 12 / 30 Interface text: Cascades | SEA ### Admission gating During peak hours, smaller changes enter the submit queue for builds and tests. The gate rejects expensive changes that could tie up resources, so developers must resubmit them outside peak hours. [7:55](https://www.youtube.com/watch?v=hX140a5klW0&t=475s) [A smaller change passes the peak-hour gate and enters the active queue for build and test. The gate rejects an expensive change, and the developer retries outside peak hours.](https://assets.ai.engineer/sites/aiecode2025/sha256/e74cbb3de9752d5137313288175beee6c5c90c2da761d87ff0331f566a1c9560.webp) INTERVENTION 1 Admission gating Submitted changes Peak-hour gate Smaller change Admit Active SQ A B S Build + test Expensive change Rejected during peak hours Developer retries outside peak hours First, control what enters the active queue during peak hours 13 / 30 Interface text: Cascades | SEA ### Admission gating: the tradeoff After Uber introduced peak-hour gating, its business-hour merge rate rose 23%, from about 130 to about 160 changes per hour. Developers whose expensive changes hit the gate had to resubmit them outside peak hours. [9:28](https://www.youtube.com/watch?v=hX140a5klW0&t=568s) [After peak-hour gating, business-hour throughput rose from about 130 to 160 lands per hour, an observed 23% increase. Developers had to resubmit expensive changes outside peak hours.](https://assets.ai.engineer/sites/aiecode2025/sha256/89cc3ed0f6cbcaf1cfa4924f31aa71d869ef05a9c56414fcee7cbdd788100cf8.webp) INTERVENTION 1 Admission gating: the tradeoff After peak-hour gating ~130 → ~160 Lands / hour Business hours +23% observed increase Developer cost Developers become the schedulers “Submit early. Or try again tonight.” Resubmit expensive changes outside peak hours. We protected capacity during peak hours, but made developers manage the wait 14 / 30 Interface text: Cascades | SEA ### Deferrals A developer submits an expensive change once during peak hours. The platform holds it in a deferred queue before validation, then automatically moves it into the active submit queue when capacity allows. [10:32](https://www.youtube.com/watch?v=hX140a5klW0&t=632s) [During peak hours, request R waits in a deferred queue before validation. The platform re-evaluates R automatically and releases it into the active queue when capacity allows.](https://assets.ai.engineer/sites/aiecode2025/sha256/1acfebc07ae53f9b80585d83d05c137df40011b7758ba910b2bc4f3159027d72.webp) INTERVENTION 2 Deferrals DURING PEAK HOURS Request R Deferred queue R Persisted Accepted, waiting No active validation yet Re-evaluate automatically When gates permit release No resubmission needed R Release Active SQ A B R Submit once. Release automatically when capacity allows. 15 / 30 Interface text: Cascades | SEA ### Deferrals: the tradeoff Developers submit once. The platform holds eligible changes until the queue has room, then releases them automatically. Deferred changes still incur validation costs. Dependencies can still force changes to run in sequence. [11:43](https://www.youtube.com/watch?v=hX140a5klW0&t=703s) [Developers submit once, and the platform releases eligible work automatically. Deferred work still waits and incurs validation costs. Dependencies can still force changes to run in sequence.](https://assets.ai.engineer/sites/aiecode2025/sha256/74eb3d336a4213645d4f3d71424ce299075ff8545a81c46159e4fb92865ed0df.webp) INTERVENTION 2 Deferrals: the tradeoff Good Submit once. Release automatically. The platform holds and releases eligible work without developer resubmission. Bad The waiting still exists Demand moves to another time. Validation cost and dependency coupling remain. We moved the wait. We had not removed serialization. 16 / 30 Interface text: Cascades | SEA ### Any target overlap could serialize work Each neighboring pair of changes from A through F shares at least one target. The overlap between C and D joins the two halves, so the strict queue rule could force all six changes to run in sequence. [12:15](https://www.youtube.com/watch?v=hX140a5klW0&t=735s) [Six changes labeled A through F form one connected chain. Under the strict overlap rule, a single overlap can join two groups so that all connected changes share an ordering constraint.](https://assets.ai.engineer/sites/aiecode2025/sha256/6e509aae1af31b2ccf86958af9f6896001d0b4ca65f1126576ed0c1abeb399ed.webp) THE REMAINING BOTTLENECK Any target overlap could serialize work STRICT · 1 CONNECTED COMPONENT A B C D E F A single overlap can join otherwise separate groups. Connected changes share an ordering constraint 17 / 30 Interface text: Cascades | SEA ### Jaccard scoring for dependency overlap Direct target overlap gets full weight. Each change’s new dependencies get half weight when compared with the other change’s targets. The largest of the three scores ranks the link and helps determine which links to keep. [13:37](https://www.youtube.com/watch?v=hX140a5klW0&t=817s) [Jaccard similarity divides shared targets by distinct targets. The ranking score takes the largest of three values: direct target similarity and two half-weighted similarities comparing each side’s new dependency edges with the other side’s targets.](https://assets.ai.engineer/sites/aiecode2025/sha256/17cddaaf1dcbdf9ed2da61f80de815ea534131a8c52cf43d5e8590fa5e96a554.webp) CONFLICT RELAXATION Jaccard scoring for dependency overlap Jaccard similarity = shared targets ÷ distinct targets score(A, B) = max( 1 × Jaccard(A, B), 0.5 × Jaccard(E_A, B), 0.5 × Jaccard(E_B, A) ) A, B Overlapping targets E_A, E_B New Dependency Edge A ranking signal. Not a failure probability. 19 / 30 Interface text: Cascades | SEA ### Buy parallelism by dropping the weakest bridges For six changes, a 50% parallelism target means three queues. Dropping the two links with the lowest overlap scores, B–C (0.08) and D–E (0.04), lets the remaining pairs A–B, C–D and E–F run in parallel. [15:04](https://www.youtube.com/watch?v=hX140a5klW0&t=904s) [Dropping B–C (0.08) and D–E (0.04) leaves three queues of two changes each: A–B (0.90), C–D (0.80) and E–F (0.70).](https://assets.ai.engineer/sites/aiecode2025/sha256/2259d9a0e084179a0acb3d4c1d7c9ebada7ad855505458facaf537a78e55e0b5.webp) CONFLICT RELAXATION Buy parallelism by dropping the weakest bridges 6 changes · 50% desired parallelism target = 3 queues RELAXED · 3 VIRTUAL QUEUES A–B: 0.90 B–C: 0.08 (drop) C–D: 0.80 D–E: 0.04 (drop) E–F: 0.70 Keep the strongest links. Stop joining independent queues at the target. 20 / 30 Interface text: Cascades | SEA ### Assign new arrivals. Leave in-flight work alone. When a change arrives, Uber checks it for conflicts with queued work and assigns it to a queue once. Changes already under validation stay in place, preserving the speculative builds already underway. [16:17](https://www.youtube.com/watch?v=hX140a5klW0&t=977s) [Two columns show when placement happens. The conflict policy assigns new work once. Work already undergoing validation keeps its queue placement.](https://assets.ai.engineer/sites/aiecode2025/sha256/31eb2c4e1f210254c370960b7f07fe9fc398dbeb0e48cdd52ee4932ef65d8a14.webp) CONFLICT RELAXATION Assign new arrivals. Leave in-flight work alone. NEW ARRIVALS Assign once Place new work using the conflict policy. IN-FLIGHT WORK Preserve placement Do not reshuffle work already being validated. Preserve useful speculative builds 21 / 30 Interface text: Cascades | SEA ### Conflict relaxation: the tradeoff During business hours in July 2026, Uber saw throughput rise from about 160 to about 250 lands per hour, a roughly 56% gain. Changes that pass separately can still fail together, so Uber needs to test the combination that lands. [17:59](https://www.youtube.com/watch?v=hX140a5klW0&t=1079s) [During business hours in July 2026, observed throughput rose from about 160 to 250 lands per hour, roughly 56%. Changes that pass individually can interact after landing and cause trunk validation to fail.](https://assets.ai.engineer/sites/aiecode2025/sha256/c39843b08c20d328705dceb8cc534006dcc642503799cf54a170d82e8fb83d98.webp) INTERVENTION 3 Conflict relaxation: the tradeoff Early throughput ~160 → ~250 Lands / hour Business hours, July 2026 ~56% observed increase The tradeoff Some combinations land untested Individually green changes can interact after landing. Trunk can turn red. Higher throughput creates a new obligation: validate what actually lands 22 / 30 Interface text: Cascades | SEA ### The cost: trunk can turn red Changes A and B each pass when tested separately against the same starting commit, H. When A lands and B follows, trunk contains both changes. That combination can fail even though both pre-merge checks passed. [19:14](https://www.youtube.com/watch?v=hX140a5klW0&t=1154s) [H + A and H + B each pass independent pre-merge validation. The post-merge diagram shows one possible outcome in which H + A + B fails.](https://assets.ai.engineer/sites/aiecode2025/sha256/581ac96124e867e8f89a6879e93ee9b389c4a4bba0b2775ddb835147fb84ed0e.webp) THE CONSEQUENCE The cost: trunk can turn red PRE-MERGE · INDEPENDENT VALIDATION H A H + A passes B H + B passes POST-MERGE · ONE POSSIBLE OUTCOME H A B H + A + B fails Which landed state can we actually trust? 23 / 30 Interface text: Cascades | SEA ### Stovepipe: one history, two moving refs Both pointers start at commit A. As new commits land, main advances through D, where validation fails, to G. verified/main stays at A until validation passes at G, then catches up to main. [20:02](https://www.youtube.com/watch?v=hX140a5klW0&t=1202s) [On a history labeled A through G, both main HEAD and verified/main HEAD point to the validated commit A. Commits B through G are faded.](https://assets.ai.engineer/sites/aiecode2025/sha256/cd2ae7b7a3e2787ef267ef8dfab3fe3a62dd4b115fa207e2dd8609f3ec3e4dcd.webp) POST-MERGE CONFIDENCE Stovepipe: one history, two moving refs main advances on landing. verified/main advances on passing validation. A is known green Both refs point to the known-good baseline. main HEAD A B C D E F G verified/main HEAD 24 / 30 Interface text: Cascades | SEA [22:25](https://www.youtube.com/watch?v=hX140a5klW0&t=1345s) [A single line connects commits A through G. D remains red. A green bracket marks validation from A to G as passed, and the main and verified/main arrows both point to G.](https://assets.ai.engineer/sites/aiecode2025/sha256/025566c26986ef051507618c35036135a2115314be000c8a3822269efce0ed1c.webp) POST-MERGE CONFIDENCE Stovepipe: one history, two moving refs main advances on landing. verified/main advances on passing validation. G passes validation verified/main advances to the passing state at G. main HEAD A B C D E F G verified/main HEAD Validate A–G: passed 24 / 30 Interface text: Cascades | SEA ### Recovering a broken trunk When commit D fails validation, Stovepipe pages the on-call engineer. The engineer gets a fix or revert approved and lands it through SubmitQueue. Stovepipe advances verified/main only after the repaired trunk passes validation. [22:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1375s) [Stovepipe alerts the on-call engineer when D fails validation. The engineer prepares a fix or revert, gets approval and lands it through SubmitQueue. Stovepipe validates the repair and advances verified/main on success.](https://assets.ai.engineer/sites/aiecode2025/sha256/a04bc76973211fd3906339437e7582ec401b404125aa8dec35b261d77c7e1da6.webp) POST-MERGE CONFIDENCE Recovering a broken trunk STOVEPIPE Detect and alert D fails validation. On-call is alerted. ON-CALL Prepare a repair Create a fix or revert. ON-CALL Review and land Get approvals and land the repair through SubmitQueue. STOVEPIPE Verify recovery Validate the repaired state. A passing result advances verified/main. 25 / 30 Interface text: Cascades | SEA ### More throughput. Shorter waits. By October 2026, Uber’s peak merge rate had risen from about 90 changes per hour in January to about 300, a roughly 233% increase. About 95% of changes landed within roughly 30 minutes. [23:40](https://www.youtube.com/watch?v=hX140a5klW0&t=1420s) [As of October 2026, peak throughput is about 300 lands per hour, up about 233% from roughly 90 per hour in January. The 95th-percentile land time is about 30 minutes.](https://assets.ai.engineer/sites/aiecode2025/sha256/2efdcb74e9a1507f0f10e5fc72f4de5c2fd87239ddb86acdbd09805779842881.webp) RESULTS · AS OF OCTOBER 2026 More throughput. Shorter waits. ~300 Peak lands / hour ~233% increase from January’s ~90/hour ~30 min Land-time p95 26 / 30 Interface text: Cascades | SEA ### SubmitQueue and Stovepipe in the open SubmitQueue tests what the code would look like after a proposed merge and lands changes that pass. Stovepipe then checks the merged code and tracks which revisions passed. Uber develops both tools in the open. [24:12](https://www.youtube.com/watch?v=hX140a5klW0&t=1452s) [Before merge, SubmitQueue validates proposed changes and lands those that pass. After merge, Stovepipe validates landed states and tracks known-good states. A QR code and the project repository address appear on the right.](https://assets.ai.engineer/sites/aiecode2025/sha256/0afab361afd484d32104d7d93ab6349ca4180fce4b5b0b909b3481e3fae7f996.webp) OPEN SOURCE SubmitQueue and Stovepipe in the open PRE-MERGE SubmitQueue Validate speculative futures. Land passing changes. POST-MERGE Stovepipe Validate landed states. Track known-good states. Explore the project github.com/uber/submitqueue 27 / 30 Interface text: Cascades | SEA ### What comes next Uber wants SubmitQueue to batch related changes so it runs fewer builds. Project-level checks would show which projects still pass when trunk breaks. Automated fixes and reverts would restore trunk health faster. [24:35](https://www.youtube.com/watch?v=hX140a5klW0&t=1475s) [Three columns outline plans to batch changes in SubmitQueue, report validation results for individual projects, and automate fixes and reverts to restore trunk health faster.](https://assets.ai.engineer/sites/aiecode2025/sha256/4de422d99aa7c1c9c310538fdd8e0e289c4f7f30a7ad08ad269270f7c1730092.webp) OPEN SOURCE & NEXT What comes next BETTER THROUGHPUT Batching for SubmitQueue Validate groups of changes together to reduce repeated work. BETTER CONFIDENCE Validate project scope Answer “is my project green?” without treating every failure as global. FASTER RECOVERY Automate fixes and reverts Invest in automated diagnosis and corrective actions to restore trunk health faster. 28 / 30 Interface text: Cascades | SEA ### A hybrid approach to validation Uber runs required checks and enforces stronger dependency constraints before merge. It continuously tests the integrated trunk after merge, so more changes can land while the wait for each change stays reasonable. [26:13](https://www.youtube.com/watch?v=hX140a5klW0&t=1573s) [Required checks and stronger dependency constraints apply before merge. Continuous checks validate the integrated trunk after merge. Highlighted text says this supports higher landing throughput while keeping land times reasonable.](https://assets.ai.engineer/sites/aiecode2025/sha256/b272214f3e8ffca6fc3b9b313616170feea8e19b7b9493c9fd909affd0a619c4.webp) CLOSING A hybrid approach to validation We retain required checks and stronger dependency constraints before merge, while continuously validating integrated trunk states after merge. This gives us the flexibility to support higher landing throughput while keeping land times reasonable. 29 / 30 Interface text: Cascades | SEA ### Thank you Explore Uber’s SubmitQueue repository, or scan a QR code to watch Merging at Scale for a closer look at how the queue works. The left code links to the WeAreDevelopers talk and the right to Seattle Systems. [27:07](https://www.youtube.com/watch?v=hX140a5klW0&t=1627s) [The SubmitQueue repository address appears above two QR codes for the talk “Merging at Scale,” subtitled “From Broken Builds to Green Mainline.” The left code is labeled WeAreDevelopers and the right code Seattle Systems.](https://assets.ai.engineer/sites/aiecode2025/sha256/87827eb2cb0d13a6b529c37ac7c9a73373a7156a7c17cbcdf3b5d845e005f6e4.webp) Thank you github.com/uber/submitqueue WeAreDevelopers Merging at Scale From Broken Builds to Green Mainline Watch talk ↗ Seattle Systems Merging at Scale From Broken Builds to Green Mainline Watch talk ↗ 30 / 30 Interface text: Cascades | SEA ## Full transcript [0:04](https://www.youtube.com/watch?v=hX140a5klW0&t=4s) Uh hello everyone. Uh we are here to [0:06](https://www.youtube.com/watch?v=hX140a5klW0&t=6s) Uh hello everyone. Uh we are here to [0:06](https://www.youtube.com/watch?v=hX140a5klW0&t=6s) Uh hello everyone. Uh we are here to present our talk uh merging at um AI [0:10](https://www.youtube.com/watch?v=hX140a5klW0&t=10s) present our talk uh merging at um AI [0:10](https://www.youtube.com/watch?v=hX140a5klW0&t=10s) present our talk uh merging at um AI scale. Uh this is essentially uh Uber's [0:13](https://www.youtube.com/watch?v=hX140a5klW0&t=13s) scale. Uh this is essentially uh Uber's [0:13](https://www.youtube.com/watch?v=hX140a5klW0&t=13s) scale. Uh this is essentially uh Uber's journey from going to a fully green [0:17](https://www.youtube.com/watch?v=hX140a5klW0&t=17s) journey from going to a fully green [0:17](https://www.youtube.com/watch?v=hX140a5klW0&t=17s) journey from going to a fully green trunk to somewhat broken trunk. [0:20](https://www.youtube.com/watch?v=hX140a5klW0&t=20s) trunk to somewhat broken trunk. [0:20](https://www.youtube.com/watch?v=hX140a5klW0&t=20s) trunk to somewhat broken trunk. Um I'm Priam Devi. I'm a staff engineer [0:23](https://www.youtube.com/watch?v=hX140a5klW0&t=23s) Um I'm Priam Devi. I'm a staff engineer [0:23](https://www.youtube.com/watch?v=hX140a5klW0&t=23s) Um I'm Priam Devi. I'm a staff engineer at Uber working in developer platform. [0:26](https://www.youtube.com/watch?v=hX140a5klW0&t=26s) at Uber working in developer platform. [0:26](https://www.youtube.com/watch?v=hX140a5klW0&t=26s) at Uber working in developer platform. Hi, I'm Maji. I'm a senior engineer [0:27](https://www.youtube.com/watch?v=hX140a5klW0&t=27s) Hi, I'm Maji. I'm a senior engineer [0:28](https://www.youtube.com/watch?v=hX140a5klW0&t=28s) Hi, I'm Maji. I'm a senior engineer working alongside Priam. uh in the [0:30](https://www.youtube.com/watch?v=hX140a5klW0&t=30s) working alongside Priam. uh in the [0:30](https://www.youtube.com/watch?v=hX140a5klW0&t=30s) working alongside Priam. uh in the developer at at Uber. [0:32](https://www.youtube.com/watch?v=hX140a5klW0&t=32s) developer at at Uber. [0:32](https://www.youtube.com/watch?v=hX140a5klW0&t=32s) developer at at Uber. Great. Um so here's the outline for the [0:35](https://www.youtube.com/watch?v=hX140a5klW0&t=35s) Great. Um so here's the outline for the [0:36](https://www.youtube.com/watch?v=hX140a5klW0&t=36s) Great. Um so here's the outline for the talk. Uh we're going to go over like how [0:38](https://www.youtube.com/watch?v=hX140a5klW0&t=38s) talk. Uh we're going to go over like how [0:38](https://www.youtube.com/watch?v=hX140a5klW0&t=38s) talk. Uh we're going to go over like how the merging um works at Uber at the [0:40](https://www.youtube.com/watch?v=hX140a5klW0&t=40s) the merging um works at Uber at the [0:40](https://www.youtube.com/watch?v=hX140a5klW0&t=40s) the merging um works at Uber at the moment and then how AI changed the [0:44](https://www.youtube.com/watch?v=hX140a5klW0&t=44s) moment and then how AI changed the [0:44](https://www.youtube.com/watch?v=hX140a5klW0&t=44s) moment and then how AI changed the landscape and then I'll respond um to [0:47](https://www.youtube.com/watch?v=hX140a5klW0&t=47s) landscape and then I'll respond um to [0:47](https://www.youtube.com/watch?v=hX140a5klW0&t=47s) landscape and then I'll respond um to that how we adapted and then discuss the [0:49](https://www.youtube.com/watch?v=hX140a5klW0&t=49s) that how we adapted and then discuss the [0:49](https://www.youtube.com/watch?v=hX140a5klW0&t=49s) that how we adapted and then discuss the results and um talk about what are we [0:52](https://www.youtube.com/watch?v=hX140a5klW0&t=52s) results and um talk about what are we [0:52](https://www.youtube.com/watch?v=hX140a5klW0&t=52s) results and um talk about what are we planning next. [0:54](https://www.youtube.com/watch?v=hX140a5klW0&t=54s) planning next. [0:54](https://www.youtube.com/watch?v=hX140a5klW0&t=54s) planning next. So at Uber uh we have a six language [0:59](https://www.youtube.com/watch?v=hX140a5klW0&t=59s) So at Uber uh we have a six language [0:59](https://www.youtube.com/watch?v=hX140a5klW0&t=59s) So at Uber uh we have a six language based monor repos where more than 5,000 [1:02](https://www.youtube.com/watch?v=hX140a5klW0&t=62s) based monor repos where more than 5,000 [1:02](https://www.youtube.com/watch?v=hX140a5klW0&t=62s) based monor repos where more than 5,000 uh developers uh are contributing over [1:05](https://www.youtube.com/watch?v=hX140a5klW0&t=65s) uh developers uh are contributing over [1:05](https://www.youtube.com/watch?v=hX140a5klW0&t=65s) uh developers uh are contributing over 36,000 commits uh 27,000 [1:09](https://www.youtube.com/watch?v=hX140a5klW0&t=69s) 36,000 commits uh 27,000 [1:09](https://www.youtube.com/watch?v=hX140a5klW0&t=69s) 36,000 commits uh 27,000 PRs uh every week and uh pretty much [1:13](https://www.youtube.com/watch?v=hX140a5klW0&t=73s) PRs uh every week and uh pretty much [1:13](https://www.youtube.com/watch?v=hX140a5klW0&t=73s) PRs uh every week and uh pretty much every changes uh to the services [1:15](https://www.youtube.com/watch?v=hX140a5klW0&t=75s) every changes uh to the services [1:15](https://www.youtube.com/watch?v=hX140a5klW0&t=75s) every changes uh to the services libraries are being done in one of these [1:18](https://www.youtube.com/watch?v=hX140a5klW0&t=78s) libraries are being done in one of these [1:18](https://www.youtube.com/watch?v=hX140a5klW0&t=78s) libraries are being done in one of these monor repos. uh for the purpose of this [1:21](https://www.youtube.com/watch?v=hX140a5klW0&t=81s) monor repos. uh for the purpose of this [1:21](https://www.youtube.com/watch?v=hX140a5klW0&t=81s) monor repos. uh for the purpose of this talk we're going to focus mostly on go [1:23](https://www.youtube.com/watch?v=hX140a5klW0&t=83s) talk we're going to focus mostly on go [1:23](https://www.youtube.com/watch?v=hX140a5klW0&t=83s) talk we're going to focus mostly on go code when we talk about anything because [1:25](https://www.youtube.com/watch?v=hX140a5klW0&t=85s) code when we talk about anything because [1:25](https://www.youtube.com/watch?v=hX140a5klW0&t=85s) code when we talk about anything because that is our the biggest one and that's [1:27](https://www.youtube.com/watch?v=hX140a5klW0&t=87s) that is our the biggest one and that's [1:27](https://www.youtube.com/watch?v=hX140a5klW0&t=87s) that is our the biggest one and that's where the most of the scaling challenges [1:29](https://www.youtube.com/watch?v=hX140a5klW0&t=89s) where the most of the scaling challenges [1:29](https://www.youtube.com/watch?v=hX140a5klW0&t=89s) where the most of the scaling challenges uh we face. [1:30](https://www.youtube.com/watch?v=hX140a5klW0&t=90s) uh we face. [1:30](https://www.youtube.com/watch?v=hX140a5klW0&t=90s) uh we face. So let's understand the problem a little [1:32](https://www.youtube.com/watch?v=hX140a5klW0&t=92s) So let's understand the problem a little [1:32](https://www.youtube.com/watch?v=hX140a5klW0&t=92s) So let's understand the problem a little bit and um and understand like why we [1:37](https://www.youtube.com/watch?v=hX140a5klW0&t=97s) bit and um and understand like why we [1:37](https://www.youtube.com/watch?v=hX140a5klW0&t=97s) bit and um and understand like why we need a merge cube here. So uh given here [1:41](https://www.youtube.com/watch?v=hX140a5klW0&t=101s) need a merge cube here. So uh given here [1:41](https://www.youtube.com/watch?v=hX140a5klW0&t=101s) need a merge cube here. So uh given here basically we have two users uh user A [1:43](https://www.youtube.com/watch?v=hX140a5klW0&t=103s) basically we have two users uh user A [1:43](https://www.youtube.com/watch?v=hX140a5klW0&t=103s) basically we have two users uh user A and B creating PR A and B um on a base [1:47](https://www.youtube.com/watch?v=hX140a5klW0&t=107s) and B creating PR A and B um on a base [1:47](https://www.youtube.com/watch?v=hX140a5klW0&t=107s) and B creating PR A and B um on a base edge uh from the trunk and when the PRs [1:50](https://www.youtube.com/watch?v=hX140a5klW0&t=110s) edge uh from the trunk and when the PRs [1:50](https://www.youtube.com/watch?v=hX140a5klW0&t=110s) edge uh from the trunk and when the PRs are created and CI is run uh the CI is [1:53](https://www.youtube.com/watch?v=hX140a5klW0&t=113s) are created and CI is run uh the CI is [1:53](https://www.youtube.com/watch?v=hX140a5klW0&t=113s) are created and CI is run uh the CI is green builds are passing everything is [1:54](https://www.youtube.com/watch?v=hX140a5klW0&t=114s) green builds are passing everything is [1:54](https://www.youtube.com/watch?v=hX140a5klW0&t=114s) green builds are passing everything is good. [1:56](https://www.youtube.com/watch?v=hX140a5klW0&t=116s) good. [1:56](https://www.youtube.com/watch?v=hX140a5klW0&t=116s) good. Now uh when a is merged into the trunk, [2:00](https://www.youtube.com/watch?v=hX140a5klW0&t=120s) Now uh when a is merged into the trunk, [2:00](https://www.youtube.com/watch?v=hX140a5klW0&t=120s) Now uh when a is merged into the trunk, we can see the bills for the merge [2:02](https://www.youtube.com/watch?v=hX140a5klW0&t=122s) we can see the bills for the merge [2:02](https://www.youtube.com/watch?v=hX140a5klW0&t=122s) we can see the bills for the merge commit are still green. Um bills are [2:05](https://www.youtube.com/watch?v=hX140a5klW0&t=125s) commit are still green. Um bills are [2:05](https://www.youtube.com/watch?v=hX140a5klW0&t=125s) commit are still green. Um bills are still passing. But when the PRB is [2:08](https://www.youtube.com/watch?v=hX140a5klW0&t=128s) still passing. But when the PRB is [2:08](https://www.youtube.com/watch?v=hX140a5klW0&t=128s) still passing. But when the PRB is merged into the trunk, um bills are no [2:11](https://www.youtube.com/watch?v=hX140a5klW0&t=131s) merged into the trunk, um bills are no [2:11](https://www.youtube.com/watch?v=hX140a5klW0&t=131s) merged into the trunk, um bills are no longer green. And the reason is that B [2:13](https://www.youtube.com/watch?v=hX140a5klW0&t=133s) longer green. And the reason is that B [2:13](https://www.youtube.com/watch?v=hX140a5klW0&t=133s) longer green. And the reason is that B was never tested uh with the changes [2:15](https://www.youtube.com/watch?v=hX140a5klW0&t=135s) was never tested uh with the changes [2:16](https://www.youtube.com/watch?v=hX140a5klW0&t=136s) was never tested uh with the changes that were merged by A. So uh we never [2:19](https://www.youtube.com/watch?v=hX140a5klW0&t=139s) that were merged by A. So uh we never [2:20](https://www.youtube.com/watch?v=hX140a5klW0&t=140s) that were merged by A. So uh we never tested the the combination of those two [2:22](https://www.youtube.com/watch?v=hX140a5klW0&t=142s) tested the the combination of those two [2:22](https://www.youtube.com/watch?v=hX140a5klW0&t=142s) tested the the combination of those two things before merging and hence the the [2:27](https://www.youtube.com/watch?v=hX140a5klW0&t=147s) things before merging and hence the the [2:27](https://www.youtube.com/watch?v=hX140a5klW0&t=147s) things before merging and hence the the when B is merged it's broken. [2:30](https://www.youtube.com/watch?v=hX140a5klW0&t=150s) when B is merged it's broken. [2:30](https://www.youtube.com/watch?v=hX140a5klW0&t=150s) when B is merged it's broken. Um now imagine when we have thousands of [2:33](https://www.youtube.com/watch?v=hX140a5klW0&t=153s) Um now imagine when we have thousands of [2:33](https://www.youtube.com/watch?v=hX140a5klW0&t=153s) Um now imagine when we have thousands of uh engineers working in a monor repo and [2:35](https://www.youtube.com/watch?v=hX140a5klW0&t=155s) uh engineers working in a monor repo and [2:35](https://www.youtube.com/watch?v=hX140a5klW0&t=155s) uh engineers working in a monor repo and everyone is making changes um like [2:38](https://www.youtube.com/watch?v=hX140a5klW0&t=158s) everyone is making changes um like [2:38](https://www.youtube.com/watch?v=hX140a5klW0&t=158s) everyone is making changes um like hundreds and thousands of those every [2:40](https://www.youtube.com/watch?v=hX140a5klW0&t=160s) hundreds and thousands of those every [2:40](https://www.youtube.com/watch?v=hX140a5klW0&t=160s) hundreds and thousands of those every day. Uh this is a constant problem uh [2:44](https://www.youtube.com/watch?v=hX140a5klW0&t=164s) day. Uh this is a constant problem uh [2:44](https://www.youtube.com/watch?v=hX140a5klW0&t=164s) day. Uh this is a constant problem uh where trunk is broken um very frequently [2:47](https://www.youtube.com/watch?v=hX140a5klW0&t=167s) where trunk is broken um very frequently [2:47](https://www.youtube.com/watch?v=hX140a5klW0&t=167s) where trunk is broken um very frequently and there is a burden on fixing and [2:50](https://www.youtube.com/watch?v=hX140a5klW0&t=170s) and there is a burden on fixing and [2:50](https://www.youtube.com/watch?v=hX140a5klW0&t=170s) and there is a burden on fixing and there is a toll across the company to [2:52](https://www.youtube.com/watch?v=hX140a5klW0&t=172s) there is a toll across the company to [2:52](https://www.youtube.com/watch?v=hX140a5klW0&t=172s) there is a toll across the company to sort of um figure it out and uh fix it. [2:56](https://www.youtube.com/watch?v=hX140a5klW0&t=176s) sort of um figure it out and uh fix it. [2:56](https://www.youtube.com/watch?v=hX140a5klW0&t=176s) sort of um figure it out and uh fix it. Um so here uh we're going to present a [2:59](https://www.youtube.com/watch?v=hX140a5klW0&t=179s) Um so here uh we're going to present a [2:59](https://www.youtube.com/watch?v=hX140a5klW0&t=179s) Um so here uh we're going to present a solution uh to that problem uh and that [3:02](https://www.youtube.com/watch?v=hX140a5klW0&t=182s) solution uh to that problem uh and that [3:02](https://www.youtube.com/watch?v=hX140a5klW0&t=182s) solution uh to that problem uh and that solution is summit Q. Um that is our [3:04](https://www.youtube.com/watch?v=hX140a5klW0&t=184s) solution is summit Q. Um that is our [3:04](https://www.youtube.com/watch?v=hX140a5klW0&t=184s) solution is summit Q. Um that is our Uber's merge Q um that we deploy uh to [3:08](https://www.youtube.com/watch?v=hX140a5klW0&t=188s) Uber's merge Q um that we deploy uh to [3:08](https://www.youtube.com/watch?v=hX140a5klW0&t=188s) Uber's merge Q um that we deploy uh to solve this problem. And in this case we [3:10](https://www.youtube.com/watch?v=hX140a5klW0&t=190s) solve this problem. And in this case we [3:10](https://www.youtube.com/watch?v=hX140a5klW0&t=190s) solve this problem. And in this case we can see our two examples here P A and [3:13](https://www.youtube.com/watch?v=hX140a5klW0&t=193s) can see our two examples here P A and [3:13](https://www.youtube.com/watch?v=hX140a5klW0&t=193s) can see our two examples here P A and PRB. And here we essentially what we are [3:16](https://www.youtube.com/watch?v=hX140a5klW0&t=196s) PRB. And here we essentially what we are [3:16](https://www.youtube.com/watch?v=hX140a5klW0&t=196s) PRB. And here we essentially what we are doing is we are predicting the futures [3:18](https://www.youtube.com/watch?v=hX140a5klW0&t=198s) doing is we are predicting the futures [3:18](https://www.youtube.com/watch?v=hX140a5klW0&t=198s) doing is we are predicting the futures that would be u that could be true on [3:20](https://www.youtube.com/watch?v=hX140a5klW0&t=200s) that would be u that could be true on [3:20](https://www.youtube.com/watch?v=hX140a5klW0&t=200s) that would be u that could be true on main. So there are like three things [3:23](https://www.youtube.com/watch?v=hX140a5klW0&t=203s) main. So there are like three things [3:23](https://www.youtube.com/watch?v=hX140a5klW0&t=203s) main. So there are like three things here. A could land on on its own to the [3:26](https://www.youtube.com/watch?v=hX140a5klW0&t=206s) here. A could land on on its own to the [3:26](https://www.youtube.com/watch?v=hX140a5klW0&t=206s) here. A could land on on its own to the main B could land on its own to the main [3:29](https://www.youtube.com/watch?v=hX140a5klW0&t=209s) main B could land on its own to the main [3:29](https://www.youtube.com/watch?v=hX140a5klW0&t=209s) main B could land on its own to the main and then the combination of A and B. In [3:32](https://www.youtube.com/watch?v=hX140a5klW0&t=212s) and then the combination of A and B. In [3:32](https://www.youtube.com/watch?v=hX140a5klW0&t=212s) and then the combination of A and B. In general I think uh what we want to do is [3:34](https://www.youtube.com/watch?v=hX140a5klW0&t=214s) general I think uh what we want to do is [3:34](https://www.youtube.com/watch?v=hX140a5klW0&t=214s) general I think uh what we want to do is that we want to land the change that is [3:35](https://www.youtube.com/watch?v=hX140a5klW0&t=215s) that we want to land the change that is [3:35](https://www.youtube.com/watch?v=hX140a5klW0&t=215s) that we want to land the change that is going to keep the trunk green as a [3:37](https://www.youtube.com/watch?v=hX140a5klW0&t=217s) going to keep the trunk green as a [3:37](https://www.youtube.com/watch?v=hX140a5klW0&t=217s) going to keep the trunk green as a whole. So in this case when we land A [3:39](https://www.youtube.com/watch?v=hX140a5klW0&t=219s) whole. So in this case when we land A [3:39](https://www.youtube.com/watch?v=hX140a5klW0&t=219s) whole. So in this case when we land A and B together the trunk is green and uh [3:43](https://www.youtube.com/watch?v=hX140a5klW0&t=223s) and B together the trunk is green and uh [3:43](https://www.youtube.com/watch?v=hX140a5klW0&t=223s) and B together the trunk is green and uh and that's how basically uh sort of [3:45](https://www.youtube.com/watch?v=hX140a5klW0&t=225s) and that's how basically uh sort of [3:45](https://www.youtube.com/watch?v=hX140a5klW0&t=225s) and that's how basically uh sort of summit Q works and keeps the trunk green [3:48](https://www.youtube.com/watch?v=hX140a5klW0&t=228s) summit Q works and keeps the trunk green [3:48](https://www.youtube.com/watch?v=hX140a5klW0&t=228s) summit Q works and keeps the trunk green for our monor repos. Now how does it [3:51](https://www.youtube.com/watch?v=hX140a5klW0&t=231s) for our monor repos. Now how does it [3:51](https://www.youtube.com/watch?v=hX140a5klW0&t=231s) for our monor repos. Now how does it scale in this uh version of it? Um there [3:54](https://www.youtube.com/watch?v=hX140a5klW0&t=234s) scale in this uh version of it? Um there [3:54](https://www.youtube.com/watch?v=hX140a5klW0&t=234s) scale in this uh version of it? Um there are three things that summit key [3:56](https://www.youtube.com/watch?v=hX140a5klW0&t=236s) are three things that summit key [3:56](https://www.youtube.com/watch?v=hX140a5klW0&t=236s) are three things that summit key employs. Uh the very first thing is [3:58](https://www.youtube.com/watch?v=hX140a5klW0&t=238s) employs. Uh the very first thing is [3:58](https://www.youtube.com/watch?v=hX140a5klW0&t=238s) employs. Uh the very first thing is target analysis. Uh what it means is [4:00](https://www.youtube.com/watch?v=hX140a5klW0&t=240s) target analysis. Uh what it means is [4:00](https://www.youtube.com/watch?v=hX140a5klW0&t=240s) target analysis. Uh what it means is when two users are working across two [4:02](https://www.youtube.com/watch?v=hX140a5klW0&t=242s) when two users are working across two [4:02](https://www.youtube.com/watch?v=hX140a5klW0&t=242s) when two users are working across two different services uh they don't really [4:05](https://www.youtube.com/watch?v=hX140a5klW0&t=245s) different services uh they don't really [4:05](https://www.youtube.com/watch?v=hX140a5klW0&t=245s) different services uh they don't really have any overlapping changes. Um so [4:07](https://www.youtube.com/watch?v=hX140a5klW0&t=247s) have any overlapping changes. Um so [4:07](https://www.youtube.com/watch?v=hX140a5klW0&t=247s) have any overlapping changes. Um so those two things can be built and tested [4:10](https://www.youtube.com/watch?v=hX140a5klW0&t=250s) those two things can be built and tested [4:10](https://www.youtube.com/watch?v=hX140a5klW0&t=250s) those two things can be built and tested independently of each other and then can [4:12](https://www.youtube.com/watch?v=hX140a5klW0&t=252s) independently of each other and then can [4:12](https://www.youtube.com/watch?v=hX140a5klW0&t=252s) independently of each other and then can be merged. So this allows us to [4:14](https://www.youtube.com/watch?v=hX140a5klW0&t=254s) be merged. So this allows us to [4:14](https://www.youtube.com/watch?v=hX140a5klW0&t=254s) be merged. So this allows us to basically get higher throughput and not [4:16](https://www.youtube.com/watch?v=hX140a5klW0&t=256s) basically get higher throughput and not [4:16](https://www.youtube.com/watch?v=hX140a5klW0&t=256s) basically get higher throughput and not serialize every change that is entering [4:18](https://www.youtube.com/watch?v=hX140a5klW0&t=258s) serialize every change that is entering [4:18](https://www.youtube.com/watch?v=hX140a5klW0&t=258s) serialize every change that is entering into the queue. [4:20](https://www.youtube.com/watch?v=hX140a5klW0&t=260s) into the queue. [4:20](https://www.youtube.com/watch?v=hX140a5klW0&t=260s) into the queue. Uh second is probabistic speculation. Um [4:23](https://www.youtube.com/watch?v=hX140a5klW0&t=263s) Uh second is probabistic speculation. Um [4:23](https://www.youtube.com/watch?v=hX140a5klW0&t=263s) Uh second is probabistic speculation. Um what it means is uh we are using [4:26](https://www.youtube.com/watch?v=hX140a5klW0&t=266s) what it means is uh we are using [4:26](https://www.youtube.com/watch?v=hX140a5klW0&t=266s) what it means is uh we are using historical um the build information and [4:29](https://www.youtube.com/watch?v=hX140a5klW0&t=269s) historical um the build information and [4:29](https://www.youtube.com/watch?v=hX140a5klW0&t=269s) historical um the build information and also the the change characteristics to [4:32](https://www.youtube.com/watch?v=hX140a5klW0&t=272s) also the the change characteristics to [4:32](https://www.youtube.com/watch?v=hX140a5klW0&t=272s) also the the change characteristics to predict uh what is the success score for [4:35](https://www.youtube.com/watch?v=hX140a5klW0&t=275s) predict uh what is the success score for [4:35](https://www.youtube.com/watch?v=hX140a5klW0&t=275s) predict uh what is the success score for a given change and then we are using [4:37](https://www.youtube.com/watch?v=hX140a5klW0&t=277s) a given change and then we are using [4:37](https://www.youtube.com/watch?v=hX140a5klW0&t=277s) a given change and then we are using that to basically decide what all builds [4:40](https://www.youtube.com/watch?v=hX140a5klW0&t=280s) that to basically decide what all builds [4:40](https://www.youtube.com/watch?v=hX140a5klW0&t=280s) that to basically decide what all builds are likely needed in order to land that [4:43](https://www.youtube.com/watch?v=hX140a5klW0&t=283s) are likely needed in order to land that [4:43](https://www.youtube.com/watch?v=hX140a5klW0&t=283s) are likely needed in order to land that change. And so in that case we [4:45](https://www.youtube.com/watch?v=hX140a5klW0&t=285s) change. And so in that case we [4:45](https://www.youtube.com/watch?v=hX140a5klW0&t=285s) change. And so in that case we prioritize the build that are most [4:47](https://www.youtube.com/watch?v=hX140a5klW0&t=287s) prioritize the build that are most [4:47](https://www.youtube.com/watch?v=hX140a5klW0&t=287s) prioritize the build that are most likely needed and are important for that [4:49](https://www.youtube.com/watch?v=hX140a5klW0&t=289s) likely needed and are important for that [4:49](https://www.youtube.com/watch?v=hX140a5klW0&t=289s) likely needed and are important for that change to be uh merged to the trunk [4:51](https://www.youtube.com/watch?v=hX140a5klW0&t=291s) change to be uh merged to the trunk [4:51](https://www.youtube.com/watch?v=hX140a5klW0&t=291s) change to be uh merged to the trunk rather than like every other build that [4:53](https://www.youtube.com/watch?v=hX140a5klW0&t=293s) rather than like every other build that [4:53](https://www.youtube.com/watch?v=hX140a5klW0&t=293s) rather than like every other build that uh comes from the speculation. The the [4:56](https://www.youtube.com/watch?v=hX140a5klW0&t=296s) uh comes from the speculation. The the [4:56](https://www.youtube.com/watch?v=hX140a5klW0&t=296s) uh comes from the speculation. The the third part is bypassing the large [4:58](https://www.youtube.com/watch?v=hX140a5klW0&t=298s) third part is bypassing the large [4:58](https://www.youtube.com/watch?v=hX140a5klW0&t=298s) third part is bypassing the large changes. Uh what it means is that if we [5:00](https://www.youtube.com/watch?v=hX140a5klW0&t=300s) changes. Uh what it means is that if we [5:00](https://www.youtube.com/watch?v=hX140a5klW0&t=300s) changes. Uh what it means is that if we have a large chain sitting in the queue [5:01](https://www.youtube.com/watch?v=hX140a5klW0&t=301s) have a large chain sitting in the queue [5:01](https://www.youtube.com/watch?v=hX140a5klW0&t=301s) have a large chain sitting in the queue which is taking longer to build and at [5:04](https://www.youtube.com/watch?v=hX140a5klW0&t=304s) which is taking longer to build and at [5:04](https://www.youtube.com/watch?v=hX140a5klW0&t=304s) which is taking longer to build and at the same time if we have a smaller [5:05](https://www.youtube.com/watch?v=hX140a5klW0&t=305s) the same time if we have a smaller [5:05](https://www.youtube.com/watch?v=hX140a5klW0&t=305s) the same time if we have a smaller changes entering into the queue um those [5:08](https://www.youtube.com/watch?v=hX140a5klW0&t=308s) changes entering into the queue um those [5:08](https://www.youtube.com/watch?v=hX140a5klW0&t=308s) changes entering into the queue um those can be built faster and so though those [5:10](https://www.youtube.com/watch?v=hX140a5klW0&t=310s) can be built faster and so though those [5:10](https://www.youtube.com/watch?v=hX140a5klW0&t=310s) can be built faster and so though those changes don't need to wait for large [5:12](https://www.youtube.com/watch?v=hX140a5klW0&t=312s) changes don't need to wait for large [5:12](https://www.youtube.com/watch?v=hX140a5klW0&t=312s) changes don't need to wait for large changes to land before they can land. So [5:14](https://www.youtube.com/watch?v=hX140a5klW0&t=314s) changes to land before they can land. So [5:14](https://www.youtube.com/watch?v=hX140a5klW0&t=314s) changes to land before they can land. So in this case we can land the smaller [5:16](https://www.youtube.com/watch?v=hX140a5klW0&t=316s) in this case we can land the smaller [5:16](https://www.youtube.com/watch?v=hX140a5klW0&t=316s) in this case we can land the smaller changes without landing the larger [5:18](https://www.youtube.com/watch?v=hX140a5klW0&t=318s) changes without landing the larger [5:18](https://www.youtube.com/watch?v=hX140a5klW0&t=318s) changes without landing the larger changes first and this optimization [5:20](https://www.youtube.com/watch?v=hX140a5klW0&t=320s) changes first and this optimization [5:20](https://www.youtube.com/watch?v=hX140a5klW0&t=320s) changes first and this optimization allows us to basically move fast and get [5:22](https://www.youtube.com/watch?v=hX140a5klW0&t=322s) allows us to basically move fast and get [5:22](https://www.youtube.com/watch?v=hX140a5klW0&t=322s) allows us to basically move fast and get a higher throughput of the queue. So [5:25](https://www.youtube.com/watch?v=hX140a5klW0&t=325s) a higher throughput of the queue. So [5:25](https://www.youtube.com/watch?v=hX140a5klW0&t=325s) a higher throughput of the queue. So this is what has worked um when it was [5:28](https://www.youtube.com/watch?v=hX140a5klW0&t=328s) this is what has worked um when it was [5:28](https://www.youtube.com/watch?v=hX140a5klW0&t=328s) this is what has worked um when it was just humans uh landing changes creating [5:31](https://www.youtube.com/watch?v=hX140a5klW0&t=331s) just humans uh landing changes creating [5:31](https://www.youtube.com/watch?v=hX140a5klW0&t=331s) just humans uh landing changes creating PRs [5:33](https://www.youtube.com/watch?v=hX140a5klW0&t=333s) PRs [5:33](https://www.youtube.com/watch?v=hX140a5klW0&t=333s) PRs but um then we had AI and the changes at [5:38](https://www.youtube.com/watch?v=hX140a5klW0&t=338s) but um then we had AI and the changes at [5:38](https://www.youtube.com/watch?v=hX140a5klW0&t=338s) but um then we had AI and the changes at which the uh PRs were created and landed [5:42](https://www.youtube.com/watch?v=hX140a5klW0&t=342s) which the uh PRs were created and landed [5:42](https://www.youtube.com/watch?v=hX140a5klW0&t=342s) which the uh PRs were created and landed it all changed. So to understand it a [5:44](https://www.youtube.com/watch?v=hX140a5klW0&t=344s) it all changed. So to understand it a [5:44](https://www.youtube.com/watch?v=hX140a5klW0&t=344s) it all changed. So to understand it a better let's look at some of the [5:45](https://www.youtube.com/watch?v=hX140a5klW0&t=345s) better let's look at some of the [5:46](https://www.youtube.com/watch?v=hX140a5klW0&t=346s) better let's look at some of the numbers. So uh in January 2026 we were [5:50](https://www.youtube.com/watch?v=hX140a5klW0&t=350s) numbers. So uh in January 2026 we were [5:50](https://www.youtube.com/watch?v=hX140a5klW0&t=350s) numbers. So uh in January 2026 we were doing uh roughly 90 requests uh every [5:54](https://www.youtube.com/watch?v=hX140a5klW0&t=354s) doing uh roughly 90 requests uh every [5:54](https://www.youtube.com/watch?v=hX140a5klW0&t=354s) doing uh roughly 90 requests uh every hour and by March we were doing 130 [5:58](https://www.youtube.com/watch?v=hX140a5klW0&t=358s) hour and by March we were doing 130 [5:58](https://www.youtube.com/watch?v=hX140a5klW0&t=358s) hour and by March we were doing 130 requests every hour. So that's roughly a [6:01](https://www.youtube.com/watch?v=hX140a5klW0&t=361s) requests every hour. So that's roughly a [6:01](https://www.youtube.com/watch?v=hX140a5klW0&t=361s) requests every hour. So that's roughly a 44% increase in two months and the [6:05](https://www.youtube.com/watch?v=hX140a5klW0&t=365s) 44% increase in two months and the [6:05](https://www.youtube.com/watch?v=hX140a5klW0&t=365s) 44% increase in two months and the reason for this increase is primarily [6:06](https://www.youtube.com/watch?v=hX140a5klW0&t=366s) reason for this increase is primarily [6:06](https://www.youtube.com/watch?v=hX140a5klW0&t=366s) reason for this increase is primarily because of the AI generated changes. Now [6:10](https://www.youtube.com/watch?v=hX140a5klW0&t=370s) because of the AI generated changes. Now [6:10](https://www.youtube.com/watch?v=hX140a5klW0&t=370s) because of the AI generated changes. Now given the way the system was built um uh [6:13](https://www.youtube.com/watch?v=hX140a5klW0&t=373s) given the way the system was built um uh [6:13](https://www.youtube.com/watch?v=hX140a5klW0&t=373s) given the way the system was built um uh it had the rate of uh these changes had [6:17](https://www.youtube.com/watch?v=hX140a5klW0&t=377s) it had the rate of uh these changes had [6:17](https://www.youtube.com/watch?v=hX140a5klW0&t=377s) it had the rate of uh these changes had effect on it. So um looking into those [6:21](https://www.youtube.com/watch?v=hX140a5klW0&t=381s) effect on it. So um looking into those [6:21](https://www.youtube.com/watch?v=hX140a5klW0&t=381s) effect on it. So um looking into those like we had 30 minutes of uh P95 line [6:23](https://www.youtube.com/watch?v=hX140a5klW0&t=383s) like we had 30 minutes of uh P95 line [6:23](https://www.youtube.com/watch?v=hX140a5klW0&t=383s) like we had 30 minutes of uh P95 line times when we were doing 90 requests [6:26](https://www.youtube.com/watch?v=hX140a5klW0&t=386s) times when we were doing 90 requests [6:26](https://www.youtube.com/watch?v=hX140a5klW0&t=386s) times when we were doing 90 requests every hour and that went to roughly 3 to [6:30](https://www.youtube.com/watch?v=hX140a5klW0&t=390s) every hour and that went to roughly 3 to [6:30](https://www.youtube.com/watch?v=hX140a5klW0&t=390s) every hour and that went to roughly 3 to four hours when we were doing uh 130 uh [6:33](https://www.youtube.com/watch?v=hX140a5klW0&t=393s) four hours when we were doing uh 130 uh [6:33](https://www.youtube.com/watch?v=hX140a5klW0&t=393s) four hours when we were doing uh 130 uh requests every hour. So in this case we [6:36](https://www.youtube.com/watch?v=hX140a5klW0&t=396s) requests every hour. So in this case we [6:36](https://www.youtube.com/watch?v=hX140a5klW0&t=396s) requests every hour. So in this case we were still keeping the trunk green uh at [6:39](https://www.youtube.com/watch?v=hX140a5klW0&t=399s) were still keeping the trunk green uh at [6:39](https://www.youtube.com/watch?v=hX140a5klW0&t=399s) were still keeping the trunk green uh at all times but at the cost of developers [6:42](https://www.youtube.com/watch?v=hX140a5klW0&t=402s) all times but at the cost of developers [6:42](https://www.youtube.com/watch?v=hX140a5klW0&t=402s) all times but at the cost of developers essentially waiting for 3 to four hours [6:44](https://www.youtube.com/watch?v=hX140a5klW0&t=404s) essentially waiting for 3 to four hours [6:44](https://www.youtube.com/watch?v=hX140a5klW0&t=404s) essentially waiting for 3 to four hours to land their changes to the trunk. Um [6:47](https://www.youtube.com/watch?v=hX140a5klW0&t=407s) to land their changes to the trunk. Um [6:47](https://www.youtube.com/watch?v=hX140a5klW0&t=407s) to land their changes to the trunk. Um now I'm going to hand over to Mry uh and [6:49](https://www.youtube.com/watch?v=hX140a5klW0&t=409s) now I'm going to hand over to Mry uh and [6:49](https://www.youtube.com/watch?v=hX140a5klW0&t=409s) now I'm going to hand over to Mry uh and she's going to talk about our response [6:51](https://www.youtube.com/watch?v=hX140a5klW0&t=411s) she's going to talk about our response [6:51](https://www.youtube.com/watch?v=hX140a5klW0&t=411s) she's going to talk about our response to it and how we adapted uh to the [6:54](https://www.youtube.com/watch?v=hX140a5klW0&t=414s) to it and how we adapted uh to the [6:54](https://www.youtube.com/watch?v=hX140a5klW0&t=414s) to it and how we adapted uh to the situation. [6:55](https://www.youtube.com/watch?v=hX140a5klW0&t=415s) situation. [6:55](https://www.youtube.com/watch?v=hX140a5klW0&t=415s) situation. Great. Thanks Pritham. So, Priam just uh [6:59](https://www.youtube.com/watch?v=hX140a5klW0&t=419s) Great. Thanks Pritham. So, Priam just uh [6:59](https://www.youtube.com/watch?v=hX140a5klW0&t=419s) Great. Thanks Pritham. So, Priam just uh walked us through a few optimizations [7:01](https://www.youtube.com/watch?v=hX140a5klW0&t=421s) walked us through a few optimizations [7:01](https://www.youtube.com/watch?v=hX140a5klW0&t=421s) walked us through a few optimizations that Submit Q had that worked so far in [7:03](https://www.youtube.com/watch?v=hX140a5klW0&t=423s) that Submit Q had that worked so far in [7:03](https://www.youtube.com/watch?v=hX140a5klW0&t=423s) that Submit Q had that worked so far in keeping up with the demand uh that [7:05](https://www.youtube.com/watch?v=hX140a5klW0&t=425s) keeping up with the demand uh that [7:06](https://www.youtube.com/watch?v=hX140a5klW0&t=426s) keeping up with the demand uh that humans generated. But once AI came into [7:08](https://www.youtube.com/watch?v=hX140a5klW0&t=428s) humans generated. But once AI came into [7:08](https://www.youtube.com/watch?v=hX140a5klW0&t=428s) humans generated. But once AI came into the picture, uh the optimizations we had [7:11](https://www.youtube.com/watch?v=hX140a5klW0&t=431s) the picture, uh the optimizations we had [7:11](https://www.youtube.com/watch?v=hX140a5klW0&t=431s) the picture, uh the optimizations we had were no longer sufficient. So we our [7:14](https://www.youtube.com/watch?v=hX140a5klW0&t=434s) were no longer sufficient. So we our [7:14](https://www.youtube.com/watch?v=hX140a5klW0&t=434s) were no longer sufficient. So we our response essentially was another set of [7:17](https://www.youtube.com/watch?v=hX140a5klW0&t=437s) response essentially was another set of [7:17](https://www.youtube.com/watch?v=hX140a5klW0&t=437s) response essentially was another set of optimizations to our existing submit Q [7:20](https://www.youtube.com/watch?v=hX140a5klW0&t=440s) optimizations to our existing submit Q [7:20](https://www.youtube.com/watch?v=hX140a5klW0&t=440s) optimizations to our existing submit Q and we approached the response in three [7:22](https://www.youtube.com/watch?v=hX140a5klW0&t=442s) and we approached the response in three [7:22](https://www.youtube.com/watch?v=hX140a5klW0&t=442s) and we approached the response in three different ways. The first was we [7:24](https://www.youtube.com/watch?v=hX140a5klW0&t=444s) different ways. The first was we [7:24](https://www.youtube.com/watch?v=hX140a5klW0&t=444s) different ways. The first was we introduced this concept of admission [7:26](https://www.youtube.com/watch?v=hX140a5klW0&t=446s) introduced this concept of admission [7:26](https://www.youtube.com/watch?v=hX140a5klW0&t=446s) introduced this concept of admission gating which helped us to shape the [7:29](https://www.youtube.com/watch?v=hX140a5klW0&t=449s) gating which helped us to shape the [7:29](https://www.youtube.com/watch?v=hX140a5klW0&t=449s) gating which helped us to shape the traffic that we accept into the queue. [7:31](https://www.youtube.com/watch?v=hX140a5klW0&t=451s) traffic that we accept into the queue. [7:31](https://www.youtube.com/watch?v=hX140a5klW0&t=451s) traffic that we accept into the queue. Uh next we introduced the concept of [7:34](https://www.youtube.com/watch?v=hX140a5klW0&t=454s) Uh next we introduced the concept of [7:34](https://www.youtube.com/watch?v=hX140a5klW0&t=454s) Uh next we introduced the concept of deferrals which is a secondary queue [7:37](https://www.youtube.com/watch?v=hX140a5klW0&t=457s) deferrals which is a secondary queue [7:37](https://www.youtube.com/watch?v=hX140a5klW0&t=457s) deferrals which is a secondary queue that we introduced that helps with [7:39](https://www.youtube.com/watch?v=hX140a5klW0&t=459s) that we introduced that helps with [7:39](https://www.youtube.com/watch?v=hX140a5klW0&t=459s) that we introduced that helps with developer experience uh from the pain [7:42](https://www.youtube.com/watch?v=hX140a5klW0&t=462s) developer experience uh from the pain [7:42](https://www.youtube.com/watch?v=hX140a5klW0&t=462s) developer experience uh from the pain that comes with admission gating. And [7:44](https://www.youtube.com/watch?v=hX140a5klW0&t=464s) that comes with admission gating. And [7:44](https://www.youtube.com/watch?v=hX140a5klW0&t=464s) that comes with admission gating. And lastly, we implemented what we call [7:46](https://www.youtube.com/watch?v=hX140a5klW0&t=466s) lastly, we implemented what we call [7:46](https://www.youtube.com/watch?v=hX140a5klW0&t=466s) lastly, we implemented what we call conflict relaxation which helps us buy [7:49](https://www.youtube.com/watch?v=hX140a5klW0&t=469s) conflict relaxation which helps us buy [7:49](https://www.youtube.com/watch?v=hX140a5klW0&t=469s) conflict relaxation which helps us buy back more parallelism uh within the [7:53](https://www.youtube.com/watch?v=hX140a5klW0&t=473s) back more parallelism uh within the [7:53](https://www.youtube.com/watch?v=hX140a5klW0&t=473s) back more parallelism uh within the queue itself. Uh next we will look at [7:56](https://www.youtube.com/watch?v=hX140a5klW0&t=476s) queue itself. Uh next we will look at [7:56](https://www.youtube.com/watch?v=hX140a5klW0&t=476s) queue itself. Uh next we will look at each of these uh interventions in a [7:59](https://www.youtube.com/watch?v=hX140a5klW0&t=479s) each of these uh interventions in a [7:59](https://www.youtube.com/watch?v=hX140a5klW0&t=479s) each of these uh interventions in a little bit of depth and understand [8:01](https://www.youtube.com/watch?v=hX140a5klW0&t=481s) little bit of depth and understand [8:01](https://www.youtube.com/watch?v=hX140a5klW0&t=481s) little bit of depth and understand exactly what each of them are doing. So [8:04](https://www.youtube.com/watch?v=hX140a5klW0&t=484s) exactly what each of them are doing. So [8:04](https://www.youtube.com/watch?v=hX140a5klW0&t=484s) exactly what each of them are doing. So the first one admission gating uh is [8:06](https://www.youtube.com/watch?v=hX140a5klW0&t=486s) the first one admission gating uh is [8:06](https://www.youtube.com/watch?v=hX140a5klW0&t=486s) the first one admission gating uh is basically what it sounds like. There is [8:08](https://www.youtube.com/watch?v=hX140a5klW0&t=488s) basically what it sounds like. There is [8:08](https://www.youtube.com/watch?v=hX140a5klW0&t=488s) basically what it sounds like. There is a gate at the admission of every request [8:11](https://www.youtube.com/watch?v=hX140a5klW0&t=491s) a gate at the admission of every request [8:11](https://www.youtube.com/watch?v=hX140a5klW0&t=491s) a gate at the admission of every request into the main submit que. We look at the [8:14](https://www.youtube.com/watch?v=hX140a5klW0&t=494s) into the main submit que. We look at the [8:14](https://www.youtube.com/watch?v=hX140a5klW0&t=494s) into the main submit que. We look at the request that comes in and we categorize [8:17](https://www.youtube.com/watch?v=hX140a5klW0&t=497s) request that comes in and we categorize [8:17](https://www.youtube.com/watch?v=hX140a5klW0&t=497s) request that comes in and we categorize it as one of two things. Either it's a [8:19](https://www.youtube.com/watch?v=hX140a5klW0&t=499s) it as one of two things. Either it's a [8:19](https://www.youtube.com/watch?v=hX140a5klW0&t=499s) it as one of two things. Either it's a small change or it's an expensive [8:21](https://www.youtube.com/watch?v=hX140a5klW0&t=501s) small change or it's an expensive [8:21](https://www.youtube.com/watch?v=hX140a5klW0&t=501s) small change or it's an expensive change. And the definition of this uh [8:24](https://www.youtube.com/watch?v=hX140a5klW0&t=504s) change. And the definition of this uh [8:24](https://www.youtube.com/watch?v=hX140a5klW0&t=504s) change. And the definition of this uh could be anything. One of the [8:25](https://www.youtube.com/watch?v=hX140a5klW0&t=505s) could be anything. One of the [8:25](https://www.youtube.com/watch?v=hX140a5klW0&t=505s) could be anything. One of the definitions that we use is the target [8:28](https://www.youtube.com/watch?v=hX140a5klW0&t=508s) definitions that we use is the target [8:28](https://www.youtube.com/watch?v=hX140a5klW0&t=508s) definitions that we use is the target analysis that Pitham talked about uh [8:31](https://www.youtube.com/watch?v=hX140a5klW0&t=511s) analysis that Pitham talked about uh [8:31](https://www.youtube.com/watch?v=hX140a5klW0&t=511s) analysis that Pitham talked about uh earlier in the slides. Uh a small change [8:34](https://www.youtube.com/watch?v=hX140a5klW0&t=514s) earlier in the slides. Uh a small change [8:34](https://www.youtube.com/watch?v=hX140a5klW0&t=514s) earlier in the slides. Uh a small change touches fewer targets. An expensive [8:36](https://www.youtube.com/watch?v=hX140a5klW0&t=516s) touches fewer targets. An expensive [8:36](https://www.youtube.com/watch?v=hX140a5klW0&t=516s) touches fewer targets. An expensive change might be something that spans [8:38](https://www.youtube.com/watch?v=hX140a5klW0&t=518s) change might be something that spans [8:38](https://www.youtube.com/watch?v=hX140a5klW0&t=518s) change might be something that spans across uh multiple targets, the entire [8:42](https://www.youtube.com/watch?v=hX140a5klW0&t=522s) across uh multiple targets, the entire [8:42](https://www.youtube.com/watch?v=hX140a5klW0&t=522s) across uh multiple targets, the entire build graph, something like that. So a [8:44](https://www.youtube.com/watch?v=hX140a5klW0&t=524s) build graph, something like that. So a [8:44](https://www.youtube.com/watch?v=hX140a5klW0&t=524s) build graph, something like that. So a smaller change is more likely to give [8:47](https://www.youtube.com/watch?v=hX140a5klW0&t=527s) smaller change is more likely to give [8:47](https://www.youtube.com/watch?v=hX140a5klW0&t=527s) smaller change is more likely to give you higher parallelism and we basically [8:51](https://www.youtube.com/watch?v=hX140a5klW0&t=531s) you higher parallelism and we basically [8:51](https://www.youtube.com/watch?v=hX140a5klW0&t=531s) you higher parallelism and we basically say admit it during peak hours and add [8:53](https://www.youtube.com/watch?v=hX140a5klW0&t=533s) say admit it during peak hours and add [8:53](https://www.youtube.com/watch?v=hX140a5klW0&t=533s) say admit it during peak hours and add it to the active uh submit Q build and [8:56](https://www.youtube.com/watch?v=hX140a5klW0&t=536s) it to the active uh submit Q build and [8:56](https://www.youtube.com/watch?v=hX140a5klW0&t=536s) it to the active uh submit Q build and test as usual uh and move along. But an [8:59](https://www.youtube.com/watch?v=hX140a5klW0&t=539s) test as usual uh and move along. But an [8:59](https://www.youtube.com/watch?v=hX140a5klW0&t=539s) test as usual uh and move along. But an expensive change has the capability to [9:02](https://www.youtube.com/watch?v=hX140a5klW0&t=542s) expensive change has the capability to [9:02](https://www.youtube.com/watch?v=hX140a5klW0&t=542s) expensive change has the capability to basically be a bottleneck by holding [9:05](https://www.youtube.com/watch?v=hX140a5klW0&t=545s) basically be a bottleneck by holding [9:06](https://www.youtube.com/watch?v=hX140a5klW0&t=546s) basically be a bottleneck by holding resources [9:07](https://www.youtube.com/watch?v=hX140a5klW0&t=547s) resources [9:07](https://www.youtube.com/watch?v=hX140a5klW0&t=547s) resources uh needing to validate against [9:09](https://www.youtube.com/watch?v=hX140a5klW0&t=549s) uh needing to validate against [9:09](https://www.youtube.com/watch?v=hX140a5klW0&t=549s) uh needing to validate against everything else that's in the queue. So [9:11](https://www.youtube.com/watch?v=hX140a5klW0&t=551s) everything else that's in the queue. So [9:11](https://www.youtube.com/watch?v=hX140a5klW0&t=551s) everything else that's in the queue. So to protect the rest of the queue, what [9:14](https://www.youtube.com/watch?v=hX140a5klW0&t=554s) to protect the rest of the queue, what [9:14](https://www.youtube.com/watch?v=hX140a5klW0&t=554s) to protect the rest of the queue, what we do uh in this uh phase is just to [9:18](https://www.youtube.com/watch?v=hX140a5klW0&t=558s) we do uh in this uh phase is just to [9:18](https://www.youtube.com/watch?v=hX140a5klW0&t=558s) we do uh in this uh phase is just to reject any expensive change during peak [9:21](https://www.youtube.com/watch?v=hX140a5klW0&t=561s) reject any expensive change during peak [9:21](https://www.youtube.com/watch?v=hX140a5klW0&t=561s) reject any expensive change during peak hours. What this means is that the [9:23](https://www.youtube.com/watch?v=hX140a5klW0&t=563s) hours. What this means is that the [9:23](https://www.youtube.com/watch?v=hX140a5klW0&t=563s) hours. What this means is that the developer has to come back and retry [9:25](https://www.youtube.com/watch?v=hX140a5klW0&t=565s) developer has to come back and retry [9:25](https://www.youtube.com/watch?v=hX140a5klW0&t=565s) developer has to come back and retry their expensive change outside peak [9:28](https://www.youtube.com/watch?v=hX140a5klW0&t=568s) their expensive change outside peak [9:28](https://www.youtube.com/watch?v=hX140a5klW0&t=568s) their expensive change outside peak hours. Uh so this gives us control into [9:31](https://www.youtube.com/watch?v=hX140a5klW0&t=571s) hours. Uh so this gives us control into [9:31](https://www.youtube.com/watch?v=hX140a5klW0&t=571s) hours. Uh so this gives us control into what enters the queue by moving the [9:33](https://www.youtube.com/watch?v=hX140a5klW0&t=573s) what enters the queue by moving the [9:34](https://www.youtube.com/watch?v=hX140a5klW0&t=574s) what enters the queue by moving the burden onto a very subset [9:37](https://www.youtube.com/watch?v=hX140a5klW0&t=577s) burden onto a very subset [9:37](https://www.youtube.com/watch?v=hX140a5klW0&t=577s) burden onto a very subset of changes and developers. [9:41](https://www.youtube.com/watch?v=hX140a5klW0&t=581s) of changes and developers. [9:41](https://www.youtube.com/watch?v=hX140a5klW0&t=581s) of changes and developers. Uh let's look at the trade-off of [9:42](https://www.youtube.com/watch?v=hX140a5klW0&t=582s) Uh let's look at the trade-off of [9:42](https://www.youtube.com/watch?v=hX140a5klW0&t=582s) Uh let's look at the trade-off of admission gating. Uh after we [9:44](https://www.youtube.com/watch?v=hX140a5klW0&t=584s) admission gating. Uh after we [9:44](https://www.youtube.com/watch?v=hX140a5klW0&t=584s) admission gating. Uh after we implemented this, we definitely observed [9:47](https://www.youtube.com/watch?v=hX140a5klW0&t=587s) implemented this, we definitely observed [9:47](https://www.youtube.com/watch?v=hX140a5klW0&t=587s) implemented this, we definitely observed an increase in our throughput. We were [9:49](https://www.youtube.com/watch?v=hX140a5klW0&t=589s) an increase in our throughput. We were [9:49](https://www.youtube.com/watch?v=hX140a5klW0&t=589s) an increase in our throughput. We were able to achieve over 20% increase. Uh [9:52](https://www.youtube.com/watch?v=hX140a5klW0&t=592s) able to achieve over 20% increase. Uh [9:52](https://www.youtube.com/watch?v=hX140a5klW0&t=592s) able to achieve over 20% increase. Uh our land rate went from 130 to about 160 [9:56](https://www.youtube.com/watch?v=hX140a5klW0&t=596s) our land rate went from 130 to about 160 [9:56](https://www.youtube.com/watch?v=hX140a5klW0&t=596s) our land rate went from 130 to about 160 during business hours. But it came at a [9:59](https://www.youtube.com/watch?v=hX140a5klW0&t=599s) during business hours. But it came at a [9:59](https://www.youtube.com/watch?v=hX140a5klW0&t=599s) during business hours. But it came at a cost and the cost is that developers [10:02](https://www.youtube.com/watch?v=hX140a5klW0&t=602s) cost and the cost is that developers [10:02](https://www.youtube.com/watch?v=hX140a5klW0&t=602s) cost and the cost is that developers become the scheduleulers. uh not all [10:05](https://www.youtube.com/watch?v=hX140a5klW0&t=605s) become the scheduleulers. uh not all [10:05](https://www.youtube.com/watch?v=hX140a5klW0&t=605s) become the scheduleulers. uh not all developers since we do protect the [10:07](https://www.youtube.com/watch?v=hX140a5klW0&t=607s) developers since we do protect the [10:07](https://www.youtube.com/watch?v=hX140a5klW0&t=607s) developers since we do protect the smaller changes. But you can imagine if [10:09](https://www.youtube.com/watch?v=hX140a5klW0&t=609s) smaller changes. But you can imagine if [10:09](https://www.youtube.com/watch?v=hX140a5klW0&t=609s) smaller changes. But you can imagine if you are a developer who owns any types [10:12](https://www.youtube.com/watch?v=hX140a5klW0&t=612s) you are a developer who owns any types [10:12](https://www.youtube.com/watch?v=hX140a5klW0&t=612s) you are a developer who owns any types of libraries uh that span the entire [10:16](https://www.youtube.com/watch?v=hX140a5klW0&t=616s) of libraries uh that span the entire [10:16](https://www.youtube.com/watch?v=hX140a5klW0&t=616s) of libraries uh that span the entire code, you will be particularly targeted [10:19](https://www.youtube.com/watch?v=hX140a5klW0&t=619s) code, you will be particularly targeted [10:19](https://www.youtube.com/watch?v=hX140a5klW0&t=619s) code, you will be particularly targeted in this because the work that you do uh [10:22](https://www.youtube.com/watch?v=hX140a5klW0&t=622s) in this because the work that you do uh [10:22](https://www.youtube.com/watch?v=hX140a5klW0&t=622s) in this because the work that you do uh naturally falls into this expensive [10:24](https://www.youtube.com/watch?v=hX140a5klW0&t=624s) naturally falls into this expensive [10:24](https://www.youtube.com/watch?v=hX140a5klW0&t=624s) naturally falls into this expensive change bucket. Uh so we managed to [10:27](https://www.youtube.com/watch?v=hX140a5klW0&t=627s) change bucket. Uh so we managed to [10:27](https://www.youtube.com/watch?v=hX140a5klW0&t=627s) change bucket. Uh so we managed to protect the capacity during peak hours [10:29](https://www.youtube.com/watch?v=hX140a5klW0&t=629s) protect the capacity during peak hours [10:29](https://www.youtube.com/watch?v=hX140a5klW0&t=629s) protect the capacity during peak hours but had to make developers wait. Uh next [10:33](https://www.youtube.com/watch?v=hX140a5klW0&t=633s) but had to make developers wait. Uh next [10:33](https://www.youtube.com/watch?v=hX140a5klW0&t=633s) but had to make developers wait. Uh next slide. So the next thing that we do is [10:36](https://www.youtube.com/watch?v=hX140a5klW0&t=636s) slide. So the next thing that we do is [10:36](https://www.youtube.com/watch?v=hX140a5klW0&t=636s) slide. So the next thing that we do is targeted more towards bettering the [10:38](https://www.youtube.com/watch?v=hX140a5klW0&t=638s) targeted more towards bettering the [10:38](https://www.youtube.com/watch?v=hX140a5klW0&t=638s) targeted more towards bettering the developer experience uh of uh the cons [10:42](https://www.youtube.com/watch?v=hX140a5klW0&t=642s) developer experience uh of uh the cons [10:42](https://www.youtube.com/watch?v=hX140a5klW0&t=642s) developer experience uh of uh the cons that come from admission gating. Uh what [10:45](https://www.youtube.com/watch?v=hX140a5klW0&t=645s) that come from admission gating. Uh what [10:45](https://www.youtube.com/watch?v=hX140a5klW0&t=645s) that come from admission gating. Uh what if instead of outright rejecting [10:48](https://www.youtube.com/watch?v=hX140a5klW0&t=648s) if instead of outright rejecting [10:48](https://www.youtube.com/watch?v=hX140a5klW0&t=648s) if instead of outright rejecting expensive changes, we are able to record [10:50](https://www.youtube.com/watch?v=hX140a5klW0&t=650s) expensive changes, we are able to record [10:50](https://www.youtube.com/watch?v=hX140a5klW0&t=650s) expensive changes, we are able to record them and replay them back into the main [10:53](https://www.youtube.com/watch?v=hX140a5klW0&t=653s) them and replay them back into the main [10:53](https://www.youtube.com/watch?v=hX140a5klW0&t=653s) them and replay them back into the main queue uh based on the status of the [10:57](https://www.youtube.com/watch?v=hX140a5klW0&t=657s) queue uh based on the status of the [10:57](https://www.youtube.com/watch?v=hX140a5klW0&t=657s) queue uh based on the status of the queue uh when we have let's say a better [10:59](https://www.youtube.com/watch?v=hX140a5klW0&t=659s) queue uh when we have let's say a better [10:59](https://www.youtube.com/watch?v=hX140a5klW0&t=659s) queue uh when we have let's say a better uh handle on the capacity. And that's [11:02](https://www.youtube.com/watch?v=hX140a5klW0&t=662s) uh handle on the capacity. And that's [11:02](https://www.youtube.com/watch?v=hX140a5klW0&t=662s) uh handle on the capacity. And that's exactly what deferrals implement. During [11:04](https://www.youtube.com/watch?v=hX140a5klW0&t=664s) exactly what deferrals implement. During [11:04](https://www.youtube.com/watch?v=hX140a5klW0&t=664s) exactly what deferrals implement. During peak hours, if an expensive request [11:06](https://www.youtube.com/watch?v=hX140a5klW0&t=666s) peak hours, if an expensive request [11:06](https://www.youtube.com/watch?v=hX140a5klW0&t=666s) peak hours, if an expensive request comes in, we no longer reject it, but [11:09](https://www.youtube.com/watch?v=hX140a5klW0&t=669s) comes in, we no longer reject it, but [11:09](https://www.youtube.com/watch?v=hX140a5klW0&t=669s) comes in, we no longer reject it, but put it into a deferred queue where it is [11:11](https://www.youtube.com/watch?v=hX140a5klW0&t=671s) put it into a deferred queue where it is [11:11](https://www.youtube.com/watch?v=hX140a5klW0&t=671s) put it into a deferred queue where it is persisted. [11:13](https://www.youtube.com/watch?v=hX140a5klW0&t=673s) persisted. [11:13](https://www.youtube.com/watch?v=hX140a5klW0&t=673s) persisted. And once the gates permit, usually upon [11:17](https://www.youtube.com/watch?v=hX140a5klW0&t=677s) And once the gates permit, usually upon [11:17](https://www.youtube.com/watch?v=hX140a5klW0&t=677s) And once the gates permit, usually upon uh for example, we can say after peak [11:20](https://www.youtube.com/watch?v=hX140a5klW0&t=680s) uh for example, we can say after peak [11:20](https://www.youtube.com/watch?v=hX140a5klW0&t=680s) uh for example, we can say after peak hours uh we put this persisted request [11:24](https://www.youtube.com/watch?v=hX140a5klW0&t=684s) hours uh we put this persisted request [11:24](https://www.youtube.com/watch?v=hX140a5klW0&t=684s) hours uh we put this persisted request back onto the active submit queue. So [11:26](https://www.youtube.com/watch?v=hX140a5klW0&t=686s) back onto the active submit queue. So [11:26](https://www.youtube.com/watch?v=hX140a5klW0&t=686s) back onto the active submit queue. So essentially we uh shape the traffic in a [11:29](https://www.youtube.com/watch?v=hX140a5klW0&t=689s) essentially we uh shape the traffic in a [11:30](https://www.youtube.com/watch?v=hX140a5klW0&t=690s) essentially we uh shape the traffic in a way where we release more expensive [11:32](https://www.youtube.com/watch?v=hX140a5klW0&t=692s) way where we release more expensive [11:32](https://www.youtube.com/watch?v=hX140a5klW0&t=692s) way where we release more expensive changes when the queue is able to handle [11:35](https://www.youtube.com/watch?v=hX140a5klW0&t=695s) changes when the queue is able to handle [11:35](https://www.youtube.com/watch?v=hX140a5klW0&t=695s) changes when the queue is able to handle it. So what this solves is a developer [11:38](https://www.youtube.com/watch?v=hX140a5klW0&t=698s) it. So what this solves is a developer [11:38](https://www.youtube.com/watch?v=hX140a5klW0&t=698s) it. So what this solves is a developer submits once and then we automatically [11:41](https://www.youtube.com/watch?v=hX140a5klW0&t=701s) submits once and then we automatically [11:41](https://www.youtube.com/watch?v=hX140a5klW0&t=701s) submits once and then we automatically release it for them when the capacity [11:43](https://www.youtube.com/watch?v=hX140a5klW0&t=703s) release it for them when the capacity [11:43](https://www.youtube.com/watch?v=hX140a5klW0&t=703s) release it for them when the capacity allows. Next slide. So the trade-off [11:46](https://www.youtube.com/watch?v=hX140a5klW0&t=706s) allows. Next slide. So the trade-off [11:46](https://www.youtube.com/watch?v=hX140a5klW0&t=706s) allows. Next slide. So the trade-off here is it solves the problem for the [11:49](https://www.youtube.com/watch?v=hX140a5klW0&t=709s) here is it solves the problem for the [11:49](https://www.youtube.com/watch?v=hX140a5klW0&t=709s) here is it solves the problem for the developer. They don't have to wake up [11:51](https://www.youtube.com/watch?v=hX140a5klW0&t=711s) developer. They don't have to wake up [11:51](https://www.youtube.com/watch?v=hX140a5klW0&t=711s) developer. They don't have to wake up early, stay up late, try to find a [11:53](https://www.youtube.com/watch?v=hX140a5klW0&t=713s) early, stay up late, try to find a [11:53](https://www.youtube.com/watch?v=hX140a5klW0&t=713s) early, stay up late, try to find a particular window where the queue looks [11:55](https://www.youtube.com/watch?v=hX140a5klW0&t=715s) particular window where the queue looks [11:55](https://www.youtube.com/watch?v=hX140a5klW0&t=715s) particular window where the queue looks good and hit the button on their end. We [11:59](https://www.youtube.com/watch?v=hX140a5klW0&t=719s) good and hit the button on their end. We [11:59](https://www.youtube.com/watch?v=hX140a5klW0&t=719s) good and hit the button on their end. We will handle that for them. But the other [12:01](https://www.youtube.com/watch?v=hX140a5klW0&t=721s) will handle that for them. But the other [12:01](https://www.youtube.com/watch?v=hX140a5klW0&t=721s) will handle that for them. But the other side of this, the bad that comes with [12:03](https://www.youtube.com/watch?v=hX140a5klW0&t=723s) side of this, the bad that comes with [12:03](https://www.youtube.com/watch?v=hX140a5klW0&t=723s) side of this, the bad that comes with this with this is that it doesn't really [12:06](https://www.youtube.com/watch?v=hX140a5klW0&t=726s) this with this is that it doesn't really [12:06](https://www.youtube.com/watch?v=hX140a5klW0&t=726s) this with this is that it doesn't really solve the waiting for the developer. You [12:09](https://www.youtube.com/watch?v=hX140a5klW0&t=729s) solve the waiting for the developer. You [12:09](https://www.youtube.com/watch?v=hX140a5klW0&t=729s) solve the waiting for the developer. You still have to wait the same amount of [12:11](https://www.youtube.com/watch?v=hX140a5klW0&t=731s) still have to wait the same amount of [12:11](https://www.youtube.com/watch?v=hX140a5klW0&t=731s) still have to wait the same amount of time for your expensive change to go in [12:15](https://www.youtube.com/watch?v=hX140a5klW0&t=735s) time for your expensive change to go in [12:15](https://www.youtube.com/watch?v=hX140a5klW0&t=735s) time for your expensive change to go in to the queue. Uh, next slide. So before [12:19](https://www.youtube.com/watch?v=hX140a5klW0&t=739s) to the queue. Uh, next slide. So before [12:19](https://www.youtube.com/watch?v=hX140a5klW0&t=739s) to the queue. Uh, next slide. So before we look at the next uh and final [12:21](https://www.youtube.com/watch?v=hX140a5klW0&t=741s) we look at the next uh and final [12:21](https://www.youtube.com/watch?v=hX140a5klW0&t=741s) we look at the next uh and final optimization that we did uh I want to [12:24](https://www.youtube.com/watch?v=hX140a5klW0&t=744s) optimization that we did uh I want to [12:24](https://www.youtube.com/watch?v=hX140a5klW0&t=744s) optimization that we did uh I want to touch a little bit about the uh on the [12:27](https://www.youtube.com/watch?v=hX140a5klW0&t=747s) touch a little bit about the uh on the [12:27](https://www.youtube.com/watch?v=hX140a5klW0&t=747s) touch a little bit about the uh on the concept of what a target overlap is uh [12:30](https://www.youtube.com/watch?v=hX140a5klW0&t=750s) concept of what a target overlap is uh [12:30](https://www.youtube.com/watch?v=hX140a5klW0&t=750s) concept of what a target overlap is uh and how that serializes work in [12:33](https://www.youtube.com/watch?v=hX140a5klW0&t=753s) and how that serializes work in [12:33](https://www.youtube.com/watch?v=hX140a5klW0&t=753s) and how that serializes work in something. Um so recall that Fam [12:35](https://www.youtube.com/watch?v=hX140a5klW0&t=755s) something. Um so recall that Fam [12:35](https://www.youtube.com/watch?v=hX140a5klW0&t=755s) something. Um so recall that Fam mentioned the target analysis that we [12:37](https://www.youtube.com/watch?v=hX140a5klW0&t=757s) mentioned the target analysis that we [12:37](https://www.youtube.com/watch?v=hX140a5klW0&t=757s) mentioned the target analysis that we perform uh to determine whether two [12:40](https://www.youtube.com/watch?v=hX140a5klW0&t=760s) perform uh to determine whether two [12:40](https://www.youtube.com/watch?v=hX140a5klW0&t=760s) perform uh to determine whether two requests should be serialized or can [12:43](https://www.youtube.com/watch?v=hX140a5klW0&t=763s) requests should be serialized or can [12:43](https://www.youtube.com/watch?v=hX140a5klW0&t=763s) requests should be serialized or can they go in parallel. So an example that [12:46](https://www.youtube.com/watch?v=hX140a5klW0&t=766s) they go in parallel. So an example that [12:46](https://www.youtube.com/watch?v=hX140a5klW0&t=766s) they go in parallel. So an example that we are looking at here uh is a strict [12:48](https://www.youtube.com/watch?v=hX140a5klW0&t=768s) we are looking at here uh is a strict [12:48](https://www.youtube.com/watch?v=hX140a5klW0&t=768s) we are looking at here uh is a strict one connected component. What this means [12:50](https://www.youtube.com/watch?v=hX140a5klW0&t=770s) one connected component. What this means [12:50](https://www.youtube.com/watch?v=hX140a5klW0&t=770s) one connected component. What this means is that each of these requests A through [12:53](https://www.youtube.com/watch?v=hX140a5klW0&t=773s) is that each of these requests A through [12:53](https://www.youtube.com/watch?v=hX140a5klW0&t=773s) is that each of these requests A through F touch some subset of targets and all [12:57](https://www.youtube.com/watch?v=hX140a5klW0&t=777s) F touch some subset of targets and all [12:57](https://www.youtube.com/watch?v=hX140a5klW0&t=777s) F touch some subset of targets and all of there's at least one overlap of [13:00](https://www.youtube.com/watch?v=hX140a5klW0&t=780s) of there's at least one overlap of [13:00](https://www.youtube.com/watch?v=hX140a5klW0&t=780s) of there's at least one overlap of targets between each pair of views. So [13:02](https://www.youtube.com/watch?v=hX140a5klW0&t=782s) targets between each pair of views. So [13:02](https://www.youtube.com/watch?v=hX140a5klW0&t=782s) targets between each pair of views. So like A and B overlap on at least one [13:05](https://www.youtube.com/watch?v=hX140a5klW0&t=785s) like A and B overlap on at least one [13:05](https://www.youtube.com/watch?v=hX140a5klW0&t=785s) like A and B overlap on at least one target, B and C overlap at least one [13:07](https://www.youtube.com/watch?v=hX140a5klW0&t=787s) target, B and C overlap at least one [13:07](https://www.youtube.com/watch?v=hX140a5klW0&t=787s) target, B and C overlap at least one target. Similarly for C, D, E, and F. [13:10](https://www.youtube.com/watch?v=hX140a5klW0&t=790s) target. Similarly for C, D, E, and F. [13:10](https://www.youtube.com/watch?v=hX140a5klW0&t=790s) target. Similarly for C, D, E, and F. And what this makes uh submit Q do is [13:13](https://www.youtube.com/watch?v=hX140a5klW0&t=793s) And what this makes uh submit Q do is [13:13](https://www.youtube.com/watch?v=hX140a5klW0&t=793s) And what this makes uh submit Q do is group all of these requests into a [13:15](https://www.youtube.com/watch?v=hX140a5klW0&t=795s) group all of these requests into a [13:16](https://www.youtube.com/watch?v=hX140a5klW0&t=796s) group all of these requests into a single component. And that means that [13:18](https://www.youtube.com/watch?v=hX140a5klW0&t=798s) single component. And that means that [13:18](https://www.youtube.com/watch?v=hX140a5klW0&t=798s) single component. And that means that all of these have to go together. And [13:22](https://www.youtube.com/watch?v=hX140a5klW0&t=802s) all of these have to go together. And [13:22](https://www.youtube.com/watch?v=hX140a5klW0&t=802s) all of these have to go together. And this is basically what prevents uh us [13:26](https://www.youtube.com/watch?v=hX140a5klW0&t=806s) this is basically what prevents uh us [13:26](https://www.youtube.com/watch?v=hX140a5klW0&t=806s) this is basically what prevents uh us from doing work faster. Uh and the next [13:31](https://www.youtube.com/watch?v=hX140a5klW0&t=811s) from doing work faster. Uh and the next [13:31](https://www.youtube.com/watch?v=hX140a5klW0&t=811s) from doing work faster. Uh and the next optimization we do is how do we break [13:33](https://www.youtube.com/watch?v=hX140a5klW0&t=813s) optimization we do is how do we break [13:33](https://www.youtube.com/watch?v=hX140a5klW0&t=813s) optimization we do is how do we break this? Is there a way where we can break [13:35](https://www.youtube.com/watch?v=hX140a5klW0&t=815s) this? Is there a way where we can break [13:35](https://www.youtube.com/watch?v=hX140a5klW0&t=815s) this? Is there a way where we can break this up in a smart way? Um next slide. [13:39](https://www.youtube.com/watch?v=hX140a5klW0&t=819s) this up in a smart way? Um next slide. [13:39](https://www.youtube.com/watch?v=hX140a5klW0&t=819s) this up in a smart way? Um next slide. So conflict relaxation is what helps us [13:42](https://www.youtube.com/watch?v=hX140a5klW0&t=822s) So conflict relaxation is what helps us [13:42](https://www.youtube.com/watch?v=hX140a5klW0&t=822s) So conflict relaxation is what helps us break these links uh in an informed [13:45](https://www.youtube.com/watch?v=hX140a5klW0&t=825s) break these links uh in an informed [13:45](https://www.youtube.com/watch?v=hX140a5klW0&t=825s) break these links uh in an informed fashion. Uh before is basically the [13:48](https://www.youtube.com/watch?v=hX140a5klW0&t=828s) fashion. Uh before is basically the [13:48](https://www.youtube.com/watch?v=hX140a5klW0&t=828s) fashion. Uh before is basically the status quo. Any relevant overlap between [13:52](https://www.youtube.com/watch?v=hX140a5klW0&t=832s) status quo. Any relevant overlap between [13:52](https://www.youtube.com/watch?v=hX140a5klW0&t=832s) status quo. Any relevant overlap between two requests uh makes them conflict with [13:55](https://www.youtube.com/watch?v=hX140a5klW0&t=835s) two requests uh makes them conflict with [13:55](https://www.youtube.com/watch?v=hX140a5klW0&t=835s) two requests uh makes them conflict with each other and makes uh them serialize [13:58](https://www.youtube.com/watch?v=hX140a5klW0&t=838s) each other and makes uh them serialize [13:58](https://www.youtube.com/watch?v=hX140a5klW0&t=838s) each other and makes uh them serialize one behind the other. But with [14:00](https://www.youtube.com/watch?v=hX140a5klW0&t=840s) one behind the other. But with [14:00](https://www.youtube.com/watch?v=hX140a5klW0&t=840s) one behind the other. But with relaxation we essentially do three [14:03](https://www.youtube.com/watch?v=hX140a5klW0&t=843s) relaxation we essentially do three [14:03](https://www.youtube.com/watch?v=hX140a5klW0&t=843s) relaxation we essentially do three steps. First we rank the overlap. So [14:05](https://www.youtube.com/watch?v=hX140a5klW0&t=845s) steps. First we rank the overlap. So [14:05](https://www.youtube.com/watch?v=hX140a5klW0&t=845s) steps. First we rank the overlap. So it's no longer a binary decision of is [14:09](https://www.youtube.com/watch?v=hX140a5klW0&t=849s) it's no longer a binary decision of is [14:09](https://www.youtube.com/watch?v=hX140a5klW0&t=849s) it's no longer a binary decision of is there at least one overlap between these [14:11](https://www.youtube.com/watch?v=hX140a5klW0&t=851s) there at least one overlap between these [14:11](https://www.youtube.com/watch?v=hX140a5klW0&t=851s) there at least one overlap between these two requests. Think of it more as a how [14:15](https://www.youtube.com/watch?v=hX140a5klW0&t=855s) two requests. Think of it more as a how [14:15](https://www.youtube.com/watch?v=hX140a5klW0&t=855s) two requests. Think of it more as a how strong is this overlap. Uh once we have [14:18](https://www.youtube.com/watch?v=hX140a5klW0&t=858s) strong is this overlap. Uh once we have [14:18](https://www.youtube.com/watch?v=hX140a5klW0&t=858s) strong is this overlap. Uh once we have that we keep the stronger links and [14:21](https://www.youtube.com/watch?v=hX140a5klW0&t=861s) that we keep the stronger links and [14:21](https://www.youtube.com/watch?v=hX140a5klW0&t=861s) that we keep the stronger links and relax the weaker ones to buy back [14:24](https://www.youtube.com/watch?v=hX140a5klW0&t=864s) relax the weaker ones to buy back [14:24](https://www.youtube.com/watch?v=hX140a5klW0&t=864s) relax the weaker ones to buy back parallelism based on a config value that [14:27](https://www.youtube.com/watch?v=hX140a5klW0&t=867s) parallelism based on a config value that [14:28](https://www.youtube.com/watch?v=hX140a5klW0&t=868s) parallelism based on a config value that we set. Um next slide. So the first step [14:31](https://www.youtube.com/watch?v=hX140a5klW0&t=871s) we set. Um next slide. So the first step [14:31](https://www.youtube.com/watch?v=hX140a5klW0&t=871s) we set. Um next slide. So the first step is essentially scoring these edges uh or [14:34](https://www.youtube.com/watch?v=hX140a5klW0&t=874s) is essentially scoring these edges uh or [14:34](https://www.youtube.com/watch?v=hX140a5klW0&t=874s) is essentially scoring these edges uh or these overlaps. We use a jakard scoring [14:37](https://www.youtube.com/watch?v=hX140a5klW0&t=877s) these overlaps. We use a jakard scoring [14:37](https://www.youtube.com/watch?v=hX140a5klW0&t=877s) these overlaps. We use a jakard scoring uh for computing this dependency [14:39](https://www.youtube.com/watch?v=hX140a5klW0&t=879s) uh for computing this dependency [14:39](https://www.youtube.com/watch?v=hX140a5klW0&t=879s) uh for computing this dependency overlap. We basically have overlapping [14:41](https://www.youtube.com/watch?v=hX140a5klW0&t=881s) overlap. We basically have overlapping [14:41](https://www.youtube.com/watch?v=hX140a5klW0&t=881s) overlap. We basically have overlapping targets. Uh we check for that and any [14:44](https://www.youtube.com/watch?v=hX140a5klW0&t=884s) targets. Uh we check for that and any [14:44](https://www.youtube.com/watch?v=hX140a5klW0&t=884s) targets. Uh we check for that and any new dependency edges that are [14:46](https://www.youtube.com/watch?v=hX140a5klW0&t=886s) new dependency edges that are [14:46](https://www.youtube.com/watch?v=hX140a5klW0&t=886s) new dependency edges that are introduced. We compute a combination of [14:49](https://www.youtube.com/watch?v=hX140a5klW0&t=889s) introduced. We compute a combination of [14:49](https://www.youtube.com/watch?v=hX140a5klW0&t=889s) introduced. We compute a combination of these two into a continuous valued score [14:51](https://www.youtube.com/watch?v=hX140a5klW0&t=891s) these two into a continuous valued score [14:51](https://www.youtube.com/watch?v=hX140a5klW0&t=891s) these two into a continuous valued score between zero and one where a higher [14:54](https://www.youtube.com/watch?v=hX140a5klW0&t=894s) between zero and one where a higher [14:54](https://www.youtube.com/watch?v=hX140a5klW0&t=894s) between zero and one where a higher value of score indicates a stronger [14:57](https://www.youtube.com/watch?v=hX140a5klW0&t=897s) value of score indicates a stronger [14:57](https://www.youtube.com/watch?v=hX140a5klW0&t=897s) value of score indicates a stronger overlap and a lower value of the score [15:00](https://www.youtube.com/watch?v=hX140a5klW0&t=900s) overlap and a lower value of the score [15:00](https://www.youtube.com/watch?v=hX140a5klW0&t=900s) overlap and a lower value of the score indicates a weaker edge with fewer [15:03](https://www.youtube.com/watch?v=hX140a5klW0&t=903s) indicates a weaker edge with fewer [15:03](https://www.youtube.com/watch?v=hX140a5klW0&t=903s) indicates a weaker edge with fewer targets that overlap between requests. [15:06](https://www.youtube.com/watch?v=hX140a5klW0&t=906s) targets that overlap between requests. [15:06](https://www.youtube.com/watch?v=hX140a5klW0&t=906s) targets that overlap between requests. Next slide. Once we are armed with this [15:08](https://www.youtube.com/watch?v=hX140a5klW0&t=908s) Next slide. Once we are armed with this [15:08](https://www.youtube.com/watch?v=hX140a5klW0&t=908s) Next slide. Once we are armed with this knowledge, we basically uh make this [15:11](https://www.youtube.com/watch?v=hX140a5klW0&t=911s) knowledge, we basically uh make this [15:11](https://www.youtube.com/watch?v=hX140a5klW0&t=911s) knowledge, we basically uh make this conflict graph again. Right? So taking [15:13](https://www.youtube.com/watch?v=hX140a5klW0&t=913s) conflict graph again. Right? So taking [15:13](https://www.youtube.com/watch?v=hX140a5klW0&t=913s) conflict graph again. Right? So taking the same example and adding some numbers [15:16](https://www.youtube.com/watch?v=hX140a5klW0&t=916s) the same example and adding some numbers [15:16](https://www.youtube.com/watch?v=hX140a5klW0&t=916s) the same example and adding some numbers to the uh conflicts that we have uh from [15:19](https://www.youtube.com/watch?v=hX140a5klW0&t=919s) to the uh conflicts that we have uh from [15:19](https://www.youtube.com/watch?v=hX140a5klW0&t=919s) to the uh conflicts that we have uh from the earlier slide. Let's say we have [15:21](https://www.youtube.com/watch?v=hX140a5klW0&t=921s) the earlier slide. Let's say we have [15:21](https://www.youtube.com/watch?v=hX140a5klW0&t=921s) the earlier slide. Let's say we have these six changes and the config value [15:24](https://www.youtube.com/watch?v=hX140a5klW0&t=924s) these six changes and the config value [15:24](https://www.youtube.com/watch?v=hX140a5klW0&t=924s) these six changes and the config value we set says we want at least 50% [15:27](https://www.youtube.com/watch?v=hX140a5klW0&t=927s) we set says we want at least 50% [15:27](https://www.youtube.com/watch?v=hX140a5klW0&t=927s) we set says we want at least 50% parallelism in the queue at all times. [15:29](https://www.youtube.com/watch?v=hX140a5klW0&t=929s) parallelism in the queue at all times. [15:29](https://www.youtube.com/watch?v=hX140a5klW0&t=929s) parallelism in the queue at all times. What this means is that we need at least [15:33](https://www.youtube.com/watch?v=hX140a5klW0&t=933s) What this means is that we need at least [15:33](https://www.youtube.com/watch?v=hX140a5klW0&t=933s) What this means is that we need at least three separate cues that are working uh [15:37](https://www.youtube.com/watch?v=hX140a5klW0&t=937s) three separate cues that are working uh [15:37](https://www.youtube.com/watch?v=hX140a5klW0&t=937s) three separate cues that are working uh independent of each other and based on [15:41](https://www.youtube.com/watch?v=hX140a5klW0&t=941s) independent of each other and based on [15:41](https://www.youtube.com/watch?v=hX140a5klW0&t=941s) independent of each other and based on the u overlap scores that we have [15:45](https://www.youtube.com/watch?v=hX140a5klW0&t=945s) the u overlap scores that we have [15:45](https://www.youtube.com/watch?v=hX140a5klW0&t=945s) the u overlap scores that we have computed we are able to make this choice [15:48](https://www.youtube.com/watch?v=hX140a5klW0&t=948s) computed we are able to make this choice [15:48](https://www.youtube.com/watch?v=hX140a5klW0&t=948s) computed we are able to make this choice of dropping certain edges in order to be [15:51](https://www.youtube.com/watch?v=hX140a5klW0&t=951s) of dropping certain edges in order to be [15:51](https://www.youtube.com/watch?v=hX140a5klW0&t=951s) of dropping certain edges in order to be able to achieve that target uh Q size [15:55](https://www.youtube.com/watch?v=hX140a5klW0&t=955s) able to achieve that target uh Q size [15:55](https://www.youtube.com/watch?v=hX140a5klW0&t=955s) able to achieve that target uh Q size that we have. So in this case uh the [15:57](https://www.youtube.com/watch?v=hX140a5klW0&t=957s) that we have. So in this case uh the [15:57](https://www.youtube.com/watch?v=hX140a5klW0&t=957s) that we have. So in this case uh the weakest edges are between B and C and B [16:00](https://www.youtube.com/watch?v=hX140a5klW0&t=960s) weakest edges are between B and C and B [16:00](https://www.youtube.com/watch?v=hX140a5klW0&t=960s) weakest edges are between B and C and B and E and we drop those two which gives [16:03](https://www.youtube.com/watch?v=hX140a5klW0&t=963s) and E and we drop those two which gives [16:03](https://www.youtube.com/watch?v=hX140a5klW0&t=963s) and E and we drop those two which gives us three separate connected components. [16:06](https://www.youtube.com/watch?v=hX140a5klW0&t=966s) us three separate connected components. [16:06](https://www.youtube.com/watch?v=hX140a5klW0&t=966s) us three separate connected components. Uh and this is how we essentially go [16:09](https://www.youtube.com/watch?v=hX140a5klW0&t=969s) Uh and this is how we essentially go [16:09](https://www.youtube.com/watch?v=hX140a5klW0&t=969s) Uh and this is how we essentially go from a serial chain of six requests to a [16:14](https://www.youtube.com/watch?v=hX140a5klW0&t=974s) from a serial chain of six requests to a [16:14](https://www.youtube.com/watch?v=hX140a5klW0&t=974s) from a serial chain of six requests to a parallelized chain where two each exist [16:16](https://www.youtube.com/watch?v=hX140a5klW0&t=976s) parallelized chain where two each exist [16:16](https://www.youtube.com/watch?v=hX140a5klW0&t=976s) parallelized chain where two each exist in three separate cues. [16:19](https://www.youtube.com/watch?v=hX140a5klW0&t=979s) in three separate cues. [16:19](https://www.youtube.com/watch?v=hX140a5klW0&t=979s) in three separate cues. Next slide. So next comes a question of [16:23](https://www.youtube.com/watch?v=hX140a5klW0&t=983s) Next slide. So next comes a question of [16:23](https://www.youtube.com/watch?v=hX140a5klW0&t=983s) Next slide. So next comes a question of um where exactly do we make this [16:25](https://www.youtube.com/watch?v=hX140a5klW0&t=985s) um where exactly do we make this [16:25](https://www.youtube.com/watch?v=hX140a5klW0&t=985s) um where exactly do we make this determination of uh dropping the edges [16:29](https://www.youtube.com/watch?v=hX140a5klW0&t=989s) determination of uh dropping the edges [16:29](https://www.youtube.com/watch?v=hX140a5klW0&t=989s) determination of uh dropping the edges right so there are two places where we [16:31](https://www.youtube.com/watch?v=hX140a5klW0&t=991s) right so there are two places where we [16:31](https://www.youtube.com/watch?v=hX140a5klW0&t=991s) right so there are two places where we can do this uh one is at a global level [16:35](https://www.youtube.com/watch?v=hX140a5klW0&t=995s) can do this uh one is at a global level [16:35](https://www.youtube.com/watch?v=hX140a5klW0&t=995s) can do this uh one is at a global level every time uh we are trying to assign uh [16:38](https://www.youtube.com/watch?v=hX140a5klW0&t=998s) every time uh we are trying to assign uh [16:38](https://www.youtube.com/watch?v=hX140a5klW0&t=998s) every time uh we are trying to assign uh config graphs and building them uh we [16:41](https://www.youtube.com/watch?v=hX140a5klW0&t=1001s) config graphs and building them uh we [16:41](https://www.youtube.com/watch?v=hX140a5klW0&t=1001s) config graphs and building them uh we can basically recomputee this every time [16:44](https://www.youtube.com/watch?v=hX140a5klW0&t=1004s) can basically recomputee this every time [16:44](https://www.youtube.com/watch?v=hX140a5klW0&t=1004s) can basically recomputee this every time and what that does is it introduces some [16:48](https://www.youtube.com/watch?v=hX140a5klW0&t=1008s) and what that does is it introduces some [16:48](https://www.youtube.com/watch?v=hX140a5klW0&t=1008s) and what that does is it introduces some level of instability into the graph [16:51](https://www.youtube.com/watch?v=hX140a5klW0&t=1011s) level of instability into the graph [16:51](https://www.youtube.com/watch?v=hX140a5klW0&t=1011s) level of instability into the graph which is determined by any new requests [16:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1013s) which is determined by any new requests [16:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1013s) which is determined by any new requests that enter the queue, any work that [16:56](https://www.youtube.com/watch?v=hX140a5klW0&t=1016s) that enter the queue, any work that [16:56](https://www.youtube.com/watch?v=hX140a5klW0&t=1016s) that enter the queue, any work that finishes and exits the queue. So uh [17:00](https://www.youtube.com/watch?v=hX140a5klW0&t=1020s) finishes and exits the queue. So uh [17:00](https://www.youtube.com/watch?v=hX140a5klW0&t=1020s) finishes and exits the queue. So uh essentially what we we can do that uh [17:02](https://www.youtube.com/watch?v=hX140a5klW0&t=1022s) essentially what we we can do that uh [17:02](https://www.youtube.com/watch?v=hX140a5klW0&t=1022s) essentially what we we can do that uh but what we chose to do instead was a [17:05](https://www.youtube.com/watch?v=hX140a5klW0&t=1025s) but what we chose to do instead was a [17:05](https://www.youtube.com/watch?v=hX140a5klW0&t=1025s) but what we chose to do instead was a more uh local assignment uh of overlaps [17:09](https://www.youtube.com/watch?v=hX140a5klW0&t=1029s) more uh local assignment uh of overlaps [17:09](https://www.youtube.com/watch?v=hX140a5klW0&t=1029s) more uh local assignment uh of overlaps and the place that we choose to do this [17:12](https://www.youtube.com/watch?v=hX140a5klW0&t=1032s) and the place that we choose to do this [17:12](https://www.youtube.com/watch?v=hX140a5klW0&t=1032s) and the place that we choose to do this at is only at the new arrival uh [17:15](https://www.youtube.com/watch?v=hX140a5klW0&t=1035s) at is only at the new arrival uh [17:15](https://www.youtube.com/watch?v=hX140a5klW0&t=1035s) at is only at the new arrival uh checkpoint. So once a new request comes [17:19](https://www.youtube.com/watch?v=hX140a5klW0&t=1039s) checkpoint. So once a new request comes [17:19](https://www.youtube.com/watch?v=hX140a5klW0&t=1039s) checkpoint. So once a new request comes in, we essentially take that and compute [17:21](https://www.youtube.com/watch?v=hX140a5klW0&t=1041s) in, we essentially take that and compute [17:22](https://www.youtube.com/watch?v=hX140a5klW0&t=1042s) in, we essentially take that and compute the edges of this with everything that [17:24](https://www.youtube.com/watch?v=hX140a5klW0&t=1044s) the edges of this with everything that [17:24](https://www.youtube.com/watch?v=hX140a5klW0&t=1044s) the edges of this with everything that exists in the queue, make the decisions [17:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1047s) exists in the queue, make the decisions [17:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1047s) exists in the queue, make the decisions and that's how it stays. The next [17:30](https://www.youtube.com/watch?v=hX140a5klW0&t=1050s) and that's how it stays. The next [17:30](https://www.youtube.com/watch?v=hX140a5klW0&t=1050s) and that's how it stays. The next decision making will only happen when [17:32](https://www.youtube.com/watch?v=hX140a5klW0&t=1052s) decision making will only happen when [17:32](https://www.youtube.com/watch?v=hX140a5klW0&t=1052s) decision making will only happen when another new arrival comes in. So the [17:35](https://www.youtube.com/watch?v=hX140a5klW0&t=1055s) another new arrival comes in. So the [17:35](https://www.youtube.com/watch?v=hX140a5klW0&t=1055s) another new arrival comes in. So the concept is simple. We assign once at [17:37](https://www.youtube.com/watch?v=hX140a5klW0&t=1057s) concept is simple. We assign once at [17:37](https://www.youtube.com/watch?v=hX140a5klW0&t=1057s) concept is simple. We assign once at entry time uh and basically reuse this. [17:40](https://www.youtube.com/watch?v=hX140a5klW0&t=1060s) entry time uh and basically reuse this. [17:40](https://www.youtube.com/watch?v=hX140a5klW0&t=1060s) entry time uh and basically reuse this. Um what this helps us do is any work [17:43](https://www.youtube.com/watch?v=hX140a5klW0&t=1063s) Um what this helps us do is any work [17:43](https://www.youtube.com/watch?v=hX140a5klW0&t=1063s) Um what this helps us do is any work that we schedule based on previous [17:47](https://www.youtube.com/watch?v=hX140a5klW0&t=1067s) that we schedule based on previous [17:47](https://www.youtube.com/watch?v=hX140a5klW0&t=1067s) that we schedule based on previous conflicts gra conflict graphs that we [17:49](https://www.youtube.com/watch?v=hX140a5klW0&t=1069s) conflicts gra conflict graphs that we [17:49](https://www.youtube.com/watch?v=hX140a5klW0&t=1069s) conflicts gra conflict graphs that we have created is preserved uh and it [17:52](https://www.youtube.com/watch?v=hX140a5klW0&t=1072s) have created is preserved uh and it [17:52](https://www.youtube.com/watch?v=hX140a5klW0&t=1072s) have created is preserved uh and it helps us hang on to speculative builds [17:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1075s) helps us hang on to speculative builds [17:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1075s) helps us hang on to speculative builds that can help us move faster uh towards [17:58](https://www.youtube.com/watch?v=hX140a5klW0&t=1078s) that can help us move faster uh towards [17:58](https://www.youtube.com/watch?v=hX140a5klW0&t=1078s) that can help us move faster uh towards a finalization of request. [18:01](https://www.youtube.com/watch?v=hX140a5klW0&t=1081s) a finalization of request. [18:01](https://www.youtube.com/watch?v=hX140a5klW0&t=1081s) a finalization of request. Next slide. So the tradeoff of this [18:03](https://www.youtube.com/watch?v=hX140a5klW0&t=1083s) Next slide. So the tradeoff of this [18:03](https://www.youtube.com/watch?v=hX140a5klW0&t=1083s) Next slide. So the tradeoff of this final um intervention was the biggest [18:07](https://www.youtube.com/watch?v=hX140a5klW0&t=1087s) final um intervention was the biggest [18:07](https://www.youtube.com/watch?v=hX140a5klW0&t=1087s) final um intervention was the biggest that we saw. uh we saw over 50% increase [18:11](https://www.youtube.com/watch?v=hX140a5klW0&t=1091s) that we saw. uh we saw over 50% increase [18:11](https://www.youtube.com/watch?v=hX140a5klW0&t=1091s) that we saw. uh we saw over 50% increase in the throughput. So we went from about [18:13](https://www.youtube.com/watch?v=hX140a5klW0&t=1093s) in the throughput. So we went from about [18:13](https://www.youtube.com/watch?v=hX140a5klW0&t=1093s) in the throughput. So we went from about 160 lines per hour to almost 250 [18:17](https://www.youtube.com/watch?v=hX140a5klW0&t=1097s) 160 lines per hour to almost 250 [18:17](https://www.youtube.com/watch?v=hX140a5klW0&t=1097s) 160 lines per hour to almost 250 sometimes even greater uh during peak [18:19](https://www.youtube.com/watch?v=hX140a5klW0&t=1099s) sometimes even greater uh during peak [18:19](https://www.youtube.com/watch?v=hX140a5klW0&t=1099s) sometimes even greater uh during peak hours. Uh so it worked in that it not [18:22](https://www.youtube.com/watch?v=hX140a5klW0&t=1102s) hours. Uh so it worked in that it not [18:22](https://www.youtube.com/watch?v=hX140a5klW0&t=1102s) hours. Uh so it worked in that it not only helped us uh significantly improve [18:25](https://www.youtube.com/watch?v=hX140a5klW0&t=1105s) only helped us uh significantly improve [18:25](https://www.youtube.com/watch?v=hX140a5klW0&t=1105s) only helped us uh significantly improve our throughputs while giving us uh [18:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1109s) our throughputs while giving us uh [18:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1109s) our throughputs while giving us uh extremely good uh P95 land times but it [18:33](https://www.youtube.com/watch?v=hX140a5klW0&t=1113s) extremely good uh P95 land times but it [18:33](https://www.youtube.com/watch?v=hX140a5klW0&t=1113s) extremely good uh P95 land times but it came at a cost as everything does and [18:36](https://www.youtube.com/watch?v=hX140a5klW0&t=1116s) came at a cost as everything does and [18:36](https://www.youtube.com/watch?v=hX140a5klW0&t=1116s) came at a cost as everything does and the trade-off that we have given away [18:38](https://www.youtube.com/watch?v=hX140a5klW0&t=1118s) the trade-off that we have given away [18:38](https://www.youtube.com/watch?v=hX140a5klW0&t=1118s) the trade-off that we have given away here is basically what Prium alluded to [18:41](https://www.youtube.com/watch?v=hX140a5klW0&t=1121s) here is basically what Prium alluded to [18:41](https://www.youtube.com/watch?v=hX140a5klW0&t=1121s) here is basically what Prium alluded to in the very first slide uh talking about [18:43](https://www.youtube.com/watch?v=hX140a5klW0&t=1123s) in the very first slide uh talking about [18:43](https://www.youtube.com/watch?v=hX140a5klW0&t=1123s) in the very first slide uh talking about the integration testing some [18:46](https://www.youtube.com/watch?v=hX140a5klW0&t=1126s) the integration testing some [18:46](https://www.youtube.com/watch?v=hX140a5klW0&t=1126s) the integration testing some combinations of changes will land [18:49](https://www.youtube.com/watch?v=hX140a5klW0&t=1129s) combinations of changes will land [18:49](https://www.youtube.com/watch?v=hX140a5klW0&t=1129s) combinations of changes will land untested [18:50](https://www.youtube.com/watch?v=hX140a5klW0&t=1130s) untested [18:50](https://www.youtube.com/watch?v=hX140a5klW0&t=1130s) untested because we are dropping edges from our [18:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1133s) because we are dropping edges from our [18:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1133s) because we are dropping edges from our conflict graph. What this means is main [18:56](https://www.youtube.com/watch?v=hX140a5klW0&t=1136s) conflict graph. What this means is main [18:56](https://www.youtube.com/watch?v=hX140a5klW0&t=1136s) conflict graph. What this means is main can now be broken. Uh and this creates [19:00](https://www.youtube.com/watch?v=hX140a5klW0&t=1140s) can now be broken. Uh and this creates [19:00](https://www.youtube.com/watch?v=hX140a5klW0&t=1140s) can now be broken. Uh and this creates essentially a new obligation where we [19:03](https://www.youtube.com/watch?v=hX140a5klW0&t=1143s) essentially a new obligation where we [19:04](https://www.youtube.com/watch?v=hX140a5klW0&t=1144s) essentially a new obligation where we have to validate uh what lands after it [19:07](https://www.youtube.com/watch?v=hX140a5klW0&t=1147s) have to validate uh what lands after it [19:08](https://www.youtube.com/watch?v=hX140a5klW0&t=1148s) have to validate uh what lands after it has landed. Uh and to talk a little bit [19:10](https://www.youtube.com/watch?v=hX140a5klW0&t=1150s) has landed. Uh and to talk a little bit [19:10](https://www.youtube.com/watch?v=hX140a5klW0&t=1150s) has landed. Uh and to talk a little bit more about how we do this, I will hand [19:13](https://www.youtube.com/watch?v=hX140a5klW0&t=1153s) more about how we do this, I will hand [19:13](https://www.youtube.com/watch?v=hX140a5klW0&t=1153s) more about how we do this, I will hand it back to Kam. [19:15](https://www.youtube.com/watch?v=hX140a5klW0&t=1155s) it back to Kam. [19:15](https://www.youtube.com/watch?v=hX140a5klW0&t=1155s) it back to Kam. Great. Um thanks Mandre uh for walking [19:18](https://www.youtube.com/watch?v=hX140a5klW0&t=1158s) Great. Um thanks Mandre uh for walking [19:18](https://www.youtube.com/watch?v=hX140a5klW0&t=1158s) Great. Um thanks Mandre uh for walking us through the uh those three um sort of [19:22](https://www.youtube.com/watch?v=hX140a5klW0&t=1162s) us through the uh those three um sort of [19:22](https://www.youtube.com/watch?v=hX140a5klW0&t=1162s) us through the uh those three um sort of the solutions that we employed. So uh [19:25](https://www.youtube.com/watch?v=hX140a5klW0&t=1165s) the solutions that we employed. So uh [19:25](https://www.youtube.com/watch?v=hX140a5klW0&t=1165s) the solutions that we employed. So uh taking back this example that we had [19:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1167s) taking back this example that we had [19:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1167s) taking back this example that we had initially we had basically two changes [19:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1169s) initially we had basically two changes [19:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1169s) initially we had basically two changes here A and B and say we do a premerge [19:33](https://www.youtube.com/watch?v=hX140a5klW0&t=1173s) here A and B and say we do a premerge [19:33](https://www.youtube.com/watch?v=hX140a5klW0&t=1173s) here A and B and say we do a premerge validation independently after relaxing [19:35](https://www.youtube.com/watch?v=hX140a5klW0&t=1175s) validation independently after relaxing [19:36](https://www.youtube.com/watch?v=hX140a5klW0&t=1176s) validation independently after relaxing those uh conflict between those two [19:38](https://www.youtube.com/watch?v=hX140a5klW0&t=1178s) those uh conflict between those two [19:38](https://www.youtube.com/watch?v=hX140a5klW0&t=1178s) those uh conflict between those two things and then say we land it. So now [19:41](https://www.youtube.com/watch?v=hX140a5klW0&t=1181s) things and then say we land it. So now [19:41](https://www.youtube.com/watch?v=hX140a5klW0&t=1181s) things and then say we land it. So now our the landed state is essentially a [19:44](https://www.youtube.com/watch?v=hX140a5klW0&t=1184s) our the landed state is essentially a [19:44](https://www.youtube.com/watch?v=hX140a5klW0&t=1184s) our the landed state is essentially a lands first then b and now the [19:47](https://www.youtube.com/watch?v=hX140a5klW0&t=1187s) lands first then b and now the [19:47](https://www.youtube.com/watch?v=hX140a5klW0&t=1187s) lands first then b and now the combination of a plus b uh fails on the [19:50](https://www.youtube.com/watch?v=hX140a5klW0&t=1190s) combination of a plus b uh fails on the [19:50](https://www.youtube.com/watch?v=hX140a5klW0&t=1190s) combination of a plus b uh fails on the trunk. Right? So this is where we [19:52](https://www.youtube.com/watch?v=hX140a5klW0&t=1192s) trunk. Right? So this is where we [19:52](https://www.youtube.com/watch?v=hX140a5klW0&t=1192s) trunk. Right? So this is where we started um at the start of this talk and [19:56](https://www.youtube.com/watch?v=hX140a5klW0&t=1196s) started um at the start of this talk and [19:56](https://www.youtube.com/watch?v=hX140a5klW0&t=1196s) started um at the start of this talk and so now we have a broken trunk again. So [20:00](https://www.youtube.com/watch?v=hX140a5klW0&t=1200s) so now we have a broken trunk again. So [20:00](https://www.youtube.com/watch?v=hX140a5klW0&t=1200s) so now we have a broken trunk again. So now how are we dealing with this? [20:03](https://www.youtube.com/watch?v=hX140a5klW0&t=1203s) now how are we dealing with this? [20:03](https://www.youtube.com/watch?v=hX140a5klW0&t=1203s) now how are we dealing with this? So we have a system uh here called [20:06](https://www.youtube.com/watch?v=hX140a5klW0&t=1206s) So we have a system uh here called [20:06](https://www.youtube.com/watch?v=hX140a5klW0&t=1206s) So we have a system uh here called stowpipe and stowpipe is our like post [20:09](https://www.youtube.com/watch?v=hX140a5klW0&t=1209s) stowpipe and stowpipe is our like post [20:10](https://www.youtube.com/watch?v=hX140a5klW0&t=1210s) stowpipe and stowpipe is our like post uh land uh validation system. Um it's [20:13](https://www.youtube.com/watch?v=hX140a5klW0&t=1213s) uh land uh validation system. Um it's [20:13](https://www.youtube.com/watch?v=hX140a5klW0&t=1213s) uh land uh validation system. Um it's responsible for validating any commit [20:14](https://www.youtube.com/watch?v=hX140a5klW0&t=1214s) responsible for validating any commit [20:14](https://www.youtube.com/watch?v=hX140a5klW0&t=1214s) responsible for validating any commit that lands on to the main and then [20:17](https://www.youtube.com/watch?v=hX140a5klW0&t=1217s) that lands on to the main and then [20:17](https://www.youtube.com/watch?v=hX140a5klW0&t=1217s) that lands on to the main and then making sure that uh it it detects [20:20](https://www.youtube.com/watch?v=hX140a5klW0&t=1220s) making sure that uh it it detects [20:20](https://www.youtube.com/watch?v=hX140a5klW0&t=1220s) making sure that uh it it detects basically if bills are broken and if so [20:23](https://www.youtube.com/watch?v=hX140a5klW0&t=1223s) basically if bills are broken and if so [20:23](https://www.youtube.com/watch?v=hX140a5klW0&t=1223s) basically if bills are broken and if so then it creates a uh recovery uh [20:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1227s) then it creates a uh recovery uh [20:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1227s) then it creates a uh recovery uh strategy for that. Now uh here we have [20:31](https://www.youtube.com/watch?v=hX140a5klW0&t=1231s) strategy for that. Now uh here we have [20:31](https://www.youtube.com/watch?v=hX140a5klW0&t=1231s) strategy for that. Now uh here we have basically two things. One is every [20:34](https://www.youtube.com/watch?v=hX140a5klW0&t=1234s) basically two things. One is every [20:34](https://www.youtube.com/watch?v=hX140a5klW0&t=1234s) basically two things. One is every change that is being landed by summit Q [20:36](https://www.youtube.com/watch?v=hX140a5klW0&t=1236s) change that is being landed by summit Q [20:36](https://www.youtube.com/watch?v=hX140a5klW0&t=1236s) change that is being landed by summit Q um lands onto the main trunk and then we [20:39](https://www.youtube.com/watch?v=hX140a5klW0&t=1239s) um lands onto the main trunk and then we [20:39](https://www.youtube.com/watch?v=hX140a5klW0&t=1239s) um lands onto the main trunk and then we have the second reference here or the uh [20:42](https://www.youtube.com/watch?v=hX140a5klW0&t=1242s) have the second reference here or the uh [20:42](https://www.youtube.com/watch?v=hX140a5klW0&t=1242s) have the second reference here or the uh trunk um the pointer if you call is the [20:45](https://www.youtube.com/watch?v=hX140a5klW0&t=1245s) trunk um the pointer if you call is the [20:45](https://www.youtube.com/watch?v=hX140a5klW0&t=1245s) trunk um the pointer if you call is the verified main and the verified main is [20:47](https://www.youtube.com/watch?v=hX140a5klW0&t=1247s) verified main and the verified main is [20:47](https://www.youtube.com/watch?v=hX140a5klW0&t=1247s) verified main and the verified main is essentially the one that is being uh [20:49](https://www.youtube.com/watch?v=hX140a5klW0&t=1249s) essentially the one that is being uh [20:50](https://www.youtube.com/watch?v=hX140a5klW0&t=1250s) essentially the one that is being uh managed uh by store. So let's take an [20:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1253s) managed uh by store. So let's take an [20:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1253s) managed uh by store. So let's take an example to understand it better. So [20:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1255s) example to understand it better. So [20:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1255s) example to understand it better. So imagine we have this comet A u where [20:59](https://www.youtube.com/watch?v=hX140a5klW0&t=1259s) imagine we have this comet A u where [20:59](https://www.youtube.com/watch?v=hX140a5klW0&t=1259s) imagine we have this comet A u where main and verified main both are pointing [21:01](https://www.youtube.com/watch?v=hX140a5klW0&t=1261s) main and verified main both are pointing [21:01](https://www.youtube.com/watch?v=hX140a5klW0&t=1261s) main and verified main both are pointing to it. This is a good known comet. Uh [21:04](https://www.youtube.com/watch?v=hX140a5klW0&t=1264s) to it. This is a good known comet. Uh [21:04](https://www.youtube.com/watch?v=hX140a5klW0&t=1264s) to it. This is a good known comet. Uh this is a building uh this is green on [21:06](https://www.youtube.com/watch?v=hX140a5klW0&t=1266s) this is a building uh this is green on [21:06](https://www.youtube.com/watch?v=hX140a5klW0&t=1266s) this is a building uh this is green on the trunk. Now let's say we have uh [21:10](https://www.youtube.com/watch?v=hX140a5klW0&t=1270s) the trunk. Now let's say we have uh [21:10](https://www.youtube.com/watch?v=hX140a5klW0&t=1270s) the trunk. Now let's say we have uh another set of comet that land onto the [21:12](https://www.youtube.com/watch?v=hX140a5klW0&t=1272s) another set of comet that land onto the [21:12](https://www.youtube.com/watch?v=hX140a5klW0&t=1272s) another set of comet that land onto the main say B C and D. Now main moves to [21:16](https://www.youtube.com/watch?v=hX140a5klW0&t=1276s) main say B C and D. Now main moves to [21:16](https://www.youtube.com/watch?v=hX140a5klW0&t=1276s) main say B C and D. Now main moves to point uh to the D uh commit. So at this [21:20](https://www.youtube.com/watch?v=hX140a5klW0&t=1280s) point uh to the D uh commit. So at this [21:20](https://www.youtube.com/watch?v=hX140a5klW0&t=1280s) point uh to the D uh commit. So at this point a STO pipe is going to take this [21:22](https://www.youtube.com/watch?v=hX140a5klW0&t=1282s) point a STO pipe is going to take this [21:22](https://www.youtube.com/watch?v=hX140a5klW0&t=1282s) point a STO pipe is going to take this range of commits essentially B, C and D [21:25](https://www.youtube.com/watch?v=hX140a5klW0&t=1285s) range of commits essentially B, C and D [21:25](https://www.youtube.com/watch?v=hX140a5klW0&t=1285s) range of commits essentially B, C and D on top of A and validate this newly uh [21:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1289s) on top of A and validate this newly uh [21:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1289s) on top of A and validate this newly uh landed changes to see if this new state [21:32](https://www.youtube.com/watch?v=hX140a5klW0&t=1292s) landed changes to see if this new state [21:32](https://www.youtube.com/watch?v=hX140a5klW0&t=1292s) landed changes to see if this new state is still buildable and is still passing [21:35](https://www.youtube.com/watch?v=hX140a5klW0&t=1295s) is still buildable and is still passing [21:35](https://www.youtube.com/watch?v=hX140a5klW0&t=1295s) is still buildable and is still passing or not. In this case we can see it [21:38](https://www.youtube.com/watch?v=hX140a5klW0&t=1298s) or not. In this case we can see it [21:38](https://www.youtube.com/watch?v=hX140a5klW0&t=1298s) or not. In this case we can see it fails. And so now store pipe is not [21:41](https://www.youtube.com/watch?v=hX140a5klW0&t=1301s) fails. And so now store pipe is not [21:41](https://www.youtube.com/watch?v=hX140a5klW0&t=1301s) fails. And so now store pipe is not going to move the verified main pointer [21:43](https://www.youtube.com/watch?v=hX140a5klW0&t=1303s) going to move the verified main pointer [21:43](https://www.youtube.com/watch?v=hX140a5klW0&t=1303s) going to move the verified main pointer in this case. it will just keep it at a [21:45](https://www.youtube.com/watch?v=hX140a5klW0&t=1305s) in this case. it will just keep it at a [21:45](https://www.youtube.com/watch?v=hX140a5klW0&t=1305s) in this case. it will just keep it at a because that's what the good known last [21:48](https://www.youtube.com/watch?v=hX140a5klW0&t=1308s) because that's what the good known last [21:48](https://www.youtube.com/watch?v=hX140a5klW0&t=1308s) because that's what the good known last uh commit is on the trunk. Now over time [21:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1313s) uh commit is on the trunk. Now over time [21:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1313s) uh commit is on the trunk. Now over time more commits uh land onto the trunk and [21:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1315s) more commits uh land onto the trunk and [21:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1315s) more commits uh land onto the trunk and main moves to commit G. So now between [21:59](https://www.youtube.com/watch?v=hX140a5klW0&t=1319s) main moves to commit G. So now between [21:59](https://www.youtube.com/watch?v=hX140a5klW0&t=1319s) main moves to commit G. So now between uh the new state that we have G and the [22:02](https://www.youtube.com/watch?v=hX140a5klW0&t=1322s) uh the new state that we have G and the [22:02](https://www.youtube.com/watch?v=hX140a5klW0&t=1322s) uh the new state that we have G and the previously uh good known state which is [22:05](https://www.youtube.com/watch?v=hX140a5klW0&t=1325s) previously uh good known state which is [22:05](https://www.youtube.com/watch?v=hX140a5klW0&t=1325s) previously uh good known state which is A we have more commits in between. So we [22:08](https://www.youtube.com/watch?v=hX140a5klW0&t=1328s) A we have more commits in between. So we [22:08](https://www.youtube.com/watch?v=hX140a5klW0&t=1328s) A we have more commits in between. So we have B C D E F and G commits in between. [22:12](https://www.youtube.com/watch?v=hX140a5klW0&t=1332s) have B C D E F and G commits in between. [22:12](https://www.youtube.com/watch?v=hX140a5klW0&t=1332s) have B C D E F and G commits in between. So now store pipe is going to take those [22:15](https://www.youtube.com/watch?v=hX140a5klW0&t=1335s) So now store pipe is going to take those [22:15](https://www.youtube.com/watch?v=hX140a5klW0&t=1335s) So now store pipe is going to take those commit uh range and basically validate [22:18](https://www.youtube.com/watch?v=hX140a5klW0&t=1338s) commit uh range and basically validate [22:18](https://www.youtube.com/watch?v=hX140a5klW0&t=1338s) commit uh range and basically validate and see if those are buildable and if [22:20](https://www.youtube.com/watch?v=hX140a5klW0&t=1340s) and see if those are buildable and if [22:20](https://www.youtube.com/watch?v=hX140a5klW0&t=1340s) and see if those are buildable and if bills are passing. So now say in this [22:24](https://www.youtube.com/watch?v=hX140a5klW0&t=1344s) bills are passing. So now say in this [22:24](https://www.youtube.com/watch?v=hX140a5klW0&t=1344s) bills are passing. So now say in this case builds are passing and so the store [22:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1347s) case builds are passing and so the store [22:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1347s) case builds are passing and so the store pipe will basically take the verified [22:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1349s) pipe will basically take the verified [22:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1349s) pipe will basically take the verified main pointer and point it to the newly [22:32](https://www.youtube.com/watch?v=hX140a5klW0&t=1352s) main pointer and point it to the newly [22:32](https://www.youtube.com/watch?v=hX140a5klW0&t=1352s) main pointer and point it to the newly uh validated uh good known uh commit uh [22:36](https://www.youtube.com/watch?v=hX140a5klW0&t=1356s) uh validated uh good known uh commit uh [22:36](https://www.youtube.com/watch?v=hX140a5klW0&t=1356s) uh validated uh good known uh commit uh g in this case. So that's what a store [22:38](https://www.youtube.com/watch?v=hX140a5klW0&t=1358s) g in this case. So that's what a store [22:38](https://www.youtube.com/watch?v=hX140a5klW0&t=1358s) g in this case. So that's what a store pipe does is it is essentially observing [22:41](https://www.youtube.com/watch?v=hX140a5klW0&t=1361s) pipe does is it is essentially observing [22:41](https://www.youtube.com/watch?v=hX140a5klW0&t=1361s) pipe does is it is essentially observing the newly uh landed commits onto the [22:43](https://www.youtube.com/watch?v=hX140a5klW0&t=1363s) the newly uh landed commits onto the [22:43](https://www.youtube.com/watch?v=hX140a5klW0&t=1363s) the newly uh landed commits onto the main uh verifying if they can build or [22:45](https://www.youtube.com/watch?v=hX140a5klW0&t=1365s) main uh verifying if they can build or [22:45](https://www.youtube.com/watch?v=hX140a5klW0&t=1365s) main uh verifying if they can build or not and if they pass move the pointer if [22:49](https://www.youtube.com/watch?v=hX140a5klW0&t=1369s) not and if they pass move the pointer if [22:49](https://www.youtube.com/watch?v=hX140a5klW0&t=1369s) not and if they pass move the pointer if not then uh continuously look for more [22:52](https://www.youtube.com/watch?v=hX140a5klW0&t=1372s) not then uh continuously look for more [22:52](https://www.youtube.com/watch?v=hX140a5klW0&t=1372s) not then uh continuously look for more changes and keep validating. [22:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1375s) changes and keep validating. [22:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1375s) changes and keep validating. So now let's look at like when uh a [22:59](https://www.youtube.com/watch?v=hX140a5klW0&t=1379s) So now let's look at like when uh a [22:59](https://www.youtube.com/watch?v=hX140a5klW0&t=1379s) So now let's look at like when uh a comet landed on main u breaks the trunk. [23:02](https://www.youtube.com/watch?v=hX140a5klW0&t=1382s) comet landed on main u breaks the trunk. [23:02](https://www.youtube.com/watch?v=hX140a5klW0&t=1382s) comet landed on main u breaks the trunk. So what happens in that case as I said [23:05](https://www.youtube.com/watch?v=hX140a5klW0&t=1385s) So what happens in that case as I said [23:05](https://www.youtube.com/watch?v=hX140a5klW0&t=1385s) So what happens in that case as I said earlier stopipe is essentially [23:06](https://www.youtube.com/watch?v=hX140a5klW0&t=1386s) earlier stopipe is essentially [23:06](https://www.youtube.com/watch?v=hX140a5klW0&t=1386s) earlier stopipe is essentially validating and then detecting that it [23:09](https://www.youtube.com/watch?v=hX140a5klW0&t=1389s) validating and then detecting that it [23:09](https://www.youtube.com/watch?v=hX140a5klW0&t=1389s) validating and then detecting that it failed and on failure it basically [23:11](https://www.youtube.com/watch?v=hX140a5klW0&t=1391s) failed and on failure it basically [23:11](https://www.youtube.com/watch?v=hX140a5klW0&t=1391s) failed and on failure it basically alerts. So on call gets paged uh and as [23:15](https://www.youtube.com/watch?v=hX140a5klW0&t=1395s) alerts. So on call gets paged uh and as [23:15](https://www.youtube.com/watch?v=hX140a5klW0&t=1395s) alerts. So on call gets paged uh and as a response to this um broken trunk on [23:18](https://www.youtube.com/watch?v=hX140a5klW0&t=1398s) a response to this um broken trunk on [23:18](https://www.youtube.com/watch?v=hX140a5klW0&t=1398s) a response to this um broken trunk on call is responsible to prepare a fix or [23:21](https://www.youtube.com/watch?v=hX140a5klW0&t=1401s) call is responsible to prepare a fix or [23:21](https://www.youtube.com/watch?v=hX140a5klW0&t=1401s) call is responsible to prepare a fix or revert that change um get it removed and [23:24](https://www.youtube.com/watch?v=hX140a5klW0&t=1404s) revert that change um get it removed and [23:24](https://www.youtube.com/watch?v=hX140a5klW0&t=1404s) revert that change um get it removed and then land through summit cube and then [23:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1407s) then land through summit cube and then [23:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1407s) then land through summit cube and then eventually it merges back into the trunk [23:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1409s) eventually it merges back into the trunk [23:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1409s) eventually it merges back into the trunk and then a stow pipe continuously [23:32](https://www.youtube.com/watch?v=hX140a5klW0&t=1412s) and then a stow pipe continuously [23:32](https://www.youtube.com/watch?v=hX140a5klW0&t=1412s) and then a stow pipe continuously validates the new state to see if it has [23:34](https://www.youtube.com/watch?v=hX140a5klW0&t=1414s) validates the new state to see if it has [23:34](https://www.youtube.com/watch?v=hX140a5klW0&t=1414s) validates the new state to see if it has been fixed or not and then accordingly [23:37](https://www.youtube.com/watch?v=hX140a5klW0&t=1417s) been fixed or not and then accordingly [23:37](https://www.youtube.com/watch?v=hX140a5klW0&t=1417s) been fixed or not and then accordingly move the pointer um to the newly landed [23:40](https://www.youtube.com/watch?v=hX140a5klW0&t=1420s) move the pointer um to the newly landed [23:40](https://www.youtube.com/watch?v=hX140a5klW0&t=1420s) move the pointer um to the newly landed comet. [23:42](https://www.youtube.com/watch?v=hX140a5klW0&t=1422s) comet. [23:42](https://www.youtube.com/watch?v=hX140a5klW0&t=1422s) comet. So uh with all of this um I want to just [23:46](https://www.youtube.com/watch?v=hX140a5klW0&t=1426s) So uh with all of this um I want to just [23:46](https://www.youtube.com/watch?v=hX140a5klW0&t=1426s) So uh with all of this um I want to just recap on basically what we have [23:47](https://www.youtube.com/watch?v=hX140a5klW0&t=1427s) recap on basically what we have [23:48](https://www.youtube.com/watch?v=hX140a5klW0&t=1428s) recap on basically what we have achieved. So uh at peak times we are [23:51](https://www.youtube.com/watch?v=hX140a5klW0&t=1431s) achieved. So uh at peak times we are [23:51](https://www.youtube.com/watch?v=hX140a5klW0&t=1431s) achieved. So uh at peak times we are able to do roughly uh 300 uh lands per [23:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1435s) able to do roughly uh 300 uh lands per [23:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1435s) able to do roughly uh 300 uh lands per hour. we are still able to maintain our [23:58](https://www.youtube.com/watch?v=hX140a5klW0&t=1438s) hour. we are still able to maintain our [23:58](https://www.youtube.com/watch?v=hX140a5klW0&t=1438s) hour. we are still able to maintain our P95 land time to 30 minutes and this is [24:01](https://www.youtube.com/watch?v=hX140a5klW0&t=1441s) P95 land time to 30 minutes and this is [24:01](https://www.youtube.com/watch?v=hX140a5klW0&t=1441s) P95 land time to 30 minutes and this is essentially a 233% increase uh since the [24:05](https://www.youtube.com/watch?v=hX140a5klW0&t=1445s) essentially a 233% increase uh since the [24:06](https://www.youtube.com/watch?v=hX140a5klW0&t=1446s) essentially a 233% increase uh since the January uh numbers that we had which [24:08](https://www.youtube.com/watch?v=hX140a5klW0&t=1448s) January uh numbers that we had which [24:08](https://www.youtube.com/watch?v=hX140a5klW0&t=1448s) January uh numbers that we had which were like 90 lines uh per hour. [24:12](https://www.youtube.com/watch?v=hX140a5klW0&t=1452s) were like 90 lines uh per hour. [24:12](https://www.youtube.com/watch?v=hX140a5klW0&t=1452s) were like 90 lines uh per hour. So u with this u we are doing all this [24:16](https://www.youtube.com/watch?v=hX140a5klW0&t=1456s) So u with this u we are doing all this [24:16](https://www.youtube.com/watch?v=hX140a5klW0&t=1456s) So u with this u we are doing all this work um in the open u we have summit Q [24:19](https://www.youtube.com/watch?v=hX140a5klW0&t=1459s) work um in the open u we have summit Q [24:19](https://www.youtube.com/watch?v=hX140a5klW0&t=1459s) work um in the open u we have summit Q and store pipe being uh developed and uh [24:23](https://www.youtube.com/watch?v=hX140a5klW0&t=1463s) and store pipe being uh developed and uh [24:24](https://www.youtube.com/watch?v=hX140a5klW0&t=1464s) and store pipe being uh developed and uh everything is in the open source. So if [24:26](https://www.youtube.com/watch?v=hX140a5klW0&t=1466s) everything is in the open source. So if [24:26](https://www.youtube.com/watch?v=hX140a5klW0&t=1466s) everything is in the open source. So if you want to explore and contribute um do [24:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1469s) you want to explore and contribute um do [24:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1469s) you want to explore and contribute um do scan the QR code explore the project and [24:32](https://www.youtube.com/watch?v=hX140a5klW0&t=1472s) scan the QR code explore the project and [24:32](https://www.youtube.com/watch?v=hX140a5klW0&t=1472s) scan the QR code explore the project and uh feel feel free to uh contribute to [24:35](https://www.youtube.com/watch?v=hX140a5klW0&t=1475s) uh feel feel free to uh contribute to [24:35](https://www.youtube.com/watch?v=hX140a5klW0&t=1475s) uh feel feel free to uh contribute to the project. Um what are we doing next? [24:39](https://www.youtube.com/watch?v=hX140a5klW0&t=1479s) the project. Um what are we doing next? [24:39](https://www.youtube.com/watch?v=hX140a5klW0&t=1479s) the project. Um what are we doing next? And um as we have already done this work [24:42](https://www.youtube.com/watch?v=hX140a5klW0&t=1482s) And um as we have already done this work [24:42](https://www.youtube.com/watch?v=hX140a5klW0&t=1482s) And um as we have already done this work uh we are still uh looking ways to uh [24:46](https://www.youtube.com/watch?v=hX140a5klW0&t=1486s) uh we are still uh looking ways to uh [24:46](https://www.youtube.com/watch?v=hX140a5klW0&t=1486s) uh we are still uh looking ways to uh make it better uh optimize for [24:48](https://www.youtube.com/watch?v=hX140a5klW0&t=1488s) make it better uh optimize for [24:48](https://www.youtube.com/watch?v=hX140a5klW0&t=1488s) make it better uh optimize for throughput handle the uh the recovery [24:52](https://www.youtube.com/watch?v=hX140a5klW0&t=1492s) throughput handle the uh the recovery [24:52](https://www.youtube.com/watch?v=hX140a5klW0&t=1492s) throughput handle the uh the recovery process in better way and also like [24:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1495s) process in better way and also like [24:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1495s) process in better way and also like improve on our like how we how we sort [24:58](https://www.youtube.com/watch?v=hX140a5klW0&t=1498s) improve on our like how we how we sort [24:58](https://www.youtube.com/watch?v=hX140a5klW0&t=1498s) improve on our like how we how we sort of tag what is green or not. So as a [25:02](https://www.youtube.com/watch?v=hX140a5klW0&t=1502s) of tag what is green or not. So as a [25:02](https://www.youtube.com/watch?v=hX140a5klW0&t=1502s) of tag what is green or not. So as a first thing we are thinking about is on [25:03](https://www.youtube.com/watch?v=hX140a5klW0&t=1503s) first thing we are thinking about is on [25:04](https://www.youtube.com/watch?v=hX140a5klW0&t=1504s) first thing we are thinking about is on summit Q side basically batching uh [25:06](https://www.youtube.com/watch?v=hX140a5klW0&t=1506s) summit Q side basically batching uh [25:06](https://www.youtube.com/watch?v=hX140a5klW0&t=1506s) summit Q side basically batching uh related changes together so that way we [25:08](https://www.youtube.com/watch?v=hX140a5klW0&t=1508s) related changes together so that way we [25:08](https://www.youtube.com/watch?v=hX140a5klW0&t=1508s) related changes together so that way we run less number of builds and we can do [25:10](https://www.youtube.com/watch?v=hX140a5klW0&t=1510s) run less number of builds and we can do [25:10](https://www.youtube.com/watch?v=hX140a5klW0&t=1510s) run less number of builds and we can do more work uh with the same uh same [25:14](https://www.youtube.com/watch?v=hX140a5klW0&t=1514s) more work uh with the same uh same [25:14](https://www.youtube.com/watch?v=hX140a5klW0&t=1514s) more work uh with the same uh same amount of resources. Second is uh today [25:17](https://www.youtube.com/watch?v=hX140a5klW0&t=1517s) amount of resources. Second is uh today [25:17](https://www.youtube.com/watch?v=hX140a5klW0&t=1517s) amount of resources. Second is uh today we have a single pointer verified main [25:19](https://www.youtube.com/watch?v=hX140a5klW0&t=1519s) we have a single pointer verified main [25:19](https://www.youtube.com/watch?v=hX140a5klW0&t=1519s) we have a single pointer verified main which points whether a repository head [25:22](https://www.youtube.com/watch?v=hX140a5klW0&t=1522s) which points whether a repository head [25:22](https://www.youtube.com/watch?v=hX140a5klW0&t=1522s) which points whether a repository head is green or not or a given commit uh is [25:24](https://www.youtube.com/watch?v=hX140a5klW0&t=1524s) is green or not or a given commit uh is [25:24](https://www.youtube.com/watch?v=hX140a5klW0&t=1524s) is green or not or a given commit uh is green or not. We want to do better where [25:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1527s) green or not. We want to do better where [25:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1527s) green or not. We want to do better where basically we can say a given service is [25:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1529s) basically we can say a given service is [25:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1529s) basically we can say a given service is green, given project is green. So even [25:31](https://www.youtube.com/watch?v=hX140a5klW0&t=1531s) green, given project is green. So even [25:31](https://www.youtube.com/watch?v=hX140a5klW0&t=1531s) green, given project is green. So even if a comet on the trunk is broken, uh we [25:36](https://www.youtube.com/watch?v=hX140a5klW0&t=1536s) if a comet on the trunk is broken, uh we [25:36](https://www.youtube.com/watch?v=hX140a5klW0&t=1536s) if a comet on the trunk is broken, uh we can essentially tell which part of the [25:38](https://www.youtube.com/watch?v=hX140a5klW0&t=1538s) can essentially tell which part of the [25:38](https://www.youtube.com/watch?v=hX140a5klW0&t=1538s) can essentially tell which part of the trunk is broken and not all of it. So [25:40](https://www.youtube.com/watch?v=hX140a5klW0&t=1540s) trunk is broken and not all of it. So [25:40](https://www.youtube.com/watch?v=hX140a5klW0&t=1540s) trunk is broken and not all of it. So that would allow us to basically do a [25:43](https://www.youtube.com/watch?v=hX140a5klW0&t=1543s) that would allow us to basically do a [25:43](https://www.youtube.com/watch?v=hX140a5klW0&t=1543s) that would allow us to basically do a better um uh sort of rollouts of those [25:46](https://www.youtube.com/watch?v=hX140a5klW0&t=1546s) better um uh sort of rollouts of those [25:46](https://www.youtube.com/watch?v=hX140a5klW0&t=1546s) better um uh sort of rollouts of those services and changes um and we don't [25:49](https://www.youtube.com/watch?v=hX140a5klW0&t=1549s) services and changes um and we don't [25:49](https://www.youtube.com/watch?v=hX140a5klW0&t=1549s) services and changes um and we don't have to block everything. And the third [25:51](https://www.youtube.com/watch?v=hX140a5klW0&t=1551s) have to block everything. And the third [25:51](https://www.youtube.com/watch?v=hX140a5klW0&t=1551s) have to block everything. And the third one is basically uh as I said the the [25:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1553s) one is basically uh as I said the the [25:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1553s) one is basically uh as I said the the recovery process from a broken trunk is [25:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1555s) recovery process from a broken trunk is [25:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1555s) recovery process from a broken trunk is manual. So uh the most of the investment [25:58](https://www.youtube.com/watch?v=hX140a5klW0&t=1558s) manual. So uh the most of the investment [25:58](https://www.youtube.com/watch?v=hX140a5klW0&t=1558s) manual. So uh the most of the investment is around automating the fixes or [26:01](https://www.youtube.com/watch?v=hX140a5klW0&t=1561s) is around automating the fixes or [26:01](https://www.youtube.com/watch?v=hX140a5klW0&t=1561s) is around automating the fixes or reverting the fixes and then landing [26:03](https://www.youtube.com/watch?v=hX140a5klW0&t=1563s) reverting the fixes and then landing [26:03](https://www.youtube.com/watch?v=hX140a5klW0&t=1563s) reverting the fixes and then landing those uh to summit queue uh in automated [26:06](https://www.youtube.com/watch?v=hX140a5klW0&t=1566s) those uh to summit queue uh in automated [26:06](https://www.youtube.com/watch?v=hX140a5klW0&t=1566s) those uh to summit queue uh in automated way so that our time to recovery can be [26:09](https://www.youtube.com/watch?v=hX140a5klW0&t=1569s) way so that our time to recovery can be [26:09](https://www.youtube.com/watch?v=hX140a5klW0&t=1569s) way so that our time to recovery can be uh reduced and we can fix the broken [26:12](https://www.youtube.com/watch?v=hX140a5klW0&t=1572s) uh reduced and we can fix the broken [26:12](https://www.youtube.com/watch?v=hX140a5klW0&t=1572s) uh reduced and we can fix the broken trunks faster. [26:14](https://www.youtube.com/watch?v=hX140a5klW0&t=1574s) trunks faster. [26:14](https://www.youtube.com/watch?v=hX140a5klW0&t=1574s) trunks faster. And so as a closing point uh I want to [26:17](https://www.youtube.com/watch?v=hX140a5klW0&t=1577s) And so as a closing point uh I want to [26:17](https://www.youtube.com/watch?v=hX140a5klW0&t=1577s) And so as a closing point uh I want to call out that we chose a hybrid approach [26:20](https://www.youtube.com/watch?v=hX140a5klW0&t=1580s) call out that we chose a hybrid approach [26:20](https://www.youtube.com/watch?v=hX140a5klW0&t=1580s) call out that we chose a hybrid approach uh to validating the changes. Um one we [26:24](https://www.youtube.com/watch?v=hX140a5klW0&t=1584s) uh to validating the changes. Um one we [26:24](https://www.youtube.com/watch?v=hX140a5klW0&t=1584s) uh to validating the changes. Um one we are still doing the premerge um uh [26:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1587s) are still doing the premerge um uh [26:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1587s) are still doing the premerge um uh validations. Um we are still uh making [26:30](https://www.youtube.com/watch?v=hX140a5klW0&t=1590s) validations. Um we are still uh making [26:30](https://www.youtube.com/watch?v=hX140a5klW0&t=1590s) validations. Um we are still uh making sure that we test the changes uh where [26:34](https://www.youtube.com/watch?v=hX140a5klW0&t=1594s) sure that we test the changes uh where [26:34](https://www.youtube.com/watch?v=hX140a5klW0&t=1594s) sure that we test the changes uh where it matters and relax the changes uh [26:37](https://www.youtube.com/watch?v=hX140a5klW0&t=1597s) it matters and relax the changes uh [26:37](https://www.youtube.com/watch?v=hX140a5klW0&t=1597s) it matters and relax the changes uh where we can take that hit and valid [26:40](https://www.youtube.com/watch?v=hX140a5klW0&t=1600s) where we can take that hit and valid [26:40](https://www.youtube.com/watch?v=hX140a5klW0&t=1600s) where we can take that hit and valid validate them later uh after they land [26:42](https://www.youtube.com/watch?v=hX140a5klW0&t=1602s) validate them later uh after they land [26:42](https://www.youtube.com/watch?v=hX140a5klW0&t=1602s) validate them later uh after they land to the uh trunk and then we are running [26:45](https://www.youtube.com/watch?v=hX140a5klW0&t=1605s) to the uh trunk and then we are running [26:45](https://www.youtube.com/watch?v=hX140a5klW0&t=1605s) to the uh trunk and then we are running the whole exhaustive uh validation with [26:47](https://www.youtube.com/watch?v=hX140a5klW0&t=1607s) the whole exhaustive uh validation with [26:47](https://www.youtube.com/watch?v=hX140a5klW0&t=1607s) the whole exhaustive uh validation with the help of a store pipe um uh post uh [26:51](https://www.youtube.com/watch?v=hX140a5klW0&t=1611s) the help of a store pipe um uh post uh [26:51](https://www.youtube.com/watch?v=hX140a5klW0&t=1611s) the help of a store pipe um uh post uh landing to the trunk and that allows us [26:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1613s) landing to the trunk and that allows us [26:53](https://www.youtube.com/watch?v=hX140a5klW0&t=1613s) landing to the trunk and that allows us to basically have this balance where we [26:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1615s) to basically have this balance where we [26:55](https://www.youtube.com/watch?v=hX140a5klW0&t=1615s) to basically have this balance where we are validating uh whatever whatever [26:57](https://www.youtube.com/watch?v=hX140a5klW0&t=1617s) are validating uh whatever whatever [26:57](https://www.youtube.com/watch?v=hX140a5klW0&t=1617s) are validating uh whatever whatever matters before and then validating [26:59](https://www.youtube.com/watch?v=hX140a5klW0&t=1619s) matters before and then validating [26:59](https://www.youtube.com/watch?v=hX140a5klW0&t=1619s) matters before and then validating everything at the end to make sure it is [27:01](https://www.youtube.com/watch?v=hX140a5klW0&t=1621s) everything at the end to make sure it is [27:01](https://www.youtube.com/watch?v=hX140a5klW0&t=1621s) everything at the end to make sure it is still good. Um and uh our consumers can [27:04](https://www.youtube.com/watch?v=hX140a5klW0&t=1624s) still good. Um and uh our consumers can [27:04](https://www.youtube.com/watch?v=hX140a5klW0&t=1624s) still good. Um and uh our consumers can consume the the good estate [27:07](https://www.youtube.com/watch?v=hX140a5klW0&t=1627s) consume the the good estate [27:07](https://www.youtube.com/watch?v=hX140a5klW0&t=1627s) consume the the good estate and with that um that would be our end [27:10](https://www.youtube.com/watch?v=hX140a5klW0&t=1630s) and with that um that would be our end [27:10](https://www.youtube.com/watch?v=hX140a5klW0&t=1630s) and with that um that would be our end of the talk. Um um if you want to learn [27:13](https://www.youtube.com/watch?v=hX140a5klW0&t=1633s) of the talk. Um um if you want to learn [27:13](https://www.youtube.com/watch?v=hX140a5klW0&t=1633s) of the talk. Um um if you want to learn more about Summit Q and inner workings [27:15](https://www.youtube.com/watch?v=hX140a5klW0&t=1635s) more about Summit Q and inner workings [27:16](https://www.youtube.com/watch?v=hX140a5klW0&t=1636s) more about Summit Q and inner workings of it, feel free to scan these QR codes. [27:19](https://www.youtube.com/watch?v=hX140a5klW0&t=1639s) of it, feel free to scan these QR codes. [27:19](https://www.youtube.com/watch?v=hX140a5klW0&t=1639s) of it, feel free to scan these QR codes. Um these are the previously uh talks [27:21](https://www.youtube.com/watch?v=hX140a5klW0&t=1641s) Um these are the previously uh talks [27:21](https://www.youtube.com/watch?v=hX140a5klW0&t=1641s) Um these are the previously uh talks that we have done um on summit Q in [27:23](https://www.youtube.com/watch?v=hX140a5klW0&t=1643s) that we have done um on summit Q in [27:23](https://www.youtube.com/watch?v=hX140a5klW0&t=1643s) that we have done um on summit Q in depth and they go over like in detail on [27:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1647s) depth and they go over like in detail on [27:27](https://www.youtube.com/watch?v=hX140a5klW0&t=1647s) depth and they go over like in detail on how how it functions internally and [27:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1649s) how how it functions internally and [27:29](https://www.youtube.com/watch?v=hX140a5klW0&t=1649s) how how it functions internally and again thank you. [27:30](https://www.youtube.com/watch?v=hX140a5klW0&t=1650s) again thank you. [27:30](https://www.youtube.com/watch?v=hX140a5klW0&t=1650s) again thank you. Thank you.