Comprehend First, Code Later: The AI Skill I Rely On Daily

[upbeat music] Hello, everyone.

Uh, today I'm gonna share with you how I use AI at Sentry, and the skill I use the most in my day-to-day work. Um, so but before we dive into that, let me tell you who I am.

Um, my name is Priscila. I'm a [REDACTED:origin] based in Vienna, Austria. I'm a mom of a [REDACTED:age], very energetic toddler. Uh, I'm a maintainer of Verdaccio, uh, an open source npm registry.

I'm a co-orga- co-organizer of ViennaJS, a very traditional meetup in Vienna, and we talk all about, uh, JavaScript. And, uh, I'm, I'm a senior software engineer at Sentry. Yesterday, someone told me that I don't look like an, a, a, a software engineer, but guess what?

I am. [laughs] Um, so my, my title, my official title is senior software engineer, but I have given me a little promotion, and I am now an agent manager. Um, no salary raise, but at least my reports, they don't complain.

Yes. Uh, this was me at work, uh, a few wee- a few weeks ago. Uh, my colleague, uh, Dominic Dorfmeister, found it funny to see me managing a couple of agents, uh, and took this picture.

Um, yeah, luckily, I have, uh, three monitors, so that works pretty well.

Um, this is my new reality. So the indust... Uh, sorry. Uh, this is how I feel actually orchestrating a bunch of agents. Uh, yeah, it's, it's weird, but it's fun.

And the industry is changing, right? That's why you all are here, and I'm also adapting. Uh, since last year, since December, uh, two thousand and twenty-five, I haven't coded anymore.

I'm only prompting. Yes. Um, and even this presentation was created by a skill. I have... Yeah, I didn't do anything. Uh. [laughs]

Um, so as you can see, these are some of my recent contributions to Sentry, uh, and I created a, a few PRs together with my favorite teammate, Claude. And it's not just, uh, bug fixes, you know.

It's also features, refactors, cross repositories contributions, so it's real. It's working. Um,

so, and this is Sentry. Um, maybe you don't know Sentry, but we are very well known for error and performance monitoring. But we ha- we have grown into a, a full observability platform.

So we have error monitoring, we have metrics, we have profiling. We have also, uh, agentic, uh, tools, like, uh, to monitor your agentic platforms. Um, yes, uh, the code base is very complex and, uh, it was founded in two thousand and ten.

It has fifteen plus years of code. We got around, uh, four hundred employees around the globe. We have one hundred K organizations depending on this code base working every day.

And as employee, I also depend on this code base working because I get my salary from it, right? So I don't want it to just, uh, ship slop code.

Um, yeah, so it's a serious business. And we vibe code as well at Sentry. Uh, recently, we had a hackathon, uh, where we could... We, we had a few days to just, uh, get ourselves familiar with AI and try out new things, and a lot of good projects came out of this, uh, week, this hackathon project.

And, uh, we have, for example, Abacos. This was created to track, uh, the usage of AI internally at Sentry. Uh, we have Warden. This is a, a code review agent.

Uh, you can have it in your PRs. Um, we have Junior. Junior is a bot we have in our Slack. Uh, and because usually people, they say like, uh, "Oh, um,

I don't like this UI. Something changed. Uh, can you go fix it?" Or, uh, "Why this was changed?" They like to, to go and share something, maybe some bug they found in Slack.

And then you can-- we can just trigger Junior, and Junior can analyze that thread and create APR and already fix the bug. And people are having a lot of fun with Junior.

Daniel can... Yeah, he loves that. [laughs] And, uh, there is also this AI SDK testing repository. Uh, this was created before this hackathon, uh, but this is basically a repository where we create tests to...

for our AI integrations. So and this was really weird because I started contributing to this repository, and, um, uh, my team told me I shouldn't code at all. I should only prompt until I would get a nice result.

So it was a, like a, a different experience. Uh, but yeah, it's working.

We are using all of these tools like internally every day. [laughs]

And yeah, at Sentry, we are going all in AI.

But quality still matters. Um, so, uh, also at Sentry last year, uh, during three months, it's li- uh, we- We use this time like a quality quarter. We use this time just to improve our code bases.

So remove all the any types from TypeScript, for example, or all the to-do, because usually... I don't know if you guys have the same, but we had a lot of to-do, uh, do something else at certain time.

And, uh, so we use this time to simplify code, and have, uh, remove unused feature flags, and really have always our, our code base in a, in a good shape.

And this is very important, right? We wanted that. This is, uh, called, uh, technical debt.

Um, so, and, uh, as I told you before, the code base, the Sentry code base is very complex, and it's a moving target. We have about 100 PRs, uh, merged every day.

We have four offices. Uh, Sentry is fair source, so we... You can contribute to Sentry. We have also contributors. Um, and we are all the time deprecating components, uh, adding new components, uh, adding new lint rules.

Uh, I don't know, you name it. Like, all the time something happens at Sentry. I'm there over six years now, and I can go on vacation, I come back, and maybe my PR, uh, is full of conflicts, and I have to solve those conflicts.

And I really need to understand. Ev- like all the time, I need to align and, and understand something. It's a daily practice.

So, and this is not new, right? Like, uh, you guys know, uh, there are studies behind it. Like 70% of a, uh, developer time is spending reading and navigating code, and et cetera.

Uh, this hasn't changed, you know? But, uh, now we have a very smart tool which can help us to understand faster, and that's how I'm using AI.

So, um, I'm using AI a lot to understand, to explore and understand. And maybe you, y- you, you may think like, uh, you just tell AI to go explore the code base, and after do something.

But, like, maybe AI understood, uh, not the, the correct thing, under- understood wrongly, you know? And you need to also understand, because maybe you need to steer the AI to go on the correct path, you know?

And yeah. So this is how I'm using AI. Uh, it made me faster, but not the way you think. May- because like before, um, let's say, um, an incident happened, and I would have to track that down.

I would have to open APR... uh, uh, open GitHub, sorry. Uh, go git blame, and then trying to understand, like, where the regression happened. And now I can just, uh, prompt a simple phrase, and I have it in a few seconds.

Or before maybe a product de- product decision. Um, why this changed? And then I would, I don't know, ch- uh, ask this question in Slack. Maybe my colleague is in another t- another country, another time zone, and I would need to wait for that answer in the next day.

And now I can just, uh, ask AI, and I have it. So it's been really useful, and this made me really productive. Uh, but, but like, the understanding part of it, like, yeah.

And my prompts, they kept repeating. Uh, so I had this idea to let, to, to let, uh, Claude analyze my cache and see, uh... So it analyzed it, uh, 116 sessions, and it classified everything in six categories: comprehension, modification, process, review, generation, and other.

And, um, guess what? Uh, this im- impressed even me.

Uh, oops. It's fine. So 67% of my AI usage was, uh, comprehension, and only 2%, uh, code generation.

Uh, so this... I was very surprised. [laughs] So because my prompts kept repeating, repeating, I created a skill for me. Uh, this skill, it's locally in my computer.

I could share it with someone if I wish. But I use... This is for me here, and it's called Catch Me Up. Um, that structures those comprehension questions into six exploration modes: architecture, convention, feature, trace, syntax, testing, and history.

So a skill is just a very detailed prompt, right? With some very clear goals. Uh, I can actually do like this here. Uh, you can see how, how is it.

But it's, it's just like it's a, an MD file, uh, with, uh, human language. And I am a very visual person. I work at Sentry a lot on the front end part of it, and I like to see things to understand.

So, um, this skill brings me like, uh, the organogram, like, uh, the structure, uh, a table for me to understand. I think it helps a lot. And, uh, I can now give you a short demo.

Um, just a minute. Um, by the way, this is my presentation running. [laughs]

Oh, it's here, actually, here. Um, I already run this, uh, skill because I don't know, maybe I would have some issues. But,

uh, do you remember that project I told to you that I should only prompt and, uh, don't code anymore? So in the beginning, I was not familiar with that project.

It was a new repository for me, and I, I used my skill for that. I said, "I am a new..." Uh, can you guys see this well? Yeah. So I said, "I am a new contributor.

Catch me up on how this repository works, and clarify whether it simulates a Sentry envelope and intercepts it during tests." Um, I am using Claude, as you can see, Opus.

Uh, and it gave h- it gave me here a summary, and this I like, this flow, like how it works. Um,

and here also answered my question, like, uh, "Does it simulate envelopes? No. Inter- intercepts real ones. They spawn collector..." Yeah, I'm not gonna read it, but this information, all of this, it's very useful.

Like, if I didn't have AI, I would have to do this myself, you know? Um, I mean, I don't like to, to, uh, just ship something I don't understand.

If it's a vibe coded project, that's fine, but this is a real serious business. It's my work. Uh,

so yes, uh, this skill is helping me a lot with that, and also to review PRs, because maybe I'm reviewing a PR of a colleague. I have a lot of context, but not enough to approve that PR, and I wanted to have that context, so I use this skill to give me that.

Uh, okay, so back to my vibe coded presentation. [laughs]

So, uh, Jack Nations wrote a blog post called Vibe Coding Our Way to Disaster, drawing on Rich Hickey's, uh, simple made easy philosophy. He proposes three phases, research, planning, and implementation.

I think you guys also heard about it o- outside, like even Claude Code has this planning mode, right? Um, so yes, I agree with all of that, but I think it's missing this step.

Like, you need to understand the research, uh, your agent did, you know? You need to understand that, and to steer, as I said, maybe it's going the wrong direction, or maybe you need to explore something else, and you, you need to have that to understand that.

Then after that, you can say, "Okay, plan that for me, and let's do the implementation. Let's go ahead."

Um, so Armin Ronacher, uh, he's, uh, the creator of Flask and a former Sentry. Now he's worked on his own startup. Today, he's gonna give a talk here, by the way.

He's around. Um, so he wrote in his blog post, "When more and more people tell me they no longer know what code is in their own code base, I feel like something is very wrong here."

And yes, I agree. Um, so what I hope you, you can take away from this presentation is that, uh, the biggest unlock from AI in a large code base isn't generation, it's comprehension.

Uh, I tracked my own usage, and I was surprised, uh, sixty-seven percent of my prompts are co- uh, comprehension, and only two percent generation. Maybe you track your own AI usage as well, and you can improve it, right?

Uh, so AI is the teammate, uh, who never gets tired of your questions, so there is no dumb questions. It's the cheapest senior engineer out there, so just go for it.

Um, yes, and align your mental model before you prompt, because, uh, you know, the code is gonna flow naturally. And don't ship, uh, slop code into the code base that pays your salary.

Uh, ship kino code, really. [laughs] This is the term, um, the industry is using. And, uh, yes, thank you. Um, try Sentry. This was a sponsored talk. We have a booth downstairs if you wanted to stop by to say hello.

If you scan this QR code, you're gonna get a three months free trial of our business plan, and I hope you enjoyed the presentation. [laughs] [applause] [upbeat music]