# Move Fast Break Nothing

Dedy Kredo, CodiumAI

AI Engineer Summit 2023

[Talk page](https://ai.engineer/talks/oJz4wveQ4oM-move-fast-break-nothing) · [Video](https://www.youtube.com/watch?v=oJz4wveQ4oM)

## Move Fast, Break Nothing: Giving Generated Code a Critic

Dedy Kredo’s AutoScraper demonstration follows a separate code critic from behavior mapping and test generation through automated repair, a failed regression check, and PR review.

Basic familiarity with Python, automated tests, and Git pull requests will help you follow the demonstration.

### A critic for generated code

[0:18](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=18s)

What checks whether generated code actually does what its developer intended? Dedy Kredo opens with **generative adversarial networks**, or [GANs](https://arxiv.org/abs/1406.2661): two networks working together, one generating and the other criticizing its output. In his telling, transformers shifted attention toward generation and became the preferred architecture for many tasks. Code, however, needs more than a plausible next output. It needs scrutiny of the behaviors that output creates.

The proposed architecture is GAN-like at the **system level**, not an adversarially trained neural network. One component generates code; a separate **code integrity** component reviews it, analyzes its behavior, and searches for edge cases. The target is not simply code that looks correct, but code that works according to the developer’s intent.

![Slide connecting Code Gen and Code Integrity with paired arrows beneath the heading Code that works as intended.](https://ai.engineer/api/talk-essay-media/b71c4384294e262a9b90ac20cbadbcdb03d5bdbdf443e9b17def1685695b0d51)

Code generation and code integrity form a feedback loop aimed at code that works as intended.

CodiumAI focuses on that critic. Kredo argues that **behavior coverage is more useful than code coverage**: the organizing question is which behaviors have been identified and tested, rather than only which code has executed. The product connects behavior discovery to test generation, code enhancement, and review. At the time of the demonstration, those capabilities appeared in JetBrains and VS Code extensions and a Git plugin; APIs for embedding them in agents were described as forthcoming.

### Starting with an untested scraping class

[3:00](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=180s)

The live demonstration begins in VS Code with the CodiumAI extension installed. Kredo reports around 200,000 installs across JetBrains and VS Code. The project is [AutoScraper](https://github.com/alirezamika/autoscraper), a Python scraper that generates extraction rules from examples of desired information. Kredo describes the demo project as having more than 5,000 GitHub stars but no tests. That absence is the practical obstacle: changing the scraper leaves the developer without a suite to catch unintended behavior. This describes his demo checkout; the current repository contains tests.

Kredo invokes CodiumAI on what he describes as a 600-line class. The extension can also operate on an individual method, but starting at class scope lets it analyze the larger unit first. It maps behaviors and then streams generated tests into the interface. Alongside the tests, it produces an explanation of the class, including example usage, components, and methods. The initial suite spans happy paths and edge cases.

### Choose behaviors, then run and repair tests

[4:56](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=296s)

The behavior map is hierarchical. Happy paths and edge cases can be expanded into sub-behaviors, and the developer can choose which deserve additional tests. This makes the map a place to exercise judgment: generation proposes a set of behaviors, while the developer decides where to investigate more deeply.

![VS Code shows AutoScraper Python code beside CodiumAI’s Behaviors Coverage panel, with Happy Path and Edge Cases sections and Go to test or Generate test controls.](https://ai.engineer/api/talk-essay-media/1a6fe5ff7ffcc7da34c4d611aa7b673405e4a8f6ed3b7d79264bce564554027b)

AutoScraper behavior coverage lists happy paths and edge cases alongside the class source.

For a scraper, a small behavior-focused test can hold the page content fixed and state the expected extraction directly. For example, this Python test uses a local HTML string to express the happy path of learning a rule from a desired value:

```python
from autoscraper import AutoScraper


def test_build_finds_requested_product_name():
    html = "<html><body><h2>Travel Mug</h2></body></html>"
    scraper = AutoScraper()

    results = scraper.build(
        html=html,
        wanted_list=["Travel Mug"],
    )

    assert "Travel Mug" in results
```

The useful unit of coverage here is the behavior expressed by the assertion. Kredo’s next action is to select additional edge cases in the map and request tests for them. During that expansion, he reports nine tests in the interface.

Next comes **Run & Autofix**. The demonstrated sequence is:

1. Execute the generated tests; some pass immediately.
2. Inspect a failing test’s execution result.
3. Have the model analyze the failure and generate a repair.
4. Run the repaired test again.

The demonstrated failing test passes on its second attempt. Kredo describes this as a reflection process, using the language of chain of thought. The observable mechanism is the execution feedback loop: a failure supplies information for a revision, which is then checked by another run.

Kredo then opens eight tests as a file and saves them into the project. The recording does not explain the change from nine tests during generation to eight at this point. Saving the suite turns the generated output into a project artifact that can be rerun when the implementation changes.

### An enhancement still has to survive the tests

[6:54](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=414s)

With a saved suite, Kredo moves to code enhancement. He selects AutoScraper’s `build` method because it contains much of the class’s central functionality, then opens CodiumAI’s code suggestions. These suggestions aim beyond linting: performance, security, best practices, and readability are all in scope.

He chooses a proposed performance improvement that replaces hashing through Python’s [`hashlib`](https://docs.python.org/3/library/hashlib.html) with [BLAKE3](https://github.com/BLAKE3-team/BLAKE3). The distinction matters: `hashlib` is a module exposing multiple hash algorithms, whereas BLAKE3 is an algorithm. The original algorithm is not identified in the narration. Kredo prepares the suggested changes, applies them, saves the code, and returns to the test suite.

The rerun does not produce a clean result. One test fails after the change, and Kredo explicitly sets it aside to continue the demonstration. The cause is not established, so the failure cannot be attributed specifically to BLAKE3. Nor is a speedup measured. At this point, the workflow has produced an applied enhancement and an unresolved regression signal—not a validated refactor. **Generating a change and validating it remain separate steps.**

### From a local change to PR review

[8:54](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=534s)

The next stage is preparing the change for review. In the CodiumAI PR Assistant, Kredo invokes `/commit` to generate a commit message and then commits the changes. Only after committing does he invoke `/review`. The assistant is expected to summarize the PR, assign a score, and look for issues the developer may have missed. Its initial assessment identifies tests as the main theme and describes the change as well structured. These are the historical IDE assistant commands shown in the recording.

### The review finds an API key

[10:06](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=606s)

The review then flags a potential security vulnerability. Kredo invokes `/improve` to request a fix and recognizes the issue: an API key has been left in the code. This extends the critic’s scope beyond functional test results. A change can have useful tests and still contain something that should not be committed.

![VS Code with a pull request assistant showing review feedback about a hardcoded API key beside Python source code.](https://ai.engineer/api/talk-essay-media/0d89e10100b804d1ab69ce4f386c292871faddbd774091ce90499dc4d9268e85)

The review flags a hardcoded API key as a potential security vulnerability.

The response is delayed, so Kredo retries; an interjection points to the room’s poor network connection. Once the result arrives, clicking the finding navigates to the key’s location. Kredo identifies a displayed fix, but the demonstration does not establish its implementation or show it being applied and validated. The security result is therefore a located issue and a proposed repair, not a completed remediation.

The demo ends with a connected workflow: map behaviors, generate and execute tests, assess changes, and review the resulting contribution. Kredo also points to the Git plugin as a way to bring this work into GitHub. The critic is intended to operate throughout development, including the transition from an individual developer’s editor to a shared review process.

### A personal closing

[11:45](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=705s)

Kredo closes by turning from the software to the people building it. CodiumAI is based in Israel, and he says Hamas attacked while the team was on the plane to the event. He condemns the attackers in dehumanizing terms and describes civilians being killed in their homes and abducted into Gaza.

He then introduces a photograph sent by his co-founder and CEO, Itamar. According to Kredo, Itamar had left his eight-month-pregnant wife at home to enter military reserve duty. The photograph shows a laptop and a rifle behind it; Kredo identifies the chart on the screen as increasing CodiumAI usage. The juxtaposition closes the talk on the circumstances in which his colleagues were continuing their work. His final declaration is: “We will prevail.”

![Slide photograph of an open laptop on rough ground, a rifle behind it, and a circular portrait inset.](https://ai.engineer/api/talk-essay-media/f50f75c7ba424b3e93ad9e90273f729010b167285a6dc633a0879db08b0eea23)

A closing slide shows a laptop with a rising chart and a rifle behind it.

## Resources

- [AutoScraper](https://github.com/alirezamika/autoscraper) — Python scraper that learns extraction rules from examples, with installation instructions and build, reuse, and persistence examples.
- [BLAKE3](https://github.com/BLAKE3-team/BLAKE3) — Official BLAKE3 implementations, usage examples, test vectors, and benchmark context.
- [Generative Adversarial Networks](https://arxiv.org/abs/1406.2661) — The original paper introducing adversarial training between a generator and a discriminator.
- [PR-Agent](https://github.com/The-PR-Agent/pr-agent) — Community-maintained open-source PR review agent with CLI and GitHub Actions setup, distinct from Qodo's primary offering.

## Slides

### Diamond Sponsors

[0:09](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=9s) [Diamond sponsors Fixie and Supabase on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/b3edd6c6072875ff1282697e19201c1cfe0bada6599bffeb282f6fe6e5368755.webp)

DIAMOND SPONSORS

FIXIE
supabase

### CodiumAI capabilities

CodiumAI positions test generation, code explanation, enhancement and review across the workflow from coding to PR review. Code generation and a code integrity API are marked as coming soon.

[1:57](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=117s) [CodiumAI features for testing, coverage, code explanation, enhancement and review, with planned code generation and a code integrity API. A diagram shows the code-to-review workflow.](https://assets.ai.engineer/sites/aiecode2025/sha256/9b163b94e755d483402637479d91a4517f3e54a81f449631546a3a27374d75e2.webp)

Generates Tests
One-click test suite

Increase Coverage
Explore relevant code behaviors

Explain Code
Get familiar with the code

Enhance Your Code
Apply code suggestions

Boost Code Reviews
Quick and improved reviews

Soon: Code Generation
Chat to generate new code

Soon
Code Integrity API

CODE → TEST → PREPARE PR → REVIEW PR

© 2023 CodiumAI

Interface text:
AIE
SUMMIT

### AutoScraper class

In VS Code, the presenter opens AutoScraper, a Python project that learns rules for extracting content from web pages. The CodiumAI extension is installed for the demo.

[2:56](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=176s) [VS Code displays AutoScraper imports and the beginning of the class documentation.](https://assets.ai.engineer/sites/aiecode2025/sha256/6761e5b85c3a1316c8891571c47e63a1d11fa9fec62c3ae92150d1667fcb5266.webp)

import hashlib
import json
import os
from collections import defaultdict
from html import unescape
from urllib.parse import urljoin, urlparse

import requests
from bs4 import BeautifulSoup

from autoscraper.utils import (
    FuzzyText,
    ResultItem,
    get_non_rec_text,
    get_random_str,
    normalize,
    text_match,
    unique_hashable,
    unique_stack_list,
)

class AutoScraper(object):
    """
    AutoScraper : A Smart, Automatic, Fast and Lightweight Web Scraper for Python.
    AutoScraper automatically learns a set of rules required to extract the needed content
        from a web page. So the programmer doesn't need to explicitly construct the rules.

    Attributes
    ----------
    stack_list: list
        List of rules learned by AutoScraper

    Methods

Interface text:
autoscraper_forked
Codium AI - auto_scraper.py[AutoScraper]
auto_scraper.py
autoscraper > auto_scraper.py > ...
You, 13 hours ago | 6 authors (alireza and others)
alireza, 3 years ago • apply fuzziness ratio on full url matching
You, 13 hours ago | 6 authors (alireza and others) | CodiumAI: Test this class
add-tests-and-enhancements*
alireza, 3 years ago
Ln 17, Col 16
Spaces: 4
UTF-8
LF
Python 3.11.3 ('autoscraper_forked-[illegible]': Pipenv)
CodiumAI

### AutoScraper methods

AutoScraper builds scraping rules from a wanted list and reuses them to find similar elements. The presenter says the project has no tests, making changes difficult.

[3:15](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=195s) [The AutoScraper class documentation lists methods for building rules and retrieving results.](https://assets.ai.engineer/sites/aiecode2025/sha256/728847e401cf9cc8bcc5001f54b0ae1b13c1ebc138294e2974c1065acce7fc24.webp)

import requests
from bs4 import BeautifulSoup

from autoscraper.utils import (
    FuzzyText,
    ResultItem,
    get_non_rec_text,
    get_random_str,
    normalize,
    text_match,
    unique_hashable,
    unique_stack_list,
)

class AutoScraper(object):
    """
    AutoScraper : A Smart, Automatic, Fast and Lightweight Web Scraper for Python.
    AutoScraper automatically learns a set of rules required to extract the needed content
        from a web page. So the programmer doesn't need to explicitly construct the rules.

    Attributes
    ----------
    stack_list: list
        List of rules learned by AutoScraper

    Methods
    -------
    build() - Learns a set of rules represented as stack_list based on the wanted_list,
        which can be reused for scraping similar elements from other web pages in the future.
    get_result_similar() - Gets similar results based on the previously learned rules.
    get_result_exact() - Gets exact results based on the previously learned rules.
    get_results() - Gets exact and similar results based on the previously learned rules.
    save() - Serializes the stack_list as JSON and saves it to disk.

Interface text:
autoscraper_forked
Codium AI - auto_scraper.py[AutoScraper]
auto_scraper.py
autoscraper > auto_scraper.py > ...
alireza, 3 years ago • apply fuzziness ratio on full url matching
You, 13 hours ago | 6 authors (alireza and others) | CodiumAI: Test this class
add-tests-and-enhancements*
alireza, 3 years ago
Ln 17, Col 16
Spaces: 4
UTF-8
LF
Python 3.11.3 ('autoscraper_forked-[illegible]': Pipenv)
CodiumAI

### Trigger class tests

The presenter points to CodiumAI’s class-level test action for AutoScraper, a 600-line class, and says it can also be triggered on individual methods.

[3:29](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=209s) [The pointer highlights CodiumAI’s Test this class action above the AutoScraper class.](https://assets.ai.engineer/sites/aiecode2025/sha256/681843d685f9e76a652a205b2aaaa157110122d9b3d94981fa4e85b1d22ec8e8.webp)

import os
from collections import defaultdict
from html import unescape
from urllib.parse import urljoin, urlparse

import requests
from bs4 import BeautifulSoup

from autoscraper.utils import (
    FuzzyText,
    ResultItem,
    get_non_rec_text,
    get_random_str,
    normalize,
    text_match,
    unique_hashable,
    unique_stack_list,
)

class AutoScraper(object):
    """
    AutoScraper : A Smart, Automatic, Fast and Lightweight Web Scraper for Python.
    AutoScraper automatically learns a set of rules required to extract the needed content
        from a web page. So the programmer doesn't need to explicitly construct the rules.

    Attributes
    ----------
    stack_list: list
        List of rules learned by AutoScraper

    Methods
    -------
    build() - Learns a set of rules represented as stack_list based on the wanted_list,

Interface text:
autoscraper_forked
Codium AI - auto_scraper.py[AutoScraper]
auto_scraper.py
autoscraper > auto_scraper.py > ...
alireza, 3 years ago • apply fuzziness ratio on full url matching
You, 13 hours ago | 6 authors (alireza and others) | CodiumAI: Test this class
add-tests-and-enhancements*
Ln 17, Col 16
Spaces: 4
UTF-8
LF
Python 3.11.3
CodiumAI

### Generating tests for AutoScraper

CodiumAI analyzes the AutoScraper class, maps its behaviors, and streams generated tests. The demo moves from a loading test suite to test data containing rules for titles, authors, and dates.

[4:00](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=240s) [CodiumAI’s test suite panel shows Pytest configuration, behavior coverage, and a loading state.](https://assets.ai.engineer/sites/aiecode2025/sha256/efb45dd7230b35a3802dcf4250472c58b2264ac423254d09186f3a8756209d79.webp)

Test Suite
Configuration
General instructions: Start typing or use hints... Hints
Test Framework: Pytest
Reference test examples: None
Auto-mock
Number of tests: 6
Regenerate
Behaviors Coverage
Total Behaviors: 16
Tests
Run All Tests
Generating...
New custom test
Copy tests
Open as a file

import hashlib
import json
import os
from collections import defaultdict
from html import unescape
from urllib.parse import urljoin, urlparse

import requests
from bs4 import BeautifulSoup

from autoscraper.utils import (
    FuzzyText,
    ResultItem,
    get_non_rec_text,
    get_random_str,
    normalize,
    text_match,
    unique_hashable,
    unique_stack_list,
)

class AutoScraper(object):
    """
    AutoScraper : A Smart, Automatic, Fast and Lightweight Web Scraper for Pyt[illegible]
    AutoScraper automatically learns a set of rules required to extract the ne[illegible]
        from a web page. So the programmer doesn't need to explicitly construc[illegible]

    Attributes
    ----------
    stack_list: list
        List of rules learned by AutoScraper

    Methods

Interface text:
autoscraper_forked
Codium AI - auto_scraper.py[AutoScraper]
auto_scraper.py
autoscraper > auto_scraper.py > ...
Test Suite
Code Explanation
Code Suggestions
codium ai
You, 13 hours ago | 6 authors (alireza and others)
CodiumAI: Test this class
alireza, 3 years ago • apply fuzziness ratio on full url [illegible]
Ln 17, Col 16
Spaces: 4
UTF-8
LF
Python 3.11.3
CodiumAI
AIE
AI Engineer SUMMIT
FIXIE
supabase

[4:46](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=286s) [A code editor beside generated test data in Codium AI.](https://assets.ai.engineer/sites/aiecode2025/sha256/5557375d1310d41e4b11fa3841256136f23a2bae6f0c5d44b41470da9044969c.webp)

stack_list = [
    {
        'content': [('div', {'class': 'title'})],
        'wanted_attr': None,
        'is_full_url': False,
        'is_non_rec_text': False,
        'url': '',
        'hash': '123456',
        'stack_id': 'rule_1234',
        'alias': 'Rule 1'
    },
    {
        'content': [('div', {'class': 'author'})],
        'wanted_attr': None,
        'is_full_url': False,
        'is_non_rec_text': False,
        'url': '',
        'hash': '789012',
        'stack_id': 'rule_5678',
        'alias': 'Rule 2'
    },
    {
        'content': [('div', {'class': 'date'})],
        'wanted_attr': None,
        'is_full_url': False,
        'is_non_rec_text': False,
        'url': '',
        'hash': '345678',
        'stack_id': 'rule_9012',
        'alias': 'Rule 3'

import hashlib
import json
import os
from collections import defaultdict
from html import unescape
from urllib.parse import urljoin, urlparse

import requests
from bs4 import BeautifulSoup

from autoscraper.utils import (
    FuzzyText,
    ResultItem,
    get_non_rec_text,
    get_random_str,
    normalize,
    text_match,
    unique_hashable,
    unique_stack_list,
)

class AutoScraper(object):
    """
    AutoScraper : A Smart, Automatic, Fast and Lightweight Web Scraper for Pyt[illegible]
    AutoScraper automatically learns a set of rules required to extract the ne[illegible]
        from a web page. So the programmer doesn't need to explicitly construc[illegible]

    Attributes
    ----------
    stack_list: list
        List of rules learned by AutoScraper

    Methods

Interface text:
autoscraper_forked
Codium AI - auto_scraper.py[AutoScraper]
auto_scraper.py
autoscraper > auto_scraper.py > ...
Test Suite 6
Code Explanation 1
Code Suggestions 2
codium ai
You, 13 hours ago | 6 authors (alireza and others)
CodiumAI: Test this class
Give me more tests
New custom test
Copy tests
Open as a file
add-tests-and-enhancements*
CodiumAI

### Behavior coverage

Codium AI maps AutoScraper behaviors across happy paths and edge cases. The demo shows links to existing tests and options to generate tests for uncovered behaviors.

[4:52](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=292s) [Codium AI lists happy-path and edge-case behaviors beside the source code.](https://assets.ai.engineer/sites/aiecode2025/sha256/aa3af2a77c1b461955bf92622f2efe7cb4995ed18f4856891a50182ab936015d.webp)

Behaviors Coverage
Total Behaviors: 18

HAPPY PATH
Building the AutoScraper object with a stack list. — Go to test
Test the behavior of getting the result for exact items when the stack list contains rules with different URLs. — Go to test
Building the scraper with a URL that requires authentication and a non-empty stack list. — Go to test
Building the scraper with an empty stack list. — Go to test
Getting the result of the scraper for similar items. — Go to test
Build the scraper with an empty stack list and valid HTML content. — Go to test
Testing the behavior when the stack list contains rules with different URLs. — Generate test

EDGE CASES
Test the behavior of getting the result for exact items when the stack list is empty. — Generate test

import hashlib
import json
import os
from collections import defaultdict
from html import unescape
from urllib.parse import urljoin, urlparse

import requests
from bs4 import BeautifulSoup

from autoscraper.utils import (
    FuzzyText,
    ResultItem,
    get_non_rec_text,
    get_random_str,
    normalize,
    text_match,
    unique_hashable,
    unique_stack_list,
)

class AutoScraper(object):
    """
    AutoScraper : A Smart, Automatic, Fast and Lightweight Web Scraper for Pyt[illegible]
    AutoScraper automatically learns a set of rules required to extract the ne[illegible]
        from a web page. So the programmer doesn't need to explicitly construc[illegible]

    Attributes
    ----------
    stack_list: list
        List of rules learned by AutoScraper

    Methods

Interface text:
autoscraper_forked
auto_scraper.py
autoscraper > auto_scraper.py > ...
You, 13 hours ago | 6 authors (alireza and others)
Codium AI - auto_scraper.py[AutoScraper]
Test Suite 6
Code Explanation 1
Code Suggestions 5
codium
Configuration
Regenerate
Give me more tests
New custom test
Copy tests
Open as a file
add-tests-and-enhancements*
CodiumAI

### Generating edge-case tests

The presenter selects edge cases for additional tests. Codium AI is generating a test for retrieving exact items when the stack list is empty.

[5:21](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=321s) [The behavior list is scrolled to edge cases, with one test generating.](https://assets.ai.engineer/sites/aiecode2025/sha256/7d9b7a3144b159e935726d33edd5c3294060ea4f1ed47354d0b877a8d05b9865.webp)

Testing the behavior when the stack list contains rules with different URLs.  Generate test

EDGE CASES
Test the behavior of getting the result for exact items when the stack list is empty.  Generating
Test the behavior of getting the result for similar items when the stack list is empty.  Generate test
Test the behavior of getting the result for exact items when the HTML content is empty.  Generate test
Test the behavior of getting the result for similar items when the HTML content is empty.  Generate test
Test the behavior of getting the result for exact items when the URL is empty.  Generate test
Test the behavior of getting the result for similar items when the URL is empty.  Generate test

OTHER
Test the behavior of saving the stack list to a file.  Generate test
Test the behavior of loading the stack list from a file.  Generate test
Test the behavior of removing rules from the stack list.  Generate test

import hashlib
import json
import os
from collections import defaultdict
from html import unescape
from urllib.parse import urljoin, urlparse

import requests
from bs4 import BeautifulSoup

from autoscraper.utils import (
    FuzzyText,
    ResultItem,
    get_non_rec_text,
    get_random_str,
    normalize,
    text_match,
    unique_hashable,
    unique_stack_list,
)

class AutoScraper(object):
    """
    AutoScraper : A Smart, Automatic, Fast and Lightweight Web Scraper for Pyt[illegible]
    AutoScraper automatically learns a set of rules required to extract the ne[illegible]
        from a web page. So the programmer doesn't need to explicitly construc[illegible]

    Attributes
    ----------
    stack_list: list
        List of rules learned by AutoScraper

    Methods

Interface text:
autoscraper_forked
auto_scraper.py
autoscraper > auto_scraper.py > ...
You, 13 hours ago | 6 authors (alireza and others)
Codium AI - auto_scraper.py[AutoScraper]
Test Suite 7
Code Explanation 1
Code Suggestions 5
codium ai
Generating...
New custom test
Copy tests
Open as a file
add-tests-and-enhancements*
CodiumAI

### Test failure analysis

Codium analyzes a failed AutoScraper test: the test expected an empty result, but the scraper returned two titles. It flags possible test and code issues and offers fixes for each.

[5:54](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=354s) [One generated test has failed and is being analyzed; another has passed.](https://assets.ai.engineer/sites/aiecode2025/sha256/f2b15a1865c75e014a577d6e647e72fbbb66a53c79511785cbf944022fe7b1b5.webp)

import hashlib
import json
import os
from collections import defaultdict
from html import unescape
from urllib.parse import urljoin, urlparse

import requests
from bs4 import BeautifulSoup

from autoscraper.utils import (
    FuzzyText,
    ResultItem,
    get_non_rec_text,
    get_random_str,
    normalize,
    text_match,
    unique_hashable,
    unique_stack_list,
)

class AutoScraper(object):
    """
    AutoScraper : A Smart, Automatic, Fast and Lightweight Web Scraper for Pyt[illegible]
    AutoScraper automatically learns a set of rules required to extract the ne[illegible]
        from a web page. So the programmer doesn't need to explicitly construc[illegible]

    Attributes
    ----------
    stack_list: list
        List of rules learned by AutoScraper

    Methods

Failed
Analyzing the failure
Build the scraper with an empty stack list and invalid HTML content.
HAPPY_PATH

def test_build_with_empty_stack_list_and_invalid_html(self):
    scraper = AutoScraper()
    html = '<html><body><div class="title">Title 1</div><div class="t[illegible]
    result = scraper.build(html=html, wanted_list=['Title 1', 'Title[illegible]

    assert result == []
    assert scraper.stack_list == []

Passed
Test the behavior of getting the result for exact items when the stack list is empty.
EDGE_CASE

def test_get_result_exact_with_empty_stack_list(self):
    # Create an instance of AutoScraper
    scraper = AutoScraper()

Interface text:
autoscraper_forked
auto_scraper.py
autoscraper > auto_scraper.py > ...
You, 13 hours ago | 6 authors (alireza and others)
Codium AI - auto_scraper.py[AutoScraper]
Test Suite 9
Code Explanation 1
Code Suggestions 5
What would you like to change?
1 / 1
Stop
What would you like to change?
1 / 1
Run and Auto-fix
Give me more tests
New custom test
Copy tests
Open as a file
add-tests-and-enhancements*
CodiumAI

[6:02](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=362s) [Codium AI displays an assertion failure for an AutoScraper test.](https://assets.ai.engineer/sites/aiecode2025/sha256/9d4d20e74c24a6cdc50761329031715862859f45c301d5f551c0e069efc0bd5e.webp)

import hashlib
import json
import os
from collections import defaultdict
from html import unescape
from urllib.parse import urljoin, urlparse

import requests
from bs4 import BeautifulSoup

from autoscraper.utils import (
    FuzzyText,
    ResultItem,
    get_non_rec_text,
    get_random_str,
    normalize,
    text_match,
    unique_hashable,
    unique_stack_list,
)

class AutoScraper(object):
    """
    AutoScraper : A Smart, Automatic, Fast and Lightweight Web Scraper for Pyt[illegible]
    AutoScraper automatically learns a set of rules required to extract the ne[illegible]
        from a web page. So the programmer doesn't need to explicitly construc[illegible]

    Attributes
    ----------
    stack_list: list
        List of rules learned by AutoScraper

    Methods

Failed
Build the scraper with an empty stack list and invalid HTML content.
HAPPY_PATH
POSSIBLE TEST ISSUE
POSSIBLE CODE ISSUE

self = <autoscraper.[illegible]_auto_scraper.TestAutoScraper object at [illegible]>

    def test_build_with_empty_stack_list_and_invalid_html(self):
        scraper = AutoScraper()
        html = '<html><body><div class="title">Title 1</div><div class="title">Title 2</div>'
        result = scraper.build(html=html, wanted_list=['Title 1', 'Title 2'])

>       assert result == []
E       AssertionError: assert ['Title 1', 'Title 2'] == []
E         Left contains 2 more items, first extra item: 'Title 1'
E         Full diff:
E         - []
E         + ['Title 1', 'Title 2']

[illegible]_auto_scraper.py:744: AssertionError

def test_build_with_empty_stack_list_and_invalid_html(self):
    scraper = AutoScraper()

Interface text:
autoscraper_forked
auto_scraper.py
autoscraper > auto_scraper.py > ...
You, 13 hours ago | 6 authors (alireza and others)
Codium AI - auto_scraper.py[AutoScraper]
Test Suite 9
Code Explanation 1
Code Suggestions 5
What would you like to change?
1 / 1
Fix test
Fix Code
Stop
Give me more tests
New custom test
Copy tests
Open as a file
add-tests-and-enhancements*
CodiumAI

## Full transcript

[0:00](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=0s) [upbeat music] Hey, good morning, everyone.

[0:18](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=18s) Let's take a step... Let's start by taking a step back. Uh, remember GANs, generative adversarial networks. They represented a very compelling architecture, in my opinion. Two neural networks working hand-in-hand, one generating and one is the critic, in order to generate high-quality outcomes.

[0:41](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=41s) Then came transformers that changed everything. We dropped the adversarial and the focus became solely on the generative,

[0:51](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=51s) and they became the state-of-the-art for a variety of use cases.

[0:57](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=57s) But code is very, very nuanced. We believe that in order to generate code that actually works as intended, the right architecture is actually GAN-like architecture.

[1:11](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=71s) And what I mean by is that... by that is not the actual neural network, it's the system. It's the concept of having two different components, one focused on the code generation piece and one that serves as the critic.

[1:26](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=86s) We call it the code integrity component. It actually analyzes the, the outcomes, the generation of the code gen, uh, component, and it reviews it, it analyzes it. It tries to figure out all the different edge cases in order to generate high-quality code that works as intended based on the developer's actual intent.

[1:50](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=110s) This is our focus at CodiumAI, on the critic piece. We help developers understand the behaviors of their code.

[2:00](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=120s) We believe that behavior coverage is a more useful metric than actual code coverage. We help them generate tests for these behaviors, enhance their code, and review their code,

[2:13](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=133s) and we do that throughout the developer life cycle, leveraging our IDE extensions for both JetBrains and VS Code

[2:21](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=141s) and our Git plugin. And then soon in the future, in the near future, we will also offer APIs for this to be able to be embedded in various agents.

[2:36](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=156s) So we're gonna focus the majority of the time in a live demo,

[2:41](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=161s) which is a risky thing to do in this, uh, situation here, but, uh, let's go for it.

[3:00](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=180s) Okay, I'm here in my VS Code. I have the CodiumAI extension installed. Uh, we now have, uh, around two hundred thousand, uh, installs across both JetBrains and VS Code.

[3:10](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=190s) I have here an open source project that's called AutoScraper. Um, it's basically a scraping class that automates the process of generating the rules for scraping information from websites. It's a very cool project.

[3:25](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=205s) It has more than five thousand, uh, GitHub stars. But the problem is that it doesn't have any tests, so it's very hard to make changes to a project where it doesn't have any, any tests because there's nothing that protects you from making changes.

[3:42](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=222s) So I'm gonna go ahead here and trigger CodiumAI on this class. This is a six hundred-line class, complex code, and you can see that I can trigger CodiumAI either on the class level or at the method level.

[3:56](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=236s) So I'm starting on the class. I'm actually gonna re-trigger it.

[4:00](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=240s) Um, the first thing that happens is that Codium analyzes the class.

[4:06](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=246s) It basically maps out different behaviors, and it starts generating tests. You can see it starts, uh, streaming the test. I already have one, one, two. I'm getting more tests.

[4:16](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=256s) You can see some of them are quite complex.

[4:20](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=260s) It also generate the code explanation, detailed code explanation that shows me how this class actually works, the example usage, the different components, the methods. Uh, very detailed.

[4:33](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=273s) And then I have all the... all my tests.

[4:40](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=280s) As you can see, we look at different, uh, examples, both happy path, edge cases, variety of cases.

[4:56](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=296s) Okay, so here I have the different behaviors

[4:59](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=299s) that were generated. Now, this is crucial. We're basically mapping the different behaviors of this class, doing both happy path, edge cases, and for each one of them, we can drill deeper down and see the sub-behaviors below them.

[5:13](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=313s) And we can generate tests for any one that is important for us. So let's pick a few and add additional tests.

[5:19](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=319s) Let's pick some, uh, uh, edge cases as well. Let's generate test here. Maybe here we'll generate another one for an edge case.

[5:30](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=330s) And you can see it's very simple. A few clicks and I'm having a... and I have a test suite that is built out. I already have nine tests here.

[5:38](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=338s) The next step will be to run these tests, so let's go ahead and do that.

[5:47](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=347s) So I'm hitting Run & Autofix. You can see some of these very complex tests are actually passing.

[5:54](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=354s) And here I have a test that actually failed. What happens in a failure is that the model actually analyzes, reflects on the failure, and then it tries to generate a fix in an automated manner.

[6:07](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=367s) So we have a fix, uh, generated. And now it's gonna be run.

[6:15](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=375s) And it passed on the second try. So this is a, this, this chain of thought, this reflection process in order to get to a high-quality test suite.

[6:26](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=386s) Okay. So I'm gonna start with these eight tests. Let's open them as a file.

[6:34](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=394s) I'm gonna save them in my project. And done.

[6:51](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=411s) I have a test suite that now protects me.

[6:54](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=414s) So now I'm gonna go ahead and take the next step. Let's use CodiumAI to actually enhance this code

[7:02](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=422s) now that I have a test suite that protects me. So I'm gonna choose a method here, the build method, that has a lot of the main functionality of the class.

[7:13](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=433s) I'm gonna trigger CodiumAI on that. And now let's focus on the code suggestions component of CodiumAI.

[7:24](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=444s) So Codium analyzes this code, and it basically recommends different improvements, enhancements. And these are deep enhancements. We're not talking about linting or, or things like that. We're talking about things related to performance, security, best practices, readability.

[7:45](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=465s) So I'm gonna look at these. Let's, uh, choose one that, uh, makes sense. Maybe the first one that looks quite important for performance. Basically, it recommends to replace, um, the hashlib with, uh, BLAKE3.

[8:01](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=481s) I'm gonna prepare the code changes and apply it to my code.

[8:09](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=489s) And now I can save this. But remember, now I have a test suite. So now actually, I can actually go to my test suite

[8:19](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=499s) and run it. And of course, it broke on me for some reason, as things happen in a demo. But,

[8:31](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=511s) uh, let's see this again. Okay. I have one test that failed.

[8:42](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=522s) I'm gonna ignore that for now. [laughs] Okay. So let's continue.

[8:54](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=534s) I created my test suite. I, I enhanced my code. The next step would be to prepare for my PR. So I'm gonna go ahead here and commit these changes.

[9:09](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=549s) And I'm gonna go to the CodiumAI PR Assistant, and I'm gonna do a /commit to get a commit message.

[9:19](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=559s) And now I have a commit message, so I can commit.

[9:27](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=567s) And now that I committed my changes, I can then go ahead to the last step

[9:32](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=572s) and prepare for the PR. So I'm gonna do a /review.

[9:37](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=577s) Um, and that's basically a review process that CodiumAI would do, and it will try to see if there's any issues, anything I may have missed. It will summarize the PR.

[9:47](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=587s) It will give it a score. And then we can see, um, if there's anything that I maybe I have missed here. Let's take a look.

[9:55](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=595s) So this is the main theme of the PR. We can see that it's tests. You can see that it's, it's basically telling me that it's pretty well-structured.

[10:06](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=606s) Let's let it continue. But it says that it does introduce a, a potential security vulnerability. So I'm gonna do /improve to try to fix that.

[10:20](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=620s) And it looks like I forgot an API key in my code.

[10:27](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=627s) So CodiumAI will then, uh, suggest a fix for this, and I can actually see the API key in my code.

[10:39](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=639s) Let's give it a second. Looks like... I'm gonna do it again.

[10:54](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=654s) And this is where I actually have the API key in my code.

[10:59](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=659s) The network is bad in this room.

[11:01](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=661s) Yeah. No, here we go. So basically, it's saying, uh, here's the API key. I'm gonna click on this, and it will launch me to where I actually forgot the API key.

[11:11](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=671s) Forgot the API key. Um, and this is the actual fix.

[11:16](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=676s) So, uh, with that, uh, I'm gonna conclude the demo so we can go back to the slides.

[11:23](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=683s) So we're able to see how we were able to, uh, use CodiumAI to map our behaviors, to generate tests, to review our code, and to do it throughout the entire life cycle.

[11:32](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=692s) We also have, as I mentioned, a Git plugin that enables us to do that inside of GitHub as well. I'm going to end, um, with a personal note.

[11:45](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=705s) So we're a company that is based in Israel. While we were on the plane on the way here, the Hamas terrorist organization launched a vicious attack on Israel.

[11:57](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=717s) The Hamas terrorists are not humans. They, they are animals.

[12:04](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=724s) Maybe not even animals. They entered into towns. They slaughtered men, women, and children. Innocent people in their home. And abducted many

[12:20](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=740s) into the Gaza Strip. This is a picture that my co-founder and CEO, Itamar, sent me.

[12:29](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=749s) He left his eight-month-pregnant wife at home and is now in military reserve duty. In the screen,

[12:37](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=757s) you can see a chart that shows the CodiumAI usage constantly increasing.

[12:43](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=763s) Behind it is his rifle. We will prevail.

[12:49](https://www.youtube.com/watch?v=oJz4wveQ4oM&t=769s) Thank you. [audience applauding] [upbeat music]
