# How I automate my own job at Hugging Face using agents

Niels Rogge

AI Engineer World's Fair 2026

[Talk page](https://ai.engineer/talks/FLUoowDJg4I-i-automate-my-own-job-at-hugging) · [Video](https://www.youtube.com/watch?v=FLUoowDJg4I)

## How I automate my own job at Hugging Face using agents

Niels Rogge turns repetitive requests for research artifacts into a nightly workflow, then uses autonomous agents to handle the replies, improve documentation, and report results.

Basic familiarity with Python, GitHub issues and pull requests, and LLM tool calling will help you follow the implementation choices.

### Getting research artifacts out of Google Drive

[0:28](https://www.youtube.com/watch?v=FLUoowDJg4I&t=28s)

What happens when an interesting research paper has public model weights, but nobody can easily find them? [Niels Rogge](https://nielsrogge.github.io/), a Belgian KU Leuven alumnus who describes five years working as a machine learning engineer at Hugging Face, encountered this repeatedly while following research on GitHub. Researchers published artifacts through Google Drive, GitHub releases, Dropbox, Zenodo, and other servers. The files existed, but finding and reusing them required following a scattered trail of links.

His intervention was straightforward: open a GitHub issue suggesting that the authors also publish their weights on Hugging Face, where hosting is free. Authors often agreed that the move made sense. This became a recurring part of the Community Science team's work—enough for Rogge to describe it as the Google Drive to the Hub team.

![Community Science slide showing a GitHub reply about moving weights and a video dataset to Hugging Face, above Niels Rogge’s Google Drive-to-hub comment.](https://ai.engineer/api/talk-essay-media/125abd5cab4d9d59af758625131f0154463878b32cdc49649ccfac2cf29ba893)

Community science as the “Google Drive -> hub team.”

The benefit goes beyond changing the download location. Hugging Face paper pages, sourced from arXiv, connect papers to their models and datasets. Metadata makes those artifacts searchable by task, language, and compatible library: someone looking for a depth estimation model or an LLM can filter for it instead of discovering it accidentally. Model cards and dataset cards supply documentation, while upload and download tooling makes the artifacts easier to use. Researchers gain visibility; downstream users gain a more direct path from a paper to its implementation.

### A repeatable procedure that outgrew one person

[3:13](https://www.youtube.com/watch?v=FLUoowDJg4I&t=193s)

The manual work consisted of repeated requests to release checkpoints or datasets, plus pull requests on the Hub to improve model cards and dataset cards. But the supply of research kept growing. Rogge points to hundreds of new arXiv papers each day and rising submission volumes at conferences such as NeurIPS. Writing each request personally could not keep pace.

The automation target was an existing procedure, with a clear branch based on where the artifacts already lived:

1. Find the paper's GitHub repository, if one is available.
2. Read its README and identify new models or datasets worth sharing.
3. If the artifacts are already on Hugging Face, inspect their cards and metadata; open a pull request when documentation needs improvement.
4. If the artifacts are absent, open a GitHub issue requesting publication on the Hub.
5. Follow up with the author.

The last step matters: creating a request starts a conversation rather than completing the work.

![Automating Community Science slide with a numbered workflow for reading a README, checking artifacts, opening a pull request or issue, and following up, beside an LLM diagram with retrieval, tools, and memory.](https://ai.engineer/api/talk-essay-media/77fdf65357b5283221bc230613ff4a09324db6cab550eac3631bfc553f5a1d31)

The outreach workflow, from finding a paper’s GitHub repository to following up with its author.

### Start with a controlled workflow

[5:05](https://www.youtube.com/watch?v=FLUoowDJg4I&t=305s)

[Anthropic's Building Effective AI Agents](https://www.anthropic.com/engineering/building-effective-agents) provides the architectural distinction Rogge uses. A **workflow** puts LLM calls inside a predefined path. An **autonomous agent** lets the model choose tools repeatedly until it considers the task complete.

| Design | Who chooses the path? | Main trade-off |
| --- | --- | --- |
| Workflow | Application code | More control; less flexibility |
| Autonomous agent | Model operating in a tool loop | More flexibility; less predictability |
| Hybrid | Code and model at different stages | Bounded autonomy where needed |

These are not mutually exclusive choices for an entire application. Different parts of the outreach process can use different amounts of autonomy.

For the initial build in 2024, Rogge chose the deterministic end of that spectrum. He took Anthropic's guidance to start simply—potentially with a single LLM API call—and avoid unnecessary agents or frameworks. His implementation replicated the manual procedure with direct LLM calls at individual steps. He later used Cursor and an Excalidraw MCP server to visualize the pipeline, but the runtime itself did not depend on an agent framework.

In Python, the routing decision can remain ordinary application code even when an LLM supplies the assessment. This compact example captures the artifact branch and produces a proposed action; it does not submit an issue or pull request:

```python
from dataclasses import dataclass
from typing import Literal

Action = Literal["request_upload", "improve_card", "no_change"]

@dataclass(frozen=True)
class ArtifactAssessment:
    on_hub: bool
    card_complete: bool
    metadata_complete: bool


def propose_action(artifact: ArtifactAssessment) -> Action:
    if not artifact.on_hub:
        return "request_upload"
    if not artifact.card_complete or not artifact.metadata_complete:
        return "improve_card"
    return "no_change"


assessment = ArtifactAssessment(
    on_hub=True,
    card_complete=True,
    metadata_complete=False,
)
proposed_action = propose_action(assessment)
assert proposed_action == "improve_card"
```

The model can interpret research material while the surrounding program controls which branch runs next. That separation is the source of the workflow's predictability.

### Run it nightly, then inspect the traces

[6:59](https://www.youtube.com/watch?v=FLUoowDJg4I&t=419s)

Deployment was a nightly cron job: a Python script calling an LLM API, reading arXiv papers, and potentially opening GitHub issues or Hugging Face pull requests while Rogge slept. He used GitHub Actions, following [Free Cron Jobs with Github Actions](https://dev.to/anshuman_bhardwaj/free-cron-jobs-with-github-actions-31d6), and valued its accessible starting point and job-management UI. Scheduled Actions can be delayed, so this suits overnight processing rather than an exact-time guarantee. Rogge reports hundreds of GitHub issues created each night.

For observability, he uses **Langfuse** to inspect prompts, inputs, outputs, cost, and latency. The schedule answers when the program runs; traces answer what the model actually did during a run. That visibility becomes especially useful when a small recurring job produces a large volume of public-facing output.

### The next bottleneck was the inbox

[8:18](https://www.youtube.com/watch?v=FLUoowDJg4I&t=498s)

Automating issue creation moved the bottleneck downstream. Researchers replied, unread GitHub notifications accumulated, and Rogge still had to work through them like a mailbox. A few months before the talk, he automated that follow-up work too.

This time he chose an autonomous agent using the **Claude Agent SDK**. The immediate influence was an Anthropic workshop at AI Engineer New York in November of the previous year. Rogge interpreted its guidance as a shift toward agents because newer models could handle more of the control flow themselves. He accepted less predictability in exchange for flexibility.

Rogge cites a Cursor talk in London that described replacing a 12,000-line custom workflow with a 200-line skill. His own experience follows the same direction: work that previously required thousands of lines of custom logic can now be expressed through an agent, a command-line tool, and a skill. The skill supplies procedural knowledge; the model chooses how to apply it to the particular reply it encounters.

### An SDK, a CLI skill, and a sandbox

[10:25](https://www.youtube.com/watch?v=FLUoowDJg4I&t=625s)

The follow-up system uses the Python Claude Agent SDK. It initially ran **Claude** models; Rogge says he switched that week to **GLM 5.2** through [Hugging Face Inference Providers](https://huggingface.co/docs/inference-providers/index). That service brings providers such as Together.ai, Fireworks, and Cerebras behind a unified interface. He describes his integration as using OpenAI- or Anthropic-compatible interfaces.

Modal hosts the execution environment. The agent primarily uses **Bash** to run the Hugging Face CLI, with a Hugging Face CLI skill supplying instructions for Hub operations. It can then comment on GitHub and post final results to the team's Slack channel. The architecture separates three concerns: the SDK runs the agent loop, the CLI performs operations, and the skill explains how to use those operations.

Model choice was partly motivated by external evaluations. Rogge cites strong Cursor Bench performance and says GLM 5.2 beat **Opus 4.8** on **PostTrainBench** while costing less. PostTrainBench concerns agents post-training small language models, not research outreach: the historical comparison used four small base models, seven evaluation benchmarks, and a one-H100, ten-hour budget per run. Its June 2026 changelog supports the historical ranking, while later auditing changed result eligibility; the talk supplies neither an outreach comparison nor a price calculation.

### One invocation, many issue-processing agents

[11:58](https://www.youtube.com/watch?v=FLUoowDJg4I&t=718s)

Modal's batch processing provides the fan-out: parallel containers each run one agent loop for one GitHub issue. Rogge describes their startup as fast and finds the arrangement well suited to background work. The unit of parallelism is a conversation that needs follow-up, rather than an arbitrary fragment of a single agent's reasoning.

Although Modal supports cron scheduling, Rogge still starts follow-up manually. He invokes a Cursor skill he calls `Process Unread Model`; **Composer 2.5**, his usual Cursor agent, then invokes the other agents. Their results ultimately go to Slack. **Manual initiation and autonomous execution coexist:** he chooses when to process the backlog, while the workers handle the individual issues.

![Automating Community Science slide labeled Invoked as a Skill, with two workflow bullets and a terminal screenshot showing a Modal job running in the background.](https://ai.engineer/api/talk-essay-media/a267a3c5f42971c38237c15f567dd7af051cd29a0fdd7061b8b223f68eb750b1)

A skill fetches unread GitHub issues and Hub notifications, then posts results to Slack.

The Slack messages list research papers and the artifacts uploaded following outreach. Paper references in model cards or dataset cards cause those papers to be indexed on the Hub, connecting the completed documentation back to discoverability. Rogge says the messages appear a few minutes after he invokes the skill.

### Recipients experience it as personal outreach

[13:44](https://www.youtube.com/watch?v=FLUoowDJg4I&t=824s)

Rogge says he does not disclose that an agent is posting. His reasoning is that people may immediately close an issue from a bot, while the generated requests resemble the messages he previously wrote himself. Recipients address him personally and thank him for the suggestion and clear guidance. He also sees what he believes are agents replying to his agents, prompting a joke about the dead internet.

Rogge reports only two negative comments among thousands of created issues to date. One recipient asked him to close the issue as slop; he says most instead accept that publishing their weights or datasets on Hugging Face is useful. That is his account of recipient feedback under an undisclosed-automation policy, rather than a measured comparison of disclosed and undisclosed outreach.

### From release requests to better model cards

[14:51](https://www.youtube.com/watch?v=FLUoowDJg4I&t=891s)

The conversations sometimes return through other channels. An Apple researcher sent Rogge a direct message after an agent requested artifacts from an Apple paper. Other outreach concerned Google DeepMind mathematics datasets. Rogge also describes an email about publishing a 400-gigabyte dataset on Hugging Face. He reports that [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR), the OCR project, migrated all its OCR models to the Hub following agent-created issues.

Uploading files is only part of the work. Margaret Mitchell and her coauthors' [Model Cards for Model Reporting](https://arxiv.org/abs/1810.03993) motivates systematic model documentation, and Hugging Face provides a default template. In the Git diff Rogge presents, the agent fills that template using the paper's GitHub README, PDF, and related source material. The mechanism is source-based completion of an existing document structure.

One generated card credited Niels of the Hugging Face Community Science team as its author, even though Rogge had not explicitly requested that attribution. Other recipients thanked him for correcting their mistakes, although the agents had made the changes. These examples show how generated documentation can acquire a human identity as it passes into the normal contribution process.

A particularly visible example was the [release request for **Tiny Recursive Models**](https://github.com/SamsungSAILMontreal/TinyRecursiveModels/issues/2), which had been trending on Hugging Face and Twitter. Rogge reports more than 60 upvotes on the agent-created issue requesting its Hub release. The request illustrates demand from downstream users as well as the researcher's opportunity for greater visibility. Rogge says he has hundreds of positive issue examples; the public release-request issue itself does not establish that publication was completed.

### Scaling useful work also requires evaluation

[17:25](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1045s)

The obvious failure mode is an agent producing low-quality issues at internet scale. Rogge explicitly raises that concern and recommends Hamel Husain's LLM Evals FAQ, along with Husain's broader free writing and paid course. The talk points readers toward evaluation practice but does not describe an implemented evaluation suite or an acceptance threshold for outgoing messages.

His architectural conclusion is scoped to this work. He sees open models such as GLM 5.2 and **DeepSeek V4** becoming capable replacements for closed models. For his use case, he now favors agents over elaborate workflows: a Hugging Face CLI, a skill explaining its use, and a sandbox provide the core execution setup. Evaluation remains necessary even as the custom orchestration code shrinks.

### Reusing discovery for Daily Papers

[18:38](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1118s)

The same underlying research-discovery workflow powers Daily Papers, an account Rogge created on X. Rogge reports that Daily Papers passed 90,000 followers without his ongoing involvement. After deployment, it posts interesting research papers and Hugging Face artifacts every four hours, or when a noteworthy release appears.

**Gemini** selects the visual to accompany a post, extending automation from finding research to choosing how to present it. Rogge reports more than 2,000 likes on a Daily Papers post about NVIDIA releasing an optimized version of GLM 5.2. This is another destination for the discovery machinery: instead of asking researchers to publish artifacts, it tells potential users what has become available.

### Making benchmarks and research concepts accessible

[19:32](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1172s)

Rogge closes with an effort to revive [Papers with Code](https://paperswithcode.co/). He describes the original service as having been acquired by Meta and subsequently discontinued; his current effort aims to make research and the state of the art easier to access again.

The scope includes benchmark tracking, with OCR as his example, and educational explanations of concepts such as mid-training and on-policy distillation. The final slide places a mid-training explanation beside an `olmOCR-Bench` chart. This extends the Community Science goal beyond making artifacts available: readers also need ways to understand the research, locate relevant evaluations, and decide what to investigate next.

![Papers with Code(.co) slide showing a post explaining mid-training beside an olmOCR-Bench line chart.](https://ai.engineer/api/talk-essay-media/dc9aee8c9dc1a95a6cb3faae31c039d01e47d882f7e91658624d386fbedc39e7)

Papers with Code combines research explanations with benchmark tracking.

## Resources

- [Building Effective AI Agents](https://www.anthropic.com/engineering/building-effective-agents) — Anthropic's guidance on choosing simple LLM applications, workflows, and autonomous agents.
- [Free Cron Jobs with Github Actions](https://dev.to/anshuman_bhardwaj/free-cron-jobs-with-github-actions-31d6) — A scheduled GitHub Actions workflow tutorial with example YAML and scheduling-delay caveats.
- [Hugging Face Skills](https://github.com/huggingface/skills) — Current installation instructions for the hf-cli skill and additional Hugging Face agent workflows.
- [Tiny Recursive Models: Hugging Face release request](https://github.com/SamsungSAILMontreal/TinyRecursiveModels/issues/2) — Rogge's public request to publish TRM checkpoints and datasets on the Hub, including upload guidance.
- [Model Cards for Model Reporting](https://arxiv.org/abs/1810.03993) — Margaret Mitchell and coauthors' framework for documenting model uses, evaluation conditions, and limitations.
- [Papers with Code](https://paperswithcode.co/) — Browse machine-learning research, methods, benchmarks, datasets, and implementations.

## Slides

### How I automate my own job at Hugging Face with agents

Niels Rogge introduces his talk on automating his work at Hugging Face with agents.

[0:21](https://www.youtube.com/watch?v=FLUoowDJg4I&t=21s) [Title slide for Niels Rogge’s talk on automating his work with agents.](https://assets.ai.engineer/sites/aiecode2025/sha256/4646b032a6a02aec417005897e1e4783473e92ccc738626cc23c9926c7d532bc.webp)

How I automate my own
Job at Hugging Face
with agents

Niels Rogge
June 2026

Hugging Face

### Hi, I’m Niels

Niels is a machine learning engineer at Hugging Face, with a background from KU Leuven. He contributes to open source, provides AI advice and makes tutorials.

[0:26](https://www.youtube.com/watch?v=FLUoowDJg4I&t=26s) [Niels introduces his background, role at Hugging Face and interests, alongside his portrait.](https://assets.ai.engineer/sites/aiecode2025/sha256/19ea1dbfe913197dcc41bedcc85d67332ae5a1ce22fbf324aec814f9168a204f.webp)

Hi, I’m Niels

From Belgium (the land of 🍟, 🍺 and 🍫)

Graduated as Business and Informations Systems Engineer
KU LEUVEN

Machine Learning Engineer at Hugging Face
Hugging Face

Loves to:
• Contribute to open-source (Transformers library)
• Consult/provide AI advisory
• Make tutorials for people (Transformers-Tutorials, YouTube)

Hugging Face

### Content

Niels starts with Hugging Face’s community science team, then covers how he automates much of its work and other efforts at Hugging Face.

[0:40](https://www.youtube.com/watch?v=FLUoowDJg4I&t=40s) [Agenda lists community science, automating community science and other efforts.](https://assets.ai.engineer/sites/aiecode2025/sha256/c17c70c6184fcc68588ce92452b389508c2843619a7085e3c362eb9bce4dd68a.webp)

Content

👋 Community science
👋 Automating community science
👋 Other efforts

Hugging Face

### Community Science: model weights

Researchers often host model weights on Google Drive, GitHub releases and other servers, which the speaker says makes their work harder to discover.

[0:59](https://www.youtube.com/watch?v=FLUoowDJg4I&t=59s) [Screenshots show model weights and checkpoints distributed across several hosting locations.](https://assets.ai.engineer/sites/aiecode2025/sha256/df9e2d1cf0a5d00788f99144c5fa8a32475dd1c2b1688c61b7d0de318d8063af.webp)

Community Science

Download Models & Weights

1. Download SMPL-related weights.
- Download basicModel_f_lbs_10_207_0_v1.0.0.pkl, basicModel_m_lbs_10_207_0_v1.0.0.pkl, and basicModel_neutral_lbs_10_207_0_v1.0.0.pkl from here (female & male) and here (neutral) to ${Project}/weights/smpl_data/smpl. Please rename them as SMPL_FEMALE.pkl, SMPL_MALE.pkl, and SMPL_NEUTRAL.pkl, respectively.
- Download others from Google drive and put them to ${Project}/weights/smpl_data/smpl.

2. Download DINOv2 pretrained weights from their official repository. We use ViT-B/14 distilled (without registers). Please put dinov2_vitb14_pretrain.pth to ${Project}/weights/dinov2. These weights will be used to initialize our encoder. You can skip this step if you are not going to train SAT-HMR.

Model Checkpoints
Three model versions of the model are available [illegible]
running

from segment_anything import sam_model_re[illegible]
sam = sam_model_registry["<model_type>"]([illegible]

Click the links below to download the checkpoint
- default or vit_h: ViT-H SAM model.
- vit_l: ViT-L SAM model.
- vit_b: ViT-B SAM model.

checkpoint-v1
Latest
qjadud1994 released this Nov 4, 2024 · 8 commits to main
ckpt-v1
Update Install guidelines
Assets 4
- zim_vit_b_2043.tar
- zim_vit_l_2092.tar
- Source code (zip)
- Source code (tar.gz)

clip_dinoiser / checkpoints /
wysoczanska Update demos
Name
..
last.pt

first_stage.pt

Interface text:
https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
README
Code of conduct
Contributing
drive.google.com/file/[illegible]
Bestand
Hulp
Hugging Face

### Community Science: research outreach

After Niels Rogge suggests publishing research artifacts on Hugging Face, a researcher agrees to move YOLOv11 weights and a video dataset from Google Drive to the Hub and submit the paper for greater discoverability.

[1:31](https://www.youtube.com/watch?v=FLUoowDJg4I&t=91s) [A research author responds positively to a suggestion to move model weights and a dataset to the Hub.](https://assets.ai.engineer/sites/aiecode2025/sha256/449c4e664de3036c73e8026130382fbfed6fe4da40391dd24cb7493ba44d17ae.webp)

Community Science

Hi @NielsRogge,

Thank you for reaching out and for your interest! I really appreciate the invitation and the detailed suggestions.

We are definitely interested in improving the discoverability of our research. I will go ahead and submit the paper to hf.co/papers as suggested.

Regarding the artifacts, migrating the YOLOv11 .pt weights and our video dataset from Google Drive to the Hugging Face Hub makes perfect sense. I will look into the documentation you provided and start setting up the separate model and dataset repositories.

Thanks again for the info!

Best regards,
Jianlin

Niels Rogge, 9:52 PM
Community Science, also known as the Google Drive -> hub team

Hugging Face

Interface text:
JYe9 yesterday
Owner

### Community Science: linked research artifacts

The Ola paper page highlights linked models and a dataset in its sidebar, helping readers find research artifacts for reproduction.

[1:51](https://www.youtube.com/watch?v=FLUoowDJg4I&t=111s) [A paper page highlights models and a dataset linked to the research.](https://assets.ai.engineer/sites/aiecode2025/sha256/868004e857d712952455622e752cc06689dfd866426e4afa936e70dfc3960334.webp)

Community Science

Ola: Pushing the Frontiers of Omni-Modal Language Model with Progressive Modality Alignment

Published on Feb 6, 2025 · Submitted by liuzuyan on Feb 7, 2025
Authors: Zuyan Liu, Yuhao Dong, Jiahui Wang, Ziwei Liu, Winston Hu, Jiwen Lu, Yongming Rao

Abstract

Ola is an omni-modal language model that achieves competitive performance across image, video, and audio understanding through progressive modality alignment and efficient cross-modal training.

Generated by Qwen/Qwen2.5-Coder-32B-Instruct

Recent advances in large language models, particularly following GPT-4o, have sparked increasing interest in developing omni-modal models capable of understanding more modalities. While some open-source alternatives have emerged, there is still a notable lag behind specialized single-modality models in performance. In this paper, we present Ola, an Omni-modal language model that achieves competitive performance across image, video, and audio understanding compared to specialized counterparts. The core design of Ola lies in its progressive modality alignment strategy that extends the supporting modality of the language model progressively. Our training pipeline begins with the most distinct modalities: image and text, then gradually expands the skill sets of the model using speech data that connects language and audio knowledge, and video data that connects all modalities. The progressive learning pipeline also enables us to maintain a relatively small size of the cross-modal alignment data, making developing omni-modal from existing vision-language models easy and less costly. Moreover, to unlock an advanced interactive experience like GPT-4o, we further design a sentence-wise decoding solution for streaming speech generation. Extensive experiments demonstrate that Ola surpasses existing open omni-modal LLMs across all modalities while achieving highly competitive performance compared to state-of-the-art specialized models of similar sizes. We aim to make Ola a fully open omni-modal understanding solution to advance future research in this emerging field. Model weights, code, and data are open-sourced at https://github.com/Ola-Omni/Ola.

Models citing this paper 5
THUdyh/Ola-7b
THUdyh/Ola-Image
THUdyh/Ola_speech_encoders
THUdyh/Ola-Video
Browse 5 models citing this paper

Datasets citing this paper 1
THUdyh/Ola-Data

Making sure researchers publish artifacts on the hub

Interface text:
Papers · arxiv:2502.04328
Copy markdown
Upvote 29
Get this paper in your agent:
hf papers read 2502.04328
Don't have the latest CLI?
Hugging Face

### Community Science: Discovery Filters

Hugging Face filters help people discover models by task, language and compatible library. The panels show options ranging from depth estimation to text-to-speech and dozens of languages.

[2:13](https://www.youtube.com/watch?v=FLUoowDJg4I&t=133s) [Three Hugging Face filter panels show tasks, libraries, and languages.](https://assets.ai.engineer/sites/aiecode2025/sha256/fa4205c570dc30de3610c73caf0dcfbce125651dc9d65fa5afcf6d22aa652265.webp)

Community Science

Tasks
Text Generation
Any-to-Any
Image-Text-to-Text
Image-to-Text
Image-to-Image
Text-to-Image
Text-to-Video
Text-to-Speech
+44

Parameters
<1B  6B  12B  32B  128B  >500B

Libraries
PyTorch
TensorFlow
JAX
Transformers
Diffusers
GGUF
MLX
Transformers.js
Safetensors
sentence-transformers
ONNX
+42

Apps
vLLM
llama.cpp
MLX LM
LM Studio
Ollama
Jan
Draw Things
+9

Multimodal
Audio-Text-to-Text
Image-Text-to-Text
Image-Text-to-Image
Image-Text-to-Video
Visual Question Answering
Document Question Answering
Video-Text-to-Text
Visual Document Retrieval
Any-to-Any

Computer Vision
Depth Estimation
Image Classification
Object Detection
Image Segmentation
Text-to-Image
Image-to-Text
Image-to-Image
Image-to-Video
Unconditional Image Generation
Video Classification
Text-to-Video
Zero-Shot Image Classification
Mask Generation
Zero-Shot Object Detection
Text-to-3D
Image-to-3D
Image Feature Extraction
Keypoint Detection
Video-to-Video

Languages
English, Chinese, French, Spanish
German, Japanese, Korean, Portuguese
Italian, Russian, Arabic, Hindi, Thai
multilingual, Turkish, Vietnamese, Indonesian
Polish, Dutch, Swedish, Ukrainian
Romanian, Finnish, Persian, Czech
Bengali, Danish, Hebrew, Greek, Malay
Nepali, Hungarian, Tamil, Urdu
Bulgarian, Catalan, Norwegian, Telugu
Swahili, Marathi, Serbian, Slovenian
Slovak, Estonian, Gujarati, Lithuanian
Burmese, Croatian, French, Latvian
Malayalam, Icelandic, Tagalog, Galician
Panjabi, Kannada, Khmer, Basque
Afrikaans, Kazakh, Amharic, Georgian

Interface text:
Hugging Face
Search models, datasets, users...
Main  Tasks  Libraries  Languages  Licenses  Other
Filter Tasks by name
Filter Languages by name
Hugging Face

### Community Science: Artifact Metadata

Hugging Face model and dataset pages display tags for tasks, libraries, languages and licenses, helping people discover relevant artifacts.

[2:31](https://www.youtube.com/watch?v=FLUoowDJg4I&t=151s) [Three model and dataset page excerpts highlight metadata tags.](https://assets.ai.engineer/sites/aiecode2025/sha256/f7c707bcfa483f13579840661debd4b7d751dc2f5d3dd80045c156d6977b8ace.webp)

Community Science

Interface text:
MiniMaxAI / MiniMax-M3
like 1.11k
Follow MiniMax 7.03k
Image-Text-to-Text
Transformers
Safetensors
minimax_m3_vl
multimodal
License: minimax-community
Model card
Files and versions
xet
Community 16

fishaudio / s2-pro
like 1.04k
Follow Fish Aud[illegible]
Text-to-Speech
Safetensors
83 languages
instruction-followin[illegible]
Model card
Files and versions
xet
Community 24

Datasets: EDS-lab / electricity-demand
like 26
Tasks: Time Series Forecasting
Modalities: Tabular Text
Formats:
Libraries: Datasets pandas Croissant +1
License: bsd-3-clause
Dataset card
Data Studio
Files and versions
xet
Comm[illegible]

Hugging Face

### Scaling Community Science: GitHub Issues

Niels Rogge used a repeatable GitHub issue template to ask researchers to release model checkpoints and datasets on Hugging Face, highlighting discoverability and links to papers.

[3:11](https://www.youtube.com/watch?v=FLUoowDJg4I&t=191s) [Two GitHub issues request release of model checkpoints and a dataset on Hugging Face.](https://assets.ai.engineer/sites/aiecode2025/sha256/1da9cf7ec575feb4372729922729feabd92bb3d14f051b00a6f40204ad5f9d54.webp)

Scaling Community Science: Github issues

Release HUVR model checkpoints on Hugging Face #1
Closed
NielsRogge opened on Jan 21

Hi @[illegible] 👋

I'm Niels and work as part of the open-source team at Hugging Face. I discovered your work on Arxiv ("Implicit Neural Representation Facilitates Unified Universal Vision Encoding") and saw that you've already set up a GitHub repository for it: https://github.com/tiktok/huvr.

I noticed in the README that the Hugging Face model weight links for your open-source ViT-B/16 and ViT-L/16 models are currently marked as "TODO". I was wondering if you are still planning to host these on the Hub?

Hosting on Hugging Face would provide better visibility and discoverability for your work. We can also help link the models directly to the Hugging Face paper page (https://huggingface.co/papers/2601.14256) so that researchers can easily find the artifacts associated with the paper.

If you're interested, you can find a guide for uploading models here. For custom architectures, you might find the PyTorchModelHubMixin useful for adding from_pretrained and push_to_hub methods directly to your model class.

Additionally, we can provide you a ZeroGPU grant if you'd like to build a Gradio demo for HUVR on Spaces, which gives you A100 GPUs for free.

Let me know if you need any guidance or help with the process!

Kind regards,
Niels

Releasing models

Release Physics-IQ on Hugging Face #1
Closed
NielsRogge opened on Jan 18, 2025

Hello @rgeirhos 👋

I'm Niels and work as part of the open-source team at Hugging Face. I discovered your work through Hugging Face's daily papers as yours got featured: https://huggingface.co/papers/2501.09038.
The paper page lets people discuss about your paper and lets them find artifacts about it (your dataset for instance), you can also claim the paper as yours which will show up on your public profile at HF.

Would you like to host the datasets you've released on https://huggingface.co/datasets?
I see you're using Google Cloud Storage for it. Hosting on Hugging Face will give you more visibility/enable better discoverability, and will also allow people to do:

from datasets import load_dataset

dataset = load_dataset("your-hf-org-or-username/your-dataset")

If you're down, leaving a guide here: https://huggingface.co/docs/datasets/loading
We also support Webdataset, useful for image/video datasets: https://huggingface.co/docs/datasets/en/loading#webdataset.

Besides that, there's the dataset viewer which allows people to quickly explore the first few rows of the data in the browser.

After uploaded, we can also link the datasets to the paper page (read here) so people can discover your work.

Let me know if you're interested/need any guidance.

Kind regards,
Niels

Releasing datasets

Interface text:
Hugging Face

### Scaling Community Science: Hub Pull Requests

Niels opens Hub pull requests to improve model and dataset documentation, adding descriptions, paper links, usage examples and citations.

[3:22](https://www.youtube.com/watch?v=FLUoowDJg4I&t=202s) [Two Hub pull requests propose improvements to a model card and a dataset card.](https://assets.ai.engineer/sites/aiecode2025/sha256/712bf73f92e84590c2004998c5f763841cb62f6edc31d1b6757c3aa0d70b24c8.webp)

Scaling Community Science: Hub pull requests

Add model details, paper links and usage information
Hi, I'm Niels, part of the community science team at Hugging Face.

I noticed this model card was missing some information. This PR:
• Adds a description of Boxer and BoxerNet.
• Links the model card to the corresponding paper, project page, and GitHub repository.
• Adds the object-detection pipeline tag for better discoverability.
• Adds a sample usage section using the CLI commands found in the README.
• Adds a citation section.

Add dataset card for PufferDrive
This PR improves the dataset card by adding:
• Metadata for the reinforcement-learning task category.
• A link to the associated paper: “Human-like autonomy emerges from self-play and a pinch of human data”.
• Links to the project page and GitHub repository (PufferDrive).
• Sample usage for data preparation.
• A BibTeX citation for the project.

Add or improve model cards
Add dataset cards
Hugging Face

Interface text:
facebook/boxer
Object Detection · facebook/boxer · 3d-object-detection · open-world-detection · 3d-visi[illegible]
Model card · Files and versions · Community 1
by nielsr · HF Staff · opened Apr 10
Discussion · Files changed · +45 -2

Datasets: daphne-cornelisse/pufferdrive_womd_train
License: mit
Dataset card · Files and versions · Community 1
by nielsr · HF Staff · opened 5 minutes ago
Discussion · Files changed · +46 -3

### Scaling Community Science: The Scale Problem

Growing research submissions make manual outreach hard to scale. The slide pairs a rising submissions chart with a NeurIPS post reporting 25,000 submissions and projecting 28,000 for 2025.

[3:32](https://www.youtube.com/watch?v=FLUoowDJg4I&t=212s) [A submissions growth chart and a post about rising NeurIPS submissions illustrate the scaling challenge.](https://assets.ai.engineer/sites/aiecode2025/sha256/42542bbba73f05d4cb00b8151a26bd8cc77c475b3ce4f26f56265e973ad93459.webp)

Scaling Community Science

Problem: not really scalable

Total number of submissions as of June 19, 2026 = 3,080,173.
Y-axis ticks: 5,000; 10,000; 15,000; 20,000; 25,000; 30,000
Timeline: Jan '95; Jan '00; Jan '05; Jan '10; Jan '15; Jan '20; Jan '25

Sharon Li
@SharonYixuanLi

NeurIPS has a record-breaking number of 25,000 submissions this year already. It's crazy but not entirely unexpected.

Here is the scaling over the years, with a conservative projection.

How do we even run a conference and review at this massive scale?

NeurIPS Submissions Over the Years with 2025 Projection
Legend: 2025 Projection (28,000)
Y-axis: Number of Submissions
Y-axis ticks: 0; 5000; 10000; 15000; 20000; 25000; 30000
X-axis: Year
Timeline: 2014; 2015; 2016; 2017; 2018; 2019; 2020; 2021; 2022; 2023; 2024; 2025

Hugging Face

Interface text:
Download CSV
3:48 AM · May 12, 2025 · 83K Views

### Automating Community Science: Outreach

The proposed agent would find a paper’s GitHub repository, check its artifacts on Hugging Face, then open a pull request or issue and follow up with the author.

[4:04](https://www.youtube.com/watch?v=FLUoowDJg4I&t=244s) [Workflow for finding a paper’s GitHub repository, checking artifacts, opening a pull request or issue, and following up with the author.](https://assets.ai.engineer/sites/aiecode2025/sha256/fc89a6894d0b7956d560f2acfff4383a5b98cdbf69115f6058d8b5430eabfd66.webp)

Automating Community Science

Idea: have an AI agent automatically reach out to authors

Workflow:
1. Identify Github URL based on arXiv paper
2. Read Github README
3. Check for new artifacts
   a. If yes => check on HF already
      i. If yes: Open PR
   b. If no: open GH issue
4. Follow-up with author

Diagram:
In → LLM → Out
Query/Results: Retrieval
Call/Response: Tools
Read/Write: Memory

Hugging Face

### Workflow or AI Agent?

A workflow follows a predefined path through LLM calls and a gate, trading flexibility for predictability. An agent loops between an LLM and its environment, gaining flexibility but losing predictability.

[5:04](https://www.youtube.com/watch?v=FLUoowDJg4I&t=304s) [Diagrams compare a predefined workflow with an agent interacting with an environment, alongside their predictability and flexibility tradeoffs.](https://assets.ai.engineer/sites/aiecode2025/sha256/92ddccf8ba90d8332ebbe9979c2c221ba240ffbda9fd40ff42ce7af806e79620.webp)

Automating Community Science

Workflow or AI agent?

Workflow diagram:
In → LLM Call 1 → Output 1 → Gate
Pass → LLM Call 2 → Output 2 → LLM Call 3 → Out
Fail → Exit

+ Predictability
+ Simpler to manage
- Flexibility

AI agent diagram:
Human ↔ LLM Call
LLM Call → Action → Environment
Environment → Feedback → LLM Call
LLM Call → Stop

+ Flexibility
- Predictability

Hugging Face

### Predictability and Flexibility

The slide plots a tradeoff: agentic workflows are more predictable, while autonomous and multi-agent systems offer more flexibility. A use case can mix elements along the curve.

[5:44](https://www.youtube.com/watch?v=FLUoowDJg4I&t=344s) [A descending curve plots predictability against flexibility, from an agentic workflow to an autonomous or multi-agent system, with a use case between them.](https://assets.ai.engineer/sites/aiecode2025/sha256/f148bd86031d21ddf1664d3b4e49a7f9ee2c976574c56b50f50590c067285175.webp)

Automating Community Science

Workflow or AI agent?
Not a binary story

Agentic Workflow
Your use case?
Autonomous agent/
Multi-agent system

Y-axis: Predictability
X-axis: Flexibility

Hugging Face

### Choosing a Workflow

The speaker chose a deterministic workflow: a gate controls a sequence of LLM calls. The slide contrasts its predictability and simpler management with an agent’s more flexible action–feedback loop.

[5:56](https://www.youtube.com/watch?v=FLUoowDJg4I&t=356s) [The workflow side of the workflow-versus-agent comparison is outlined, emphasizing predictability and simpler management.](https://assets.ai.engineer/sites/aiecode2025/sha256/d5490c4819e5149e73a956f520083806f1bbb4d879a78fddb90043c30fe87d64.webp)

Automating Community Science

Workflow or AI agent?

Workflow:
In → LLM Call 1 → Output 1 → Gate
Pass → LLM Call 2 → Output 2 → LLM Call 3 → Out
Fail → Exit
+ Predictability
+ Simpler to manage
- Flexibility

AI agent:
Human ↔ LLM Call
Action
Environment
Feedback
Stop
+ Flexibility
- Predictability

Hugging Face

### Workflow and Framework Guidance

The speaker chose a predictable, easier-to-manage workflow, accepting less flexibility. Anthropic’s guidance recommends starting with direct LLM API calls and understanding any framework’s underlying code.

[6:02](https://www.youtube.com/watch?v=FLUoowDJg4I&t=362s) [The highlighted workflow comparison appears beside an excerpt from Anthropic’s Building Effective Agents recommending starting with direct LLM API use.](https://assets.ai.engineer/sites/aiecode2025/sha256/859e32547d05c8de869d3c90b02b20884fa3e4c54210d1f1ec40f8b879406139.webp)

Automating Community Science

Workflow

In → LLM Call 1 → Gate
Output 1
Pass → LLM Call 2 → LLM Call 3 → Out
Output 2
Fail → Exit

+ Predictability
+ Simpler to manage
- Flexibility

When and how to use frameworks

There are many frameworks that make agentic systems easier to implement, including:
• The Claude Agent SDK;
• Strands Agents SDK by AWS;
• Rivet, a drag and drop GUI LLM workflow builder; and
• Vellum, another GUI tool for building and testing complex workflows.

These frameworks make it easy to get started by simplifying standard low-level tasks like calling LLMs, defining and parsing tools, and chaining calls together. However, they often create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug. They can also make it tempting to add complexity when a simpler setup would suffice.

We suggest that developers start by using LLM APIs directly: many patterns can be implemented in a few lines of code. If you do use a framework, ensure you understand the underlying code. Incorrect assumptions about what's under the hood are a common source of customer error.

See our cookbook for some sample implementations.

“Building effective agents” (Anthropic, 2024)

Hugging Face

### Outreach Processing Pipeline

The pipeline finds URLs in paper-related sources, extracts GitHub links from project pages, and aggregates five classifications. It can open issues or Hugging Face PRs before saving results to the Hub.

[6:28](https://www.youtube.com/watch?v=FLUoowDJg4I&t=388s) [A seven-step pipeline finds URLs, identifies GitHub repositories, classifies artifacts, takes a majority vote, opens issues and pull requests when appropriate, and persists results.](https://assets.ai.engineer/sites/aiecode2025/sha256/0892211ecdf96db2d6bf1514895b00c2fa511a83bbc867d34e3a14dcc91eb90d.webp)

Automating Community Science

Outreach Workflow

Single Paper Processing Pipeline (main.py)
uv run main.py --arxiv_id <id> --open_issue --open_prs

Step 1: Find URLs
From abstract, HF comments, HF Papers API, PDF (first 2 pages)

Step 2: Project -> GitHub
If only project URL, LLM extracts GitHub from project HTML

Step 3: Classify
NEW_ARTIFACTS
NO_ARTIFACTS
NO_CODE
NO_CODE_YET
x N=5 in parallel

Step 4: Majority Vote
LLM aggregates the 5 ensemble outputs into final result

Step 5: GitHub Issue
If reach-out decided & --open_issue: open issue on repo

Step 6: HF PRs
If new artifacts & --open_prs: open PRs on HF cards

Step 7: Persist
Merge results into HF dataset, push to Hub

Hugging Face

### Nightly deployment with GitHub Actions

A nightly GitHub Actions cron job runs a Python script with an LLM API to read hundreds of arXiv papers. The workflow may open GitHub issues or Hugging Face pull requests.

[6:57](https://www.youtube.com/watch?v=FLUoowDJg4I&t=417s) [A slide describes a nightly cron job that parses hundreds of arXiv papers, alongside GitHub Actions screenshots.](https://assets.ai.engineer/sites/aiecode2025/sha256/20214d4e22ac44c06cd677e3833182642f57363f369a017b82a1c5f635c524cd.webp)

Automating Community Science

Deployment: Simple CRON Job

Parses 100s of arXiv papers every night

Free Cron Jobs with Github Actions
#github #javascript #webdev #beginners

Scheduling tasks can be a handy way to automate work, and Cron Jobs is the way to do it. We all have that one recurring task we wish could be done automatically without manual intervention. Not to forget, "That is also for FREE".

Cron jobs are meant for performing regularly scheduled actions such as backups, report generation, and so on. Each of those tasks should be configured to recur indefinitely (for example: once a day/week/month); you can define the point in time within that interval when the job should start.

Hugging Face

Interface text:
huggingface / paper-parsing
Issues · Pull requests 5 · Actions · Projects · Wiki · Security and quality · Insights · Settings
New workflow
Parse Papers
main.yml
1,241 workflow runs
This workflow has a workflow_dispatch event trigger.
parse-papers-huggingface/new-cs-papers-2026-06-19-100-128
Parse Papers #1611: Manually run by github-actions Bot
parse-papers-huggingface/new-cs-papers-2026-06-19-50-100
Parse Papers #1610: Manually run by github-actions Bot
parse-papers-huggingface/new-cs-papers-2026-06-19-0-50
Parse Papers #1609: Manually run by github-actions Bot
parse-papers-2026-06-18
Parse Papers #1608: Manually run by github-actions Bot

### Tracing with Langfuse

The speaker uses Langfuse to inspect LLM inputs, outputs, prompts, costs and latency. The slide pairs an activity dashboard with a table of individual traces.

[7:50](https://www.youtube.com/watch?v=FLUoowDJg4I&t=470s) [A slide shows Langfuse dashboards for tracing and monitoring model activity.](https://assets.ai.engineer/sites/aiecode2025/sha256/fb7d496e43f2faa3ed14bdc912d9bb91ac4cd9ced3367df992a4f0413b6b7609.webp)

Automating Community Science

Tracing: Langfuse

Traces
270 Total traces tracked
Step 4: 119
generate: 107
Step 1: 11
improve metad...: 11
Generate pul...: 11
Axis ticks: 0, 30, 60, 90, 120

Model costs
$194.48 Total cost
Model
Tokens
[illegible]

Observations by time
Observations by Level
6.35K Observations tracked
Legend: DEFAULT 6.292K, ERROR 61
Y-axis ticks: 0, 400, 800, 1.2K, 1.6K
Timeline: [illegible]

Model Usage
Cost by model
$194.48
Y-axis ticks: $0.00, $15.00, $30.00, $45.00, $60.00

langfuse
by ClickHouse

Hugging Face

Interface text:
Hugging Face / Hobby / Paper Parsing
Home
Past 1 day
Env default
Filters
Tracing
Start Time
Type
Name
Trace Name
Input
Output
Metadata
Rows per page 50
Page 1
[Other interface text illegible]

### Unread GitHub notifications

The speaker’s agents open GitHub issues every night, and replies leave him with many unread notifications. He later automated the follow-up as well.

[8:17](https://www.youtube.com/watch?v=FLUoowDJg4I&t=497s) [A slide illustrates many unread GitHub notifications using three overlapping inbox screenshots.](https://assets.ai.engineer/sites/aiecode2025/sha256/52a960ec911604e7d27dda99fbb41b172822267232581021468b87ea07a0d14d.webp)

Automating Community Science

Results: many, many unread GitHub notifications 😰

ZhaoZ77/EEGDancer #1
Release EEGDancer on Hugging Face
OpenDFM/Xcientist #1
Release Xcientist Paper-Graph dataset on Hugging Face
AaronJi/MeGan #1
Release MeGan on Hugging Face
gong-io/call-playbook #3
Release Call Playbook Dataset on Hugging Face
Y-Research-SBU/NeuroSonic #1
Release NeuroSonic models on Hugging Face
AgibotTech/GE-Sim-V2 #1
Release GE-Sim 2.0 on Hugging Face
ShenzhiYang2000/OPRD #1
Release OPRD model checkpoints on Hugging Face

AgentACE-AI/ProAct #1
Release ProActEval on Hugging Face
IBM/Evoflux #1
Release Evoflux artifacts (models, dataset) on Hugging [illegible]
Hongcheng-Gao/SpatialWorld #1
Release SpatialWorld dataset on Hugging Face
jiang0/MyPCBench #2
Release MyPCBench tasks on Hugging Face
Cynthiazyx123/SAMA #1
Release artifacts (models, dataset) on Hugging Face
microsoft/RPG-ZeroRepo #3
Release RPG-Encoder artifacts on Hugging Face
InternScience/SciReason #1
Release artifacts (SciReasoner-8B model, instruct tun[illegible]
zexiJia/1d-Distance #1
Release CMMS regressor on Hugging Face
HongyuanLuke/frequencylaw #1
Release Textual Frequency Paired Dataset (TFPD) on [illegible]
awakening-ai/ReactMotion #1
Release ReactMotionNet on Hugging Face

BatFaceWayne/Multi4D #1
Release Multi4D on Hugging Face
Raven-July/CFPO #1
Release CFPO models on Hugging Face
HanjiangHu/NBF-LLM #1
Release NBF-LLM artifacts (model, dataset) on Hugging Face
HanjiangHu/VLESA #1
Release VLESA artifacts (EgoSafety dataset and safety filter) [illegible]
Dominator-Index/RMNP #1
Release RMNP on Hugging Face
khanak0509/AdversaBench #1
Release AdversaBench dataset on Hugging Face
Zixind/GFPO_LHC #1
Release CMS 2016 Trigger Control Datasets on Hugging Face

Hugging Face

Interface text:
All
Unread
is:unread
Select all

### Workflow or AI agent?

For GitHub issue follow-up, the speaker chose an autonomous agent: an LLM loops through actions and feedback. It offers more flexibility than a workflow, with less predictability.

[8:56](https://www.youtube.com/watch?v=FLUoowDJg4I&t=536s) [A slide compares a fixed workflow with an agent loop, contrasting predictability and flexibility.](https://assets.ai.engineer/sites/aiecode2025/sha256/816eca7a332941a560a8e7be2d057eb5458d4883c268f19a9347ce2d6022ee4e.webp)

Automating Community Science

Workflow or AI agent?

Workflow:
In → LLM Call 1 → Output 1 → Gate
Pass → LLM Call 2 → Output 2 → LLM Call 3 → Out
Fail → Exit

+ Predictability
+ Simpler to manage
- Flexibility

AI agent:
Human ↔ LLM Call
LLM Call → Action → Environment
Environment → Feedback → LLM Call
LLM Call → Stop

+ Flexibility
- Predictability

Hugging Face

### Cursor example: replacing code with a skill

Cursor replaced a 12,000-line workflow with a 200-line skill. The speaker sees a similar opportunity with an agent, CLI and skill; the slide lists flexibility as an agent advantage and predictability as a drawback.

[9:48](https://www.youtube.com/watch?v=FLUoowDJg4I&t=588s) [An agent comparison slide includes a Cursor talk about replacing 12,000 lines of code with a 200-line skill.](https://assets.ai.engineer/sites/aiecode2025/sha256/f82c8d1359527e9b28f3973e74eb656de021bc0445670da2d02d73278e9d1478.webp)

Automating Community Science

AI Engineer EUROPE
CURSOR
[illegible code]
12K Lines
200 Lines
Replacing 12K LoC with a 200 LoC Skill — David Gomes, Cursor
Watch

Cursor, 2026

or
AI agent?

Human ↔ LLM Call
LLM Call → Environment: Action
Environment → LLM Call: Feedback
LLM Call → Stop

+ Flexibility
- Predictability

Hugging Face

### Community science agent architecture

A Modal-hosted Claude Agents SDK agent uses Bash and an hf-CLI skill for Hugging Face commands and GitHub follow-ups. It uses GLM-5.2 through Inference Providers and posts results to Slack.

[10:23](https://www.youtube.com/watch?v=FLUoowDJg4I&t=623s) [Diagram of an agent using the Claude Agents SDK, a command-line skill, Modal, and GLM-5.2 for GitHub follow-ups.](https://assets.ai.engineer/sites/aiecode2025/sha256/02a3498400fb88ca709c22adc6063d9ca2bcd302f2c669fd49c47e3c31ea05b0.webp)

Automating Community Science

Agent: automates GitHub follow-ups + Slack posting

GitHub issue URL

Modal

Claude Agents SDK

e.g. Bash(hf models info ...)
e.g. Bash (gh comment ...)

hf-CLI
SKILL.md

GLM-5.2

Inference Providers
Replicate · Together AI · SambaNova · fal · Novita · Groq · Cohere · Fireworks · Hyperbolic · Cerebras

Hugging Face

### Switching to GLM-5.2

The speaker replaces Claude with GLM-5.2, citing Cursor and PostTrainBench results. In a separate SDPO reproduction, alphaXiv reports a cost of $6.21 for GLM 5.2 versus $46.35 for Opus 4.8.

[11:34](https://www.youtube.com/watch?v=FLUoowDJg4I&t=694s) [Slide announcing the switch to GLM-5.2 with screenshots of benchmark and cost comparisons.](https://assets.ai.engineer/sites/aiecode2025/sha256/82ebf161a25059620ca283ed5517e5d336fd4d9e99bef2b93f7d6500a6f200c4.webp)

Automating Community Science

Agent: replaced Claude by GLM-5.2

Lee Robinson @leerob · Jun 25
You can now try GLM 5.2 in Cursor!

Excited to see more useful open models, thank you to Fireworks for partnering here. Results from our evals ↓

CursorBench 3.1 score
Y-axis ticks: 75%, 70%, 65%, 60%, 55%, 50%, 45%, 40%, 35%, 30%
X-axis: Average cost per task
X-axis ticks: $20, $16, $12, $8, $4, $0
Model labels: Fable 5 high; Composer 2.5; GPT-5.5 medium; Opus 4.8 high; GLM 5.2 high; Gemini 3.5 Flash; Sonnet 4.6 high; Kimi 2.6

Hardik Bhatnagar @hrdkbhatnagar
New #1 on PostTrainBench: GLM 5.2 (Max reasoning) hits 34.29%, narrowly beating Opus 4.8 Max (34.08%)

What makes GLM 5.2 interesting: zero failed runs across 84 runs (vs ~10% failure rate for Opus agents). The most reliable agent we've seen

Leaderboard: posttrainbench.com
Leaderboard table: [illegible]

alphaXiv @askalphaxiv · 19h
Here’s a fun comparison between GLM 5.2 and Opus 4.8 on a one-shot reproduction of the SDPO paper

This is a hard task: the model must resolve messy verl issues and then run ablations to completion and confirm the paper’s claims.

- GLM 5.2 costs $6.21 while Opus 4.8 cost us $46.35
- Both models spent a bulk of their tokens resolving initial verl issues. GLM 5.2 attempted 14 failed runs before first success while Opus 4.8 attempted 9 runs.
- GLM 5.2 surprisingly took 2.65M tokens (excl re-reads) compared to 4.53M tokens for Opus 4.8

Interface text:
163 · 217 · 2.7K · 270K
Hugging Face

### Parallel deployment on Modal

Modal batch processing runs each GitHub issue in a separate container, with one agent loop per issue. The dashboard shows multiple successful calls running in parallel.

[11:54](https://www.youtube.com/watch?v=FLUoowDJg4I&t=714s) [Slide describing Modal batch processing with a separate container for each GitHub issue, alongside a dashboard of successful calls.](https://assets.ai.engineer/sites/aiecode2025/sha256/c662ff0955600904fad91154b1f10f13d58589494ade7da467783aa5ca94d73e.webp)

Automating Community Science

Deployment
• Modal batch processing

Each GitHub issue:
• Separate container
• Massively parallel!

Hugging Face

Interface text:
github-issues-agent / process_single_issue
Containers: 0 live
Calls: 0 running
1h  Jun 19, 2:24 PM – now

Function call results
Y-axis ticks: 20, 10
Timeline: 02:25, 02:30, 02:35, 02:40, 02:45, 02:50, 02:55, 03 PM, 03:05, 03:10

Function Calls  Containers  Metrics  Details  Files
Status: Any

Enqueued (CEST) | Started (CEST) | Startup | Execution | Status
Jun 19, 15:14:43 | Jun 19, 15:14:50 | 2.28s | 54.22s | Succeeded
Jun 19, 15:14:43 | Jun 19, 15:14:50 | 2.61s | 1m 18s | Succeeded
Jun 19, 15:14:43 | Jun 19, 15:14:50 | 2.20s | 54.67s | Succeeded
Jun 19, 15:14:43 | Jun 19, 15:14:49 | 1.74s | 52.85s | Succeeded
Jun 19, 15:14:43 | Jun 19, 15:14:49 | 1.39s | 1m 6s | Succeeded
Jun 19, 15:14:43 | Jun 19, 15:14:49 | 2.61s | 1m 0s | Succeeded
Jun 19, 15:14:43 | Jun 19, 15:14:49 | 1.78s | 1m 52s | Succeeded
Jun 19, 15:14:43 | Jun 19, 15:14:49 | 2.28s | 1m 21s | Succeeded
Jun 19, 15:14:43 | Jun 19, 15:14:49 | 1.68s | 59.13s | Succeeded
Jun 19, 15:14:43 | Jun 19, 15:14:48 | 3.01s | 1m 13s | Succeeded
Jun 19, 15:14:43 | Jun 19, 15:14:48 | 1.60s | 39.42s | Succeeded
Jun 19, 15:14:43 | Jun 19, 15:14:48 | 2.51s | 39.93s | Succeeded

## Full transcript

[0:00](https://www.youtube.com/watch?v=FLUoowDJg4I&t=0s) [upbeat music] Okay.

[0:17](https://www.youtube.com/watch?v=FLUoowDJg4I&t=17s) All right. Hello, everyone. Thanks for coming by. Today, I'll talk about how I automate my own job, uh, at Hugging Face using agents.

[0:28](https://www.youtube.com/watch?v=FLUoowDJg4I&t=28s) Um, short introduction. I'm just, uh, Niels from Belgium, the land of beer, fries, and chocolate. I studied at KU Leuven, uh, and I'm a machine learning engineer at Hugging Face for five years now.

[0:42](https://www.youtube.com/watch?v=FLUoowDJg4I&t=42s) Uh, today, I'll talk about the community science team at Hugging Face, which is the team I'm part of. Uh, then I'll talk about how I automate large parts of the community science team, and finally, uh, also discuss some other efforts, uh, that we do at Hugging Face.

[0:58](https://www.youtube.com/watch?v=FLUoowDJg4I&t=58s) So let's start with the community science team at Hugging Face. So basically, this started when I was sen- uh, I was seeing, like, trending research, uh, pop- passing by on GitHub.

[1:09](https://www.youtube.com/watch?v=FLUoowDJg4I&t=69s) And a lot of times when I saw new interesting work, um, the weights were not available on Hugging Face, sadly. Like, researchers use either Google Drive or they use GitHub releases, they use Dropbox, they use Zenodo or other servers to put their, um, artifacts on, and this hurts, uh, discoverability of their work.

[1:29](https://www.youtube.com/watch?v=FLUoowDJg4I&t=89s) It's, like, not easily, uh, visible or discoverable. And when I then open a GitHub issue to say, like, "Actually, you could put your weights on Hugging Face for free," most of the time people, uh, reply to me like, "Yeah, migrating the weights, uh, from Google Drive to Hugging Face actually makes perfect sense."

[1:46](https://www.youtube.com/watch?v=FLUoowDJg4I&t=106s) So yeah, the community science team can also, uh, be described as the Google Drive to the Hub team.

[1:52](https://www.youtube.com/watch?v=FLUoowDJg4I&t=112s) Um, why? Because on Hugging Face, we have these paper pages, uh, and every single paper is sourced from arXiv. And then on the right side, you can basically list the linked artifacts, like the, the linked models or datasets, so people can easily, uh, reproduce your paper or find the models or datasets.

[2:11](https://www.youtube.com/watch?v=FLUoowDJg4I&t=131s) So yeah, you can see them on the right side. Um, this improves the discoverability of your work because we have these metadata tags or filters on the hub, so you can easily find, for example, a depth estimation model, an LLM if you're interested.

[2:25](https://www.youtube.com/watch?v=FLUoowDJg4I&t=145s) You can find them by language. You can tag them, uh, with the library they are compatible with and so on. So this improves the discoverability of your work. So these are, yeah, the metadata tags that you can add to every single model on Hugging Face or every single, uh, dataset.

[2:40](https://www.youtube.com/watch?v=FLUoowDJg4I&t=160s) So yeah, this is, like, the main problem that we saw. Like, uh, lots of people, lots of researchers are, like, using third-party services to publish their work. We have the Hugging Face platform, which is like a centralized place where people can find machine learning artifacts.

[2:53](https://www.youtube.com/watch?v=FLUoowDJg4I&t=173s) Uh, it also improves with documentation because you can add a model card or a dataset card. We have tooling so you can easily upload or download stuff from Hugging Face.

[3:02](https://www.youtube.com/watch?v=FLUoowDJg4I&t=182s) Uh, and it might also help res- uh, researchers in promoting their work. So it's basically a win-win, uh, both for researchers and then other people using the research.

[3:13](https://www.youtube.com/watch?v=FLUoowDJg4I&t=193s) So yeah, these are the typical GitHub issues that I was opening. Uh, I always had, like, the same template. I just asked, "Could you please release these checkpoints on Hugging Face?

[3:20](https://www.youtube.com/watch?v=FLUoowDJg4I&t=200s) Could you please release this dataset on Hugging Face?" And then I also opened PRs, pull requests, on Hugging Face to add dataset cards or model cards to improve the documentation of those artifacts.

[3:33](https://www.youtube.com/watch?v=FLUoowDJg4I&t=213s) But there's a problem. Uh, it's not really scalable for me to open all these GitHub issues or pull requests because every single day there are, like, hundreds of research papers coming out on arXiv, especially now with the AI boom.

[3:45](https://www.youtube.com/watch?v=FLUoowDJg4I&t=225s) Um, yeah, also NeurIPS, for example, a major AI conference, they are seeing a massive amount of papers. So can we automate this? Can we scale the community science team with agents?

[3:56](https://www.youtube.com/watch?v=FLUoowDJg4I&t=236s) So that's the second part of my talk. How can we, yeah, scale this, uh, to a massive amount of research papers?

[4:05](https://www.youtube.com/watch?v=FLUoowDJg4I&t=245s) So the idea is pretty simple. Uh, we should have an AI agent which can help me do this outreach to all these researchers which publish, uh, models or datasets, uh, as part of their research work, and then, yeah, do the outreach in an automated way.

[4:21](https://www.youtube.com/watch?v=FLUoowDJg4I&t=261s) So this is the typical workflow that I was following. So basically, whenever I saw a research paper, I first tried to find the GitHub URL, uh, of that paper, if it's available.

[4:31](https://www.youtube.com/watch?v=FLUoowDJg4I&t=271s) Then I, I read the README of that GitHub, uh, file, and then I basically check if there's anything new, uh, interesting to be shared on Hugging Face. Uh, it could be that it's on Hugging Face already.

[4:43](https://www.youtube.com/watch?v=FLUoowDJg4I&t=283s) In that case, I check whether the model cards or dataset cards are already properly, uh, present, whether the metadata tags, for example, are there. If, uh, not, then I might open a pull request.

[4:54](https://www.youtube.com/watch?v=FLUoowDJg4I&t=294s) Otherwise, if the artifacts are not yet on Hugging Face, I open a GitHub issue. And then finally, I also follow up with the author. So that's kind of the workflow that I had to automate, uh, with agents.

[5:05](https://www.youtube.com/watch?v=FLUoowDJg4I&t=305s) And there are several ways to solve this. Uh, you could, uh, go with a workflow. Uh, these pictures are, by the way, taken from the blog post, Building effective agents by Anthropic, which is a really great read, uh, read.

[5:17](https://www.youtube.com/watch?v=FLUoowDJg4I&t=317s) Um, so on the left side, you see, yeah, a workflow which is more deterministic. You basically use LLM APIs within steps of a predefined path or pipeline, uh, which is more predictable.

[5:28](https://www.youtube.com/watch?v=FLUoowDJg4I&t=328s) It's more deterministic. You have more control over it. Of course, it's less flex- flexible. And then on the other hand, you could have a fully-fledged autonom- uh, autonomous agent, which is an LLM in a loop that calls tools until it's done, which is more flexible but also less, uh, predictable.

[5:45](https://www.youtube.com/watch?v=FLUoowDJg4I&t=345s) Uh, at the time, yeah, of course, it doesn't have to be a binary story. Uh, you can have a workflow on one hand, you can have a fully autonomous agent on the other hand, but you could, you could of course also mix and match these type of things, uh, for your use case.

[5:57](https://www.youtube.com/watch?v=FLUoowDJg4I&t=357s) In my case, I went for, um, a pretty deterministic workflow. Uh, why? Because at the time that I was building this, this was in 2024, was at the time that Anthropic, uh, wrote their blog post, Building effective agents, and there they actually said, "Try to avoid building agents if you really don't have to.

[6:15](https://www.youtube.com/watch?v=FLUoowDJg4I&t=375s) Start simple. Start with a single LLM API."

[6:19](https://www.youtube.com/watch?v=FLUoowDJg4I&t=379s) Uh, avoid frameworks. Uh, and actually, I think those were great tips. So at the time, I started building a workflow which basically replicated the workflow that I was doing when I was doing this outreach.

[6:30](https://www.youtube.com/watch?v=FLUoowDJg4I&t=390s) So yeah, this is the whole, uh, pipeline. This is created using the Excalidraw MCP server and Cursor. It's pretty nice to create a visualization of your code. Uh, I'm not gonna go into the details, but basically it just replicates, um, the workflow that I was doing when doing the outreach.

[6:45](https://www.youtube.com/watch?v=FLUoowDJg4I&t=405s) And I use LLM APIs in each of the steps without any framework, without any agent framework. So it made it quite, uh, deterministic, and I had a lot of control over, uh, how this goes.

[6:59](https://www.youtube.com/watch?v=FLUoowDJg4I&t=419s) Um, in terms of deployment of this, uh, workflow, it's a simple cron job. So a cron is just something that runs regularly. In my case, I run it once every night.

[7:08](https://www.youtube.com/watch?v=FLUoowDJg4I&t=428s) So when I'm sleeping, there is this agent, but technically it's just a cron job. It's a Python script with an LLM API, which is gonna read all these hundreds of arXiv papers, uh, and then it might open GitHub issues, or it might open pull requests on Hugging Face.

[7:23](https://www.youtube.com/watch?v=FLUoowDJg4I&t=443s) I'm using GitHub Actions for this. Uh, I saw this very nice blog post, Free Cron Jobs with GitHub Actions, and actually it's probably the best entry point if you wanna set up cron jobs, um, because GitHub has a pretty generous tier if you wanna get started with, like, putting simple cron jobs, uh, up there.

[7:40](https://www.youtube.com/watch?v=FLUoowDJg4I&t=460s) And yeah, it makes it really easy for me, uh, in the UI to manage all these cron jobs. And so yeah, every night I have, uh, hundreds of, uh, GitHub issues being, uh, created.

[7:52](https://www.youtube.com/watch?v=FLUoowDJg4I&t=472s) For the tracing part, um, I'm using Langfuse. Uh, yeah, Langfuse also has a, a booth here. Um, Langfuse is pretty great. Um, I use it mostly for the tracing part, so the observability part, just to see what is the LLM doing, what are the inputs, what are the outputs, what are the prompts, uh, how much does it

[8:11](https://www.youtube.com/watch?v=FLUoowDJg4I&t=491s) cost, latency, and so on. Um, so yeah, uh, I definitely recommend it.

[8:18](https://www.youtube.com/watch?v=FLUoowDJg4I&t=498s) Um, but yeah, as my agents are opening so many GitHub issues every night, I then end up with a l- a massive amount of unread GitHub notifications because people reply to those GitHub issues.

[8:30](https://www.youtube.com/watch?v=FLUoowDJg4I&t=510s) And that's, uh, a lot of work to then reply to all of those issues. It's kind of like going through your, uh, mailbox.

[8:37](https://www.youtube.com/watch?v=FLUoowDJg4I&t=517s) So you could wonder, could we also, um, automate the follow-up to those GitHub, uh, issues? Because initially I was still -- the GitHub issue creation was done, uh, by an agent, but I was still the one involved in then doing the follow-up.

[8:51](https://www.youtube.com/watch?v=FLUoowDJg4I&t=531s) Uh, now a few months ago, I also automated the, the follow-up to those GitHub issues.

[8:57](https://www.youtube.com/watch?v=FLUoowDJg4I&t=537s) Again, you could think, how should you solve this? Should you go for a more deterministic workflow, or can you go for a fully autonomous agent, uh, an LLM in a loop which runs with some tools and skills?

[9:08](https://www.youtube.com/watch?v=FLUoowDJg4I&t=548s) Um, well, here I went for kind of a fully autonomous agent. Uh, so it's kind of flexible. It's a bit less predictable, but it works quite well. Um, I went for this because, uh, in November of last year at AI Engineer in New York, there was a pretty nice workshop by Anthropic on the Claude Agents SDK.

[9:28](https://www.youtube.com/watch?v=FLUoowDJg4I&t=568s) And there they were actually saying that agents might be better than workflows. So they were kind of contradicting themselves. But they also said that models have become so good that you might actually now start to work with fully autonomous agents rather than a workflow.

[9:42](https://www.youtube.com/watch?v=FLUoowDJg4I&t=582s) So this is why I went with this approach, and I actually am using the Claude, uh, Agents SDK for this use case. Uh, there was another pretty nice talk, uh, by Cursor, also at AI Engineer.

[9:54](https://www.youtube.com/watch?v=FLUoowDJg4I&t=594s) This was in the European version in London a few months ago. There they talked about how they replaced 12,000 lines of custom code, pretty sophisticated workflow, with a very simple 200 lines of code skill.

[10:07](https://www.youtube.com/watch?v=FLUoowDJg4I&t=607s) Uh, actually, it's pretty similar for me. Like, I can rep- uh, replace a lot of custom code, thousands of lines of code, with nowadays just a simple agent with maybe a CLI as a tool, uh, and a skill, and that's it.

[10:21](https://www.youtube.com/watch?v=FLUoowDJg4I&t=621s) Um, because the models have become so good.

[10:25](https://www.youtube.com/watch?v=FLUoowDJg4I&t=625s) So yeah, in terms of the, uh, architecture, this is a bit what it looks like. Um, so it's actually just the Claude Agents SDK, which is, I would say, a pretty good Python SDK for building an agent.

[10:37](https://www.youtube.com/watch?v=FLUoowDJg4I&t=637s) Initially, I was using the Claude models. Uh, but then I, since actually this week, I'm using the GLM 5.2 model via Hugging Face Inference Providers. So Hugging Face does offer a service, uh, which basically wraps a lot of inference providers like Together.ai, Fireworks, Cerebras, and so on.

[10:55](https://www.youtube.com/watch?v=FLUoowDJg4I&t=655s) So you can use a lot of open models, uh, in a unified way. It's OpenAI compatible, uh, or Anthropic compatible. And then, uh, I deploy this on a Modal.

[11:05](https://www.youtube.com/watch?v=FLUoowDJg4I&t=665s) Modal is also present here today. Um, and it's mainly using, uh, Bash as a tool, so the terminal to, uh, basically, um, do Hugging Face commands because it's using the Hugging Face CLI quite a bit.

[11:18](https://www.youtube.com/watch?v=FLUoowDJg4I&t=678s) So I, I combine it with the Hugging Face CLI skill, which is actually all it needs. And then, um, it might com-comment something on GitHub as a follow-up. And it also actually does the posting on Slack because eventually I also want to see the final results on our Slack channel, uh, from Hugging Face.

[11:36](https://www.youtube.com/watch?v=FLUoowDJg4I&t=696s) So yeah, given that there's also a lot of hype on GLM 5.2 recently. For example, Cursor, uh, saw great performance on their Cursor Bench. Post-training Bench is another one, uh, where it actually beats Opus 4.8 and it's cheaper.

[11:49](https://www.youtube.com/watch?v=FLUoowDJg4I&t=709s) So yeah, there's no reason not to use GLM 5.2, uh, especially given that I work at Hugging Face now. Um, for the deployment, as I said before, I use Modal.

[11:58](https://www.youtube.com/watch?v=FLUoowDJg4I&t=718s) Um, it's pretty great if you wanna deploy agents. Uh, in my case, I'm using the batch processing feature. So they allow you to spin up a massive amount of containers all in parallel.

[12:09](https://www.youtube.com/watch?v=FLUoowDJg4I&t=729s) Every single container is basically one agent loop that is processing one GitHub issue. Uh, it's super easy to use, I have to say. Um, and the startups are also pretty fast.

[12:21](https://www.youtube.com/watch?v=FLUoowDJg4I&t=741s) So I definitely recommend it if you're building, uh, agents that are like, for example, running in the background, running overnight, for example.

[12:29](https://www.youtube.com/watch?v=FLUoowDJg4I&t=749s) Um, and then the way I invoke it, yeah, technically, I could also just, uh, deploy this as a cron job Modal, for example, has support for this. But typically the follow-up on the GitHub issues, I still do that actually manually by invoking it as a skill.

[12:44](https://www.youtube.com/watch?v=FLUoowDJg4I&t=764s) So I created a skill for this in Cursor, uh, I call it Process Unread Model. And then what it's gonna do is it's actually gonna invoke an agent, in this case Composer 2.5, which is like the agent that I'm mostly using in Cursor, which is again gonna invoke all the other agents.

[13:00](https://www.youtube.com/watch?v=FLUoowDJg4I&t=780s) So that's, this is kind of the loop that people are talking about. And then finally, it's gonna post, uh, all the results on our Slack channel.

[13:09](https://www.youtube.com/watch?v=FLUoowDJg4I&t=789s) Uh, so yeah, and this is actually what it just post. So what it does is it basically just post a huge amount of Hugging Face papers, uh, which are these research papers which people can, uh, make available on the Hugging Face, because every time someone mentions it in a model card or dataset card, we index it on

[13:26](https://www.youtube.com/watch?v=FLUoowDJg4I&t=806s) the hub. And then it just posts all the artifacts that people have been uploading based on the outreach that we do via GitHub. Um, so yeah, I do this still, uh, in a manual form.

[13:36](https://www.youtube.com/watch?v=FLUoowDJg4I&t=816s) So I just invoke the skill and then after a few minutes, these messages, uh, appear on our Slack channel.

[13:44](https://www.youtube.com/watch?v=FLUoowDJg4I&t=824s) Um, yeah, I just included some fun results because to be honest, it's quite fun to see people interacting with the agents. Um, to be honest, I don't disclose that it's an agent.

[13:56](https://www.youtube.com/watch?v=FLUoowDJg4I&t=836s) Why? Because I think if people know it's a bot, then they might quickly like close the issue. And to be honest, they post exactly the same stuff as I was doing before manually.

[14:05](https://www.youtube.com/watch?v=FLUoowDJg4I&t=845s) So I don't actually see any reason to, to do that. Um, so and then you see replies like this, uh, "Hi, Niels. Thanks a lot for your su-suggestion and the clear guidance."

[14:16](https://www.youtube.com/watch?v=FLUoowDJg4I&t=856s) I actually also oftentimes see people using an agent to reply to my agents. Uh, so it's kind of the, dead internet nowadays. Um, but people, yeah, make all their artifacts available on Hugging Face.

[14:27](https://www.youtube.com/watch?v=FLUoowDJg4I&t=867s) And out of the thousands of issues that are being created on Hugging Face, actually so far, I've only had two, uh, negative comments. One [REDACTED:gender] saying, uh, yeah, "Please close this slop."

[14:37](https://www.youtube.com/watch?v=FLUoowDJg4I&t=877s) So he closed the issue and then another one. But most of the people, they just say, "Yeah, actually it makes perfect sense to make my weights or my datasets available on Hugging Face.

[14:45](https://www.youtube.com/watch?v=FLUoowDJg4I&t=885s) Like, why didn't I think of this?" Um, so it's kind of a win-win, I would say.

[14:51](https://www.youtube.com/watch?v=FLUoowDJg4I&t=891s) Uh, I oftentimes I'll also post fun results on our Slack channel. Like for example, one time someone, a researcher from Apple, uh, sent me a DM like, "I saw you reached out to me."

[15:01](https://www.youtube.com/watch?v=FLUoowDJg4I&t=901s) But yeah, technically it's my agent just posting a GitHub issue, uh, regarding publishing a new Apple, uh, the artifacts of an Apple paper on Hugging Face. Or for example, it reaches out to Google DeepMind to, uh, publish, um, mathematics datasets.

[15:17](https://www.youtube.com/watch?v=FLUoowDJg4I&t=917s) Um, so a lot of times like I receive emails, the one on the right side, where yeah, they want to publish a 400 gigabytes dataset on Hugging Face, but this was also my agent just, uh, opening GitHub issues.

[15:29](https://www.youtube.com/watch?v=FLUoowDJg4I&t=929s) Um, yeah, this is another fun result. So, uh, PaddleOCR, it's like a [REDACTED:origin], uh, company. They migrated all their OCR models to Hugging Face based on outreach by, uh, the agents that create is-issues for me.

[15:44](https://www.youtube.com/watch?v=FLUoowDJg4I&t=944s) So, um, yeah, it's pretty nice. Another fun result is like whether when it completes the default template of model cards on Hugging Face. So, uh, Meg Mitchell, who also works at, uh, Hugging Face, she has a famous paper called "Model Cards for Model Reporting," making sure that anyone documents their models, uh, in a proper way.

[16:04](https://www.youtube.com/watch?v=FLUoowDJg4I&t=964s) And so we do provide this template, which you can see on the left side in the Git diff. And then, uh, the agent is just completing that template based on the content that it finds based on that paper, like the GitHub README, uh, the PDF itself, and so on.

[16:21](https://www.youtube.com/watch?v=FLUoowDJg4I&t=981s) Um, yeah, it's also quite funny to see, for example, in this case that it, uh, included me in, uh, the model card. It said, uh, "Model card author's Niels, part of the Hugging Face Community Science team."

[16:32](https://www.youtube.com/watch?v=FLUoowDJg4I&t=992s) I never prompted it this way, but it's pretty fun to see. Or people, uh, replying, "Thank you for helping me fix my mistakes." So those are all done by, uh, the agents.

[16:44](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1004s) Uh, I think the most popular GitHub issue that was created was this, uh, paper, uh, tiny recursive models, which you might have seen was quite trending, um, both on Hugging Face but also on Twitter.

[16:56](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1016s) Uh, so yeah, more than 60 people actually up voted that issue so that the model was released on Hugging Face. So this is again, I think the win-win, so it's both a win for the researcher making their research more discoverable on Hugging Face, but it's also, yeah, better for the people then who want to build on top

[17:12](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1032s) of that research and want to use them.

[17:16](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1036s) Uh, so yeah, I have hundreds of GitHub issues where I think I can show, uh, nice results, um, where people interact with the agents.

[17:25](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1045s) You might also wonder, yeah, how to avoid slop because you might think, okay, you have an agent, uh, spamming the whole internet with your GitHub issues, like should you even do this?

[17:34](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1054s) Again, I already talked about the win-win. Um, but a blog post that I highly recommend if you wanna avoid that your agent is just posting slop is, um, the LLM Evals FAQ, uh, by Hamel Husain.

[17:48](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1068s) Uh, I would say he's like the main expert when it comes to LLM evaluation. He also has like a, a paid course, but he also publishes a lot of stuff for free online, including this blog post.

[17:59](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1079s) So I highly recommend to go through it if you wanna learn more about how to evaluate your agents.

[18:06](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1086s) So my conclusion would be, um, that open models are actually getting great, especially now with GLM 5.2, you have DeepSeek V4 and so on. So, um, yeah, we, we are able to now replace closed source models by open ones.

[18:20](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1100s) Uh, for my use case, I would say agents are actually better than, uh, workflows. Uh, they only need a single CLI, which is the Hugging Face CLI. They need a single skill, the Hugging Face CLI skill, and a sandbox, and that's all they need to do their work.

[18:32](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1112s) And finally, yeah, don't forget about evaluation. Um-

[18:38](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1118s) Finally, uh, I can also discuss some other efforts that we do as part of the Community Science, uh, team, um, very shortly. Um, so I have a Twitter account that I created.

[18:49](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1129s) It's called Daily Papers, and it actually uses the exact same workflow as my agents behind the scenes to post interesting research papers on X. It, uh, recently crossed 90,000 followers without any involvement of me.

[19:02](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1142s) I just deployed this, uh, and it posts interesting research papers and artifacts from Hugging Face every four hours or every time someone, uh, releases something cool on Hugging Face.

[19:13](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1153s) Um, so yeah. And I have like Gemini, uh, determining the best visual to tweet or to include in the tweet. Like, for example, this recent tweet, uh, where it tweeted out that NVIDIA released an optimized version of GLM 5.2, got more than 2,000 likes, so that's pretty cool, uh, to see.

[19:32](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1172s) And a final effort that I'm working on right now is a revival of, uh, Papers With Code, which is a website that once existed, then it was acquired by Meta, uh, and then sadly it, uh, died.

[19:43](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1183s) So I'm t- I'm trying to re- revive it in making, uh, research and state-of-the-art easier accessible. Um, for now it lives at paperswithcode.co.

[19:54](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1194s) Uh, so yeah, you can find benchmarks over there. Uh, for example, for OCR models, own OCR benches like Popular Benchmark. But I'm also, uh, making it an educational resource so that people can learn about technical terms like mid-training, uh, on-policy distillation, and so on.

[20:12](https://www.youtube.com/watch?v=FLUoowDJg4I&t=1212s) So yeah, uh, that was it for my talk. I hope, uh, you learned something. Thanks all for your attention. [audience applauding] [outro jingle]
