AI Engineer World's Fair 2026
How I automate my own job at Hugging Face using agents
Read the talk
Automating research outreach at Hugging Face
Niels Rogge turns repetitive requests for research artifacts into a nightly workflow, then uses autonomous agents to handle the replies, improve documentation, and report results.
From a talk by Niels Rogge
Before you start: Basic familiarity with Python, GitHub issues and pull requests, and LLM tool calling will help you follow the implementation choices.
Getting research artifacts out of Google Drive
What happens when an interesting research paper has public model weights, but nobody can easily find them? Niels Rogge, a Belgian KU Leuven alumnus who describes five years working as a machine learning engineer at Hugging Face, encountered this repeatedly while following research on GitHub. Researchers published artifacts through Google Drive, GitHub releases, Dropbox, Zenodo, and other servers. The files existed, but finding and reusing them required following a scattered trail of links.
His intervention was straightforward: open a GitHub issue suggesting that the authors also publish their weights on Hugging Face, where hosting is free. Authors often agreed that the move made sense. This became a recurring part of the Community Science team's work—enough for Rogge to describe it as the Google Drive to the Hub team.
The benefit goes beyond changing the download location. Hugging Face paper pages, sourced from arXiv, connect papers to their models and datasets. Metadata makes those artifacts searchable by task, language, and compatible library: someone looking for a depth estimation model or an LLM can filter for it instead of discovering it accidentally. Model cards and dataset cards supply documentation, while upload and download tooling makes the artifacts easier to use. Researchers gain visibility; downstream users gain a more direct path from a paper to its implementation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
A repeatable procedure that outgrew one person
The manual work consisted of repeated requests to release checkpoints or datasets, plus pull requests on the Hub to improve model cards and dataset cards. But the supply of research kept growing. Rogge points to hundreds of new arXiv papers each day and rising submission volumes at conferences such as NeurIPS. Writing each request personally could not keep pace.
The automation target was an existing procedure, with a clear branch based on where the artifacts already lived:
- Find the paper's GitHub repository, if one is available.
- Read its README and identify new models or datasets worth sharing.
- If the artifacts are already on Hugging Face, inspect their cards and metadata; open a pull request when documentation needs improvement.
- If the artifacts are absent, open a GitHub issue requesting publication on the Hub.
- Follow up with the author.
The last step matters: creating a request starts a conversation rather than completing the work.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Start with a controlled workflow
Anthropic's Building Effective AI Agents provides the architectural distinction Rogge uses. A workflow puts LLM calls inside a predefined path. An autonomous agent lets the model choose tools repeatedly until it considers the task complete.
| Design | Who chooses the path? | Main trade-off |
|---|---|---|
| Workflow | Application code | More control; less flexibility |
| Autonomous agent | Model operating in a tool loop | More flexibility; less predictability |
| Hybrid | Code and model at different stages | Bounded autonomy where needed |
These are not mutually exclusive choices for an entire application. Different parts of the outreach process can use different amounts of autonomy.
For the initial build in 2024, Rogge chose the deterministic end of that spectrum. He took Anthropic's guidance to start simply—potentially with a single LLM API call—and avoid unnecessary agents or frameworks. His implementation replicated the manual procedure with direct LLM calls at individual steps. He later used Cursor and an Excalidraw MCP server to visualize the pipeline, but the runtime itself did not depend on an agent framework.
In Python, the routing decision can remain ordinary application code even when an LLM supplies the assessment. This compact example captures the artifact branch and produces a proposed action; it does not submit an issue or pull request:
python
from dataclasses import dataclass
from typing import Literal
Action = Literal["request_upload", "improve_card", "no_change"]
@dataclass(frozen=True)
class ArtifactAssessment:
on_hub: bool
card_complete: bool
metadata_complete: bool
def propose_action(artifact: ArtifactAssessment) -> Action:
if not artifact.on_hub:
return "request_upload"
if not artifact.card_complete or not artifact.metadata_complete:
return "improve_card"
return "no_change"
assessment = ArtifactAssessment(
on_hub=True,
card_complete=True,
metadata_complete=False,
)
proposed_action = propose_action(assessment)
assert proposed_action == "improve_card"
The model can interpret research material while the surrounding program controls which branch runs next. That separation is the source of the workflow's predictability.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Run it nightly, then inspect the traces
Deployment was a nightly cron job: a Python script calling an LLM API, reading arXiv papers, and potentially opening GitHub issues or Hugging Face pull requests while Rogge slept. He used GitHub Actions, following Free Cron Jobs with Github Actions, and valued its accessible starting point and job-management UI. Scheduled Actions can be delayed, so this suits overnight processing rather than an exact-time guarantee. Rogge reports hundreds of GitHub issues created each night.
For observability, he uses Langfuse to inspect prompts, inputs, outputs, cost, and latency. The schedule answers when the program runs; traces answer what the model actually did during a run. That visibility becomes especially useful when a small recurring job produces a large volume of public-facing output.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The next bottleneck was the inbox
Automating issue creation moved the bottleneck downstream. Researchers replied, unread GitHub notifications accumulated, and Rogge still had to work through them like a mailbox. A few months before the talk, he automated that follow-up work too.
This time he chose an autonomous agent using the Claude Agent SDK. The immediate influence was an Anthropic workshop at AI Engineer New York in November of the previous year. Rogge interpreted its guidance as a shift toward agents because newer models could handle more of the control flow themselves. He accepted less predictability in exchange for flexibility.
Rogge cites a Cursor talk in London that described replacing a 12,000-line custom workflow with a 200-line skill. His own experience follows the same direction: work that previously required thousands of lines of custom logic can now be expressed through an agent, a command-line tool, and a skill. The skill supplies procedural knowledge; the model chooses how to apply it to the particular reply it encounters.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
An SDK, a CLI skill, and a sandbox
The follow-up system uses the Python Claude Agent SDK. It initially ran Claude models; Rogge says he switched that week to GLM 5.2 through Hugging Face Inference Providers. That service brings providers such as Together.ai, Fireworks, and Cerebras behind a unified interface. He describes his integration as using OpenAI- or Anthropic-compatible interfaces.
Modal hosts the execution environment. The agent primarily uses Bash to run the Hugging Face CLI, with a Hugging Face CLI skill supplying instructions for Hub operations. It can then comment on GitHub and post final results to the team's Slack channel. The architecture separates three concerns: the SDK runs the agent loop, the CLI performs operations, and the skill explains how to use those operations.
Model choice was partly motivated by external evaluations. Rogge cites strong Cursor Bench performance and says GLM 5.2 beat Opus 4.8 on PostTrainBench while costing less. PostTrainBench concerns agents post-training small language models, not research outreach: the historical comparison used four small base models, seven evaluation benchmarks, and a one-H100, ten-hour budget per run. Its June 2026 changelog supports the historical ranking, while later auditing changed result eligibility; the talk supplies neither an outreach comparison nor a price calculation.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
One invocation, many issue-processing agents
Modal's batch processing provides the fan-out: parallel containers each run one agent loop for one GitHub issue. Rogge describes their startup as fast and finds the arrangement well suited to background work. The unit of parallelism is a conversation that needs follow-up, rather than an arbitrary fragment of a single agent's reasoning.
Although Modal supports cron scheduling, Rogge still starts follow-up manually. He invokes a Cursor skill he calls Process Unread Model; Composer 2.5, his usual Cursor agent, then invokes the other agents. Their results ultimately go to Slack. Manual initiation and autonomous execution coexist: he chooses when to process the backlog, while the workers handle the individual issues.
The Slack messages list research papers and the artifacts uploaded following outreach. Paper references in model cards or dataset cards cause those papers to be indexed on the Hub, connecting the completed documentation back to discoverability. Rogge says the messages appear a few minutes after he invokes the skill.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Recipients experience it as personal outreach
Rogge says he does not disclose that an agent is posting. His reasoning is that people may immediately close an issue from a bot, while the generated requests resemble the messages he previously wrote himself. Recipients address him personally and thank him for the suggestion and clear guidance. He also sees what he believes are agents replying to his agents, prompting a joke about the dead internet.
Rogge reports only two negative comments among thousands of created issues to date. One recipient asked him to close the issue as slop; he says most instead accept that publishing their weights or datasets on Hugging Face is useful. That is his account of recipient feedback under an undisclosed-automation policy, rather than a measured comparison of disclosed and undisclosed outreach.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From release requests to better model cards
The conversations sometimes return through other channels. An Apple researcher sent Rogge a direct message after an agent requested artifacts from an Apple paper. Other outreach concerned Google DeepMind mathematics datasets. Rogge also describes an email about publishing a 400-gigabyte dataset on Hugging Face. He reports that PaddleOCR, the OCR project, migrated all its OCR models to the Hub following agent-created issues.
Uploading files is only part of the work. Margaret Mitchell and her coauthors' Model Cards for Model Reporting motivates systematic model documentation, and Hugging Face provides a default template. In the Git diff Rogge presents, the agent fills that template using the paper's GitHub README, PDF, and related source material. The mechanism is source-based completion of an existing document structure.
One generated card credited Niels of the Hugging Face Community Science team as its author, even though Rogge had not explicitly requested that attribution. Other recipients thanked him for correcting their mistakes, although the agents had made the changes. These examples show how generated documentation can acquire a human identity as it passes into the normal contribution process.
A particularly visible example was the release request for Tiny Recursive Models, which had been trending on Hugging Face and Twitter. Rogge reports more than 60 upvotes on the agent-created issue requesting its Hub release. The request illustrates demand from downstream users as well as the researcher's opportunity for greater visibility. Rogge says he has hundreds of positive issue examples; the public release-request issue itself does not establish that publication was completed.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Scaling useful work also requires evaluation
The obvious failure mode is an agent producing low-quality issues at internet scale. Rogge explicitly raises that concern and recommends Hamel Husain's LLM Evals FAQ, along with Husain's broader free writing and paid course. The talk points readers toward evaluation practice but does not describe an implemented evaluation suite or an acceptance threshold for outgoing messages.
His architectural conclusion is scoped to this work. He sees open models such as GLM 5.2 and DeepSeek V4 becoming capable replacements for closed models. For his use case, he now favors agents over elaborate workflows: a Hugging Face CLI, a skill explaining its use, and a sandbox provide the core execution setup. Evaluation remains necessary even as the custom orchestration code shrinks.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Reusing discovery for Daily Papers
The same underlying research-discovery workflow powers Daily Papers, an account Rogge created on X. Rogge reports that Daily Papers passed 90,000 followers without his ongoing involvement. After deployment, it posts interesting research papers and Hugging Face artifacts every four hours, or when a noteworthy release appears.
Gemini selects the visual to accompany a post, extending automation from finding research to choosing how to present it. Rogge reports more than 2,000 likes on a Daily Papers post about NVIDIA releasing an optimized version of GLM 5.2. This is another destination for the discovery machinery: instead of asking researchers to publish artifacts, it tells potential users what has become available.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Making benchmarks and research concepts accessible
Rogge closes with an effort to revive Papers with Code. He describes the original service as having been acquired by Meta and subsequently discontinued; his current effort aims to make research and the state of the art easier to access again.
The scope includes benchmark tracking, with OCR as his example, and educational explanations of concepts such as mid-training and on-policy distillation. The final slide places a mid-training explanation beside an olmOCR-Bench chart. This extends the Community Science goal beyond making artifacts available: readers also need ways to understand the research, locate relevant evaluations, and decide what to investigate next.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Anthropic's guidance on choosing simple LLM applications, workflows, and autonomous agents.
A scheduled GitHub Actions workflow tutorial with example YAML and scheduling-delay caveats.
Rogge's public request to publish TRM checkpoints and datasets on the Hub, including upload guidance.
Margaret Mitchell and coauthors' framework for documenting model uses, evaluation conditions, and limitations.
Browse machine-learning research, methods, benchmarks, datasets, and implementations.
Updates since the talk
Current installation instructions for the hf-cli skill and additional Hugging Face agent workflows.
Read the complete timestamped transcript
- 0:00
[upbeat music] Okay.
- 0:17
All right. Hello, everyone. Thanks for coming by. Today, I'll talk about how I automate my own job, uh, at Hugging Face using agents.
- 0:28
Um, short introduction. I'm just, uh, Niels from Belgium, the land of beer, fries, and chocolate. I studied at KU Leuven, uh, and I'm a machine learning engineer at Hugging Face for five years now.
- 0:42
Uh, today, I'll talk about the community science team at Hugging Face, which is the team I'm part of. Uh, then I'll talk about how I automate large parts of the community science team, and finally, uh, also discuss some other efforts, uh, that we do at Hugging Face.
- 0:58
So let's start with the community science team at Hugging Face. So basically, this started when I was sen- uh, I was seeing, like, trending research, uh, pop- passing by on GitHub.
- 1:09
And a lot of times when I saw new interesting work, um, the weights were not available on Hugging Face, sadly. Like, researchers use either Google Drive or they use GitHub releases, they use Dropbox, they use Zenodo or other servers to put their, um, artifacts on, and this hurts, uh, discoverability of their work.
- 1:29
It's, like, not easily, uh, visible or discoverable. And when I then open a GitHub issue to say, like, "Actually, you could put your weights on Hugging Face for free," most of the time people, uh, reply to me like, "Yeah, migrating the weights, uh, from Google Drive to Hugging Face actually makes perfect sense."
- 1:46
So yeah, the community science team can also, uh, be described as the Google Drive to the Hub team.
- 1:52
Um, why? Because on Hugging Face, we have these paper pages, uh, and every single paper is sourced from arXiv. And then on the right side, you can basically list the linked artifacts, like the, the linked models or datasets, so people can easily, uh, reproduce your paper or find the models or datasets.
- 2:11
So yeah, you can see them on the right side. Um, this improves the discoverability of your work because we have these metadata tags or filters on the hub, so you can easily find, for example, a depth estimation model, an LLM if you're interested.
- 2:25
You can find them by language. You can tag them, uh, with the library they are compatible with and so on. So this improves the discoverability of your work. So these are, yeah, the metadata tags that you can add to every single model on Hugging Face or every single, uh, dataset.
- 2:40
So yeah, this is, like, the main problem that we saw. Like, uh, lots of people, lots of researchers are, like, using third-party services to publish their work. We have the Hugging Face platform, which is like a centralized place where people can find machine learning artifacts.
- 2:53
Uh, it also improves with documentation because you can add a model card or a dataset card. We have tooling so you can easily upload or download stuff from Hugging Face.
- 3:02
Uh, and it might also help res- uh, researchers in promoting their work. So it's basically a win-win, uh, both for researchers and then other people using the research.
- 3:13
So yeah, these are the typical GitHub issues that I was opening. Uh, I always had, like, the same template. I just asked, "Could you please release these checkpoints on Hugging Face?
- 3:20
Could you please release this dataset on Hugging Face?" And then I also opened PRs, pull requests, on Hugging Face to add dataset cards or model cards to improve the documentation of those artifacts.
- 3:33
But there's a problem. Uh, it's not really scalable for me to open all these GitHub issues or pull requests because every single day there are, like, hundreds of research papers coming out on arXiv, especially now with the AI boom.
- 3:45
Um, yeah, also NeurIPS, for example, a major AI conference, they are seeing a massive amount of papers. So can we automate this? Can we scale the community science team with agents?
- 3:56
So that's the second part of my talk. How can we, yeah, scale this, uh, to a massive amount of research papers?
- 4:05
So the idea is pretty simple. Uh, we should have an AI agent which can help me do this outreach to all these researchers which publish, uh, models or datasets, uh, as part of their research work, and then, yeah, do the outreach in an automated way.
- 4:21
So this is the typical workflow that I was following. So basically, whenever I saw a research paper, I first tried to find the GitHub URL, uh, of that paper, if it's available.
- 4:31
Then I, I read the README of that GitHub, uh, file, and then I basically check if there's anything new, uh, interesting to be shared on Hugging Face. Uh, it could be that it's on Hugging Face already.
- 4:43
In that case, I check whether the model cards or dataset cards are already properly, uh, present, whether the metadata tags, for example, are there. If, uh, not, then I might open a pull request.
- 4:54
Otherwise, if the artifacts are not yet on Hugging Face, I open a GitHub issue. And then finally, I also follow up with the author. So that's kind of the workflow that I had to automate, uh, with agents.
- 5:05
And there are several ways to solve this. Uh, you could, uh, go with a workflow. Uh, these pictures are, by the way, taken from the blog post, Building effective agents by Anthropic, which is a really great read, uh, read.
- 5:17
Um, so on the left side, you see, yeah, a workflow which is more deterministic. You basically use LLM APIs within steps of a predefined path or pipeline, uh, which is more predictable.
- 5:28
It's more deterministic. You have more control over it. Of course, it's less flex- flexible. And then on the other hand, you could have a fully-fledged autonom- uh, autonomous agent, which is an LLM in a loop that calls tools until it's done, which is more flexible but also less, uh, predictable.
- 5:45
Uh, at the time, yeah, of course, it doesn't have to be a binary story. Uh, you can have a workflow on one hand, you can have a fully autonomous agent on the other hand, but you could, you could of course also mix and match these type of things, uh, for your use case.
- 5:57
In my case, I went for, um, a pretty deterministic workflow. Uh, why? Because at the time that I was building this, this was in 2024, was at the time that Anthropic, uh, wrote their blog post, Building effective agents, and there they actually said, "Try to avoid building agents if you really don't have to.
- 6:15
Start simple. Start with a single LLM API."
- 6:19
Uh, avoid frameworks. Uh, and actually, I think those were great tips. So at the time, I started building a workflow which basically replicated the workflow that I was doing when I was doing this outreach.
- 6:30
So yeah, this is the whole, uh, pipeline. This is created using the Excalidraw MCP server and Cursor. It's pretty nice to create a visualization of your code. Uh, I'm not gonna go into the details, but basically it just replicates, um, the workflow that I was doing when doing the outreach.
- 6:45
And I use LLM APIs and then each of the steps without any framework, without any agent framework. So it made it quite, uh, deterministic, and I had a lot of control over, uh, how this goes.
- 6:59
Um, in terms of deployment of this, uh, workflow, it's a simple cron job. So a cron is just something that runs regularly. In my case, I run it once every night.
- 7:08
So when I'm sleeping, there is this agent, but technically it's just a cron job. It's a Python script with an LLM API, which is gonna read all these hundreds of arXiv papers, uh, and then it might open GitHub issues, or it might open pull requests on Hugging Face.
- 7:23
I'm using GitHub Actions for this. Uh, I saw this very nice blog post, Free Cron Jobs with GitHub Actions, and actually it's probably the best entry point if you wanna set up cron jobs, um, because GitHub has a pretty generous tier if you wanna get started with, like, putting simple cron jobs, uh, up there.
- 7:40
And yeah, it makes it really easy for me, uh, in the UI to manage all these cron jobs. And so yeah, every night I have, uh, hundreds of, uh, GitHub issues being, uh, created.
- 7:52
For the tracing part, um, I'm using Langfuse. Uh, yeah, Langfuse also has a, a booth here. Um, Langfuse is pretty great. Um, I use it mostly for the tracing part, so the observability part, just to see what is the LLM doing, what are the inputs, what are the outputs, what are the prompts, uh, how much does it
- 8:11
cost, latency, and so on. Um, so yeah, uh, I definitely recommend it.
- 8:18
Um, but yeah, as my agents are opening so many GitHub issues every night, I then end up with a l- a massive amount of unread GitHub notifications because people reply to those GitHub issues.
- 8:30
And that's, uh, a lot of work to then reply to all of those issues. It's kind of like going through your, uh, mailbox.
- 8:37
So you could wonder, could we also, um, automate the follow-up to those GitHub, uh, issues? Because initially I was still -- the GitHub issue creation was done, uh, by an agent, but I was still the one involved in then doing the follow-up.
- 8:51
Uh, now a few months ago, I also automated the, the follow-up to those GitHub issues.
- 8:57
Again, you could think, how should you solve this? Should you go for a more deterministic workflow, or can you go for a fully autonomous agent, uh, an LLM in a loop which runs with some tools and skills?
- 9:08
Um, well, here I went for kind of a fully autonomous agent. Uh, so it's kind of flexible. It's a bit less predictable, but it works quite well. Um, I went for this because, uh, in November of last year at AI Engineer in New York, there was a pretty nice workshop by Anthropic on the Claude Agents SDK.
- 9:28
And there they were actually saying that agents might be better than workflows. So they were kind of contradicting themselves. But they also said that models have become so good that you might actually now start to work with fully autonomous agents rather than a workflow.
- 9:42
So this is why I went with this approach, and I actually am using the Claude, uh, Agents SDK for this use case. Uh, there was another pretty nice talk, uh, by Cursor, also at AI Engineer.
- 9:54
This was in the European version in London a few months ago. There they talked about how they replaced 12,000 lines of custom code, pretty sophisticated workflow, with a very simple 200 lines of code skill.
- 10:07
Uh, actually, it's pretty similar for me. Like, I can rep- uh, replace a lot of custom code, thousands of lines of code, with nowadays just a simple agent with maybe a CLI as a tool, uh, and a skill, and that's it.
- 10:21
Um, because the models have become so good.
- 10:25
So yeah, in terms of the, uh, architecture, this is a bit what it looks like. Um, so it's actually just the Claude Agents SDK, which is, I would say, a pretty good Python SDK for building an agent.
- 10:37
Initially, I was using the Claude models. Uh, but then I, since actually this week, I'm using the GLM 5.2 model via Hugging Face inference providers. So Hugging Face does offer a service, uh, which basically wraps a lot of inference providers like Together.ai, Fireworks, Cerebras, and so on.
- 10:55
So you can use a lot of open models, uh, in a unified way. It's OpenAI compatible, uh, or Anthropic compatible. And then, uh, I deploy this on a Modal.
- 11:05
Modal is also present here today. Um, and it's mainly using, uh, Bash as a tool, so the terminal to, uh, basically, um, do Hugging Face commands because it's using the Hugging Face CLI quite a bit.
- 11:18
So I, I combine it with the Hugging Face CLI skill, which is actually all it needs. And then, um, it might com-comment something on GitHub as a follow-up. And it also actually does the posting on Slack because eventually I also want to see the final results on our Slack channel, uh, from Hugging Face.
- 11:36
So yeah, given that there's also a lot of hype on GLM 5.2 recently. For example, Cursor, uh, saw great performance on their Cursor Bench. Post-training Bench is another one, uh, where it actually beats Opus 4.8 and it's cheaper.
- 11:49
So yeah, there's no reason not to use GLM 5.2, uh, especially given that I work at Hugging Face now. Um, for the deployment, as I said before, I use Modal.
- 11:58
Um, it's pretty great if you wanna deploy agents. Uh, in my case, I'm using the batch processing feature. So they allow you to spin up a massive amount of containers all in parallel.
- 12:09
Every single container is basically one agent loop that is processing one GitHub issue. Uh, it's super easy to use, I have to say. Um, and the startups are also pretty fast.
- 12:21
So I definitely recommend it if you're building, uh, agents that are like, for example, running in the background, running overnight, for example.
- 12:29
Um, and then the way I invoke it, yeah, technically, I could also just, uh, deploy this as a cron job model, for example, has support for this. But typically the follow-up on the GitHub issues, I still do that actually manually by invoking it as a skill.
- 12:44
So I created a skill for this in Cursor, uh, I call it Process Unread Model. And then what it's gonna do is it's actually gonna invoke an agent, in this case Composer 2.5, which is like the agent that I'm mostly using in Cursor, which is again gonna invoke all the other agents.
- 13:00
So that's, this is kind of the loop that people are talking about. And then finally, it's gonna post, uh, all the results on our Slack channel.
- 13:09
Uh, so yeah, and this is actually what it just post. So what it does is it basically just post a huge amount of Hugging Face papers, uh, which are these research papers which people can, uh, make available on the Hugging Face, because every time someone mentions it in a model card or dataset card, we index it on
- 13:26
the hub. And then it just posts all the artifacts that people have been uploading based on the outreach that we do via GitHub. Um, so yeah, I do this still, uh, in a manual form.
- 13:36
So I just invoke the skill and then after a few minutes, these messages, uh, appear on our Slack channel.
- 13:44
Um, yeah, I just included some fun results because to be honest, it's quite fun to see people interacting with the agents. Um, to be honest, I don't disclose that it's an agent.
- 13:56
Why? Because I think if people know it's a bot, then they might quickly like close the issue. And to be honest, they post exactly the same stuff as I was doing before manually.
- 14:05
So I don't actually see any reason to, to do that. Um, so and then you see replies like this, uh, "Hi, Niels. Thanks a lot for your su-suggestion and the clear guidance."
- 14:16
I actually also oftentimes see people using an agent to reply to my agents. Uh, so it's kind of the, that internet nowadays. Um, but people, yeah, make all their artifacts available on Hugging Face.
- 14:27
And out of the thousands of issues that are being created on Hugging Face, actually so far, I've only had two, uh, negative comments. One [REDACTED:gender] saying, uh, yeah, "Please close this slop."
- 14:37
So he closed the issue and then another one. But most of the people, they just say, "Yeah, actually it makes perfect sense to make my weights or my datasets available on Hugging Face.
- 14:45
Like, why didn't I think of this?" Um, so it's kind of a win-win, I would say.
- 14:51
Uh, I oftentimes I'll also post fun results on our Slack channel. Like for example, one time someone, a researcher from Apple, uh, sent me a DM like, "I saw you reached out to me."
- 15:01
But yeah, technically it's my agent just posting a GitHub issue, uh, regarding publishing a new Apple, uh, the artifacts of an Apple paper on Hugging Face. Or for example, it reaches out to Google DeepMind to, uh, publish, um, mathematics datasets.
- 15:17
Um, so a lot of times like I receive emails, the one on the right side, where yeah, they want to publish a 400 gigabytes dataset on Hugging Face, but this was also my agent just, uh, opening GitHub issues.
- 15:29
Um, yeah, this is another fun result. So, uh, PaddleOCR, it's like a [REDACTED:origin], uh, company. They migrated all their OCR models to Hugging Face based on outreach by, uh, the agents that create is-issues for me.
- 15:44
So, um, yeah, it's pretty nice. Another fun result is like whether when it completes the default template of model cards on Hugging Face. So, uh, Meg Mitchell, who also works at, uh, Hugging Face, she has a famous paper called "Model Cards for Model Reporting," making sure that anyone documents their models, uh, in a proper way.
- 16:04
And so we do provide this template, which you can see on the left side in the Git diff. And then, uh, the agent is just completing that template based on the content that it finds based on that paper, like the GitHub README, uh, the PDF itself, and so on.
- 16:21
Um, yeah, it's also quite funny to see, for example, in this case that it, uh, included me in, uh, the model card. It said, uh, "Model card author's Niels, part of the Hugging Face Community Science team."
- 16:32
I never prompted it this way, but it's pretty fun to see. Or people, uh, replying, "Thank you for helping me fix my mistakes." So those are all done by, uh, the agents.
- 16:44
Uh, I think the most popular GitHub issue that was created was this, uh, paper, uh, tiny recursive models, which you might have seen was quite trending, um, both on Hugging Face but also on Twitter.
- 16:56
Uh, so yeah, more than 60 people actually up voted that issue so that the model was released on Hugging Face. So this is again, I think the win-win, so it's both a win for the researcher making their research more discoverable on Hugging Face, but it's also, yeah, better for the people then who want to build on top
- 17:12
of that research and want to use them.
- 17:16
Uh, so yeah, I have hundreds of GitHub issues where I think I can show, uh, nice results, um, where people interact with the agents.
- 17:25
You might also wonder, yeah, how to avoid slop because you might think, okay, you have an agent, uh, spamming the whole internet with your GitHub issues, like should you even do this?
- 17:34
Again, I already talked about the win-win. Um, but a blog post that I highly recommend if you wanna avoid that your agent is just posting slop is, um, the LLM Evals FAQ, uh, by Hamel Hussein.
- 17:48
Uh, I would say he's like the main expert when it comes to LLM evaluation. He also has like a, a paid course, but he also publishes a lot of stuff for free online, including this blog post.
- 17:59
So I highly recommend to go through it if you wanna learn more about how to evaluate your agents.
- 18:06
So my conclusion would be, um, that open models are actually getting great, especially now with GLM 5.2, you have DeepSeek V4 and so on. So, um, yeah, we, we are able to now replace closed source models by open ones.
- 18:20
Uh, for my use case, I would say agents are actually better than, uh, workflows. Uh, they only need a single CLI, which is the Hugging Face CLI. They need a single skill, the Hugging Face CLI skill, and a sandbox, and that's all they need to do their work.
- 18:32
And finally, yeah, don't forget about evaluation. Um-
- 18:38
Finally, uh, I can also discuss some other efforts that we do as part of the Community Science, uh, team, um, very shortly. Um, so I have a Twitter account that I created.
- 18:49
It's called Daily Papers, and it actually uses the exact same workflow as my agents behind the scenes to post interesting research papers on X. It, uh, recently crossed 90,000 followers without any involvement of me.
- 19:02
I just deployed this, uh, and it posts interesting research papers and artifacts from Hugging Face every four hours or every time someone, uh, releases something cool on Hugging Face.
- 19:13
Um, so yeah. And I have like Gemini, uh, determining the best visual to tweet or to include in the tweet. Like, for example, this recent tweet, uh, where it tweeted out that NVIDIA released an optimized version of GLM 5.2, got more than 2,000 likes, so that's pretty cool, uh, to see.
- 19:32
And a final effort that I'm working on right now is a revival of, uh, Papers With Code, which is a website that once existed, then it was acquired by Meta, uh, and then sadly it, uh, died.
- 19:43
So I'm t- I'm trying to re- revive it in making, uh, research and state-of-the-art easier accessible. Um, for now it lives at paperswithcode.co.
- 19:54
Uh, so yeah, you can find benchmarks over there. Uh, for example, for OCR models, own OCR benches like Popular Benchmark. But I'm also, uh, making it an educational resource so that people can learn about technical terms like mid-training, uh, on-policy distillation, and so on.
- 20:12
So yeah, uh, that was it for my talk. I hope, uh, you learned something. Thanks all for your attention. [audience applauding] [outro jingle]