The Cyber Reasoning Challenge (CRC) is an AIxCC-style competition in which student teams design a cyber reasoning system (CRS) to find and patch vulnerabilities in real software targets. Unlike a traditional CTF, teams do not solve challenges by hand. Their CRS must autonomously discover a crashing proof of vulnerability and produce a patch that compiles, preserves the program’s intended behavior, and actually fixes the bug rather than hiding a single crash.
To keep the event focused on system design, we have used the evaluation infrastructure from Team Atlanta, the first-place team in DARPA’s AI Cyber Challenge. Teams therefore do not need to build the evaluation pipeline, scoring, or target-harnessing stack from scratch. They can spend their time on the CRS itself: how it searches, localizes, and repairs bugs.
We also provide a starter kit with two working CRS baseline (codex and claude code). Teams can run it out of the box to understand the workflow, then replace or extend the finder and patcher with their own components.
CSAW CTF is designed for students who are trying to break into the field of security, as well as advanced students and industry professionals who want to practice their skills.
Open to all skill levels. Teams compete within their respective global regions.
Challenge Details
CSAW CTF is one of the oldest and biggest CTFs with 1216 teams with 1+ points in 2021. Designed as an entry-level, jeopardy-style CTF, this competition is for students who are trying to break into the field of security, as well as for advanced students and industry professionals who want to practice their skills.
CSAW CTF occurs over two rounds: a 48-hour Qualifying Round in September and a 36-hour Final Round in November.
Challenge writers who are associated with or alumni of active teams have recused themselves from playing in or supporting teams playing in CSAW CTF.
Judging Criteria
Jeopardy-style scoring. Teams earn points by solving challenges across standard CTF categories. The team with the most points at the end of each round advances.
Top teams from the Qualifying Round are invited to the Final Round.
Registration Guidelines
Register via the CTF platform. Teams compete within their respective global regions.
Contact csawctf@osiris.cyber.nyu.edu for registration questions.
Submission Guidelines
Challenges are submitted through the CTF platform during the competition window. Flags must be submitted in the correct format as specified per challenge. All submissions are final once entered.
Timeline
Items
48-hour Qualifying round begins
UTC
ONLINE
Qualifying Round Ends
UTC
ONLINE
Finalist Notification
EST
End of September
36-hour Final Round Begins
EST
Hybrid mode, region dependent
Final Round Ends
EST
Note
All times listed in UTC or EST as noted per event. Final Round format (online vs in-person) depends on region.
This competition uses the NYU CTF Lite, a streamlined benchmark of 50 challenges spanning six categories, adapted from the original NYU CTF Bench. To support easy integration with LLM-based agents, all challenges are provided in the standardized NYU CTF Bench format, fully compatible with the nyuctf pypi package for loading and interacting with autonomous agent frameworks.
While the true flags are included in accompanying metadata .json files, agents must independently solve each challenge and verify that the extracted flag matches the ground truth—no hardcoded answers allowed. A baseline agent system is provided in this repository, allowing competitors to build upon it with their own enhancements.
To request an API Key, please email nyuctf@gmail.com with all team members’ name, email and affiliation.
Rules
Team Participation: Teams of up to 3 people are allowed. Individual participation is also possible, but teamwork is highly recommended.
Agentic Framework: Participants are encouraged to analyze the general patterns of the challenges to optimize their agentic systems to make it specific for CTF automation; however, the final solutions must be generated entirely by an autonomous, LLM-powered agent, with no human-in-the-loop during execution. Participants are allowed and encouraged to use any techniques applicable to building effective agentic AI systems, including but not limited to prompt engineering, multi-agent, tool-augmented reasoning, and retrieval-augmented generation (RAG). These techniques may be applied broadly or tailored to specific challenge categories, but must remain generalizable—challenge-specific hints or hardcoded solutions are strictly prohibited. All the prompts used for challenge solutions must not include direct solutions from human players from any source; each solution that violates this rule will not be counted as solved. Participants must supply their own API tokens or model deployments for use within their autonomous frameworks. Any agentic framework may be used—including doing enhancements on open-source agentic frameworks, or custom-building systems from scratch. These frameworks must support full automation and may integrate real-time or pre-installed cybersecurity tools such as apk2jar, apktool, Ghidra, Hopper, Burp Suite, and Wireshark. All aspects of model selection, tool configuration, and system design are open-ended and left to the discretion of the participants.
Model Requirements: Participants are free to use any language model architecture for their agentic systems, including models accessed via API service providers (e.g., OpenAI, Anthropic), self-hosted open-source models (e.g., LLaMA, Qwen), or custom fine-tuned variants. There are no restrictions on model size, origin, or hosting setup. However, all models must be free of contamination, meaning they must not have been trained on or contain leaked solutions or flags from the competition dataset. Any evidence of flag leakage or training contamination will result in disqualification.
Evaluation: will be based entirely on the number of challenges successfully solved by the autonomous agent. Each correctly solved challenge contributes to the team’s final score, with no partial credit. The accuracy of the extracted flag, as verified against the ground truth, is the sole criterion for success.
Submissions: For each solved CTF challenge, participants must submit the full trajectory generated by their autonomous agent, including the agent’s thoughts, actions, observations, and the final flag, in a machine-loadable format (e.g., JSON or structured log). The extracted flag must exactly match the ground-truth flag provided in the metadata. Manual editing or tampering of the agent outputs is strictly prohibited and will result in disqualification. In addition, participants must provide a well-documented Git repository containing the complete codebase of their agentic framework. Open-source is encouraged, but a private repository shared with the organizer is also doable. This repository should include all dependencies, configurations, and tools used, along with detailed technical documentation outlining the participant’s approach—such as prompting techniques, model usage, agent architecture, tool integration, and any other implementation details. If a custom or fine-tuned model is used, training code and model weights should also be provided for validation.
API Keys and Data: All competitors may request API keys from OpenAI, Anthropic, and Gemini from the organizer, with an initial combined budget of up to $100 in credits every month during the competition. This budget may be extended as needed. Requesting API keys will automatically register participants for the competition. All competitors are required to open-source the code and data used in their submissions.
Judging Criteria
100 points in total, the final grade would be the weighted sum of all the judging criteria
Challenge Solved (50%): The number of CTF challenges solved by the participants, based on the score of each puzzle.
Creativity (30%): The methods used for finding the vulnerabilities and solving the challenges. Adding innovative features to the framework, and trying unique approaches are all vectors for evaluation. Ultimately, be sure to include a summary about how the puzzle was solved by the LLM. Using your own agent instead of the agent provided in the competition will give contestants a bonus under that judging criteria.
Presentation Quality (20% – 10% for writeups, 10% for final presentation): The quality of the final presentation. It should use the same approach that was suggested by the generative large language model you used. The presentation can be in the form of a recorded video or live demonstration, and contestants should use slides to present their findings and thoughts for the final presentation as the reference of grading.
Penalty items (deduction of 10% of the challenge score for each rule violation): The final solution must be provided by the automation framework with prompt engineering techniques, even if the participants come up with the proper solutions by themselves. Penalty items will be applied if the final solution does not come from the generative AI, even if participants find the correct solution independently. No points will be awarded for this challenge when participants use online writeups and source code to form or train the agent.
Registration Guidelines
This competition is open to the public and will run until all the 50 NYU CTF challenges are solved.
No registration is required. The first submission with a valid and verifiable team information including team members’ name and contact email will be registered for the competition.
Submission Guidelines
For each solved CTF challenge, participants must submit the full trajectory generated by their autonomous agent, including the agent’s thoughts, actions, observations, and the final flag, in a machine-loadable format (e.g., JSON or structured log). The extracted flag must exactly match the ground-truth flag provided in the metadata. Manual editing or tampering of the agent outputs is strictly prohibited and will result in disqualification. In addition, participants must provide a well-documented Git repository containing the complete codebase of their agentic framework. Open-source is encouraged, but a private repository shared with the organizer is also doable. This repository should include all dependencies, configurations, and tools used, along with detailed technical documentation outlining the participant’s approach—such as prompting techniques, model usage, agent architecture, tool integration, and any other implementation details. If a custom or fine-tuned model is used, training code and model weights should also be provided for validation.
Timeline
Items
Start Date
EST
Finalists Cut off
EST
Finalists Announcement
EST
Note
Monthly leaderboard update is the 30th of every month.
Each challenge has its own details for the competition. Those details are given on the GitHub for this competition here.
Challenge 1: The first challenges will focus on leveraging generative AI to add hardware Trojans to an AES core. Three difficulties of challenge will be provided: easy, medium, and hard, with each worth increasing points. Teams can submit all three difficulties of challenge to earn the most points.
Judging Criteria
Each challenge will have its own rubric regarding how points can be awarded. Challenges will have their base functionality automatically graded, and manual judging will take place over the following days to ensure all rules were followed, as well as to award additional points for completing further objectives. These extra points will be awarded for each competition for things like “most creative use of AI”. Please refer to each challenge's rubric for how this will be done.
Each submission must also be fully open source, guidelines for this are here.
Registration Guidelines
Please fill out the registration Google Form to register your team. Teams must consist of:
Up to 4 student team members
One team advisor/mentor
Timeline
Items
First challenge given
EST
Second challenge given
EST
All challenges due
EST
Finalists announced
EST
Final challenge given
EST
Winners announced
EST
Note
Note: The challenges do not need to be completed within the month they are given, this is just when we are releasing them. We will also update a monthly leaderboard in this repository as the competition progresses. All teams will have up until the 1 October deadline to submit their challenges to earn points and be considered as a finalist team.
It’s time to think a little differently about the capabilities of generative AI.
Description
With the rising popularity of large language models (LLMs), the capabilities of new models include identifying software vulnerabilities and generating code to exploit them. Capture the Flag (CTF) events are cybersecurity competitions where players solve challenges to identify vulnerabilities and reveal 'flags' to score points.
Your job in this competition is to use generative autonomous AI to solve CTF challenges. An autonomous framework will follow your prompts and, powered by the LLM, autonomously perform steps to Capture the Flag (i.e. no human interaction!). For this competition, you can either bring your own autonomous framework (a.k.a. agent) to the table, or make feature enhancements to a provided baseline agent. We will offer one baseline agent and provide technical support.
Large language models such as ChatGPT, Claude, and other open-source models will help your agent in navigating these challenges. The LLM CTF Attack Competition challenges will be drawn from previous CTF competitions and will include common categories (pwn, web, rev, forensics, misc.).
A successful submission will include:
All the prompts and responses from the language model – this is typically provided in an agent’s transcripts/trajectories/logs output or your conversation history.
A brief write-up that details of your strategies, any formats are accepted as far as your idea is clearly addressed. That will contribute to your presentation quality points.
ESC is an educational, research-oriented tournament aimed at hacking into the hardware of embedded systems. First run in 2008, it is the oldest hardware security competition in the world, and 2025 represents ESC's 18-year anniversary.
Past ESC competitions have focused on data exfiltration attacks against IoT devices, hacking the firmware of a RISC-V Wi-Fi access point and executing acoustic side channel on 3D printing devices.
The 2025 ESC competition centers on side channel attacks (SCAs) and fault injection attacks (FIAs), techniques used to extract sensitive data from infrastructure-critical cyber physical systems. Participants will engage in a series of challenges that involve designing and executing these attacks, as well as investigating methods to defend against them. The ChipWhisperer Nano will serve as the target hardware platform, providing teams with a safe and controlled setting to showcase their expertise in launching and mitigating these advanced attack strategies.
The event comprises a qualification and a final/competition phase where teams will be able to explore several variations of fault injections and side channels, as well as suggest mitigations for the discovered exploits.
CSAW CTF is one of the oldest and biggest CTFs with 1216 teams with 1+ points in 2021. Designed as an entry-level, jeopardy-style CTF, this competition is for students who are trying to break into the field of security, as well as for advanced students and industry professionals who want to practice their skills.
CSAW CTF occurs over two rounds: a Qualifying Round in September and a Final Round in November.
This "Best Paper Award" assesses the top scholarly security research from the previous year. The focus of this competition is on research that has a practical impact. With eligibility limited to already published papers or camera-ready papers, CSAW has a reputation for drawing some of the best security research worldwide.
It’s time to think a little differently about the capabilities of generative AI.
Description
With the rise of large language models (LLMs), AI systems are now capable of identifying software vulnerabilities and generating exploit code—skills that align closely with the goals of Capture the Flag (CTF) competitions, where participants solve security challenges to uncover hidden “flags.”
In this competition, your task is to build your own agentic AI to solve CTF challenges autonomously—that is, create or extend an AI agent powered by LLMs to analyze and exploit challenges without human intervention. You may either bring your own agent framework or enhance a provided baseline agent, with technical support available. Your agent can leverage LLMs, either API-based models such as GPT, Claude and Gemini, or open-source models deployed locally to navigate common CTF categories such as crypto, forensics, pwn, reverse, web, and misc.
A successful submission includes the full logs of prompts and model responses, along with a brief write-up describing your prompting strategies, agent enhancements, and system design choices.
