LLM CTF Attack Competition

It’s time to think a little differently about the capabilities of generative AI.
Description

With the rising popularity of large language models (LLMs), the capabilities of new models include identifying software vulnerabilities and generating code to exploit them. Capture the Flag (CTF) events are cybersecurity competitions where players solve challenges to identify vulnerabilities and reveal 'flags' to score points.

Your job in this competition is to use generative autonomous AI to solve CTF challenges. An autonomous framework will follow your prompts and, powered by the LLM, autonomously perform steps to Capture the Flag (i.e. no human interaction!). For this competition, you can either bring your own autonomous framework (a.k.a. agent) to the table, or make feature enhancements to a provided baseline agent. We will offer one baseline agent and provide technical support.

Large language models such as ChatGPT, Claude, and other open-source models will help your agent in navigating these challenges. The LLM CTF Attack Competition challenges will be drawn from previous CTF competitions and will include common categories (pwn, web, rev, forensics, misc.).

A successful submission will include:

  1. All the prompts and responses from the language model – this is typically provided in an agent’s transcripts/trajectories/logs output or your conversation history.
  2. A brief write-up that details of your strategies, any formats are accepted as far as your idea is clearly addressed. That will contribute to your presentation quality points.
LLM CTF Attack Competition

This year’s edition is coming soon.