Exploring DeepSeek-R1 s Agentic Capabilities Through Code Actions
I ran a quick experiment examining how DeepSeek-R1 carries out on agentic jobs, despite not supporting tool use natively, and I was quite impressed by initial outcomes. This experiment runs DeepSeek-R1 in a single-agent setup, where the design not just plans the actions but likewise develops the actions as executable Python code. On a subset1 of the GAIA recognition split, DeepSeek-R1 outperforms Claude 3.5 Sonnet by 12.5% absolute, from 53.1% to 65.6% appropriate, and other designs by an even larger margin:
The experiment followed design usage guidelines from the DeepSeek-R1 paper and the design card: Don't utilize few-shot examples, avoid adding a system prompt, and set the temperature to 0.5 - 0.7 (0.6 was utilized). You can find additional assessment details here.
Approach
DeepSeek-R1's strong coding capabilities allow it to serve as a representative without being clearly trained for tool use. By allowing the model to produce actions as Python code, it can flexibly interact with environments through code execution.
Tools are implemented as Python code that is included straight in the prompt. This can be a basic function definition or a module of a larger plan - any legitimate Python code. The design then produces code actions that call these tools.
Results from performing these actions feed back to the design as follow-up messages, driving the next steps up until a last answer is reached. The representative framework is an easy iterative coding loop that moderates the conversation in between the model and its environment.
Conversations
DeepSeek-R1 is utilized as chat model in my experiment, where the design autonomously pulls additional context from its environment by utilizing tools e.g. by a search engine or fetching data from web pages. This drives the conversation with the environment that continues till a final response is reached.
In contrast, o1 designs are understood to carry out poorly when used as chat models i.e. they do not attempt to pull context during a discussion. According to the connected article, o1 models perform best when they have the full context available, with clear instructions on what to do with it.
Initially, I also tried a complete context in a single prompt approach at each step (with results from previous steps included), however this caused considerably lower ratings on the GAIA subset. Switching to the conversational approach explained above, I was able to reach the reported 65.6% efficiency.
This raises an intriguing question about the claim that o1 isn't a chat model - maybe this observation was more appropriate to older o1 designs that did not have tool use capabilities? After all, isn't tool use support an essential mechanism for making it possible for models to pull extra context from their environment? This conversational approach certainly seems efficient for DeepSeek-R1, though I still need to perform similar try outs o1 models.
Generalization
Although DeepSeek-R1 was mainly trained with RL on mathematics and coding jobs, it is amazing that generalization to agentic tasks with tool usage through code actions works so well. This capability to generalize to agentic jobs reminds of current research by DeepMind that shows that RL generalizes whereas SFT remembers, although generalization to tool usage wasn't examined because work.
Despite its capability to generalize to tool usage, DeepSeek-R1 typically produces long thinking traces at each action, compared to other models in my experiments, restricting the effectiveness of this model in a single-agent setup. Even simpler tasks often take a long period of time to finish. Further RL on agentic tool use, be it via code actions or not, could be one alternative to improve effectiveness.
Underthinking
I likewise observed the underthinking phenomon with DeepSeek-R1. This is when a reasoning design regularly changes in between different thinking thoughts without adequately exploring promising courses to reach a right service. This was a major factor for overly long reasoning traces produced by DeepSeek-R1. This can be seen in the tape-recorded traces that are available for download.
Future experiments
Another typical application of reasoning models is to use them for preparing just, while utilizing other designs for generating code actions. This might be a prospective new function of freeact, if this separation of functions shows helpful for more complex tasks.
I'm likewise curious about how thinking designs that currently support tool use (like o1, o3, ...) perform in a single-agent setup, demo.qkseo.in with and without producing code actions. Recent developments like OpenAI's Deep Research or Hugging Face's open-source Deep Research, which also uses code actions, look fascinating.