Exploring Firefunction-v2 - Function Calling Model

Firefunction-v2, an open weights function calling model based on the Llama 3 architecture. This model has demonstrated competitive performance, particularly in function calling capabilities, making it a noteworthy option for developers and researchers alike.

Performance Metrics

Firefunction-v2 showcases impressive results compared to its competitors. According to various public benchmarks, Firefunction-v2 scores 0.81, slightly outperforming GPT-4o, which scores 0.80. This achievement highlights its potential effectiveness in real-world applications.

Optimized for Real-World Scenarios

One of the key features of Firefunction-v2 is its optimization for real-world scenarios. This model is specifically designed to handle:

• Multi-Turn Conversations: The ability to engage in extended dialogues.
• Instruction Following: Effectively processing and executing user instructions.
• Parallel Function Calling: Simultaneously managing multiple tasks.

In terms of multi-turn instruction capability, Firefunction-v2 retains the strengths of Llama 3, scoring 0.84 versus Llama 3's 0.89 on the MT benchmark. Moreover, it consistently surpasses Llama 3 in function calling tasks, with scores of 0.51 compared to Llama 3's 0.30 on the Nexus parallel multi-function evaluation.

More information: Firefunction-v2 Launch Blog Post