The ever-evolving world of artificial intelligence (AI) continues to astound us with its groundbreaking applications across numerous fields. In today's spotlight lies a captivating development from the cutting edge of software engineering (SWE): 'Diversity Empowers Intelligence,' a novel approach integrating the collective prowess of multiple software engineering agent frameworks. Authored by Kexun Zhang, Weiran Yao, Zuxin Liu, et al., this transformational concept pushes boundaries within the realm of collaborative AI systems. Their work not only emphasizes the power of diverse AI synergies but also highlights how such collaboration could revolutionize intricate SWE challenges.
As large language models (LLMs) evolve beyond conversational bots, they now serve as the backbone of versatile AI agents, including those dedicated to SWE. These SWE agents possess remarkable aptitude in addressing myriad real-life coding conundrums arising in repositories like GitHub. One standout achievement entails a state-of-the-art open-source SWE agent successfully tackling more than a quarter (~27%) of authentic GitHub issues featured in the SWE-Bench Lite test suite. However, despite their impressive individual accomplishments, these LLM-driven SWE agents display distinct areas of strength and weaknesses. Thus, exploiting the full spectrum of their proficiencies emerges as a critical challenge.
To tackle this issue, the researchers introduce the DEI ('Diversity Empowered Intelligence') framework. This innovative system operates as a metastrategy atop pre-existing SWE agent architectures, orchestrating ensembles of agents towards optimized solution attainment. By harmonizing disparate yet complementary skills among various SWE agents, the proposed DEI paradigm significantly outclasses any single agent's output. As a result, a cluster of previously independent, open-source SWE agents – individually capping a ~27.3% resolution success rate on the SWE-Bench Lite benchmark – boosts their overall efficiency via DEI integration, achieving a staggeringly higher 34.3% resolve rate. Consequently, a colossal 25% enhancement ensues, eclipsing most proprietary alternatives. Moreover, the topmost performing ensemble flaunts a superlative 55% resolution triumph, sealing first place on the SWE-Bench Lite leaderboard.
This inspiring exploration further fortifies the burgeoning academic discourse surrounding cooperative AI mechanisms, accentuating their unparalleled capacity to confront and vanquish complicated SWE obstacles. With the dawn of DEI, the future of intelligent software design appears brighter than ever before.
References: ArXiv Paper Link: http://arxiv.org/abs/2408.07060v1 Authors: Kexun Zhang, Weiran Yao, Zuxin Liu, Yihao Feng, Zhiwei Liu, Rithesh Murthy, Tian Lan, Lei Li, Renze Lou, Jiacheng Xu, Bo Pang, Yingbo Zhou, Silvio Savarese, Huan Wang, Caiming Xiong Organizations: Salesforce AI Research & Carnegie Mellon University
Source arXiv: http://arxiv.org/abs/2408.07060v1