Empowering LLMs with Toolkits: An Open-Source Intelligence Acquisition Method

The acquisition of cybersecurity threat intelligence is a critical task in the implementation of effective security defense strategies. Recently, advancements in large language model (LLM) technology have led to remarkable capabilities in natural language processing and understanding. In this paper, we introduce an LLM-based approach for open-source intelligence (OSINT) acquisition. This approach autonomously obtains OSINT based on user requirements, eliminating the need for manual scanning or querying, thus saving significant time and effort. To further address the knowledge limitations and timeliness challenges inherent in LLMs when handling threat intelligence, we propose a framework that integrates chain-of-thought techniques to assist LLMs in utilizing tools to acquire OSINT. Based on this framework, we have developed a threat intelligence acquisition agent capable of decomposing logical reasoning problems into multiple steps and gradually solving them using appropriate tools, along with a toolkit for the agent to dynamically access during the problem-solving process. To validate the effectiveness of our approach, we have designed four evaluation metrics to assess the agent’s performance and constructed a test set. Experimental results indicate that the agent achieves high accuracy rates in OSINT acquisition tasks, with a substantial improvement noted over its baseline large language model counterpart in specific intelligence acquisition scenarios.