Poster Abstract: Tasking Heterogeneous Sensor Systems with LLMs
Despite the extensive use of sensors enabling intelligent applications, the complementary potential of co-existing sensor systems is often not fully utilized, limiting more advanced applications. This paper introduces a novel solution using Large Language Models (LLMs) to coordinate sensor systems for handling complex user queries. It defines a sensor language for sensor systems, including vocabulary set and grammar rules, analogous to natural language components, enabling LLMs to translate user intentions into sensor coordination plans. Preliminary results show that our approach significantly outperforms the existing solution at plan generation, execution and response generation stages.
Authors
- Bufang Yang (ORCID: https://orcid.org/0000-0003-0032-2539)
- Neiwen Ling (ORCID: https://orcid.org/0000-0003-2072-1502)
- Zhenyu Yan (ORCID: https://orcid.org/0000-0002-4433-5211)
- Zhihe Zhao
- Kaiwei Liu (ORCID: https://orcid.org/0009-0002-4108-0898)
- Guoliang Xing (ORCID: https://orcid.org/0000-0003-1772-7751)
- Yunqi Guo (ORCID: https://orcid.org/0000-0002-1852-3825)
- Xiaozhe Ren (ORCID: https://orcid.org/0000-0002-0432-5510)
- Xin Jiang (ORCID: https://orcid.org/0000-0002-9117-8247)
- Xian Shuai (ORCID: https://orcid.org/0000-0002-6750-6706)
- Lilin Xu (ORCID: https://orcid.org/0009-0007-5203-7496)
Institutions
- Chinese University of Hong Kong (HK)
- Huawei Technologies (China) (CN)
Publication Details
- Published
- 2024-11-04
- DOI
- https://doi.org/10.1145/3666025.3699428
- Citations
- 5
- Primary Topic
- Fault Detection and Control Systems
- Type
- article
- Field-Weighted Citation Impact
- 1.64
Funders
- National Natural Science Foundation of China