Using Patent Data and Language Models to Analyze Company Advantages and Reduce Attorney Workload

This paper focuses on patent analysis by using deep learning and customized language model to help human beings to deal with labor-intensive, time-consuming. and knowledge-intensive paperwork. The idea is to train a customized language model. Pat-DistilRoBERTa, who is a patent specialist, and it can map a patent document to a proper high-dimensional vector for later mathematical analysis, therefore, we do not need to deal with text-based data afterward. We anticipate can better understand and represent a patent document. As a comparison, we employ a pre-trained DistilRoBERTa model as a generalist for patent analysis. We conducted numerous experiments on semiconductor patents of leading global companies, specifically TSMC and Samsung. These experiments show that we can easily leverage the language models to appropriately embed a professional technique document for different downstream tasks. For example, Pat-DistilRoBERTa improved the F1-score by 5% to 15% when classifying context and achieved 99% accuracy in classifying patents by IPC. We anticipate that such AI-assisted analysis with specialist language model can assist patent attorneys in reducing their workload as technology improves.