Challenge
Large Language Models enable firms to deliver 24-7, high-quality automated customer service. Yet to date, research into LLMs has predominantly focused upon the design of ever-more technically advanced models, rather than the implementation of existing LLMs into apps and services used by businesses.
Approach
This project examines how to implement a LLM conversational recommendation system in a resource efficient manner for small-to-medium size enterprises, balancing latency, cost, and quality concerns.
Results
Results demonstrate the feasibility of using prompt-based learning alone in achieving a satisfactory user experience. However, relatively high costs and latency mean the current system design is suited only to higher-profit margin business settings. The use of an additional LLM for the Retrieval-Augmented Generation technique (i.e., to fetch relevant information prior to generating a response) was identified as a major driver of costs and latency. Results demonstrate a valid route for LLM SME implementation, while outlining notable constraints. A pre-print is available to download on arXiv.
Project Status
Final paper under review
Lead Researcher
Dr. Joseph Ollier
