Welcome back to our series on generative AI assistants and their role in enterprise contexts. In our previous article, we discussed the potential of AI assistants such as ChatGPT, Claude, and Gemini, highlighting their capabilities, architecture, and processing workflows. We also clarified terminology and emphasized the need for advanced configuration and customization when implementing these assistants in enterprise companies. In this second part, we will delve into the various generative AI assistants available on the market, examining their benefits and drawbacks.
A generative AI assistant solution consists of two primary components: the Large Language Model (LLM) and the AI assistant application. The LLM is responsible for analyzing and responding to user queries based on the datasets it was trained on. The AI assistant application serves as a wrapper around the LLM, acting as the interface through which users interact with the LLM.
Customization options for these solutions are available at both the LLM and AI assistant application levels. For the LLM, you can either use it as is, trained on a vast array of general datasets, or further specialize it for your specific domain use-cases. This can be achieved through fine-tuning the model on your own curated dataset or employing prompt-engineering techniques to optimize the LLM's output based on its existing general knowledge. A hybrid approach combining these techniques is also possible.
At the AI assistant application level, there are multiple ways to construct an application that orchestrates how information is fed into the LLM and interacts with other components, such as connectors to ERP systems. This layer is crucial, especially in large-scale enterprise environments where the volume and complexity of queries demand robust, scalable solutions. So while the LLM is the foundation of generative AI assistants, the application layer is the part enterprises can and should influence the most, as it directly impacts cost efficiency and performance. The design of this layer must prioritize cost efficiency and performance to manage potential escalations in operational expenses effectively.
Many executives recognize that AI should not be treated as a separate business component but rather as the core fuel powering all components. However, in practice, AI is often initially adopted as a new, separate component before being integrated into the broader business ecosystem. This approach may work for a time, but as technology advances and use-cases expand, it can create technical debt for companies.
Based on current evidence, we can make two key assumptions:
With these assumptions in mind, it's essential to design and develop AI assistant applications with future complex scenarios in mind, as they will eventually become part of larger, interconnected AI systems.
Let's examine the current options provided by some of the pioneers and major players in the generative AI market for building your own AI assistant on top of their models.
After analyzing the different AI assistant application offerings from some of the most important players in the AI market, we can identify two key considerations. At the Large Language Model (LLM) layer, companies must first decide between the ease of deployment and the increased control needed. If they need a simple model deployment option, they can choose to consume models like GPT through the provider's own platform. However, this approach results in minimal control over the LLM and raises data privacy concerns. For most EU-enterprises, this option is likely not viable due to the need for increased control over the LLM and data access.
The second option for companies that require increased control is the self-deployment option. This approach allows them to deploy the models in their own cloud or on-premise infrastructure. While this option offers more control, it comes with potential disadvantages such as increased maintenance costs and the need for specialized software engineering teams.
At the application layer, enterprises must also make further trade-offs if they want to use the AI assistants provided by these companies out-of-the-box. Each of these assistants is suitable for personal productivity and use cases like synthesis, research, ideation, and writing. However, the number of users who will use the assistants is an important factor to consider. Depending on the business size, some companies might find offerings like Anthropic more appealing, while enterprises might prefer ChatGPT due to its scalability for large teams with many users.
There is also the use case of Gemini and Microsoft Co-pilot, which embed their assistants into their product offerings. This unified ecosystem experience can be attractive to organizations seeking seamless integration. However, if we ignore this use case, we are left with AI assistants that can be used for various use cases.
Apart from security and data privacy concerns, the real return on investment (ROI) will only become apparent if these solutions are integrated into the broader enterprise landscape, including ERP systems, third-party apps, and other data sources. While off-the-shelf AI products are attractive for their quick deployment capabilities and potential cost savings, their customization limitations can pose significant drawbacks. These limitations may lead to a dependency on the providers' terms and continuity, and there are potential risks with data privacy due to the necessity of processing data externally.
Another major issue is that most AI assistants from OpenAI, Anthropic, or Mistral lack output auditing capabilities. This lack of transparency can make it difficult for users to understand how the assistant generated the output, the steps it took, or the sources it leveraged. For tasks like financial report analysis, the inability to verify the output and sources might be a deal-breaker for many companies.
Cost optimization is another issue enterprises might encounter when leveraging such solutions. Since there is no one-size-fits-all LLM, companies may achieve better results and cost efficiency by using different LLMs for different tasks depending on their complexities and suitability. This level of customization is generally not possible with out-of-the-box solutions.
The alternative for companies is to develop their AI assistant apps from scratch or work with an AI assistant toolkit from a provider, which can be customized according to their needs and use cases. Custom-built AI solutions offer the highest level of customization, allowing companies to tailor features to their operational needs and maintain control over their data. However, this approach can come with higher development and maintenance costs, the need for substantial technical expertise, and the risks of project failure or integration issues.
In conclusion, whether a business should opt for an off-the-shelf product or invest in a custom solution depends on various factors. These factors include budget constraints, specific business and domain needs, available technical expertise, and desired speed of deployment. While ready-made solutions can provide quick and easy gains in productivity, they might fall short in delivering long-term value without significant customization. Custom AI solutions, while costly and complex to develop, offer tailored functionality that can significantly enhance operational efficiency and competitive advantage. In the third part of this blog series, we will delve into the development of custom generative AI assistants and provide best practices and recommendations for their implementation and adoption in enterprises.
