Within Devoteam we have been running several Knowledge Management proof-of-concept experiments, using various GenAI solutions. This post aims to outline the strengths and weaknesses we have identified while using these solutions.
(Note that these observations are mine only. My experience lies primarily with Amazon Q and less so with Gemini and Overlayer. Any omissions or errors are mine.)
Amazon Q Business
Amazon Q was released in April this year, and consists of four sub-products: Developer, Business, Apps and Quicksight. We are using Q Business which is a tool made for knowledge management. Q Business can ingest private, enterprise data from many different sources (e.g. Google Drive, Confluence, Salesforce, etc.) into a RAG and then provide ChatGPT-like answers to questions based on that internal data. All this is done while maintaining the security and access controls associated with the underlying data..
Google Gemini
As Devoteam is a Google Workspace client, and a strong Google partner, Gemini (formerly Bard) is also being used as an internal AI-based knowledge management tool. Gemini is built into the Google Workspace suite of tools, so integration with these tools is seemless. Gemini does not currently support integration with other, non-Google Workspace tools (e.g. Microsoft Sharepoint or Atlassian Confluence).
Overlayer
Overlayer is a third-party AI-based knowledge management tool using the Mistral LLM. We have been working with the Overlayer team to build an in-house knowledge management tool using Overlayer and Mistral. Overlayer has an advantage of not being assiociated with any of the large AI vendors so is free to develop features that work with any vendor or tool.
Our Data
Devoteam stores its knowledge in several different tools. Our primary tool is Google Drive. But we also use Jira, Confluence, SalesForce, Google Mail, Google Chat and Meta Workplace. The total amount of data we have is in the order of hundreds of terabytes (+/- 400 TBs).
Our Amazon Q proof of concept focused on a small subset of this data. Primarly that was the “UK Sales” Google Drive. We also ingested Confluence, Jira and web data where possible. The amount of this data was in the order of hundreds of gigabytes (+/- 200 GBs).
Comparison Table
Choose the Knowledge Management Tools Based on Your Needs
There are still significant challenges to creating a reliable, accurate and secure AI-based Enterprise Knowledge solution.
The comparison of AI-based knowledge management tools reveals distinct strengths and weaknesses in Amazon Q Business, Google Gemini, and Overlayer. Each tool operates at different scales, with varying levels of reliability in prompt engineering and security considerations. While Amazon Q Business struggles with significant scale issues and unreliable data, Google Gemini provides user-specific models but faces challenges in prompt engineering and supporting non-Google data sources.
On the other hand, Overlayer offers curated solutions but requires meticulous labeling. Security measures are paramount across all tools, with varying approaches to user permissions. As the quest for efficient knowledge management continues, understanding these nuances is crucial to make informed decisions on tool selection and implementation strategies.
This is Part 6 of our AI for Enterprise Knowledge series. Start with the introduction in Part 1, see best practices for creating Enterprise Knowledge AI assistant in Part 2, learn about the Importance of data management in Part 3, explore cost management in Part 4, and discover the Sales Knowledge use case in Part 5.
