If you’re in any way involved in technology in your organization, there’s a good chance that Gen AI has been discussed at length over the course of the last year.
If you’re not in a tech-related role, it’s also likely that you or one of your immediate colleagues have experimented with the technology in some way. Further to this, you have probably interacted with a generative AI tool whether you realize it or not.
Use cases are growing, but it’s important to understand how and when the technology can really deliver value before jumping in. Here’s some tips to help you out.
1. Internalize innovation
Many of the current use cases for generative AI are focused on automating workflows and delivering efficiencies for employees by reducing and eliminating tedious tasks and rework. Document or email drafting capabilities are the obvious and perhaps most immediately recognizable, but similar capabilities for more high-value tasks are leading to significant optimization.
An example of this is automatic information collation for customer queries. Based on omnichannel and historical data, generative AI tools can quickly deliver contextual information that assists customer service representatives and advisors. The underlying LLM can draft a summary of the customer’s unique query and situation and deliver this to the rep so that the query can be dealt with efficiently – skipping rudimentary information-gathering tasks. The same technology can be deployed for internal information gathering, such as reporting and content generation, removing the need to search internally for subject matter experts and contributors.
Another potential use case is routing customer queries directly through a chatbot experience, but with the chatbot directly calling APIs to deliver relevant information. This activity is often assisted by Retrieval Augmented Generation (RAG) systems, that enable chatbots to access information outside of their training data to bring in external sources of relevant information.
2. Start small
When developing a product or an internal use case using generative AI, the usual issues around engineering and validation exist, just the same as for any machine learning model. Generative AI tools are underpinned by large language models (LLMs), which can be prone to hallucinations if the right guardrails are not in place, such as ensuring outcomes are assessed by a human. While RAG systems can never eliminate hallucinations, they can reduce the likelihood of them occurring by pulling more relevant information.
Another challenge with LLMs is that they require a huge amount of runtime computing resources, such as GPUs. When hosted by a cloud provider, such as Azure or AWS, costs can very quickly add up, especially when the user base is making thousands of requests per day. Just to put this into perspective, it’s estimated that it costs OpenAI $700,000 per day to keep ChatGPT running.
Starting small and scaling up is the best way to mitigate these issues. Validate outcomes and assess costs so you know when it’s time to scale back. It will become clear pretty quickly if you overreach. For example, if a chatbot is delivering the wrong information, it’s likely that it is not pulling from the right sources of data, so retraining may be required, or RAG systems can be brought in to ensure the right information is provided.
3. Avoid Pandora’s chatbots
Chatbots are a use case that many organizations are adopting today, but without proper guardrails, they can deliver the opposite of their primary function, particularly in a customer-facing context.
A common complaint from consumers when trying to resolve an issue is that they are not able to reach a human operator when the situation requires it. This is a problem that can be overcome by automatically bringing in a customer services representative once escalation criteria are met. Natural language processing can be harnessed to examine tone and language and flag that there is a requirement for human intervention. In industries that may involve sensitive scenarios or vulnerable end users, such as healthcare and financial services, these considerations are vital for meeting the needs of customers.
Again, the key is not to overreach with any customer-facing Gen AI tools, which is why it’s best to first experiment with internal use cases. Whether the use case is internal or external, filtering is an important step to ensuring Gen AI tools are being used appropriately. For example, building in filters that make sure the questions asked are appropriate to the use case.
It’s no good asking an internal chatbot focused on customer data for recipe recommendations—along with other more closely related questions. Filters that ask users to rephrase questions and even suggest more appropriate language are some examples that will help narrow down the use case. The same filtering methods can also protect against harmful or biased responses being delivered to users.
4. The right model at the right time
There are many foundation models to choose from and matching the right LLM is an important component in creating a Gen AI solution. Licenses are a key consideration, as some models won’t be covered when it comes to certain use cases. This is more a due diligence point and should be a point of consideration during the early stages of exploration.
As generative AI evolves, its use cases will expand also. The next big breakthrough in the field is set to be multi-modal models, in which text, voice, audio and visual data can be merged to produce new content in various mediums. As with every use case, guardrails will need to be put in place to avoid mimicking real people and intellectual property, but these capabilities will surely be realized in the next 5 years. Text-to-image generative AI models are already being used in the creative industries and there is much discussion around training data, which of course involves artistic IP.
As new Gen AI use cases and capabilities emerge, understanding the legal and regulatory landscape will be crucial to future-proofing solutions. For this reason, it’s prudent to define specific use cases and start small and safely while the technology evolves.
