Generative AI is transforming business – here’s how you can capitalise on it
- Written by Warren Schilpzand, Area Vice President of Australia and New Zealand at DataStax
The rise of generative AI applications like ChatGPT and Google Gemini has taken the world by storm. Easy-to-use AI can transform the experiences businesses serve their customers and employees and, together with vector search, provide real-time insights into what those customers expect.
However, the idea is not to build and train AI models from scratch. Foundational models, combined with vector search, can take real time data and transform this into experiences for customers and employees alike. For business managers, this allows you to focus on running the company, not messing with technology.
It’s all about transforming the interactions your customers have with you using generative AI, exceeding their expectations and giving them an experience that will surprise and delight.
So how can businesses capitalise on the growth and potential of generative AI while ensuring their data remains secure? The answer is through using a retrieval augmented generation (RAG) pattern that combines in house structured and unstructured data with templated prompts to provide real time AI responses to customers and employees.
The great thing is you don’t need to build an AI model from first principles. Services like DataStax’s database-as-a-service Astra DB, combined with our partnership with ThirdAI, mean you can use the data you have to with a foundational AI model, easily and quickly.
Man bites dog: combining vector search and AI
Artificial intelligence is on the cusp of making real transformations for the global economy, with breakthroughs estimated to bring about a $US7 trillion increase in global GDP and a lift in productivity growth by 1.5 percentage points over a 10-year period, says Goldman Sachs Research.
According to Goldman Sachs’ senior US software analyst, Kash Rangan, generative AI can streamline business workflows, automate routine tasks, and give rise to a new generation of business applications. It can also lead to better customer service, but the key is to have the right tools in place so your company can surf the generative AI wave.
One of the key developments in using generative AI and real-time data to improve business processes and customer service is the rise of vector search, which moves the game on from the traditional keyword search methods everyone is used to.
Where keyword searches simply match documents based on the occurrence of specific terms, vector search focuses on the semantic meaning and similarity of data points. How does this play out? Keyword search, for example, would have a hard time, and would deliver false positive results for searches such as “dog bites man” and “man bites dog.” This is because those terms include the same words but have opposite meanings.
For a customer service organisation, vector search combined with AI and real-time data is a game changer. Using your own customer data and real-time insights into their behaviour, you will be able to provide an appropriate answer to queries such as “I need a new phone,” or “my old device is broken” that use completely different wording but have the same meaning.
Traditional chatbots would struggle with queries like the one about a broken phone, but this is not a limitation faced by AI-enabled vector search.
Minimising the cost of generative AI
One of the challenges faced by organisations wanting to build their own AI is that it’s computationally, and therefore financially, expensive. AI needs a lot of processing power, and processing power costs money.
This comes down to how large language models work, by breaking down queries into a series of contextual or semantic blocks called ‘tokens.’ The tokens represent words, or parts of words, and the AI can output coherent responses by understanding the complex linguistic constructs represented by the tokens.
Processing the tokens takes a lot of memory which, in turn, can limit the ability of the AI to remember previous conversations and then build subsequent responses based on those conversations.
Vector search, such as is incorporated into DataStax’ Astra DB, solves this problem by only retrieving the most semantically relevant data, minimising the number of tokens needed to answer a series of evolving queries.
For example, a chatbot designed to respond to questions and answers about a software product would, using vector search, not need to have the entire Q&A repository in memory. Instead, it could question the knowledge base semantically to provide the right answer, limiting the number of tokens needed and, as a result, the processing power required and the cost of answering the query.
For businesses wanting to capitalise on generative AI, the key is using a large language model combined with real-time data and vector search to provide the best answers to your customer’s queries, and limiting how much it costs you to provide that service.
