A hush fell over the crowd as the software developer began to build service chatbots in real time. Will the test code work as expected? The Wi-Fi seems to be too slow to run the simulation! “Unplug it and plug it back in,” a jokester types into the session’s Q&A chat. Dom Divakaruni, the Head of Product for Azure OpenAI, and Pablo Castro, a top engineer and manager of Azure Cognitive Search, were eager to show the world the software they had kept well under wraps until this moment.
First, a little background. Artificial intelligence (AI) is a field of computer science that seeks to create intelligent machines that can replicate or exceed human intelligence. Though work in this field dates back to the mid-1950s, the past few years have seen a frenzy of development unmatched by anything in recent history; ChatGPT alone reached 100 million users in a mere two months, greatly outpacing the adoption of any other modern technology.
Modern AI projects have generally relied on large language models (LLMs), which are massive data sets of natural language that can include billions of parameters. LLMs are an essential component in, for example, predicting the next word in a sentence and comparing inputs to known words, both processes you may not be aware that Microsoft uses to check spelling and grammar in their Word application. LLMs compare new text to text that they already have and analyze the similarities.
AI has historically used LLMs to make 1:1 matches of user-input queries to existing data. To get the specific result you need, you have to input a very specific prompt. This poses a problem when you’re not sure of the exact language used in the original text. Newer applications like ChatGPT and GPT-4 make use of OpenAI, a world-wide collaborative research laboratory that develops LLMs and publishes them in Microsoft Azure. More than 4,500 Microsoft customers are already using OpenAI services. This means that Microsoft is in a unique position to compile all of the machine learning of these services and use that information to not only reliably return search results using LLMs, but to generate new content in the apps, suites, and programs we already use. Microsoft’s Azure OpenAI promises to be your co-pilot in completing complex language tasks including summarizing information, creating documents, and generating code, while keeping true to their reputation for prioritizing enterprise security, privacy, and compliance.
No longer will the LLM be limited to information already input by services like ChatGPT and GPT-4. For the first time, Microsoft will allow users to easily add their own data to the learning model to deliver results relevant to their particular business needs. These LLMs will come pre-configured, and all manual additions and adjustments to that model reside solely in the user’s tenant. That is, the data added to the learning model stays securely in your tenant and is not added to any third-party model, keeping your data secure.
In their six months of user testing, existing Microsoft clients who were given a preview of the product reported back with great success. Three common use cases emerged from that testing:
In the live demonstration during the announcement event, Microsoft engineer Pablo Castro walked the crowd through two scenarios related to a chat session. First, he presented a scenario in which an employee wants to know whether their employer-provided insurance policy covers annual eye exams. The chatbot queried the standard LLM and, because the LLM doesn’t have access to the employee’s particular insurance policy, returned a null result, “I don’t have access to the specific information you’re looking for.” When Castro added the insurance policies to the data set and ran the same query, the chatbot now replied, “Yes, eye exams are covered1 subject to certain limitations and exclusions2” and provided two citations from source texts from the data set. By uploading information specific to the example company via Blob storage, uploading files, Azure Cognitive Search, or Cosmos DB, the AI chatbot was able to return information specific to the employer and specific to the employee’s needs. This chatbot with access to the company-specific data set was then easily published as an Azure App Services web app, making it accessible for all employees of the company.
In his second example, Castro walked the crowd through a complex process known as vector-based retrieval that makes it easier than ever to find the results you’re looking for. Microsoft’s Azure OpenAI service translates both the data set and the question prompt into vector data, derives a similarity rating between the two vectors, and returns results with highly similar semantic meanings.
That is, not only will a new AI model return examples of the exact phrase that was searched (e.g., search the data for the phrase maroon car), but also examples that carry a similar meaning (in this case, also include examples that reference a “red vehicle”). The infinite complexity of human speech makes this quite a dauting task, indeed.
In his live demonstration, Castro input the unwieldy prompt, “Does my plan cover things that help me better perceive mechanical waves reaching the sides of my head?” As with the first example, the result was inconclusive because those exact words were not found in the employee insurance plan. BUT after adding Azure Cognitive Search with vector support to the query box, the chatbot was quick to say, “Yes, hearing aids are included in your insurance plan.” Castro also pointed out that the Azure Cognitive Search function is also highly customizable with filters to make sure that the correct data finds its way to the user.
Azure’s OpenAI Service is gradually rolling out to users with limited availability for most products by this summer!
Katie grew up in Alabama, where sweet tea flows freely and seasons are marked by their proximity to college football (Roll Tide). After graduating from a small liberal arts college, she moved to Washington, DC and balanced her life between restaurant management, freelance cocktail consulting, and finishing her Ph.D. at American University’s Behavior, Cognition, and Neuroscience program. Central to her success in each of these arenas was (a) an ability to proactively prioritize and organize complex projects with many moving parts, (b) a detail-oriented focus on keeping everything on track, and (c) a commitment to seeing each project through to completion. Katie has now brought these skills to Cloudforce as a Project Manager, where she works with clients to ensure that their needs are addressed on time and under budget, and with a result that exceeds expectations.
As a self-proclaimed futurist, I’d much rather focus my attention on what’s to come versus dwelling on the past. Not surprisingly, all roads start (and end) with artificial intelligence (AI), and...
The true measure of technology’s excellence lies in its accessibility, empowering even the less tech-savvy. And Microsoft nailed it. Welcome to the era of enhanced productivity with Microsoft's...
The Foundation of Inclusive AI: Understanding the Significance of Inclusive Data Sets In grounding a discussion on the intersection of AI inclusion and innovation at Microsoft’s Ignite...
