ChatGPT and Large Language Models with MATLAB

So it's really a pleasure to be here talking to you today. It's actually my first time presenting at Math so I hope this is the first of many. And let's talk about ChatGPT today. So as you might know, AI is becoming part of all the stages in the automotive industry. So we have examples from research and development. We have examples from manufacturing. And I'm sure you're going to see lots of cool applications at the exhibition and there will be other presentations in the afternoon.

But the question now is, we have this new disruptive AI technology called ChatGPT. Is it also going to disrupt the automotive industry? And we are starting to see some early adopters. So for example, Mercedes is trialing incorporating ChatGPT into their voice assistant. So users can try things like, what activities can I do at the beach? And then, get some nice recommendations.

But let's take a step back and understand what is ChatGPT. So it's an AI chatbot that has been developed by OpenAI. It's built on top of these models, GPT-3.5 and GPT-4, and it's focused on human alignment. So what is human alignment? Human alignment is when you train a model to make sure the output aligns with human expectations. So when you ask a question, you kind of expect some answer.

So GPT stands for generative pre-trained transformers, and they are a type of large language models. So what are large language models? So they are a type of artificial intelligence model. They are trained on huge amounts of data. So we are talking something to the scale of trillions of words. And they are networks with millions to trillions of parameters.

And large language models, even though it might look like it's something new, it's not something new. It has been there for a while. So the first LLM, or Large Language Models, they were GPT-1, BERT, and I remember when I was doing my PhD, I had to deal with those models that only have millions of parameters and I was desperate trying to figure out, how am I going to run a model this size in my machine?

But guess what? Now the models have billions of parameters. And we even have models like GPT-4 that we don't know the size because it's not public. But we can only guess it's going to be much, much larger than what we have with GPT-3.

There are different types of models. So models like BERT and RoBERTa, they are what we call encoder models. So they are not the type of model that you would use to ask a question. You would fine tune these models for a specific task, like classification.

The other ones are what we call generative models. So these are the ones you throw some text and then you get some text back. And these days, when people talk about large language models, they are usually referring to this type of generative models. And for this presentation, I'm mostly going to be focusing on generative models. At the end, I will talk a little bit about BERT.

One interesting thing about large language models is that when they generate some output, they don't come up with a reasoning plan of how am I going to answer this question. They just generate word by word. So given a current context, what is the most probable word to follow up next?

And that's done over and over and over again until the full text is generated in what we call autoregressive generation. So they do not understand texts in the way humans do. What they do is they recognize patterns in the data that they were trained on.

And another interesting thing about LLMs is that they are based on what we call transformer. So transformer is a deep learning architecture. So we have models like the GPT variations, even models like BERT, they are part of this LLM's universe.

If you notice, GPT-4 is not completely inside the LLM's circle because it also works with images. So it's not only an LLM. What they have in common is that they are all based on this transformer architecture, transformer can work with different modalities of data, it can work with video, image, audio text. When we talk about large language models, it's mostly about text data.

There are different ways of interacting with this type of model. So if the model is proprietary or a closed model, usually, you are going to access this model via some API. So you send some data to the API, you get some data back. But you can also have open source models that you just download it and run it yourself on your own premises.

So large language models, they are great at things like natural language understanding. And while it might not always generate the correct answer, it does seem to understand what you're asking. And it's great as an assistant to generate text and to generate code.

Well, of course, there are some limitations. So for example, if you ask for something, it's usually-- it is not possible to get a citation. So it's not able to say, here is where I found this information. You can verify it here. And that's a big problem because these models have what we call hallucinations.

So what is a hallucination? It's just when the model generates something that is factually incorrect. And in some cases, hallucination can be useful. So imagine you're trying to search for jokes about cars. I'm not going to tell any jokes. I'm not good at telling jokes. You can search it yourself.

So in that case, you want hallucination. You want creativity. But if you want a code-- the model to generate some code that does some precise computation, then no. Then hallucination is very harmful.

But it's still-- it can help you solve some specific problems if you know how to use those models. So it can help generating text. It can help classifying text. It's great at summarizing text and retrieving information. But the important thing is you need to take ownership of the model output. You need to have a way of verifying that the output is correct.

There are different approaches for leveraging these models. I'm going to mention some of them. So you can do what is called prompt engineering by directly querying the model. You can do what is called retrieval augmented generation. And that is feeding some bits of your own data so that ChatGPT is able to give a more focused answer.

And we also fine tuning. We can fine tune these models. So you can adapt the model to some domain specific data, for example. I'm going to talk a little bit about each one of them.

So while you can use Matlab to connect to the ChatGPT API, it's very easy to do so. For some things, you're just going to want to use the chat interface. So for example, if you want to try generating some Matlab code, just use the web interface. You get the output code if your request is simple enough probably to be correct.

But notice here that I didn't just ask for the code. I also asked the code to be optimized. I asked it to use vectorization. And this process of crafting and refining the inputs you give to the model is what we call prompt engineering. So there is no magic about prompt engineering. It can become more complex depending on your problem. But the more you try it, the better you're going to become at it.

And then, of course, for some other things, you're actually want to use the ChatGPT programmatically. So imagine you have thousands of nodes that you need to summarize. So you could just copy and paste each node into ChatGPT, ask ChatGPT for a summary, and then, you would have to save each file individually manually. That would take ages.

So the other option is you build a Matlab script that will read each file inside the loop. And then, you could just send a request to the ChatGPT API for the summarization. And then, you just automatically save each file. So that will be a way of automating your workflow. So every time you see there is a repetition in your interactions with ChatGPT, there is a potential for automation.

Another approach that you can try is using ChatGPT to ask questions about your own documents. So let's say I have a model XYZ that has some type of fault. And I know that ChatGPT does not have information about this model XYZ. So I'm going to have to provide ChatGPT with some information before it can answer my question.

So what I can do is I can look at my reports, but I cannot just feed all my documents into GPT. If you have tried ChatGPT, you know there's a limit in context, how much information you can feed to ChatGPT. So what we can do is we can filter based-- on my query, filter what are the relevant reports for that query. And you can do that using things like text analytics toolbox.

And then, you can feed both your query and your documents to ChatGPT. And then, you should be able to get a more focused answer. So you can use Retrieval Augmented Generation, or RAG, when you want to tailor the model's output towards your own data.

And the last approach I want to talk about today is fine tuning large language models. So if you have very specific text-- domain specific text, you might want to adapt the model to your own data. And you probably know ChatGPT is a closed model. You cannot just take the model, find it in yourself.

But if you're working with transformer models, we have support for it in Matlab. So you can fine tune those type of models inside Matlab. And one thing to remember is if you're working with huge models to the scale of billions of parameters, then you might have to look into what we call parameter efficient fine tuning techniques.

And I also told you I will go back and talk a little bit about BERT. So if your problem is very well scoped, then maybe you just need a smaller, more simple model. So BERT can be used by a fine tuning if you have a small label data set for things like document classification, word classification, question answering. So not all the problems need ChatGPT to be solved.

And I also want to announce something that we are working on. It's the Matlab AI Chat Playground. So this is a web app where you're going to be able to interact with an AI chatbot. All you're going to need is a MathWorks account to be able to log in and use it.

It's not ready yet. But you're going to be able to use this AI chat bot to generate Matlab code, for example. And then, you have next to it a live editor that you can just run the code and see if the generated code is correct.

In the next slide there will be a QR code. And I want you to grab your phone, scan it, if you want to have early access to this AI Chat Playground, OK? So thank you so much for paying attention to my presentation. It was a pleasure talking to you today.

[APPLAUSE]