NodeJS: Finetuning LLMs

Neural Network

  1. Type of Machine Learning model inspired by the way human brain works. 
  2. They are made up of layers of connected nodes called "neurons" which process data
  3. They improve over time by recognizing patterns and making decisions

LLM

L: Large
LM: Language Model
  1. Its called "Large" because they have billions of parameters.
  2. "Language Model" because they deal with tasks that are language related like question-answering, translation, sentiment analysis etc
  3. LLM is a Deep Learning Model that is trained on vast amount of data to Understand, Generate and Respond to human-like text
  4. LLM is just one powerful tool inside the large toolbox of NLP
    Ex: Grammarly is an NLP system but not an LLM because it isn’t trained on a large scale using deep learning models like transformers. However, it can still analyze and process human language.

Transformers? What is that? 😕

Imagine you're reading a long book, and you want to understand how different sentences relate to each other. A Transformer is like a smart highlighter that:

  1. Focuses on important words in a sentence, even if they’re far apart.
  2. Understands meaning based on context, not just word order.
  3. Processes everything at once (not word by word like older models).

For example:
Sentence: "Alice saw Bob at the park, and she waved at him."
 A Transformer knows that "she" refers to Alice and "him" refers to Bob, even though those words are not next to each other.

This is done using a technique called "attention", which helps the model "pay attention" to the right words. That’s why transformers power advanced AI like ChatGPT and Google’s BERT! 



 
A Basic LLM on Nodejs

Run: npm i openai dotenv fs
  1. Cannot fine tune models on free version. Be sure to purchase from https://platform.openai.com/
  2. Once you get the API key, update .env file. Create a file index.js
    OPENAI_API_KEY="123"
import OpenAI from "openai";
import dotenv from "dotenv";

dotenv.config();
const openai = new OpenAI();

const completion = await openai.chat.completions.create({
    model: "gpt-3.5-turbo",
    max_tokens: 50, //limits the output length
    messages: [{
            role: "system",
            content: "Chan is an insurance company chatbot that only answers
questions about insurance policies, coverage, and related topics."
        },{
            role: "user",
            content: "Do accidents increase my premium?",
        }],
});
console.log(completion.choices[0].message.content);

Run: node index.js

And that's it! we have our first LLM integrated.

FINETUNING

Just 2 steps to fine-tune an LLM. At the end of fine-tuning, I would have created my own version of LLM that is modified and trained on the dataset specific to my company/need.

2 Steps:
  1. Upload the file (.jsonl format) containing your training data.
  2. Create a finetuning job by passing the ID received from previous step
Create a file with .jsonl extension (training_data.jsonl)
{"messages": [{"role": "system", "content": "Chan is an insurance company chatbot that only answers questions about insurance policies, coverage, and related topics."}, {"role": "user", "content": "What does renters insurance cover?"}, {"role": "assistant", "content": "At Chan, Renters insurance typically covers personal property, liability, and additional living expenses in case of an emergency. Check your policy for specifics."}]}
{"messages": [{"role": "system", "content": "Chan is an insurance company chatbot that only answers questions about insurance policies, coverage, and related topics."}, {"role": "user", "content": "How is my premium determined?"}, {"role": "assistant", "content": "At Chan, Your premium is determined by factors such as your driving history, the type of car, your location, and your coverage options."}]}
{"messages": [{"role": "system", "content": "Chan is an insurance company chatbot that only answers questions about insurance policies, coverage, and related topics."}, {"role": "user", "content": "What is the weather today?"}, {"role": "assistant", "content": "Sorry, Please ask a question about Chan insurance."}]}
{"messages": [{"role": "system", "content": "Chan is an insurance company chatbot that only answers questions about insurance policies, coverage, and related topics."}, {"role": "user", "content": "Who won the last soccer world cup?"}, {"role": "assistant", "content": "Sorry, Please ask a question about Chan insurance."}]}

STEP 1: Upload
import OpenAI from "openai";
import dotenv from "dotenv";

dotenv.config();
const openai = new OpenAI();

const upload = async () => {
    const file = await openai.files.create({
        file: fs.createReadStream("./training_data.jsonl"),
        purpose: 'fine-tune'
    })
    console.log(file)
    return file;
}
upload();
Note: Make a note of the fileID received as a response. Make sure not to run uploads multiple times

import OpenAI from "openai";
import dotenv from "dotenv";

dotenv.config();
const openai = new OpenAI();

const upload = async () => {
    const file = await openai.files.create({
        file: fs.createReadStream("./training_data.jsonl"),
        purpose: 'fine-tune'
    })
    console.log(file)
    return file;
}
//upload();

const createFineTune = async() => {
    try {
      const model = await openai.fineTuning.jobs.create({
        training_file: '<fileID>',
        model: 'gpt-3.5-turbo'
      })
      console.log('response: ', model)
    } catch (err) {
      console.log('error: ', err)
    }
}
createFineTune();
Note: Make note of the Job Id if you like to check the progress later. But not necessary

You need to wait for some time. It may take anywhere between 20min to 1hr to complete the job. You can view the progress on the openAI playground

Or if you wish to check the progress from your code, make sure to note down the Job ID from create fine tune step. Then run this:
const checkFineTuneModelCreationStatus = async() => {
    try {
        const response = await openai.fineTuning.jobs.retrieve("ftjob-ABCD")
        console.log('data: ', response)
      } catch (err) {
        console.log('error:', err)
      }
}
checkFineTuneModelCreationStatus();

Everytime a set of new training data is uploaded, the file ID changes. So everything needs to be run again. This is a drawback of finetuning approach.

Once the job is successful, it can be tested like before. Any insurance related question should be answered with "At Chan" prefix. Any non-insurance question should be answered with 'Sorry, Please ask a question about Chan insurance."






Comments

Post a Comment

Popular posts from this blog

Inside the JavaScript Memory Box: Visualizing Variables, References, and Copies

Image
There is a difference in the way JavaScript refences variables based on whether they are primitive or non-primitive Primitive Types: string, number, bigint, Boolean, symbol, null, undefined Non-Primitive: Arrays, Objects Each variable in JS is stored in a memory “box” Assignments: But when another variable tries to assign this value, JS thinks “wait, is this a primitive or non-primitive? Because if it is primitive, I will create a new box for this new variable with the same content. But if it’s not, I will just create a refence to the existing box” PRIMITIVE: Analogy: Let's say you have a note that says 5. You give a copy of this note to your friend.  Friend now updates this copy to 10. But your copy still says 5 let a = 5; let b = a; b = 10; //a remains unchanged NON-PRIMITIVE: Analogy: Let's say you have a note with address "123 Elm Street". In that house lives a cat named "Tom". You give a copy of this note to your friend. Friend now goes t...
Read more

React & State Management: All Concepts

Section 1: Fundamentals 1. Counter with increment, decrement, reset — useState Remember: prev is needed when the new state depends on the old state. When you're setting a hardcoded value like 0, you just pass 0 directly. Why:  reason prev matters — if multiple state updates happen in the same render cycle, React batches them. Using prev guarantees you're working off the latest state, not a stale snapshot. 2. Toggle show/hide paragraph — useState Remember: onClick={toggle} — pass the function reference, not the call. onClick={() => toggle()} — also fine, arrow function that calls toggle when clicked. onClick={toggle()} — calls toggle immediately during render, setVisible triggers re-render, toggle calls again, infinite loop. Why: JSX evaluates expressions inside {} during render. toggle() is an expression that executes immediately. React doesn't get a chance to attach it as a handler — it just runs it right there, state updates, component re-renders, runs again. ...
Read more

React: Communication between components

Parent to Child: Props Child to Parent Callback Props (advanced props) Controlled Components useImperative hook  Nested Tree: useContext Sibling: Lift State Up PARENT TO CHILD: Props App.jsx Users.jsx <User name ={user} /> function User({ name }) {  return <h1> {name} </h1> } CHILD TO PARENT a) Callback Props (Advanced Props) Parent.jsx Child1.jsx export function Parent() {   function sayHello () {     alert(“hello triggered”)   }   <Child1 onSayHello ={ sayHello } /> } function Child1({ onSayHello }) {   return <button onClick={ onSayhello() }>          Click       </button> } b) Controlled Components Usually used in forms. Controlled components mean the parent owns the state, the child renders inputs, and every ch...
Read more