Vector Embeddings and Vector Storage

OVERVIEW

VECTOR EMBEDDINGS

Ex: A simple text "Dog" could be represented in a vector format. This is the format Machine Learning understands. 

OpenAI already gives us an API we can use to convert any string or an array into a vector format

To visualize a vector:
Image a 2-D graph plotted to represent the words "book","page","dog","cat"

When plotted on a graph, "book" and "page" have a similar semantic meaning. They are plotted closer. Same goes to "dog" and "cat".

Vector is similar to an array. But not 2-D. Our example just has X, Y axis. 
Imagine thousands of dimensions to plot a text, which our brain cannot really visualize. 
This same word "dog" in vector representation has ~1500 items in the array. 1500 Dimensions!, each representing some learned feature about "dog" based on a machine learning model.

Why So Many Dimensions?

  • Each dimension captures a different aspect of meaning, context, or relationship.
  • More dimensions allow for finer distinctions between similar concepts.
  • However, too many dimensions can lead to inefficiencies (this is called the "curse of dimensionality").

    • OpenAI Embedding API:

    We do not need to worry about the logic behind how they are embedded since OpenAI does this job for us.
    All we need to do is, 
    make an API call to: POST https://api.openai.com/v1/embeddings
    with the text you want to embed: 
    {
     "input": "cat",
     "model": "text-embedding-ada-002",
    "encoding_format": "float"
    }

    And make sure to send API key in the Authorization bearer token


    curl https://api.openai.com/v1/embeddings \
      -H "Authorization: Bearer $OPENAI_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{
        "input": "cat",
        "model": "text-embedding-ada-002",
        "encoding_format": "float"
      }'

    VECTOR STORAGE

    OpenAI does not provide a way to store this vector. We need to store this in a DataBase that is designed to store and efficiently search Vector Embeddings.

    We have various providers like:
    FAISS (from Meta), Pinecone, SingleStore etc.

    Here, we will be using SingleStore
    Create your account (i used gmail SSO)

    Create a DataBase
    1. Create a user


    2. My DB name: db_kavyashree_e5db0

    3. navigate to SQL Editor where you can enter SQL query to create a table for the DB

    CREATE TABLE IF NOT EXISTS myVectorTable (
        text TEXT,
        vector BLOB
    )

    There will be two columns for our table (myVectorTable)
    1. One is the text we want to embed
    2. And 2nd is the vector form of this text as returned by OpenAI API response
    Click on "run"

    4. Insert data into the table
    In the same SQL editor, enter
    INSERT INTO myVectorTable (text, vector) 
    VALUES ("cat", JSON_ARRAY_PACK(" <COPY_PASTE_YOUR_VECTOR_EMBEDDING_HERE> "))
    "JSON_ARRAY_PACK" is to tell the vector storage that this is a vector

    Data is inserted

    You can add all the 4 data (dog, cat, book, paper)
    I inserted items in the below order
    cat, dog, book, rat, paper

    Notice how i inserted rat away from cat and dog.


    5. Let us perform a search to test if we can find the closest match to what we are searching for. 
    Let's say i want to search for "animal"

    Even to do this, we first need a text embedding of "animal"

    1. Back to SQL editor
    SELECT text, dot_product(vector, JSON_ARRAY_PACK("[]")) as score
    FROM myVectorTable
    order by score desc
    limit 5;

    In place of empty array, insert "animal" vector

    Now you will see that animals in our table are ranked highest. This is how vector search works.

    Comments

    Post a Comment

    Popular posts from this blog

    Inside the JavaScript Memory Box: Visualizing Variables, References, and Copies

    Image
    There is a difference in the way JavaScript refences variables based on whether they are primitive or non-primitive Primitive Types: string, number, bigint, Boolean, symbol, null, undefined Non-Primitive: Arrays, Objects Each variable in JS is stored in a memory “box” Assignments: But when another variable tries to assign this value, JS thinks “wait, is this a primitive or non-primitive? Because if it is primitive, I will create a new box for this new variable with the same content. But if it’s not, I will just create a refence to the existing box” PRIMITIVE: Analogy: Let's say you have a note that says 5. You give a copy of this note to your friend.  Friend now updates this copy to 10. But your copy still says 5 let a = 5; let b = a; b = 10; //a remains unchanged NON-PRIMITIVE: Analogy: Let's say you have a note with address "123 Elm Street". In that house lives a cat named "Tom". You give a copy of this note to your friend. Friend now goes t...
    Read more

    React & State Management: All Concepts

    Section 1: Fundamentals 1. Counter with increment, decrement, reset — useState Remember: prev is needed when the new state depends on the old state. When you're setting a hardcoded value like 0, you just pass 0 directly. Why:  reason prev matters — if multiple state updates happen in the same render cycle, React batches them. Using prev guarantees you're working off the latest state, not a stale snapshot. 2. Toggle show/hide paragraph — useState Remember: onClick={toggle} — pass the function reference, not the call. onClick={() => toggle()} — also fine, arrow function that calls toggle when clicked. onClick={toggle()} — calls toggle immediately during render, setVisible triggers re-render, toggle calls again, infinite loop. Why: JSX evaluates expressions inside {} during render. toggle() is an expression that executes immediately. React doesn't get a chance to attach it as a handler — it just runs it right there, state updates, component re-renders, runs again. ...
    Read more

    React: Communication between components

    Parent to Child: Props Child to Parent Callback Props (advanced props) Controlled Components useImperative hook  Nested Tree: useContext Sibling: Lift State Up PARENT TO CHILD: Props App.jsx Users.jsx <User name ={user} /> function User({ name }) {  return <h1> {name} </h1> } CHILD TO PARENT a) Callback Props (Advanced Props) Parent.jsx Child1.jsx export function Parent() {   function sayHello () {     alert(“hello triggered”)   }   <Child1 onSayHello ={ sayHello } /> } function Child1({ onSayHello }) {   return <button onClick={ onSayhello() }>          Click       </button> } b) Controlled Components Usually used in forms. Controlled components mean the parent owns the state, the child renders inputs, and every ch...
    Read more