Chat With Website Ai Chatbot in Python App [Source Code]

In This Blog, We Are Going to build a Chat With Website From an Url Bot In Python Using Langchain & Openai API with Streamlit GUI. We Are Going to provide Source code For That so you don’t Have To Worry Just copy it and Follow Guide And That’s It You Are Ready to Make Your Ai Within a Few Minutes

What is Chat with a Website?

With Chat With Website Bot you Can just Enter The Any Website Url And After That You Can Ask Question Based on That website and With the Help Of Gpt And Langchain It Will Extract Information From That Page And Tell You An Answer.

You Can Use This For Research, Summarization, Writing Blogs Or Any other Kinds Of Result Or May be You Can Scale this Project And Make your Own Saas Business Out Of it.

Chat with Website Chatbot App With Python [Source Code]

We Are going to Use Streamlit For Design Our UI And Langchain & Gpt For Building Chatting Funactionality.

So Firstly You Have To Install Following Libraries :

requirements.txt

langchain==0.1.4
langchain_community==0.0.16
langchain_core==0.1.17
langchain_openai==0.0.5
python-dotenv==1.0.1
streamlit==1.30.0
chromadb==0.4.22
beautifulsoup4=4.12.2=py310hca03da5_0

You Can Use This Command To Install All Required Libraries, But Before You Need To create requirements.txt file Then Paste The All Libraries into That then Use This Below Command In Terminal

pip install -r requirements.txt

Now After Installation Of All Required Libraries Its time For Application Code Which is Streamlit Code

Create File Named App.py And Paste Following Code into it :

App.py

import streamlit as st
from langchain_core.messages import AIMessage, HumanMessage
from langchain_community.document_loaders import WebBaseLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from dotenv import load_dotenv
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain.chains import create_history_aware_retriever, create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain


load_dotenv()

def get_vectorstore_from_url(url):
    # get the text in document form
    loader = WebBaseLoader(url)
    document = loader.load()
    
    # split the document into chunks
    text_splitter = RecursiveCharacterTextSplitter()
    document_chunks = text_splitter.split_documents(document)
    
    # create a vectorstore from the chunks
    vector_store = Chroma.from_documents(document_chunks, OpenAIEmbeddings())

    return vector_store

def get_context_retriever_chain(vector_store):
    llm = ChatOpenAI()
    
    retriever = vector_store.as_retriever()
    
    prompt = ChatPromptTemplate.from_messages([
      MessagesPlaceholder(variable_name="chat_history"),
      ("user", "{input}"),
      ("user", "Given the above conversation, generate a search query to look up in order to get information relevant to the conversation")
    ])
    
    retriever_chain = create_history_aware_retriever(llm, retriever, prompt)
    
    return retriever_chain
    
def get_conversational_rag_chain(retriever_chain): 
    
    llm = ChatOpenAI()
    
    prompt = ChatPromptTemplate.from_messages([
      ("system", "Answer the user's questions based on the below context:\n\n{context}"),
      MessagesPlaceholder(variable_name="chat_history"),
      ("user", "{input}"),
    ])
    
    stuff_documents_chain = create_stuff_documents_chain(llm,prompt)
    
    return create_retrieval_chain(retriever_chain, stuff_documents_chain)

def get_response(user_input):
    retriever_chain = get_context_retriever_chain(st.session_state.vector_store)
    conversation_rag_chain = get_conversational_rag_chain(retriever_chain)
    
    response = conversation_rag_chain.invoke({
        "chat_history": st.session_state.chat_history,
        "input": user_query
    })
    
    return response['answer']

# app config
st.set_page_config(page_title="Chat with websites", page_icon="")
st.title("Chat with websites")

# sidebar
with st.sidebar:
    st.header("Settings")
    website_url = st.text_input("Website URL")

if website_url is None or website_url == "":
    st.info("Please enter a website URL")

else:
    # session state
    if "chat_history" not in st.session_state:
        st.session_state.chat_history = [
            AIMessage(content="Hello, I am a bot. How can I help you?"),
        ]
    if "vector_store" not in st.session_state:
        st.session_state.vector_store = get_vectorstore_from_url(website_url)    

    # user input
    user_query = st.chat_input("Type your message here...")
    if user_query is not None and user_query != "":
        response = get_response(user_query)
        st.session_state.chat_history.append(HumanMessage(content=user_query))
        st.session_state.chat_history.append(AIMessage(content=response))
        
       

    # conversation
    for message in st.session_state.chat_history:
        if isinstance(message, AIMessage):
            with st.chat_message("AI"):
                st.write(message.content)
        elif isinstance(message, HumanMessage):
            with st.chat_message("Human"):
                st.write(message.content)

Save This file And Now Create Another File Called .env , In This File We Are Going To Keep Our OpenAi API Key :

.env

OPENAI_API_KEY= [your-openai-api-key]

Replace Your Own Key With This [your-openai-api-key] .

How To Get Your OpenAi Api Key?

1. Create an OpenAI account‍
2. Verify your account‍
3. Log into your account‍
4. Navigate to the API section.
5. Generate a new API key.
6. Copy That Key And Replace With This [your-openai-api-key]

You Will Get 5$ Free Api credit on New Openai Account

So That’s it Just Save Your Files And Its Time To Run Our App. Use following command To Run Our App :

Run the following command in Terminal To Run App :

streamlit run app.py

How Chat with a Website App Going to Work?

1. You provide the website URL: You input the link to the website you want to analyze.
2. The chatbot processes the website: The chatbot uses its AI capabilities to read, analyze, and understand the website’s content, including text, images, and structure.
3. You ask your questions: You can then ask the chatbot questions about the website, such as:
   • What is the main topic of this website?
4. The chatbot answers your questions: Based on its analysis of the website, the chatbot provides you with informative answers to your questions

Features Of Chat with a Website Bot?

• Information extraction: Extracting key details like facts, figures, dates, and contact information based on your questions.
• Summarization: Providing concise summaries of the website’s content to save you time and highlight key points.

Conclusion :

You can Use This Code For Anything And Also you can Scale It and Convert it into Saas Business As you Choice. So But Some Website Might Be Blocked But it Works With Most Of Sites.