Natural Language Processing
Fundamental Concepts
- Tokenization
- Stemming
- Lemmatization
- POS Tagging
- Named Entity Recognition
- Stopword Removal
- Syntax
- Dependency Parsing
- Parsing
- Chunking
Text Processing & Cleaning
- Text Normalization
- Bag of Words
- TF-IDF
- N-grams
- Word Embeddings
- Sentence Embeddings
- Document Similarity
- Cosine Similarity
- Text Vectorization
- Noise Removal
Tools, Libraries & APIs
- NLTK
- spaCy
- TextBlob
- Hugging Face Transformers
- Gensim
- OpenAI
- CoreNLP
- FastText
- Flair NLP
- ElasticSearch + NLP
Program(s)
- Build a Chatbot Using NLP
- Extracting Meaning from Text Using NLP in Python
- Extracting Email Addresses Using NLP in Python
- Extracting Names of People, Cities, and Countries Using NLP
- Format Email Messages Using NLP
- N-gram program
- Resume Skill Extraction Using NLP
- Sentiment Analysis in NLP
- Optimizing Travel Routes Using NLP & TSP Algorithm in Python
Program: Build a Chatbot Using NLP
Below is a Python program that uses Natural Language Processing (NLP) techniques to create a simple conversational chatbot. The chatbot can understand user input and respond appropriately.
import reimport randomimport nltkfrom nltk.tokenize import word_tokenizefrom nltk.corpus import stopwordsfrom nltk.stem import WordNetLemmatizer
# Download necessary NLTK resourcesnltk.download('punkt')nltk.download('stopwords')nltk.download('wordnet')
# Predefined responses for the chatbotresponses = {    "greeting": ["Hello! How can I help you?", "Hi there! What can I do for you?", "Hey! What's on your mind?"],    "farewell": ["Goodbye! Have a great day!", "See you later!", "Bye! Take care!"],    "thanks": ["You're welcome!", "No problem!", "Happy to help!"],    "default": ["I'm sorry, I didn't understand that.", "Could you please rephrase that?", "I'm not sure what you mean."]}
def preprocess_input(user_input):    """    Preprocess the user input by cleaning, tokenizing, and lemmatizing it.    """    # Convert text to lowercase    user_input = user_input.lower()
    # Remove special characters and numbers    user_input = re.sub(r'[^a-zA-Z\s]', '', user_input)
    # Tokenize words    words = word_tokenize(user_input)
    # Remove stopwords    stop_words = set(stopwords.words('english'))    filtered_words = [word for word in words if word not in stop_words]
    # Lemmatize words    lemmatizer = WordNetLemmatizer()    lemmatized_words = [lemmatizer.lemmatize(word) for word in filtered_words]
    return lemmatized_words
def get_response(intent):    """    Get a random response based on the detected intent.    """    return random.choice(responses.get(intent, responses["default"]))
def detect_intent(user_input):    """    Detect the intent of the user input based on keywords.    """    greeting_keywords = ["hello", "hi", "hey"]    farewell_keywords = ["bye", "goodbye", "see you"]    thanks_keywords = ["thanks", "thank you", "appreciate"]
    if any(word in user_input for word in greeting_keywords):        return "greeting"    elif any(word in user_input for word in farewell_keywords):        return "farewell"    elif any(word in user_input for word in thanks_keywords):        return "thanks"    else:        return "default"
def chatbot():    """    Run the chatbot and interact with the user.    """    print("Chatbot: Hello! I'm your friendly chatbot. Type 'exit' to end the conversation.")    while True:        user_input = input("You: ")        if user_input.lower() == "exit":            print("Chatbot: Goodbye!")            break
        # Preprocess user input        processed_input = preprocess_input(user_input)
        # Detect intent        intent = detect_intent(processed_input)
        # Get and print response        response = get_response(intent)        print(f"Chatbot: {response}")
# Run the chatbotchatbot()Explanation of the Program
- 
Preprocessing User Input: - The preprocess_inputfunction cleans the user input by converting it to lowercase, removing special characters and numbers, tokenizing words, removing stopwords, and lemmatizing words.
 
- The 
- 
Detecting Intent: - The detect_intentfunction identifies the user’s intent based on predefined keywords. For example, if the input contains “hello” or “hi,” the intent is classified as a greeting.
 
- The 
- 
Generating Responses: - The get_responsefunction selects a random response from the predefined responses based on the detected intent.
 
- The 
- 
Chatbot Interaction: - The chatbotfunction runs the chatbot, allowing the user to interact with it. The conversation continues until the user types “exit.”
 
- The 
Example Conversation
Chatbot: Hello! I'm your friendly chatbot. Type 'exit' to end the conversation.You: Hi there!Chatbot: Hi there! What can I do for you?You: How are you?Chatbot: I'm sorry, I didn't understand that.You: Thank you for your help!Chatbot: You're welcome!You: GoodbyeChatbot: Goodbye! Have a great day!Benefits of Using NLP for Chatbots
- Improved Understanding: NLP techniques like tokenization and lemmatization help the chatbot better understand user input.
- Personalized Responses: By detecting intent, the chatbot can provide more relevant and personalized responses.
- Scalability: This program can be extended to handle more complex conversations and integrate with APIs for advanced functionality.
This program is a simple yet effective way to build a chatbot using NLP techniques.