Natural Language Processing
Fundamental Concepts
- Tokenization
- Stemming
- Lemmatization
- POS Tagging
- Named Entity Recognition
- Stopword Removal
- Syntax
- Dependency Parsing
- Parsing
- Chunking
Text Processing & Cleaning
- Text Normalization
- Bag of Words
- TF-IDF
- N-grams
- Word Embeddings
- Sentence Embeddings
- Document Similarity
- Cosine Similarity
- Text Vectorization
- Noise Removal
Tools, Libraries & APIs
- NLTK
- spaCy
- TextBlob
- Hugging Face Transformers
- Gensim
- OpenAI
- CoreNLP
- FastText
- Flair NLP
- ElasticSearch + NLP
Program(s)
- Build a Chatbot Using NLP
- Extracting Meaning from Text Using NLP in Python
- Extracting Email Addresses Using NLP in Python
- Extracting Names of People, Cities, and Countries Using NLP
- Format Email Messages Using NLP
- N-gram program
- Resume Skill Extraction Using NLP
- Sentiment Analysis in NLP
- Optimizing Travel Routes Using NLP & TSP Algorithm in Python
Program: Build a Chatbot Using NLP
Below is a Python program that uses Natural Language Processing (NLP) techniques to create a simple conversational chatbot. The chatbot can understand user input and respond appropriately.
import reimport randomimport nltkfrom nltk.tokenize import word_tokenizefrom nltk.corpus import stopwordsfrom nltk.stem import WordNetLemmatizer
# Download necessary NLTK resourcesnltk.download('punkt')nltk.download('stopwords')nltk.download('wordnet')
# Predefined responses for the chatbotresponses = { "greeting": ["Hello! How can I help you?", "Hi there! What can I do for you?", "Hey! What's on your mind?"], "farewell": ["Goodbye! Have a great day!", "See you later!", "Bye! Take care!"], "thanks": ["You're welcome!", "No problem!", "Happy to help!"], "default": ["I'm sorry, I didn't understand that.", "Could you please rephrase that?", "I'm not sure what you mean."]}
def preprocess_input(user_input): """ Preprocess the user input by cleaning, tokenizing, and lemmatizing it. """ # Convert text to lowercase user_input = user_input.lower()
# Remove special characters and numbers user_input = re.sub(r'[^a-zA-Z\s]', '', user_input)
# Tokenize words words = word_tokenize(user_input)
# Remove stopwords stop_words = set(stopwords.words('english')) filtered_words = [word for word in words if word not in stop_words]
# Lemmatize words lemmatizer = WordNetLemmatizer() lemmatized_words = [lemmatizer.lemmatize(word) for word in filtered_words]
return lemmatized_words
def get_response(intent): """ Get a random response based on the detected intent. """ return random.choice(responses.get(intent, responses["default"]))
def detect_intent(user_input): """ Detect the intent of the user input based on keywords. """ greeting_keywords = ["hello", "hi", "hey"] farewell_keywords = ["bye", "goodbye", "see you"] thanks_keywords = ["thanks", "thank you", "appreciate"]
if any(word in user_input for word in greeting_keywords): return "greeting" elif any(word in user_input for word in farewell_keywords): return "farewell" elif any(word in user_input for word in thanks_keywords): return "thanks" else: return "default"
def chatbot(): """ Run the chatbot and interact with the user. """ print("Chatbot: Hello! I'm your friendly chatbot. Type 'exit' to end the conversation.") while True: user_input = input("You: ") if user_input.lower() == "exit": print("Chatbot: Goodbye!") break
# Preprocess user input processed_input = preprocess_input(user_input)
# Detect intent intent = detect_intent(processed_input)
# Get and print response response = get_response(intent) print(f"Chatbot: {response}")
# Run the chatbotchatbot()
Explanation of the Program
-
Preprocessing User Input:
- The
preprocess_input
function cleans the user input by converting it to lowercase, removing special characters and numbers, tokenizing words, removing stopwords, and lemmatizing words.
- The
-
Detecting Intent:
- The
detect_intent
function identifies the user’s intent based on predefined keywords. For example, if the input contains “hello” or “hi,” the intent is classified as a greeting.
- The
-
Generating Responses:
- The
get_response
function selects a random response from the predefined responses based on the detected intent.
- The
-
Chatbot Interaction:
- The
chatbot
function runs the chatbot, allowing the user to interact with it. The conversation continues until the user types “exit.”
- The
Example Conversation
Chatbot: Hello! I'm your friendly chatbot. Type 'exit' to end the conversation.You: Hi there!Chatbot: Hi there! What can I do for you?You: How are you?Chatbot: I'm sorry, I didn't understand that.You: Thank you for your help!Chatbot: You're welcome!You: GoodbyeChatbot: Goodbye! Have a great day!
Benefits of Using NLP for Chatbots
- Improved Understanding: NLP techniques like tokenization and lemmatization help the chatbot better understand user input.
- Personalized Responses: By detecting intent, the chatbot can provide more relevant and personalized responses.
- Scalability: This program can be extended to handle more complex conversations and integrate with APIs for advanced functionality.
This program is a simple yet effective way to build a chatbot using NLP techniques.