Natural Language Processing
Fundamental Concepts
- Tokenization
- Stemming
- Lemmatization
- POS Tagging
- Named Entity Recognition
- Stopword Removal
- Syntax
- Dependency Parsing
- Parsing
- Chunking
Text Processing & Cleaning
- Text Normalization
- Bag of Words
- TF-IDF
- N-grams
- Word Embeddings
- Sentence Embeddings
- Document Similarity
- Cosine Similarity
- Text Vectorization
- Noise Removal
Tools, Libraries & APIs
- NLTK
- spaCy
- TextBlob
- Hugging Face Transformers
- Gensim
- OpenAI
- CoreNLP
- FastText
- Flair NLP
- ElasticSearch + NLP
Program(s)
- Build a Chatbot Using NLP
- Extracting Meaning from Text Using NLP in Python
- Extracting Email Addresses Using NLP in Python
- Extracting Names of People, Cities, and Countries Using NLP
- Format Email Messages Using NLP
- N-gram program
- Resume Skill Extraction Using NLP
- Sentiment Analysis in NLP
- Optimizing Travel Routes Using NLP & TSP Algorithm in Python
Python Program to Extract the Meaning of a Paragraph Using NLP
This program utilizes spaCy and TextBlob to extract the meaning of a paragraph by performing text summarization and keyword extraction.
1. Install Required Libraries
Run the following command to install necessary libraries if not installed:
pip install spacy textblob nltk
python -m spacy download en_core_web_sm
2. Python Program
import spacy
from textblob import TextBlob
import nltk
from collections import Counter
from nltk.corpus import stopwords
import string
# Download stopwords
nltk.download('stopwords')
# Load spaCy English model
nlp = spacy.load("en_core_web_sm")
def extract_meaning(paragraph):
"""
Function to extract the meaning of a paragraph using NLP techniques
- Summarization using TextBlob
- Keyword extraction using spaCy and NLTK
"""
# Step 1: Text Summarization
blob = TextBlob(paragraph)
sentences = blob.sentences
summary = " ".join(str(sentence) for sentence in sentences[:2]) # Extract first 2 sentences as summary
# Step 2: Keyword Extraction
doc = nlp(paragraph)
words = [token.text.lower() for token in doc if token.is_alpha] # Extract words, ignore punctuation
stop_words = set(stopwords.words('english'))
keywords = [word for word in words if word not in stop_words] # Remove stopwords
most_common_keywords = [word for word, freq in Counter(keywords).most_common(5)] # Top 5 keywords
# Step 3: Extract Sentiment
sentiment = blob.sentiment.polarity
if sentiment > 0:
sentiment_analysis = "Positive"
elif sentiment < 0:
sentiment_analysis = "Negative"
else:
sentiment_analysis = "Neutral"
# Print results
print("\n--- Extracted Meaning ---")
print(f"Summary: {summary}")
print(f"Top Keywords: {', '.join(most_common_keywords)}")
print(f"Sentiment: {sentiment_analysis}")
# Example Usage
if __name__ == "__main__":
paragraph = input("Enter a paragraph: ")
extract_meaning(paragraph)
3. Explanation of the Code
- Text Summarization: Extracts the first two sentences from the paragraph as a summary.
- Keyword Extraction:
- Uses spaCy to tokenize words.
- Removes stopwords using NLTK.
- Finds the top 5 most common keywords.
- Sentiment Analysis:
- Uses TextBlob to determine sentiment polarity.
- Classifies sentiment as positive, negative, or neutral.
4. Example Output
Input:
Enter a paragraph: Artificial Intelligence is transforming industries. It enables automation, improves efficiency, and enhances decision-making. AI-driven systems can analyze vast amounts of data quickly. However, ethical concerns regarding AI remain a challenge.
Output:
--- Extracted Meaning ---
Summary: Artificial Intelligence is transforming industries. It enables automation, improves efficiency, and enhances decision-making.
Top Keywords: artificial, intelligence, transforming, automation, efficiency
Sentiment: Positive
5. Where to Use This Program?
- Summarizing long documents for quick insights.
- Keyword extraction for SEO and content optimization.
- Sentiment analysis for understanding opinions in customer reviews or social media posts.
- Chatbots and AI Assistants to comprehend text input.
Would you like any enhancements, such as topic classification or concept extraction? 😊