Text summarization in NLP means telling a long story in short with a limited number of words and convey an important message in brief.

There can be many strategies to make the large message short and giving the most important information forward, one of them is calculating word frequencies and then normalizing the word frequencies by dividing by the maximum frequency.

After that finding the sentences with high frequencies and taking the most important sentences to convey the message.

Why do we need automatic summarization?

Time optimization, as it takes lesser time to get the gist of the text in…

“With public sentiment, nothing can fail, without it, nothing can succeed”-Abraham Lincoln

Sentiment analysis is a field in which we try to use natural language processing for text analysis, to identify emotion for a service, product, or entity. Sentiment analysis or opinion mining is used to determine whether data is positive, negative, or neutral.

Types of Sentiment Analysis

Subjective or objective classification:

Any objective text or speech is related to the facts, but any subjective text or speech has feelings. They are opposites.

Objective: “The weather is Sunny”.

Subjective: “I love sunny weather!”

In this type of sentiment analysis, we…

Sentiment Analysis is a field that has a lot of scope and application into recommendation systems.

Be it movie reviews, stock market, product, or groups, sentiments play a huge role in analyzing the trend and future of a product or service.

To analyze sentiments, different fields or domains may have totally different rules, like for example, the polarity of the words in reviews will decide sentiments, while for the stock market a different set of rules like a day’s hike or dip in its share price will decide the sentiment of news.

Here, I will take an example of the…

In one of my last blogs, I tried to explain text generation through a simple Maximum Likelihood model.

In this blog, I will try to explain how can we do the same through the Bidirectional LSTM model.

In one of my blogs of RNN, we talked about all types of RNNs but they had a shortcoming, i.e, dependency on context only from the past.

Bidirectional LSTMs can be used to train two sides, instead of one side of the input sequence. First from left to right on the input sequence and the second in reversed order of the input sequence…

When you are creating a chatbot or in any other NLP applications many challenges do come like getting a text from images, converting speech to text or text to speech, language translation, etc.

Let's have a sneak-peek into each of such tasks and how these can be easily done.

OCR-Optical Character Recognition

We often have a requirement to extract text from pictures, we have a library easyocr to do this job easily for you.

This library has support for various languages from English, Hindi, Frech, Chinese, Kannada, Malayalam, and so on.

It uses 3 main components Resnet, transfer learning model…

There is an old saying “Accuracy builds credibility”-Jim Rohn.

However, accuracy in machine learning may mean a totally different thing and we may have to use different methods to validate a model.

When we develop a classification model, we need to measure how good it is to predict. For its evaluation, we need to know what do we mean by good predictions.

There are some metrics that measure and evaluate the model on its accuracy of actually predicting the class and also improves it.

Let us see a Confusion matrix that defines a number of rightly predicted happy, sad, and…

In Natural Language Processing (NLP), it is not only important to make sense of words but context too.

N-grams are one of the ways to understand the language in terms of context to better understand the meaning of words written or spoken.

For example, “I need to book a ticket to Australia.” versus “I want to read a book of Shakespeare.”:

If you have a stack of websites or a stack of pdf files and you want to have answers to your questions, it looks like a hell of a task.

What if we can do it in a few lines of code using BERT.

BERT is a pre-trained transformer-based model. Here we will be using bert-squad1.1, this model is pre-trained on squad (Stanford Question Answer Dataset) dataset, which is a typical benchmark problem for question-answer models. It consists of more than 100,000 questions based on Wikipedia snippets. Also, it is annotated with the corresponding text span i.e …