skip to Main Content

This program is to find similarities between the a sentences and words and how they are similar in synonyms I have downloaded the nltk when i first coded it was run and there were no errors but after some days when i run the programenter image description here

import nltk
nltk.download('stopwords')
nltk.download('wordnet')
from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
from nltk.corpus import wordnet as wn

filtered_uploaded_sentences = []
uploaded_sentence_synset = []
database_word_synset = []

uploaded_doc_sentence=" The issue of text semantics, such as word semantics and sentence semantics has received increasing attentions in recent years. However, rare research focuses on the document-level semantic matching due to its complexity. Long documents usually have sophisticated structure and massive information, which causes hardship to measure their semantic similarity. The semantic similarity between words, sentences, texts, and documents is widely studied in various fields, including natural language processing, document semantic comparison, artificial intelligence, semantic web, and semantic search engines. "
database_word=["car","complete",'focus',"semantics"]

stopwords = stopwords.words('english')
uploaded_sentence_words_tokenized = word_tokenize(uploaded_doc_sentence)

#filtering the sentence and synset

for word in uploaded_sentence_words_tokenized:
    if word not in stopwords:      
        filtered_uploaded_sentences.append(word)
print (filtered_uploaded_sentences)

for sentences_are in filtered_uploaded_sentences:
    uploaded_sentence_synset.append(wn.synsets(sentences_are))
    
print(uploaded_sentence_synset)

#for finding similrity in the words

for databasewords in database_word:
    database_word_synset.append(wn.synsets(databasewords)[0])
    
print(database_word_synset)

Index Error: list index out of range
this error comes when the uploaded_doc_sentence is short and long sentence is used the error accorded

check.append(wn.wup_similarity(data,sen[0]))

I want to compare the sentence and the words and the results are stored. this type

#the similarity main function for words

for data in database_word_synset:
    for sen in uploaded_sentence_synset :
        check.append(wn.wup_similarity(data,sen[0]))
print(check)

2

Answers


  1. Chosen as BEST ANSWER

    by removing the empty [] list blocks from the list and making the multi dimension list into single dimension list the problem was solver

    list2 = [x for x in main_sen if x != []]
    print(list2)
    result=list()
    for t in list2: 
        for x in t: 
            result.append(x)
    

    enter image description here


  2. The problem is that there are empty lists contained in uploaded_sentence_synset. I’m not sure what you’re trying to do, but modify the last block of code to:

    for data in database_word_synset:
        for sen in uploaded_sentence_synset:
            if sen:
                check.append(wn.wup_similarity(data, sen[0]))
    

    Without the if-else block, you’re essentially trying to index the first element of a list, giving you an IndexError.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search