IMAP email IDLE stops waiting timeout duration, instead loops instantaneously in Python – Telegram API

I have a program to check for new emails in an inbox using IMAP IDLE. Whenever I run it, it works perfectly at first. But after some time, it stops pausing for the IDLE timeout time and instead rapidly loops through the first part.

from imapclient import IMAPClient
from imap_tools import MailBox, AND

while True:
    try:
        #this is the part that's getting looped every half second when it breaks
        telegram('try portion starting again')

        # Wait for up to 540 seconds for an IDLE response
        responses = server.idle_check(timeout=540)
        if responses:
            telegram('Email Recieved')

            for response in responses:

                if response[0] not in checked_uids:

                    checked_uids.append(response[0])

                    uid = str(response[0])

                    with MailBox(HOST).login(USERNAME, PASSWORD) as mailbox:
                        for msg in mailbox.fetch(AND(uid=uid)):

                            body_text=msg.text

                            telegram(body_text)

Where telegram() sends a telegram message to me.

So when this is working correctly, I get a message containing the body of a new email, and every 540 seconds I get a message that the try: portion has started again.

In my testing, it works just like this for some time then all of a sudden, for no reason I can discern, I get the "try portion starting again" message twice a second repeated to infinity. Meaning, seemingly the responses = server.idle_check(timeout=540) portion stops working as intended

edit: here is the catch section of the code I’m using. Super simple. I’m also not getting any error messages while the program is going crazy and cycling so quickly

except KeyboardInterrupt:
    break
except Exception as e:
    print(e)
    telegram(e)