Question 215408 in Redis Question posted in
The official documentation can be found here.

Redis – Inconsistent Flask Session

I am planning to deploy a program via waitress for development purposes. I decided to use Flask Session to be able to create a unique identity per user.

app = Flask(__name__)
app.secret_key = secrets.token_hex(16)
app.config['UPLOAD_FOLDER'] = 'uploads'
app.config['SESSION_TYPE'] = 'filesystem'
app.config['SESSION_FILE_DIR'] = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'session_files')

Session(app)



@app.route('/upload', methods=['POST'])
def upload():
    files = request.files.getlist('files')
    if files:
        if 'user_folder' not in session:
            session['user_folder'] = str(uuid.uuid4())
            user_upload_folder = os.path.join(app.config['UPLOAD_FOLDER'], session['user_folder'])
            os.makedirs(user_upload_folder, exist_ok=True)
        else:
            user_upload_folder = os.path.join(app.config['UPLOAD_FOLDER'], session['user_folder'])

The problem with the code is that when I went and ran waitress and tested the code, I saw in the designated folder that the multiple files uploaded were separated into different folders when I am only expecting one unique folder per each instance. Does it help that I also checked the root of [‘SESSION_FILE_DIR’] and I noticed there were two different files created. Any solutions would be great

EDIT

Code I used to run the flask program

if __name__ == '__main__':
    #serve(app, host='127.0.0.1', port=8080)
    app.run(debug=True, use_reloader=False)

I used waitress serve() to run the program but for debugging purposes I use the app.run().

EDIT 2

Here is my updated code when I attempted to implement redis along with waitress

app = Flask(__name__)
app.secret_key = secrets.token_hex(16)
app.config['UPLOAD_FOLDER'] = 'uploads'
app.config['SESSION_TYPE'] = 'redis'
redis_client = redis.Redis()
app.config['SESSION_REDIS'] = redis.from_url('redis://localhost:6379')
app.config['SESSION_COOKIE_SECURE'] = True  

Session(app)


def upload():
    pdf_files = request.files.getlist('pdf_files')
    if pdf_files:
        if not redis_client.exists('user_folder:' + session.sid):
            user_folder = str(uuid.uuid4())
            user_upload_folder = os.path.join(app.config['UPLOAD_FOLDER'], user_folder)
            D_user_upload_folder = user_upload_folder
            os.makedirs(user_upload_folder, exist_ok=True)
            # Store the user folder in Redis
            test = redis_client.set('user_folder:' + session.sid, user_upload_folder)
            print("Filename: ", test)  
        else:
            user_upload_folder = redis_client.get('user_folder:' + session.sid)
            D_user_upload_folder = user_upload_folder.decode()
            #user_upload_folder = os.path.join(app.config['UPLOAD_FOLDER'], user_folder)

        for pdf_file in pdf_files:
            filename = secure_filename(pdf_file.headers['Content-Disposition'].split('filename=')[1].strip('"'))
            file_path = os.path.join(D_user_upload_folder, filename)
            pdf_file.save(file_path)
        #Expermienting checking maybe the subprocesses were uploading the files simultaneously  
        if len(pdf_files) == 1:
            return jsonify({'message': 'File uploaded successfully.'}), 200
        
        else:
            # Call the upload function again for the remaining files
            remaining_files = pdf_files[1:]
            with app.test_client() as client:
                client.post('/upload', data={'pdf_files': remaining_files})

The code seemed to work fine on the local machine but when I tested it on a remote computer. Multiple folders were created again per each file

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    So, I have already managed to solve my problem.

    Redis is part of the solution as it allows the data I have to be synchronized when used in other computers.

    But it does not stop the problem of waitress making multiple calls in one function which is the root cause and reason why every time I upload a file multiple folders are created.

    To solve this problem I first made sure that the creation of the folder is called during the creation of the main html page (In my case it is called "index()")

    @app.route('/')
def index():
    global current_folder
    create_user_folder()
    return render_template('index.html', filenames=[])

    I created a global variable just to save me the hassle of passing parameters of the current name of the folder.

    Since waitress creates multiple workers with each of their own unique identities. I used a thread-local variable to check if the thread has already executed the function.

    import threading
# Create a thread-local variable to store whether the function has been executed for the current user within the thread
thread_local = threading.local() 

def create_user_folder():
    global current_folder

    # Check if the function has already been executed for the current user in the current thread
    if hasattr(thread_local, 'user_folder'):
        session['user_folder'] = thread_local.user_folder
        return
    '''
    More code that checks if there is user_id and folder already created 

    if not create the folder and assign them to their respected variables.
    '''

    This allowed me, to check if the function has already been executed and along with making the execution of this function at the initial creation of the main page, it has allowed me to prevent the multiple workers from creating multiple folders when called.


  2. 0

    The problem lies in the fact that Waitress, the WSGI server you’re using, can spawn multiple worker processes to handle incoming requests. Each worker process has its own memory space and session storage, which can result in multiple folders being created for the same session.
    Try using redis:

    import redis
....
app.config['SESSION_TYPE'] = 'redis'
app.config['SESSION_REDIS'] = redis.from_url('redis://localhost:6379')
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search