Question 53892 in Javascript Question posted in
A very good W3school tutorial can be found here.

Select and read a file from user's filesystem – Javascript

I need to:

  • Enable the user to select a plain text file from the user’s filesystem.
  • Make it available to Pyodide.
  • Read its contents line-by-line in Pyodide (in the Python code).

The code to read currently replaced by dummy code: inp_str = 'ACGTACGT'. The actual use case involves complicated processing of the text file, but the minimal working example simply converts the input to lowercase.

<!doctype html>
<html>
  <head>
      <script src="https://cdn.jsdelivr.net/pyodide/v0.22.1/full/pyodide.js"></script>
  </head>
  <body>
    Analyze input <br>
    <script type="text/javascript">
      async function main(){
          let pyodide = await loadPyodide();
          let txt = pyodide.runPython(`
    # Need to replace the line below with code for the user to select
    # the file from the user's filesystem and read 
    # its contents line-by-line:
    inp_str = 'ACGTACGT'

    out_str = inp_str.lower()
    with open('/out.txt', 'w') as fh:
        print(out_str, file=fh)
    with open('/out.txt', 'rt') as fh:
        out = fh.read()
    out
`);

          const blob = new Blob([txt], {type : 'application/text'});
          let url = window.URL.createObjectURL(blob);
          
          var downloadLink = document.createElement("a");
          downloadLink.href = url;
          downloadLink.text = "Download output";
          downloadLink.download = "out.txt";
          document.body.appendChild(downloadLink);
      }
      main();
    </script>
  </body>
</html>

We have external users that may not be advanced computer users. We can specify they need to use Google Chrome browser, but not specific releases like Chrome Canary. We cannot ask them to manually enable the File System API.

Based on the suggestion by TachyonicBytes, I tried the code below. Now I got the error below. I also cannot see something like select file button, or any obvious code for it:

Uncaught (in promise) DOMException: Failed to execute 'showDirectoryPicker' on 'Window': Must be handling a user gesture to show a file picker.
    at main (file:///Users/foo/bar/upload_nativefs.html:10:35)
    at file:///Users/foo/bar/upload_nativefs.html:26:7

This is line 10 referred to in the error message:

const dirHandle = await showDirectoryPicker();

And the full page is pasted below:

<!doctype html>
<html>
  <head>
      <script src="https://cdn.jsdelivr.net/pyodide/v0.22.1/full/pyodide.js"></script>
  </head>
  <body>
    Analyze input <br>
    <script type="text/javascript">
      async function main(){
          const dirHandle = await showDirectoryPicker();
          if ((await dirHandle.queryPermission({ mode: "readwrite" })) !== "granted") {
              if (
                  (await dirHandle.requestPermission({ mode: "readwrite" })) !== "granted"
              ) {
                  throw Error("Unable to read and write directory");
              }
          }    
          let pyodide = await loadPyodide();
          const nativefs = await pyodide.mountNativeFS("/mount_dir", dirHandle);
          
          pyodide.runPython(`
  import os
  print(os.listdir('/mount_dir'))
`);
      }
      main();
    </script>
  </body>
</html>
Tags:

3

Answers


  1. 0

    The main documentation page for pyodide and the filesystem is here.

    Assuming you are using Chrome, which has the File System Access API implemented, it seems that you can mount the filesystem with this code:

    const dirHandle = await showDirectoryPicker();

if ((await dirHandle.queryPermission({ mode: "readwrite" })) !== "granted") {
  if (
    (await dirHandle.requestPermission({ mode: "readwrite" })) !== "granted"
  ) {
    throw Error("Unable to read and write directory");
  }
}

const nativefs = await pyodide.mountNativeFS("/mount_dir", dirHandle);

pyodide.runPython(`
  import os
  print(os.listdir('/mount_dir'))
`);

    Afterwards, read your file and get the input string as you normally would:

    with open("file") as f:
    f.read()

    Now, in order to be sure you have maximum chances of this working, get the Chrome Canary, and manually enable the File System API: navigate to chrome://flags/#native-file-system-api and change the value to Enabled.

    If you are not doing this for yourself, unfortunately, not all browsers can access the native filesystem, for security reasons, so you may have to go with an alternative solution, like IDBFS.

    Login or Signup to reply.
  2. 0

    I know you are looking for a pure pyodide solution but I would like to suggest you to have a look into pyscript solution which provides very minimal solution as per your requirement and loads the file in the virtual file system which is the file system that runs inside the browser.

    Pyscript supports the Python Standard Library File and Directory APIs. These APIs access storage within the Virtual File System. The virtual file system is provided by Emscripten File System API.

    Live Demo :

    <html>
  <head>
    <title>Read File</title>
    <meta charset="utf-8">
    <link rel="stylesheet" href="https://pyscript.net/latest/pyscript.css" />
    <script defer src="https://pyscript.net/latest/pyscript.js"></script>
  </head>
  <body>
    <label for="myfile">Select a file:</label>
    
    <input type="file" id="fileinput" name="fileInput"/>
    <br />
    <br />
    <div id="print_output"></div>
    <br />
    <p>File Content:</p>
    <div id="content">
    </div>

    <py-script output="print_output">
      import asyncio  
      import pyodide
      import js
      
      async def get_file(e):
        files = e.target.files.to_py()
        for file in files:
          file_content = await file.text()
          js.document.getElementById("content").innerHTML = file_content

      def main():
        get_file_proxy = pyodide.ffi.create_proxy(get_file)
        js.document.getElementById("fileinput").addEventListener("change", get_file_proxy)

      main()
    </py-script>
  </body>
</html>

    But if you still want to make it work in a pure pyodide way, You can modify it as per the documentation mentioned here.

    Login or Signup to reply.
  3. 0

    So, I am adding another answer, because I think I fully solved it.

    You error comes from the fact that accessing the filesystem with await showDirectoryPicker(); has to be invoked by the user (cannot be invoked directly). So, the solution is to use a user event (such as clicking a button) for the user to trigger the function. I added the full solution here.

    Basically, I replaced the main call with the button.

    <!doctype html>
<html>
  <head>
      <script src="https://cdn.jsdelivr.net/pyodide/v0.22.1/full/pyodide.js"></script>
  </head>
  <body>
    <button>Analyze input</button>
    <script type="text/javascript">
      async function main(){
          const dirHandle = await showDirectoryPicker();
          if ((await dirHandle.queryPermission({ mode: "readwrite" })) !== "granted") {
              if (
                  (await dirHandle.requestPermission({ mode: "readwrite" })) !== "granted"
              ) {
                  throw Error("Unable to read and write directory");
              }
          }
          let pyodide = await loadPyodide();
          const nativefs = await pyodide.mountNativeFS("/mount_dir", dirHandle);

          pyodide.runPython(`
            import os
            print(os.listdir('/mount_dir'))
          `);
      }

      const button = document.querySelector('button');
      button.addEventListener('click', main);
    </script>
  </body>
</html>

    On a fairly recent Chromium, after you click the button, you will get the system file picker in order to select a directory to mount. Afterward, you will have some pop-ups explaining what is happening. Something like this:

    enter image description here

    After you accept, the end result will look like this:

    enter image description here

    , meaning your browser has access to the filesystem, and you can use the other python functions to work with it.

    Edit:

    Now that I understand the problem a little better, I made the code get the file selection picker. Unfortunately, it seems that the filesystem api makes it hard to get the directory from the file, so there is still a call to the directory picker. I made did some error handling in case the file is not in the selected directory, but for now, the ergonomics are lacking in that respect.

    The javascript looks like this:

          // Check if handle exists inside directory our directory handle
      async function inDirectory(directoryHandle, fileHandle) {
        return await directoryHandle.resolve(fileHandle) === null;
      }

      async function main() {
        const dirHandle = await showDirectoryPicker();
        const [fileHandle] = await showOpenFilePicker();
        const bad = await inDirectory(dirHandle, fileHandle);
        if (bad) {
          throw Error("The file is not in the mounted directory");
          return;
        }

        const fileData = await fileHandle.getFile();
        console.log(fileData.name);
          if ((await fileHandle.queryPermission({ mode: "readwrite" })) !== "granted") {
              if (
                  (await fileHandle.requestPermission({ mode: "readwrite" })) !== "granted"
              ) {
                  throw Error("Unable to read and write directory");
              }
          }
          let pyodide = await loadPyodide();
          const nativefs = await pyodide.mountNativeFS("/mount_dir", dirHandle);

          pyodide.runPython(`
            import os
            print(os.listdir('/mount_dir'))
            print(open('/mount_dir/${fileData.name}').read())
          `);
      }

      const button = document.querySelector('button');
      button.addEventListener('click', main);

    I also updated the repo with it. If this is still lacking in some respect, please detail the question even further.

    Edit 2:

    Well, the code ended up being much simpler. I did away with the directory entirely, now it asks you to pick a file, it reads the contents into JS memory, it constructs the Python conversion function and then it just calls the function and puts the result in Blob.

    The code to construct the download link appears after the Analyze button has been pressed.

    This is the code, as usually, updated in the repo as well:

          async function main() {
        // Get the file contents into JS
        const [fileHandle] = await showOpenFilePicker();
        const fileData = await fileHandle.getFile();
        const contents = await fileData.text();

        // Create the Python convert toy function
        let pyodide = await loadPyodide();
        let convert = pyodide.runPython(`
from pyodide.ffi import to_js
def convert(contents):
    return to_js(contents.lower())
convert
      `);

        let result = convert(contents);
        console.log(result);

        const blob = new Blob([result], {type : 'application/text'});

        let url = window.URL.createObjectURL(blob);

        var downloadLink = document.createElement("a");
        downloadLink.href = url;
        downloadLink.text = "Download output";
        downloadLink.download = "out.txt";
        document.body.appendChild(downloadLink);

      }
      const button = document.querySelector('button');
      button.addEventListener('click', main);

    The code ended up much, much cleaner.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search