Question 183349 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – Where are files located in Sagemaker Studio?

I have seen answers to this question and the answer is always that these files are located in

/home/ec2-user/Sagemaker/

however when I’m in Terminal in Sagemaker Studio I don’t see this. It seems that Sagemaker Studio has a different directory than whichever Sagemaker service other askers are using. This is what my command line looks like when I do pwd in the directory:

sagemaker-user@studio$ pwd
/home/sagemaker-user/sagepipeline_nonomethod/source_dir
sagemaker-user@studio$

When I do

os.chdir('/home/sagemaker-user/sagepipeline_nonomethod/source_dir')

inside my jupyter notebook however I get an error that says there is no such directory. So there must be some prefix to the directory that I am missing…

Tags:

2

Answers


  1. 0

    This occurs because your files and notebooks in SageMaker Studio exist separately to the machine running the kernel to execute your code.

    SageMaker Studio is, in essence, just a theme for Jupyter with a few added features. In the below diagram, you can see that your Studio interface and files are hosted by the "JupyterServer App", while the instances that execute your code are separate machines running containers.

    The Amazon SageMaker Studio Notebooks documentation provides a helpful overview of this architecture. See the diagram below.

    Your question appears to be specific to an issue you’re having, however. If you create a cell in your notebook and execute !pwd and !ls, you should find that the files available match those in the SageMaker Studio sidebar.

    Login or Signup to reply.
  2. 0

    In short, you should be able to access the directory at /root/sagepipeline_nonomethod/source_dir within your notebook. Explaination follows:

    If you open a terminal in the Launcher under Utilities and files, you will arrive at a system terminal. It’s the compute environment living in SageMaker Studio, instead of the container where your notebook (kernel) lives.

    The /home/sagemaker-user directory is a mountpoint of an EFS, where your Jupyter file browser points to.

    sagemaker-user@studio$ df -h
Filesystem         Size  Used Avail Use% Mounted on
overlay             32G  5.3M   32G   1% /
tmpfs               64M     0   64M   0% /dev
tmpfs              1.9G     0  1.9G   0% /sys/fs/cgroup
shm                395M     0  395M   0% /dev/shm
/dev/nvme0n1p1     160G   26G  135G  16% /opt/.sagemakerinternal
127.0.0.1:/200005  8.0E   40M  8.0E   1% /home/sagemaker-user
devtmpfs           1.9G     0  1.9G   0% /dev/tty
tmpfs              1.9G     0  1.9G   0% /proc/acpi
tmpfs              1.9G     0  1.9G   0% /sys/firmware

    To access the terminal of the container environment, go to Launcher > Notebooks and compute resources > Open image terminal

    The same EFS is mounted on /root

    root@datascience-1-0-ml-t3-medium-1abf3407f667f989be9d86559395:~# df -h
Filesystem         Size  Used Avail Use% Mounted on
overlay             32G   40K   32G   1% /
tmpfs               64M     0   64M   0% /dev
tmpfs              1.9G     0  1.9G   0% /sys/fs/cgroup
shm                395M     0  395M   0% /dev/shm
127.0.0.1:/200005  8.0E   40M  8.0E   1% /root
/dev/nvme0n1p1     160G   24G  137G  15% /opt/.sagemakerinternal
devtmpfs           1.9G     0  1.9G   0% /dev/tty
tmpfs              1.9G     0  1.9G   0% /proc/acpi
tmpfs              1.9G     0  1.9G   0% /sys/firmware
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search