How to mount AWS S3 Bucket on AWS EC2 Instances

Please share it on

Few days before I was trying to mount AWS S3 buckets on EC2 Instances which is running amazon linux operating system (RPM based linux). This quick and dirty guide explains you how to mount S3 buckets on EC2 instances.

Login into your AWS Console and go to S3

1. Create a S3 bucket.

2. Under Security Credentials, create AWS Access key ans Security key

AWS Access key and security key

Now create a file /etc/passwd-s3fs which is readable by only root user and it’s group.

#vim /etc/passwd-s3fs

AccessKeyId:SecretAccessKey

replace AccessKeyId and SecretAccessKey with your id, then save and quit the file.

In case, if you are trying to mount S3 buckets of different AWS users or buckets having different AccessKeyId and SecretAccessKey Id, then include bucket name in the file.

my-s3-bucket:AccessKeyId:SecretAccessKey

#chmod 400 /etc/passwd-s3fs
#chown root:root /etc/passwd-s3fs
#mkdir /mnt/s3buckets

3. S3FS – Here I’m using S3FS, a third party package to mount s3 buckets. Before installing it, you need to satisfy the following prerequisites.

Install the following dependencies

yum install gcc libstdc++-devel gcc-c++ curl-devel libxml2-devel openssl-devel mailcap

Install Fuse from source code before that remove fuse and its development packages if any from your server

#yum remove fuse fuse-devel

# cd /usr/local/src

# wget http://sourceforge.net/projects/fuse/files/fuse-2.X/2.9.1/fuse-2.9.1.tar.gz

# tar -xzvf fuse-2.9.1.tar.gz

# cd fuse-2.9.1

#./configure –prefix=/usr
#make
#make install
#ldconfig
# export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig/

# pkg-config –modversion fuse
2.9.1

# ldconfig -v | grep fuse
libfuse.so.2 -> libfuse.so.2.9.1

Now compile and install S3FS

# cd /usr/local/src

#wget http://s3fs.googlecode.com/files/s3fs-1.73.tar.gz

#tar -xzvf s3fs-1.73.tar.gz

#cd s3fs-1.73
#./configure –prefix=/usr
#make
#make install
#ldconfig

The package S3FS is installed successfully.

[root@w3bootstrap s3fs-1.73]# s3fs –version
Amazon Simple Storage Service File System 1.73
Copyright (C) 2010 Randy Rizun <rrizun@gmail.com>
License GPL2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Now mount the bucket under the /mnt/s3buckets directory by using the following command

#s3fs w3bootstrap-bucket /mnt/s3buckets -o allow_other -o use_cache=/tmp

Check the mounted partitions

#mount

If you want to mount the S3 bucket permanently, add the following entry in the /etc/fstab

s3fs#w3bootstrap-bucket /mnt/s3buckets fuse allow_other,use_cache=tmp 0 0

where

w3bootstrap-bucket – is the S3 bucket name

/mnt/s3buckets  – is the directory where we mount the S3 bucket

fuse – File System

allow_other – It allows other users to access the file

use_cache – Local folder to use for local file cache

Now check the disk free space, you can see the s3 bucket entries

#df -h

That’s all! Happy Computing!

Please share it on

Leave a Reply

Your email address will not be published. Required fields are marked *