A client wanted to run their antivirus on an EC2 instance to scan their S3 buckets.
Logged into the AWS console saw there’s a large number of S3 buckets. You know what I’m not going to do? I’m not going to do this manually.
Let’s get started create an EC2 instance that has (or install) AWS CLI tools.
Now set up the EC2’s IAM role to access S3 resources. Select the EC2 instance you’ve created and goto Attach/Replace IAM Role.
Click Create new IAM role
Finally, go back to your EC2 instance and attach it.
Now, SSH into the EC2 instance and open up vim (if you’re cool), paste the code below and run it.
#!/bin/bash BucketCount=`aws s3api list-buckets --query "Buckets.Name" | grep '"' | cut -d'"' -f2 | wc -l` S3Buckets=`aws s3api list-buckets --query "Buckets.Name" | grep '"' | cut -d'"' -f2` if [ `ls -l /home/user/buckets/ | grep 'drw' | wc -l` != $BucketCount ]; then echo "missing buckets" for bucket in $S3Buckets; do if [[ ! -d "/home/user/buckets/$bucket" ]]; then echo "creating mount dir and mounting $bucket" mkdir "/home/user/buckets/$bucket" /bin/s3fs -o use_path_request_style -o url=http://s3.amazonaws.com "$bucket" "/root/buckets/$bucket" fi done fi
This script was designed to be ran in a cronjob, it will check for new buckets every time. If a new bucket is detected it will echo that it’s missing buckets, mkdir a mounting point for the new bucket, then use S3FS to mount it. The goal is to allow the AV to scan new buckets in perpetuity.