Use AWS S3 to Update Server Files

This post is about a nice trick that a colleague of mine had found.

We have a server running on a production data center, and so the access to this server is through several security layers. We run some POC test code on this server, and had to update the application binary several times. As this was a POC stage project, we had no CI/CD process to automate deployment of the binary, and we've had to upload the code manually. But uploading the code was extremely slow and complicated, due to the multiple security layers.

The solution we've finally used was to upload the application binary to a AWS S3 bucket, and use a python code to download it on the production server.

The first step is to create an AWS S3 bucket.
In his case I've create a bucket named binary-bucket.

Then, upload the binary to this bucket.

To retrieve the application binary on the production server, we' have use a short python script.
Notice that due to the security tiers, we could not directly access the AWS S3, so we've used a proxy instead.

import boto3
from botocore.config import Config

BUCKET_NAME = 'binary-bucket'
BINARY_FILE = 'app.bin'
PROXY = '10.1.1.1:443'

s3 = boto3.client('s3', config=Config(proxies={'https': PROXY}))
obj = s3.get_object(Bucket=BUCKET_NAME, Key=BINARY_FILE)
data = obj['Body'].read()
with open(BINARY_FILE, 'wb') as code:
    code.write(data)

The last thing to do, is to use an AWS access key for the AWS authentication.
This is done by creating of a configuration file inthe home folder: ~/.aws/config

[default]
aws_access_key_id=A12DEHAA72B4PAAATJRA
aws_secret_access_key=TelYU6M3Dfg/ssl3PWJAPSwXg/rJw9kD44Rdd7sq

and that's it. The binary is downloaded within seconds.

Update: Upload to AWS S3

After a week of work, I've found that another useful task is to retrieve files back from the production server. So I've added an upload script.

import boto3
from botocore.config import Config

BUCKET_NAME = 'binary-bucket'
UPLOAD_FILE = 'app.data'
PROXY = '10.1.1.1:443'

s3 = boto3.client('s3', config=Config(proxies={'https': PROXY}))
s3.upload_file(UPLOAD_FILE, BUCKET_NAME, UPLOAD_FILE)

and we're done.