How to Upload Pictures from a PhoneGap App to Amazon S3

In my previous post, I shared a sample application demonstrating how to upload pictures from a PhoneGap application to a Node.js server.

If your application deals with lots of images, it may be a good idea to host them on a dedicated storage infrastructure like Amazon S3, and let your own server focus on delivering the dynamic data. With the high availability and scalability features of S3, your application can deliver these images fast and reliably.

In this post, I’ll share a sample application demonstrating how to upload images from a PhoneGap application directly to an S3 bucket.

Here is a quick video:

Unsecured Approach

Here is a first (and unsecured) version of the upload module. With this version, you can upload images from a PhoneGap application directly to an S3 bucket without any other server involved.

var s3Uploader = (function () {

    var s3URI = encodeURI("https://YOUR_S3_BUCKET.s3.amazonaws.com/"),
        policyBase64 = "YOUR_BASE64_ENCODED_POLICY_FILE",
        signature = "YOUR_BASE64_ENCODED_SIGNATURE",
        awsKey = 'YOUR_AWS_USER_KEY',
        acl = "public-read";

    function upload(imageURI, fileName) {

        var deferred = $.Deferred(),
            ft = new FileTransfer(),
            options = new FileUploadOptions();

        options.fileKey = "file";
        options.fileName = fileName;
        options.mimeType = "image/jpeg";
        options.chunkedMode = false;
        options.params = {
            "key": fileName,
            "AWSAccessKeyId": awsKey,
            "acl": acl,
            "policy": policyBase64,
            "signature": signature,
            "Content-Type": "image/jpeg"
        };

        ft.upload(imageURI, s3URI,
            function (e) {
                deferred.resolve(e);
            },
            function (e) {
                deferred.reject(e);
            }, options);

        return deferred.promise();

    }

    return {
        upload: upload
    }

}());

 
A FileTransfer object is used to POST the picture along with associated data (the fields in options.params) to the S3 server. The key to making this work is to post the exact fields and values expected by S3.

Here is a quick description of these fields:

  • AWSAccessKeyId: The access key identifier of the AWS user whose credentials are used to upload the picture. It is highly recommended that you don’t use your root AWS user for this. Instead, create a new AWS user with a set of permissions limited to uploading files to your bucket. (You can create new users and assign permission in the IAM console: https://console.aws.amazon.com/iam/home?#users).
  • acl: Access control policy to apply to the uploaded file. We want our application to be able to access these images, so we specify “public-read”.
  • policy: A base-64 encoded string representing the policy document (more on this in the next section).
  • signature: A base-64 encoded string representing the policy document signed with the secret key of the AWS user whose credentials are used to upload the picture.
  • Content-Type: the type of file uploaded.

The Policy Document

A policy document is a JSON object that defines a set of rules that govern the upload to S3.

There are two things you specify in the policy document:

  • expiration: A GMT timestamp. Any upload attempted after the policy document’s expiration will fail.
  • conditions: The set of rules that govern the upload. You can specify rules to constrain the file name, the file size, etc. (More information here).

For example, the policy document below expires on 12/31/2020 and defines the following rules:

  • The file must be uploaded to the phonegap-demo bucket
  • The file name can start with any combination of characters
  • The acl must be “public-read”
  • The file type content type can be anything
  • The maximum file size is: 512K
{ 
    "expiration": "2020-12-31T12:00:00.000Z",
    "conditions": [
        {"bucket": "phonegap-demo"},
        ["starts-with", "$key", ""],
        {"acl": 'public-read'},
        ["starts-with", "$Content-Type", ""],
        ["content-length-range", 0, 524288000]
    ]
};

Encoding and Signing

Remember that as part of the upload you must POST the base-64 encoded policy and signature.

Here is a quick Node.js app (signing-util.js) that outputs the base-64 policy document and signature to the console.

var crypto = require('crypto'),
    secret = "YOUR_AWS_USER_SECRET_KEY",
    policy,
    policyBase64,
    signature;

policy = {
    "expiration": "2020-12-31T12:00:00.000Z",
    "conditions": [
        {"bucket": "phonegap-demo"},
        ["starts-with", "$key", ""],
        {"acl": 'public-read'},
        ["starts-with", "$Content-Type", ""],
        ["content-length-range", 0, 524288000]
    ]
};

policyBase64 = new Buffer(JSON.stringify(policy), 'utf8').toString('base64');
console.log("Policy Base64:");
console.log(policyBase64);

signature = crypto.createHmac('sha1', secret).update(policyBase64).digest('base64');
console.log("Signature:");
console.log(signature);

Important Security Consideration

Encoding the policy and the signature with a distant expiration date, and then hardcoding these values in your client application works fine, however this approach presents a security risk. Malicious developers can easily obtain these values by either looking at your JavaScript code or by sniffing the POST fields on the network. With these values, they can essentially hijack your S3 bucket, and upload files outside your app.

Secured Approach with Server

A solution to this problem is to use your server to sign the policy document “on demand”. The workflow works as follows:

  1. The user selects a picture to upload
  2. The app makes a request to your server to generate a base-64 policy and signature. Because that request’s endpoint is on your server, you can secure it using your standard authentication and authorization mechanism.
  3. The app uses these values to upload the picture to S3.
The picture is still uploaded directly from the client application to S3. Your server is only used to sign keys on demand.

Securing the Policy Document

The advantage of having the server sign the policy document on-demand is that the policy document is specific to one upload and can embed some stricter security restrictions.

For example, you can set the policy document to expire just a few minutes after it was requested, leaving enough time for the app to perform the upload but making the policy document useless after that.

Optionally, you could also set a condition on the filename, and for example, restrict the upload to a file with a specific name.

In the code below, I turned my Node.js signing utility (signing-util.js) into a signing server (signing-server.js) that your app can access to generate the encoded policy and signature on demand. The policy document expires after five minutes, and only allows the upload of a file with a specific file name passed as a parameter to the signing request.

var express = require('express'),
    http = require('http'),
    path = require('path'),
    crypto = require('crypto'),
    app = express(),
    bucket = "YOUR_BUCKET_NAME",
    awsKey = "YOUR_AWS_USER_KEY",
    secret = "YOUR_AWS_USER_SECRET";

app.use(express.logger("dev"));
app.use(express.methodOverride());
app.use(express.bodyParser());
app.use(app.router);

function sign(req, res, next) {

    var fileName = req.body.fileName,
        expiration = new Date(new Date().getTime() + 1000 * 60 * 5).toISOString();

    var policy =
    { "expiration": expiration,
        "conditions": [
            {"bucket": bucket},
            {"key": fileName},
            {"acl": 'public-read'},
            ["starts-with", "$Content-Type", ""],
            ["content-length-range", 0, 524288000]
        ]};

    policyBase64 = new Buffer(JSON.stringify(policy), 'utf8').toString('base64');
    signature = crypto.createHmac('sha1', secret).update(policyBase64).digest('base64');
    res.json({bucket: bucket, awsKey: awsKey, policy: policyBase64, signature: signature});

}

// DON'T FORGET TO SECURE THIS ENDPOINT WITH APPROPRIATE AUTHENTICATION/AUTHORIZATION MECHANISM
app.post('/signing', sign);

app.listen(3000, function () {
    console.log('Server listening on port 3000');
});
Again the key for this solution to be safe is that you secure the /signing endpoint with your standard authentication and authorization mechanism.

And here is the updated client application:

var s3Uploader = (function () {

    var signingURI = "http://192.168.1.8:3000/signing";

    function upload(imageURI, fileName) {

        var deferred = $.Deferred(),
            ft = new FileTransfer(),
            options = new FileUploadOptions();

        options.fileKey = "file";
        options.fileName = fileName;
        options.mimeType = "image/jpeg";
        options.chunkedMode = false;

        $.ajax({url: signingURI, data: {"fileName": fileName}, dataType: "json", type: "POST"})
            .done(function (data) {
                options.params = {
                    "key": fileName,
                    "AWSAccessKeyId": data.awsKey,
                    "acl": "public-read",
                    "policy": data.policy,
                    "signature": data.signature,
                    "Content-Type": "image/jpeg"
                };

                ft.upload(imageURI, "https://" + data.bucket + ".s3.amazonaws.com/",
                    function (e) {
                        deferred.resolve(e);
                    },
                    function (e) {
                        alert("Upload failed");
                        deferred.reject(e);
                    }, options);

            })
            .fail(function (error) {
                console.log(JSON.stringify(error));
            });

        return deferred.promise();

    }

    return {
        upload: upload
    }

}());

Source Code

The complete secured and unsecured version of the application is available in this GitHub repository.

22 Responses to How to Upload Pictures from a PhoneGap App to Amazon S3

  1. App Development Company October 15, 2013 at 8:10 am #

    Thanks for sharing this wonderful information. I would like to see How we can get create an social networking sites with the help of PhoneGap. I think it include a lot of development with the help of PHP.

  2. Jason Levinsohn October 22, 2013 at 7:34 pm #

    Hi Christophe,

    Great Article. Did you run into a 405 Method Not Allowed Error at all. This is the one hurdle I can’t seem to get around. Amazon S3 Docs say you have to be the owner of the bucket to perform this operation. Even though I have a user set up with IAM, I tried changing the credentials back to root and still no dice. I wonder does it throw this error if you have something wrong with the policy? I was able to upload files successfully using javascript in a browser, so it must have something to do with PhoneGap.

    Thanks,

    Jason

  3. Kamol October 23, 2013 at 10:23 am #

    Jason,

    open your CORS Configuration Editor in your S3 Bucket and add these:

    *
    GET
    POST
    PUT
    3000
    *

    Regards,
    Kamol

  4. Kamol October 23, 2013 at 10:25 am #

    Hi Christophe,

    Thank you for your article.

    I am having another issue. I was able to upload to S3, but not open the file it gives:

    Comment Form is loading comments…

    /**/

    How did you setup your Bucket Policy?

  5. Jason Levinsohn October 23, 2013 at 9:20 pm #

    Here it is:
    {
    “expiration” : 1382578477,
    “conditions” : [
    { "bucket", "levsdelight" },
    { "acl", "public-read" },
    ["starts-with", "$key", ""],
    ["starts-with", "$Content-Type', ""],
    ["content-length-range", 0, 524288000]
    ]
    }

  6. Jason Levinsohn October 23, 2013 at 9:41 pm #

    Also I realized the expiration wasn’t ISO time like your and it wasn’t a string value, still have the same problem though.

  7. Jason Levinsohn October 23, 2013 at 9:46 pm #

    Kamol,

    Thanks. I think I already had that in my CORS config.

    *
    GET
    POST
    PUT
    3000
    *

  8. Kamol October 23, 2013 at 10:40 pm #

    Jason,

    Did you try to use http vs https?
    e.g.:
    var aws_url = encodeURI(“http://” + obj.bucket + “.s3.amazonaws.com/”);

  9. Jason Levinsohn October 24, 2013 at 11:17 am #

    Kamol,

    I have not tried that. I left my phone at home today :/ but I’ll give that a shot and let you know. I was also trying to upload the file into a sub-directory of img/ so I’ll also start out simple and just try to upload it to the main folder.

  10. Terren October 29, 2013 at 10:03 am #

    how did you make the image files so small (~100kb) that were uploaded to the s3 bucket?

  11. Nathan November 5, 2013 at 9:41 am #

    Would this work with large files such as videos?

  12. Jason Levinsohn November 7, 2013 at 1:52 pm #

    I finally got this working. I am using Python to sign my requests so it was a little different. Make sure that policy document is spot on. Let me know if you guys have any questions :) Nathan, I believe S3 allows files up to 5 TB at a time. http://aws.amazon.com/s3/faqs/#How_much_data_can_I_store

  13. BPARSONS January 8, 2014 at 1:26 pm #

    We’ve been able to use this code for a working upload on Android, but iOS doesn’t upload anything to the server. Any ideas?

  14. Derek January 14, 2014 at 3:27 pm #

    Pretty cool. How does your signing-server.js approach compare to using either IdentityTVM or AnonymousTVM hosted on your own server?

    http://aws.amazon.com/code/7351543942956566

  15. Lucas Lobosque January 30, 2014 at 6:25 pm #

    I’m getting an ABORT_ERR.
    Did you ever get this error? Do you have any idea on how to fix it?

  16. Chris Sells February 10, 2014 at 10:03 pm #

    I’ve got a server that produces URLs that for use in uploading a File object using HTTP PUT:

    https://52e5905a095ffb0e581f1e9a.s3.amazonaws.com/c367775f-b5e5-40fb-8f11-d…IT4X3L44WN4Q&Expires=1392190685&Signature=0F3KtCdJ%2BdKFcr0d0u7UPTcWFPM%3D

    How do I use the FileTransfer object with a PUT URL like this? Can I use FileTransfer to do a PUT? Can I use a normal XHR object to do a file upload inside a PhoneGap app?

  17. patchwork kilim March 24, 2014 at 5:13 am #

    Would this work with large

  18. Deepan May 1, 2014 at 12:24 pm #

    @ Christophe Coenraets

    Hi Sir, iam regularly checking tour posts, its adds + to me,

    i have problem on browsing and uploading image to php server

    am choosing by native.xxx.getPicture(), DestinationType as FILE_URI and SourceType as PHOTOLIBRAY,

    imageURI as like content:// (mediastore).. and i have used resolveFilesystemURI aswell but its returns not physical uri of choosen image.

    please guide me right solution , iam using cordova 3.3.0….

  19. Kas May 18, 2014 at 9:22 pm #

    Hi, what is YOUR_BASE64_ENCODED_POLICY_FILE ? is it the JS file? and also YOUR_BASE64_ENCODED_SIGNATURE is that the JS file on server side? or wht? thanks.

  20. Avinash July 8, 2014 at 10:02 am #

    Hi, I am using cordova 3.5 for ios and android. I am facing some issue while using “var crypto = require(‘crypto’)”. Do I need to add any library or something

    • Avinash July 8, 2014 at 10:06 am #

      Any help would be appreciated. Thanks

      • David July 30, 2014 at 4:16 pm #

        The require(”) code is all server side nodejs. You dont run it client side.

Leave a Reply