Bash tool for MMS Backup

 
The MongoDB Management Service (MMS) is an essential part of any production MongoDB deployment, providing three key functionalities: monitoring, backups, and automation. It is available as a Cloud service or On-Premise. In addition to the powerful web interface, REST APIs are available for programmatic access, scripting, and integration with other systems.

A common use case is to trigger (manually or via cron) the restoration of the latest snapshot from MMS Backup, download the resulting tarballs and store them somewhere for safe keeping (e.g. on tape or to remote datacenter). Sometimes these are kept locally (when using MMS Cloud) for faster restores, or used to automatically reload a test/staging environment with actual production data, scrubbing sensitive information if necessary.

When asked how exactly, we typically refer users to the snapshots and restoreJobs APIs. However, implementing this on your own in a robust manner is non-trivial, especially for users without programming background or the time to do so as it involves parsing the JSON returned via the API. Fortunately, there are community tools like mms-api, a command line tool for the MMS API written in Ruby.

Sadly, installing 3rd party tools that involve external dependencies like Ruby or Python libraries can be challenging or even impossible in some Enterprise environments. Thus a pure Shell/Bash only tool is needed and I’m happy to say that we now have this in the form of mrestore.

Using it to download the most recent snapshot is a simple one-liner like the following:

$ ./mrestore.sh --server-url https://mms.mongodb.com \
                --user admin@localhost.com \
                --api-key 9d2fb094-108a-4c63-9ce6-5f79bbd8bd50 \
                --group-id 54c64146ae9fbe3d7f32c726 \
                --cluster-id 54c641560cf294969781b5c3

Which produces output like the following:

Cluster type    : REPLICA_SET
Replica set name: mms-app

Latest snapshot ID: 54cc87420cf25152251c0353
Created on        : 2015-01-31T07:40:23Z
Complete?         : true

MongoDB version   : 2.6.7
Data size         : 1.4 MB
Storage size      : 4.41 GB
File size         : 7.49 GB (uncompressed)

Restore job ID: 54ccbe6e0cf2d19b280496b2
Waiting for restore job....
Status: FINISHED

Downloading restore tarball(s) to ./...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 12.3M    0 12.3M    0     0   400k      0 --:--:--  0:00:31 --:--:--  526k
Wrote to './54cc85f3504d16cb0bf4085d-mms-app-1422690023.tar.gz' (12.3 MB)

Sharded clusters are also supported and produce output like the following (one tarball per shard + one for config server):

Cluster type: SHARDED_REPLICA_SET
Cluster name: Cluster 3

Latest snapshot ID: 54cc9bc70cf2c0b4053c592b
Created on        : 2015-01-31T09:08:56Z
Complete?         : true

Part              : 0
Type name         : REPLICA_SET
Replica set name  : shard01
MongoDB version   : 2.6.7
Data size         : 109.2 MB
Storage size      : 126.1 MB
File size         : 256.0 MB (uncompressed)

Part              : 1
Type name         : REPLICA_SET
Replica set name  : shard02
MongoDB version   : 2.6.7
Data size         : 64.0 MB
Storage size      : 92.3 MB
File size         : 256.0 MB (uncompressed)

Part              : 2
Type name         : CONFIG_SERVER
MongoDB version   : 2.6.7
Data size         : 20 KB
Storage size      : 20.1 MB
File size         : 128.0 MB (uncompressed)

Batch ID: 54ccbed20cf2d19b280496cc
Waiting for restore job....
Status: FINISHED

Downloading restore tarball(s) to ./...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  130k    0  130k    0     0   208k      0 --:--:-- --:--:-- --:--:--  208k
Wrote to './54cc85f3504d16cb0bf4085d-config-4f2e12eeb9c155a8c7118d78115134c2-1422695336.tar.gz' (132 KB)

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 13.1M    0 13.1M    0     0  3416k      0 --:--:--  0:00:03 --:--:-- 3416k
Wrote to './54cc85f3504d16cb0bf4085d-shard02-1422695336.tar.gz' (13.1 MB)

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.9M    0 21.9M    0     0  3338k      0 --:--:--  0:00:06 --:--:-- 3795k
Wrote to './54cc85f3504d16cb0bf4085d-shard01-1422695336.tar.gz' (21.9 MB)

For details, refer to the README. Note that this an unofficial tool that I’m maintaining in my spare time. Do try it out and comment here if it works for you (or not)!