USGS – Storage Gateway

Executive summary

USGS wanted to explore patterns of use for AWS Storage Gateway and provide turn-key solutions to its customers for remote site back up, infrequently and frequently accessed file storage and eventually for Virtual Tape Libraries.

Enquizit provided configuration information along with runbooks to integrate several popular backup packages. Configuration details were also provided for deploying one or multiple AWS storage gateways on VMware and configuring Windows Servers to use the storage space on the gateways.

Business Challenge / Problem Detail

The USGS Cloud Hosting Solutions (CHS) Virtual Data Center (VDC) provides a federally approved secure environment for data hosting, high-performance computer modeling, application research and development, and a testing platform where project owners can leverage centralized services. The Cloud Team was in the process of implementing secure geographic survival for critical data produced and owned by its users spread across several cost centers and geographical locations. Given the diverse needs and technical setup for users, it was natural that the solution provided was as simple to use and as flexible as possible, while providing seamless and excellent survival capabilities. USGS wanted to explore patterns of use for AWS Storage Gateway and provide turn-key solutions to its customers for remote site back up, infrequently and frequently accessed file storage and eventually for Virtual Tape Libraries.

Output / Our Solution

During the requirements analysis phase, Enquizit’s storage team finalized all parameters for the deployment of AWS Storage Gateway/Offsite Backup Solution. Success metrics for the POC were discussed and agreed upon. Our Architects conducted a feasibility study on leading products such as Commvault, HP Data Protector, Zmanda Enterprise and SoftNAS, based on the needs of our customer. Our recommendations along with a basic implementation plan were discussed and finalized during several collaborative session with the larger USGS technical team. Enquizit provided configuration information along with runbooks to integrate several popular backup packages. Configuration details were also provided for deploying one or multiple AWS storage gateways on VMware and configuring Windows Servers to use the storage space on the gateways. Enquizit provided a simple, flexible, and accessible solution without compromising infrastructure or security of USGS’s data.

Special attention was paid to USGS’s extremely remote site which have very slow and often unreliable Network connections to the internet and USGS core Data Centers. They still can produce large amounts of data as users work. That data has to be asynchronously backed up without impacting the user work flow and productivity.

Enquizit Role

Enquizit’s Cloud Storage team conducted the following activities to implement UDGS’ Storage Gateway:

Gather business and Technical requirements

Gather information about the current technical state

Design target state architecture on AWS

Analyze current backup setup

Document desired end point in terms of

Setup AWS security (as necessary

Setup S3 Buckets

Setup AWS Storage gateway

Setup virtual servers (as necessary)

Configure Alerts and Notifications

Document process to connect backup software to connect to Storage Gateway and S3

Assist with Agent deployment to all desired hosts, as time permits

Setup verification and Testing

Tooling (Tools and Technologies)

EC2, S3, EBS, Glacier, VPC, CloudWatch, Storage Gateway, CommVault

Expected Challenges

Official AWS support for the third party products

Slow and very gradual consensus building

'Slow' links to remote turned out much slower than initially estimated by USGS, complicating replication further

Client:

US. Geological Survey

Industry:

Government
Problem Statement

USGS wanted to explore patterns of use for AWS Storage Gateway and provide turn-key solutions to its customers for remote site back up, infrequently and frequently accessed file storage and eventually for Virtual Tape Libraries.

Goals and Benefits

To provide the USGS team with a turnkey Storage/Back Up solution that will enable project owners throughout the USGS the ability to store and back up their current environments into the cloud.

The project must also demonstrate the ability for the USGS to lower their storage and disaster recovery costs, alongside the ability to expand back up capacity inexpensively. In addition to those:

A working PoC backup setup utilizing AWS Storage Gateway

Configured representative instances of Windows Server to serve Windows, MacOS and Linux clients

Successful test backup and restores using the new Setup

Core Partners:

Amazon Web Services (AWS)

READ MORE

Learn more about our work and we think about key issues in different industries.