Recently we have started the process of moving our production environment to AWS.
To achieve this aim there are a number of services that can be utilised, these include VPNs, AWS DirectConnect and Snowball.
We chose to do an initial dataload using a AWS snowball device.
For those that don’t know about snowball its a storage that AWS ship to you that can be plugged into your network, and then data can be copied to it. Note: the data copied onto the device is encrypted, the encryption is done by a client you have to download, its not encrypted by the device itself.
The device can then be shipped back to AWS where they will load the data onto S3 for you.
To start the process you login to the AWS console and create a job within the Snowball area.
Once done, AWS ship you the device.
The device is a lot larger than I was expecting and actually proved problematic fitting into a normal 19 inch rack. Luckily we had space within a rack to stand up the device. Apart from this the device is cleverly designed with two flaps on the front and back that need to be raised to access the front and rear panels.
The back flap has the powerscord and network cables carefully wrapped around a specially designed storage location. Although the unit shipped with a US power cable!!. Anyway plugged in the power and the network cable into the ports on the rear panel. (there are ports for copper and fibre spf+).
There is a power button on the front above the LCD to power on the device. Once on the device initially is set to DHCP, but the network can be configured manually from the front panel. I manually set the network parameters, and ensured I could ping the device from a server on the network.
Now the device does not just present network storage, to access the device you have to use a special client tool that needs to be downloaded from aws. This at least on the windows system I was using was a java tool.
In addition to the tool, you will need an access code and a manifest file from AWS. You will use these two and the snowball client to access the device.
Once you have the client downloaded and installed on the windows device. I found the easiest way was to go to the folder that contains the snowball.bat file that we will use. Below I list out the commands I used to connect, test and copy to the device with an explanation of each.
.\snowball.bat start –i <snowball ip> -m "E:\Snowball\jobName\xxnxnxnnxnnxnxnxn_manifest.bin" -u xxxxx-nnnnn-bbbbb-ccccc-ggggg
The command above extracts the manfiest and sets up a secure channel to talk to the device from the server. Once this is in place, we can then use additional command
.\snowball ls
This will list the folders on the device, you should find that the device has a folder with the same name as the job you created in the aws console.
s3://jobname
Now I created a subfolder
.\snowball.bat mkdir s3://jobname/subfolder
Now you can run a test, before you waste time on a full run
.\snowball test -r -t 5 “\\sourceserver\folder"
Then to copy the files to the device, this will create version3 within the subfolder on the snowball device.
.\snowball cp --recursive \\sourceserver\version3 s3://jobname/subfolder
Snowball fight – Part 2 (multiple parallel copies)
Ref: https://docs.aws.amazon.com/AWSImportExport/latest/ug/using-client-commands.html
No comments:
Post a Comment