Avere FXT Container / Bucket setup - Best Practice

Doug Soltesz -

When adding a Container / Bucket to the SwiftStack object storage system for use as an Avere filesystem, Container Sharding middleware should be enabled.  Container Sharding middleware is a SwiftStack enhancement to the S3 API allowing for billions of objects to be stored in a virtual container.  Container Sharding is only available when used with systems such as Avere that utilized the Amazon S3 best practices for naming of objects.  

Further information on Amazon S3 best practices for key naming can be found here.
 http://docs.aws.amazon.com/AmazonS3/latest/dev/request-rate-perf-considerations.html

 

Container Sharding must be enabled on a new Account/User that does not have existing data in it.  In addition this account should only be used for approved systems, use of Container Sharding with unapproved applications may result in 404s, misplaced objects and other undesirable actions.

 

To Enable Container Sharding:

First Logon to  the SwiftStack Controller console and select Manage in the desired cluster. 

ManageCluster.jpg

 

 

Next select Middleware on the left panel.

avere6.png



Click on Specialized Auto-Sharding to enable this feature.  (Please note that in order to run Avere Swift3 middleware should also be enabled.)
MiddlewareShard.jpg




Click the Checkbox for Enabled and click the Submit Button.
avere1.png




Next a new account must be created.  Click on Users & Accounts in the top right corner.
avere2.png




Click on Create New User
avere3.png



Create a Username and assign a password.  Click the Checkbox for Auto-sharding Enabled.  Then click Submit.  Multiple Users can be created and deployed at the same time.
avere4.png



Push the new user to the Cluster.
avere5.png



Once the Deploy User Job has finished successfully, click Manage in the top right.
UserDeploy.jpg



In the Deploy tab, Select the Deploy Config to Swift Nodes:
DeployConfig.jpg



Once the deploy is complete, the User/Account is ready to be used.
DeployComplete.jpg




Please note that if additional accounts are created needing Auto-Sharding in the future, both the User and Swift Nodes deployment must be pushed again.  To redeploy the Swift Nodes config, return to the Deploy Tab.  Although no changes are noted, you must click the Deploy Config to Swift Nodes after each new user needing Auto-Sharding is created.
Inline image 8




To create a Bucket for use by Avere log into the SwiftStack console with the Username & Password combination created above.  The SwiftStack console can be found at http(s)://your.swiftstack.cluster/console.  Click the Plus Sign + below Containers to add a new Bucket/Container.
WebConsole.jpg



Give the Bucket a name and choose the Storage Policy.  It is not currently recommended to use Erasure Coded Storage Policies with Avere FXT clusters.  Erasure Coding performance for small objects is half the performance of Replica's and such should not be used for high performance driven applications.
BucketName.jpg



Once Data has been written to the Container/Bucket, multiple sharded containers will be automatically created.  These will use the same Storage Policy as the base Container.  Please note that the base container will not show any data, and the actually objects for Avere FXT will be in the sharded containers.  Data should not be written to any of the sharded containers.  These are for use only by SwiftStack Middleware.  In the example below, shardertest is the base container, and all of the sub-containers are the Auto-Sharder Middleware.  Clicking on these you will see the actual objects.
ContainerwData.jpg
Multiple Containers can be created in the Auto-Sharder enabled account.  Each Container/Bucket will be mapped to a single Filesystem in the Avere product.
Have more questions? Submit a request

Comments

Powered by Zendesk