When adding a Container / Bucket to the SwiftStack object storage system for use as an Avere filesystem, Container Sharding middleware should be enabled. Container Sharding middleware is a SwiftStack enhancement to the S3 API allowing for billions of objects to be stored in a virtual container. Container Sharding is only available when used with systems such as Avere that utilized the Amazon S3 best practices for naming of objects.
Further information on Amazon S3 best practices for key naming can be found here.
Container Sharding must be enabled on a new Account/User that does not have existing data in it. In addition this account should only be used for approved systems, use of Container Sharding with unapproved applications may result in 404s, misplaced objects and other undesirable actions.
To Enable Container Sharding:
First Logon to the SwiftStack Controller console and select Manage in the desired cluster.
Next select Middleware on the left panel.
Click the Checkbox for Enabled and click the Submit Button.
Next a new account must be created. Click on Users & Accounts in the top right corner.
Click on Create New User
Create a Username and assign a password. Click the Checkbox for Auto-sharding Enabled. Then click Submit. Multiple Users can be created and deployed at the same time.
Push the new user to the Cluster.
Once the Deploy User Job has finished successfully, click Manage in the top right.
In the Deploy tab, Select the Deploy Config to Swift Nodes:
Once the deploy is complete, the User/Account is ready to be used.
Please note that if additional accounts are created needing Auto-Sharding in the future, both the User and Swift Nodes deployment must be pushed again. To redeploy the Swift Nodes config, return to the Deploy Tab. Although no changes are noted, you must click the Deploy Config to Swift Nodes after each new user needing Auto-Sharding is created.
To create a Bucket for use by Avere log into the SwiftStack console with the Username & Password combination created above. The SwiftStack console can be found at http(s)://your.swiftstack.cluster/console. Click the Plus Sign + below Containers to add a new Bucket/Container.
Give the Bucket a name and choose the Storage Policy. It is not currently recommended to use Erasure Coded Storage Policies with Avere FXT clusters. Erasure Coding performance for small objects is half the performance of Replica's and such should not be used for high performance driven applications.
Once Data has been written to the Container/Bucket, multiple sharded containers will be automatically created. These will use the same Storage Policy as the base Container. Please note that the base container will not show any data, and the actually objects for Avere FXT will be in the sharded containers. Data should not be written to any of the sharded containers. These are for use only by SwiftStack Middleware. In the example below, shardertest is the base container, and all of the sub-containers are the Auto-Sharder Middleware. Clicking on these you will see the actual objects.