Storage Providers
StateMesh provides support to attach persistent storage to your Containers and Virtual Machines. We currently support following types of storage providers:
Provider | Recommended Workload | Reliability | Security | Pricing |
---|---|---|---|---|
StateMesh | Enterprise Medium Personal | Very High | Very High | GB/h Data Transfer OUT |
S3 Bucket | Enterprise Medium Personal | High | Very High | Price of underlying provider |
Network | Medium Personal | High | High | GB/h Data Transfer OUT |
StateMesh
This provider can create persistent volumes using our own storage system owned by the StateMesh organization. The data we store is encrypted and replicated across multiple machines for fault tolerance.
This is typically used for sensitive data, because nodes never have access to this data.
Pricing is calculated using the amount of storage reserved in Gb/h
and data transferred OUT from StateMesh.
Network
This storage provider stores volume date on multiple nodes in the StateMesh network. It's a block storage option that is replicated and encrypted across the network. Use this option when your data is not sensitive or does not require a high level of resiliency.
S3 Bucket
This provider can create volumes using an S3 bucket that you provide. This is call Bring Your Own Storage (BYOS) and is useful for storing sensitive data. StateMesh does not store any data from your S3 bucket, only the metadata required to mount the volume.
StateMesh provides a high-performance POSIX cloud filesystem that allows almost all kinds of object storage to be used as massive local disks and to be mounted and accessed on different applications and virtual machines. The cloud filesystem is low-latency, provides high throughput, and is suitable for storing large amounts of data.
Our internal benchmarks show that the StateMesh S3 cloud filesystem can provide 10x more throughput compared to traditional cloud storage filesystems and other S3-backed filesystems.
AWS S3
AWS S3 supports [two styles of endpoint URI](https://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html: virtual hosted-style and path-style. The difference is:
- Virtual-hosted-style:
https://<bucket>.s3.<region>.amazonaws.com
- Path-style:
https://s3.<region>.amazonaws.com/<bucket>
The <region>
should be replaced with specific region code, e.g. the region code of US East (N. Virginia) is us-east-1. All the available region codes can be found here.
For AWS users in China, you need add .cn to the host, i.e. amazonaws.com.cn, and check this document for region code.
Azure Blob Storage
To use Azure Blob Storage as data storage of JuiceFS, please check the documentation to learn how to view the storage account name and access key.
The S3 Bucket URL
is set in the format https://<container>.<endpoint>
, please replace <container>
with the name of the actual blob container and <endpoint>
with core.windows.net
(Azure Global) or core.chinacloudapi.cn
(Azure China).
Clouflare R2
R2 is Cloudflare's object storage service and provides an S3-compatible API, so usage is the same as Amazon S3. Please refer to Documentation to learn how to create Access Key and Secret Key.
Alibaba Cloud OSS
Please follow this document to learn how to get access key and secret key.
OSS provides multiple endpoints for each region, depending on your network (e.g. public or internal network). Please choose an appropriate endpoint.
Tencent Cloud COS
The naming rule of bucket in Tencent Cloud is <bucket>-<APPID>
, so you must append APPID to the bucket name. Please follow this document to learn how to get APPID.
The full format of S3 Bucket URL
is https://<bucket>-<APPID>.cos.<region>.myqcloud.com
, and please replace <region>
with specific region code. E.g. the region code of Shanghai is ap-shanghai
. You could find all available region codes here.
Huawei Cloud OBS
Please follow this document to learn how to get access key and secret key.
The full format of S3 Bucket URL
is https://<bucket>.obs.<region>.myhuaweicloud.com
, and please replace <region>
with specific region code. E.g. the region code of Beijing 1 is cn-north-1
. You could find all available region codes here.
Baidu Object Storage
Please follow this document to learn how to get access key and secret key.
The full format of S3 Bucket URL
is https://<bucket>.<region>.bcebos.com
, and please replace <region>
with specific region code. E.g. the region code of Beijing is bj
. You could find all available region codes here.
Scaleway Object Storage
Please follow this document to learn how to get access key and secret key.
The full format of S3 Bucket URL
is https://<bucket>.s3.<region>.scw.cloud
. Remember to replace <region>
with specific region code, e.g. the region code of "Amsterdam, The Netherlands" is nl-ams
. All available region codes can be found here.
Vultr Object Storage
Please find the access and secret keys for object storage in the customer portal.
The full format of S3 Bucket URL
is https://<bucket>.<region>.vultrobjects.com
.
DigitalOcean Spaces
Please follow this [document] to learn how to get access key and secret key.
The full format of S3 Bucket URL
is https://<space-name>.<region>.digitaloceanspaces.com
. Please replace <region>
with specific region code, e.g. nyc3
. All available region codes can be found here.
IBM Cloud Object Storage
When using IBM Cloud Object Storage, you first need to create an API key and an instance ID. The "API key" and "instance ID" are the equivalent of access key and secret key, respectively.
IBM Cloud Object Storage provides multiple endpoints for each region, depending on your network (e.g. public or private)
Oracle Cloud Object Storage
Oracle Cloud Object Storage supports S3 compatible access. Please refer to official documentation for more information.
The full format of S3 Bucket URL
is https://<bucket>.<endpoint>
. The endpoint format for this object storage is: ${namespace}.compat.objectstorage.${region}.oraclecloud.com
Ceph RGW
Ceph Object Gateway is an object storage interface built on top of librados
to provide applications with a RESTful gateway to Ceph Storage Clusters. Ceph Object Gateway supports S3-compatible interface.
The full format of S3 Bucket URL
is http://<bucket>.<endpoint>
(virtual hosted-style).
MinIO
MinIO is an open source lightweight object storage, compatible with Amazon S3 API.
For S3 Bucket URL
we only support path-style MinIO URI addresses e.g. https://<endpoint>:9000/myfs
When using Multi-Node MinIO deployment, consider setting using a DNS address in the service endpoint, resolving to all MinIO Node IPs, as a simple load-balancer, e.g. http://minio.example.com:9000/myjfs
WebDAV
WebDAV is an extension of the Hypertext Transfer Protocol (HTTP) that facilitates collaborative editing and management of documents stored on the WWW server among users.
The S3 Bucket URL
needs to be set to the endpoint of WebDAV. If basic authorization is enabled, username and password should be provided as S3 Access Key
and S3 Secret
MySQL
MySQL is one of the popular open source relational databases, often used as the database of choice for web applications. MySQL-compatible MariaDB, TiDB, etc. can be used as data storage.
When using MySQL as a data storage, you need to create a database in advance and add the desired permissions, specify the access address through the S3 Bucket URL
, specify the user name through S3 Access Key
and specify the password through S3 Secret
.
The full format of S3 Bucket URL
is (<host>:3306)/<database-name>
PostgreSQL
PostgreSQL is a powerful open source relational database with a complete ecology and rich application scenarios. Other databases compatible with the PostgreSQL protocol (such as CockroachDB, etc.) can also be used as data storage.
When creating a file system, you need to create a database and add the corresponding read and write permissions.
Use the S3 Bucket URL
to specify the address of the data, use S3 Access Key
to specify the username, and use S3 Secret
to specify the password.
The full format of S3 Bucket URL
is <host>:<port>/<db>[?parameters]