Skip to main content
Version: Weekly Build

Object Storage

Object Storage provides an S3-compatible API for files, backups, assets, and other unstructured data. It is intended for workloads that need bucket-based object storage instead of a filesystem or relational database.

PropertyValueNotes
Provider names3Use this in landscape provider definitions.
Versionv1Current schema version exposed by the provider.
CategoryStorageShown in the managed services catalog.
ScopeglobalAvailable at team scope rather than being tied to a single workspace runtime.
Team singletonfalseTeams can create multiple Object Storage service instances.
Pause supportfalseThis provider does not support pausing.

Preview Feature

This provider is currently a preview feature. It is not enabled by default and needs to be enabled by your operator. As a preview, the provider is still evolving, its schema, plans, and behavior may change in the future.

Capabilities​

The service is highly available: the underlying Ceph storage is set up redundantly and replicated at the storage layer.

No backups yet

Automated backups of the stored data are not yet available for this provider. Until they are, you are responsible for backing up important data yourself.

Schema​

Config​

FieldTypeRequired on createNotes
accessKeystringYesCluster-unique access key. Must be exactly 20 uppercase letters or digits.
userDisplayNamestringNoDefault: My S3 User. Friendly label for the generated user.
initialBucketNamestringYesCluster-unique initial bucket name. If it is already taken, the bucket is not created.

Secrets​

FieldTypeRequired on createNotes
secretKeystringYesSecret access key. Must be exactly 40 alphanumeric characters.

Details / Output​

FieldTypeAvailabilityNotes
urlstringExposed after provisioningS3-compatible endpoint URL. For Codesphere-managed S3, this is always http://rgw-load-balancer.rook-ceph.svc.cluster.local.
userIdstringExposed after provisioningInternal identifier of the generated object-storage user.

Plan​

The provider exposes one plan, Generic (id: 0), which allows all quota parameters to be adjusted. Example plan: Generic (id: 0).

ParameterTypeDefaultMinimumMaximumStaticNotes
maxBucketsinteger5011000NoMaximum number of buckets.
maxObjectsinteger100000110000000NoMaximum number of objects.
maxSizeKbinteger10000000110000000000NoTotal size limit in KB.
maxReadOpsPerSinteger1000110000NoMaximum read operations per second.
maxWriteOpsPerSinteger1000110000NoMaximum write operations per second.
maxReadBytesPerSinteger100000000110000000000NoMaximum read throughput in bytes per second.
maxWriteBytesPerSinteger100000000110000000000NoMaximum write throughput in bytes per second.

Example in a Landscape​

schemaVersion: v0.2

run:
  uploads:
    provider:
      name: s3
      version: v1
    plan:
      id: 0
      parameters:
        maxBuckets: 50
        maxObjects: 100000
        maxSizeKb: 10000000
        maxReadOpsPerS: 1000
        maxWriteOpsPerS: 1000
        maxReadBytesPerS: 100000000
        maxWriteBytesPerS: 100000000
    config:
      accessKey: "${{ workspace.env.S3_ACCESS_KEY }}"
      userDisplayName: "Landscape Upload User"
      initialBucketName: "${{ workspace.env.S3_BUCKET }}"
    secrets:
      secretKey: "${{ vault.s3SecretKey }}"

Within other runtimes, configure your S3 client with the returned url, the configured accessKey, and the stored secretKey. For Codesphere-managed S3, the endpoint is always http://rgw-load-balancer.rook-ceph.svc.cluster.local. The same S3-compatible endpoint is available from other Codesphere runtimes, including reactives, managed containers, and workloads inside a Virtual Cluster.

Connecting to an Object Store​

Prerequisites​

Once your Object Store is deployed, you can connect to it from your Codesphere workspaces. Each service lists non-sensitive connection details in their respective settings page, in the overview tab (or in the details property in the public API payload).

info

Before connecting, make sure the service is synchronized.

Connecting via Terminal (mc)​

The MinIO Client (mc) is a robust tool for interacting with S3-compatible APIs.

# Install mc
nix-env -iA nixpkgs.minio-client

# Configure alias
mc alias set my-storage http://rgw-load-balancer.rook-ceph.svc.cluster.local "$ACCESS_KEY" "$SECRET_KEY"

# List buckets
mc ls my-storage

# Copy file
mc cp myfile.txt my-storage/my-bucket/

Connecting via Node.js​

Using the AWS SDK for JavaScript v3 (@aws-sdk/client-s3):

const { S3 } = require("@aws-sdk/client-s3");

const s3 = new S3({
    endpoint: "http://rgw-load-balancer.rook-ceph.svc.cluster.local",
    region: "us-east-1",
    credentials: {
        accessKeyId: "YOUR_ACCESS_KEY",
        secretAccessKey: "YOUR_SECRET_KEY"
    },
    forcePathStyle: true,
    tls: false
});

const { Buckets } = await s3.listBuckets({});
console.log(Buckets);