Files - File Web Service (cloud storage)

crates.io

Purpose

The purpose of this service is to be your file web service (cloud storage) for any JSON client including JavaScript front-ends like React with fetch.

Files supports aws s3, wasabi, yandex, and digital ocean.

Files is built to be simple and blazing fast with JWT verification, bucket management, and object management.

To use this service you need to have a running broker server.

Features

Use

  1. create a user on broker with the following scopes - for full permissions files:full or granular permissions files:provider, files:get_object, files:put_object, files:delete_object, files:create_bucket, files:list_bucket, or files:delete_bucket
  2. login to broker and get a JWT
  3. attach the JWT as an Authorization: Bearer {token} to the following JSON API endpoints
  4. Add your credentials for your provider(s)

| Provider | Region | |--- | --- | | wasabi | wa-us-east-1 | | wasabi | wa-us-east-2 | | wasabi | wa-us-west-1 | | wasabi | wa-eu-central-1 | | aws | us-east-1 | | aws | us-east-2 | | aws | us-west-1 | | aws | us-west-2 | | aws | ca-central-1 | | aws | ap-south-1 | | aws | ap-northeast-1 | | aws | ap-northeast-2 | | aws | ap-northeast-3 | | aws | cn-north-1 | | aws | cn-northwest-1 | | aws | eu-north-1 | | aws | eu-central-1 | | aws | eu-west-1 | | aws | eu-west-2 | | aws | eu-west-3 | | aws | me-south-1 | | aws | sa-east-1 | | do | nyc3 | | do | ams3 | | do | spg1 | | do | fra1 | | yandex | ru-central1 |

Add/Update Provider

html POST /provider - authenticated endpoint (Authorization: Bearer {jwt}) example: json { "name": "aws", "access_key": "AKIAIOSFODNN7EXAMPLE", "secret_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" } will return: 200 or 500 or 400 or 401

Get Object

html POST /get - authenticated endpoint (Authorization: Bearer {jwt}) example: json { "object_path": "/test.pdf", "region": "wa-us-east-1", "bucket": "test" } will return: 200 or 500 or 400 or 401

200 - will return a file stream (application/octet-stream)

Put Object

html POST /put - authenticated endpoint (Authorization: Bearer {jwt}) example: json { "object_path": "/test.pdf", "region": "wa-us-east-1", "bucket": "test", "file": "dGhpc2lzYXN0cmluZw==" } - file is a base64 encoded binary file - this is doable with the FileReader class in the browser

will return: the provider's status code (e.g. 200, 400)

Delete Object

html POST /del - authenticated endpoint (Authorization: Bearer {jwt}) example: json { "object_path": "/test.pdf", "region": "wa-us-east-1", "bucket": "test" } will return: the provider's status code (e.g. 204, 409)

Create Bucket

html POST /create_bucket - authenticated endpoint (Authorization: Bearer {jwt}) example: json { "region": "wa-us-east-1", "bucket": "test" } will return: 200 with no body response or the provider's status code (e.g. 409) and the provider's error json { "error": "provider's response as an xml string" } - note: if the bucket already exists the request will return 200

Delete Bucket

html POST /delete_bucket - authenticated endpoint (Authorization: Bearer {jwt}) example: json { "region": "wa-us-east-1", "bucket": "test" } will return: the provider's status code (e.g. 204, 409) - note: all objects in the bucket have to be deleted before the bucket can be deleted (based on provider's rules)

List Bucket

html POST /list_bucket - authenticated endpoint (Authorization: Bearer {jwt}) example: json { "region": "wa-us-east-1", "bucket": "test", "prefix": "/", "delimiter": "/" } - delimiter is an optional field

will return: 200, 400, 500, 401

200 - will return an array of objects json { "objects": [{ "key": "/test.pdf", "e_tag": "33a64df551425fcc55e4d42a148795d9f25f89d4", "storage_class": "STANDARD", "size": 100, "last_modified": "2021-04-14T22:34:04.000Z" }] }

Health Check

html GET or HEAD / - public endpoint

will return: 200

Install

cargo install portal

Service

There is an example systemctl service for Ubuntu called files.service in the code

TechStack

Inspiration