s3rs

an S3 Rich Support client - rust edition 2018 - multi config (please put config files under ~/.config/s3rs) - interactive command line tool - easy to debug with http protocol - AWS4, AWS2 support - support http redirect for multi region of AWS S3 - support automatically multipart upload

current status:

| COMMAND | FUNCTION | CEPH | AWS | |----------------------------------------------------------------------|------------------------------------------------|------|-----| | ls | list all buckets | O | O | | mb bucket | create bucket | O | O | | rb bucket | delete bucket | O | O | | put s3://bucket/object | upload the file | O | O | | put s3://bucket | upload the file (file name as object name) | O | O | | put test s3://bucket/object | upload a test file sepcific object name | O | O | | get s3://bucket/object file | download object | O | O | | get s3://bucket/object | download object in current folder | O | O | | cat s3://bucket/object | show the object content | O | O | | del s3://bucket/object [delete-marker:true] [secure-delete:true] | delete the object (with flag) | O | O | | | delete-marker used in AWS | | | | | secure-delete used in BIGTERA(customized CEPH) | | | | tag list s3://bucket/object | list tag(s) to the object | O | O | | tag ls s3://bucket/object | list tag(s) to the object | O | O | | tag add s3://bucket/object key1=value1 [key2=value2] ... | add tag(s) to the object | O | O | | tag put s3://bucket/object key1=value1 [key2=value2] ... | add tag(s) to the object | O | O | | tag del s3://bucket/object | remove tag(s) from the object | O | O | | tag rm s3://bucket/object | remove tag(s) from the object | O | O | | /uri?query | give the orignal url | O | O |

| HIGH LEVEL COMMAND | INTEGRATE FUNCTIONS | CEPH | AWS | |---------------------------|------------------------------------------------------------------------------------------|------|-----| | la | list all objects | O | O | | ls bucket | list objects in the bucket | O | O | | ls s3://bucket | list objects in the bucket | O | O | | ls s3://bucket/prefix | list objects match prefix in the bucket | O | O | | ll | list all objects details (storage class, modify time, etag) | O | O | | ll s3://bucket | list objects detail in the bucket | O | O | | ll s3://bucket/prefix | list objects match prefix detail in the bucket | O | O | | info s3://bucket | acl(ceph, aws), location(ceph, aws), versioning(ceph, aws), uploads(ceph), version(ceph) | O | O |

| CEPH OPS API (use system keys) | CEPH ONLY FUNCTION | |--------------------------------|-----------------------| | usage s3://bucket | show the bucket usage |

| SHELL SETTING COMMAND | FUNCTION | |-------------------------------|------------------------------------------| | s3_type [ceph/aws/aws4/aws2] | change the api for different S3 providor | | log [trace/debug/info/erro] | change the log level | | | - Info : for Http header and body | | | - debug: for auth signature hash info | | | - trace: more detail about rust | | logout | logout and reselect user | | Ctrl + d | logout and reselect user |

| S3 TYPE | AUTH TYPE | FORMAT | URL STYLE | |---------|-----------|--------|----------------------| | ceph | aws4 | json | path-style | | aws | aws4 | xml | virtual-hosted–style |

Download excutable binary

Download the link as follow and unzip - https://github.com/yanganto/s3rs/releases/download/v0.2.6/s3rs-v0.2.6-linux.zip

Install via Crate.io

Install rust tools rustup and cargo - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Additional package for ubuntu - apt-get install libssl-dev pkg-config openssl

It is easy to install s3rs via cargo as following command. - cargo install s3rs

Set up the path for rust binnary - export PATH=$PATH:~/.cargo/bin

The config file will auto setup when binary first time launch - s3rs

edit the config file at ~/.s3rs.toml - vi ~/.s3rs.toml

Develop

Install rust tools rustup and cargo - curl https://sh.rustup.rs -sSf | sh - git clone https://github.com/yanganto/s3rs.git - cargo test - cargo build - The excutable binary will in ./target/debug/s3rs

Demo

• A short demo video
• snapshot