atomic-server

https://user-images.githubusercontent.com/2183313/139728539-d69b899f-6f9b-44cb-a1b7-bbab68beac0c.mp4

Status: Beta. Breaking changes are expected until 1.0.

Atomic-server is a graph database server for storing and sharing Atomic Data. Demo on atomicdata.dev

• ⚛️ Dynamic schema validation / type checking using Atomic Schema. Combines safety of structured data with the
• 🚀 Fast (1ms responses on my laptop)
• 🪶 Lightweight (15MB binary, no runtime dependencies)
• 💻 Runs everywhere (linux, windows, mac, arm)
• 🌐 Embedded server with support for HTTP / HTTPS / HTTP2.0 and Built-in LetsEncrypt handshake.
• 🎛️ Browser GUI included powered by atomic-data-browser. Features dynamic forms, tables, authentication, theming and more.
• 💾 Event-sourced versioning / history powered by Atomic Commits
• 🔄 Synchronization using websockets: communicates state changes with a client. Send a wss request to /ws to open a webscocket.
• 🧰 Many serialization options: to JSON, JSON-AD, and various Linked Data / RDF formats (RDF/XML, N-Triples / Turtle / JSON-LD).
• 🔎 Full-text search with fuzzy search and various operators, often <3ms responses.
• 📖 Pagination, sorting and filtering using Atomic Collections
• 🔐 Authorization (read / write permissions) and Hierarchical structures powered by Atomic Hierarchy
• 📲 Invite and sharing system with Atomic Invites
• 📂 File management: Upload, download and preview attachments.

Powered by Rust, atomic-lib, actix-web, sled, tantivy and more.

README Contents

• When should you use this
• When not to use this
• Installation & getting started
• Initial setup and configuration
• Usage
• Testing
• Performance benchmarking
• FAQ & Troubleshooting

When should you use this

• You want to make (high-value) datasets as easily accessible as possible
• You want to specify and share a common vocabulary / ontology / schema for some specific domain or dataset. Example classes here.
• You want to use and share linked data, but don't want to deal with most of the complexities of RDF, SPARQL, Triple Stores, Named Graphs and Blank Nodes.
• You are interested in re-decentralizing the web or want want to work with tech that improves data ownership and interoperability.
• You like living on the edge (this application is not production ready)

When not to use this

• If you need stability or reliability, look further (for now).
• If you need to store large amounts of data (for now). The server still lacks collection caching, which makes filtering / sorting slow on large collections #114
• You're dealing with sensitive private data. The authorization mechanisms are relatively new and not rigorously tested.
• Complex query requirements. Check out NEO4j, Apache Jena or maybe TerminusDB.

Installation & getting started

You can run atomic-server in four ways:

• From a published binary (probably the quickest)
• Using docker
• Using Cargo from crates.io: cargo install atomic-server
• Manually from source

When you're running atomic-server, go to Initial setup and configuration

Install using cargo

```sh

Install from source using cargo, and add it to your path

cargo install atomic-server

Check the available options and commands

atomic-server --help

Run it!

atomic-server ```

Run using docker

The dockerfile is located in the project root, above this server folder.

• Run: docker run -p 80:80 -p 443:443 -v atomic-storage:/atomic-storage joepmeneer/atomic-server
• If you want to update, run docker pull joepmeneer/atomic-server and docker should fetch the latest version.

Install from source

```sh

Clone this repoo

git clone git@github.com:joepio/atomic-data-rust.git cd atomic-data-rust/server

Optional, but recommended: Create a new .env using the template.

cp default.env .env

Run the server. It creates a store in ~/.config/atomic/db by default

cargo run ```

Troubleshooting compiling from source:

```sh

If pkg-config or libssl-dev is not installed, make sure to install them

sudo apt-get install -y pkg-config libssl-dev --fix-missing ```

Initial setup and configuration

• You can configure the server by passing arguments (see atomic-server --help), or by setting ENV variables.
• The server loads the .env from the current path by default. Create a .env file from the default template in your current directory with atomic-server setup-env.
• After running the server, check the logs and take note of the Agent Subject and Private key. You should use these in the atomic-cli and atomic-data-browser clients for authorization.
• A directory is made: ~/.config/atomic, which stores your newly created Agent keys, your data, the HTTPS certificates and a folder for public static files.
• Visit http://localhost:9883/setup to register your first (admin) user. You can use an existing Agent, or create a new one.

Running using a tunneling service (easy mode)

If you want to make your atomic-server available on the web, but don't want (or cannot) deal with setting up port-forwarding and DNS, you can use a tunneling service. It's the easiest way to get your server to run on the web, yet still have full control over your server.

• Create an account on some tunneling service, such as tunnelto.dev (which we will use here). Make sure to reserve a subdomain, you want it to remain stable.
• tunnelto --port 9883 --subdomain joepio --key YOUR_API_KEY
• atomic-server --domain joepio.tunnelto.dev --custom-server-url 'https://joepio.tunnelto.dev' --initialize

HTTPS Setup on a VPS (static IP required)

You'll probably want to make your Atomic Data available through HTTPS on some server. You can use the embedded HTTPS / TLS setup powered by LetsEncrypt, acme_lib and rustls.

You can do this by passing these flags:

Run the server: atomic-server --https --email some@example.com --domain example.com.

You can also set these things using a .env or by setting them some other way.

Make sure the server is accessible at ATOMIC_DOMAIN at port 80, because Let's Encrypt will send an HTTP request to this server's /.well-known directory to check the keys. The default Ports are 9883 for HTTP, and 9884 for HTTPS. If you're running the server publicly, set these to 80 and 433: atomic-server --https --port 80 --port-https 433. It will now initialize the certificate. Read the logs, watch for errors.

HTTPS certificates are automatically renewed when the server is restarted, and the certs are 4 weeks or older. They are stored in your .config/atomic/ dir.

Usage

There are three ways to interact with this server:

• GUI: Use the atomic-data-browser JS frontend by visiting localhost:9883.
• API: We have a subset of the API documented using Swagger / OpenAPI. Check out ./example_requests.http for various HTTP requests to the server. Also, read the Atomic Data Docs. You can also try the react boilerplate to build your own front-end app using @tomic/lib and @tomic/react.
• CLI: The atomic-cli terminal app

Use atomic-cli as client

atomic-cli is a useful terminal tool for interacting with atomic-server. It makes it easy to query and edit Atomic Data from the command line. Check it out.

API

You can fetch individual items by sending a GET request to their URL.

```sh

Fetch as JSON-AD (de facto standard for Atomic Data)

curl -i -H "Accept: application/ad+json" https://atomicdata.dev/properties/shortname

Fetch as JSON-LD

curl -i -H "Accept: application/ld+json" https://atomicdata.dev/properties/shortname

Fetch as JSON

curl -i -H "Accept: application/json" https://atomicdata.dev/properties/shortname

Fetch as Turtle / N3

curl -i -H "Accept: text/turtle" https://atomicdata.dev/properties/shortname ```

Check out ./example_requests.http for more things that you can do. Also, read the Atomic Data Docs to learn more about Collections, Commits, JSON-AD and other concepts used here.

FAQ & Troubleshooting

Can / should I create backups?

Yes! You should. Especially before installing a newer Atomic-Server version, it might be imcompatible with the previous database model and could corrupt the database. Run atomic-server export to create a backup in your ~/.config/atomic/backups folder. Import them using atomic-server import -p ~/.config/atomic/backups/${date}.json.

I lost the key / secret to my Root Agent, and the /setup invite is no longer usable! What now?

You can run atomic-server --initialize to recreate the /setup invite. It will be reset to 1 usage.

How do I migrate my data to a new domain?

There are no helper functions for this, but you could atomic-server export your JSON-AD, and find + replace your old domain with the new one. This could especially be helpful if you're running at localhost:9883 and want to move to a live server.

How do I reset my database?

Remove the db folder in your atomic config: rm -rf ~/.config/atomic/db.

How do I make my data private, yet available online?

This is not yet possible. See #13.

Collections are empty / TPF is not working / search is empty

You might have a problem with your indexes. Try rebuilding the indexes using atomic-server --rebuild-index.

I get a failed to retrieve error when opening

Try re-initializing atomic server atomic-server --initialize.