Content on this page
To onboard to Sneller Cloud, you will need to have a Sneller Cloud account. We will create that in the next step. You also need to have Terraform installed which is a tool to automate infrastructure.
All examples are written for Linux and should also work on MacOS. All examples have also been tested in WSL2 (Windows Subsystem for Linux).
Register with Sneller Cloud
You have to register on https://console.sneller.ai/ in order to use Sneller Cloud. You can do so either with a Google account or a GitHub handle.
After completing the registration, the onboarding process starts. You will receive an email called “Welcome to Sneller” that contains a (bearer) token that you should provide for the next step:
Installing Terraform is straightforward since it is a single executable. Follow the Install Terraform instructions to download it.
Now it is time to setup Terraform. It uses both the AWS provider and the Sneller provider. The AWS provider uses the current user’s AWS credentials, so make sure you have sufficient rights and are logged in to the proper account. You can verify your AWS user and account ID using1:
aws sts get-caller-identity
You can download the Terraform scripts as follows:
git clone https://github.com/snellerinc/examples cd examples/terraform/cloud-onboarding
In a nutshell these are the actions that Terraform will perform (see here for full details):
- create both a source and an ingestion bucket (in your own AWS account)
- create a Sneller IAM role allowing Sneller to access the S3 source and ingestion buckets
- create a test table called
tutorial/table1with some sample data
- setup S3 event notifications
Simply execute the commands below in order to complete the integration between your AWS account and Sneller Cloud:
terraform init # needed only once terraform apply
After a successful run, you will see the following output:
... ... ... Apply complete! Resources: 16 added, 0 changed, 0 destroyed. Outputs: database = "tutorial" sneller_endpoint = "https://snellerd-production.us-east-1.sneller.ai" sneller_ingest = "abcd-sneller-ingest" sneller_source = "abcd-sneller-source" sneller_token = "SA0M1ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHI" table = "table1"
Run a test query
Sneller will automatically pick up table definitions and ingest all data for the test table. We can now run some queries. First we’ll setup some environment variables to ensure that we use the correct values:
export SNELLER_DATABASE=$(terraform output -json database | jq -r '.') export SNELLER_TABLE=$(terraform output -json table | jq -r '.') export SNELLER_ENDPOINT=$(terraform output -json sneller_endpoint | jq -r '.') export SNELLER_TOKEN=$(terraform output -json sneller_token | jq -r '.')
Now run the following command to determine the number of items for the test table.
curl -H "Authorization: Bearer $SNELLER_TOKEN" \ -H "Accept: application/json" \ -s "$SNELLER_ENDPOINT/query?database=$SNELLER_DATABASE" \ --data-raw "SELECT COUNT(*) FROM $SNELLER_TABLE"
You can also find some more examples here for S3 data sources that are easy to ingest.
For full details on the Terraform scripts used in this onboarding introduction, please read the detailed explanation.
This requires that the AWS CLI has been installed locally. ↩︎