As part of the continued expansion in Chainlink node infrastructure, we’re thrilled to work with Amazon Web Services (AWS) to simplify the process for data providers, enterprises, and DevOps. With the newly launched AWS Chainlink Quickstart, operators now have a one-click workflow to easily deploy a Chainlink oracle node on AWS and sell real-world data across multiple blockchain networks.

Through AWS Chainlink Quickstart, data providers can sell their data and APIs simultaneously across numerous public and private blockchain environments without needing any prior blockchain experience. Additionally, DevOps teams, traditional enterprises, telecommunication firms, individual developers, and DAOs can instantly launch their own Chainlink nodes to provision oracle services for blockchain applications or on behalf of data provider clients. The unique workflow of AWS empowers data providers and node operators across all market segments and helps them easily grow their addressable market by giving them direct access to any emerging blockchain network.

In this article, we’ll cover the opportunity for data providers, enterprises, and DevOps teams to launch a Chainlink node, dive into the infrastructure that is deployed through the AWS Chainlink Quickstart, and provide a technical walk-through to launch a Chainlink node on AWS in minutes. If you are a data provider and want to learn more about becoming blockchain-enabled through AWS, refer to the following documentation: https://aws-quickstart.github.io/quickstart-chainlinklabs-chainlink-node/

Why Data and APIs Are Critical to Blockchain Applications

Similar to the Internet’s evolution from email to the world wide web, blockchains—shared backend computing and database infrastructure—are transitioning from purely supporting new forms of decentralized money to now enabling more tamper-proof, reliable, and efficient automation of multi-party business processes. The key driver in the evolution of blockchain applications is hybrid smart contracts—data-driven business logic running on top of blockchains that can ingest inputs from and send outputs to non-blockchain data providers and traditional systems, as well leverage advanced off-chain computations for scalability and privacy enhancements. 

Nearly all hybrid smart contract applications require external data and APIs to harness real-world value, making support for hybrid smart contract infrastructure essential for all traditional data and service providers. Whether it’s financial market data to trigger DeFi smart contracts, IoT data to automate blockchain-based supply chains, or eSports results to settle decentralized prediction markets, there’s a large and growing list of opportunities for data providers to monetize their APIs within the growing hybrid smart contract economy. 

Chainlink is a decentralized oracle network, serving as tamper-proof blockchain middleware that enables data transfer between APIs and blockchain networks, as well as a security framework to be sure API data is highly reliable in automating the execution of high-value applications. Chainlink is the most widely used blockchain oracle solution, supplying high-quality financial market data to support the growing space of decentralized finance (DeFi). Numerous data providers are already using Chainlink to sell data to smart contract applications, such as supplying weather data for parametric crop insurance, election results for prediction markets, market analytics for options contracts, luxury car benchmarks for tokenized real-world assets, and much more. Additionally, a growing number of enterprises and DevOps are also launching Chainlink nodes to provide smart contracts access to real-world data.

Entering the hybrid smart contract economy is now even easier, thanks to the new AWS Chainlink Quickstart abstracting away all the technical complexities. 

A Single Gateway to All Blockchains

AWS Chainlink Quickstart is a special workflow that gives data providers a future-proof strategy for blockchain adoption: one-click access to any new or existing blockchain, both public and private chains. The framework can quickly map any data provider’s API directly to a Chainlink node, empowering them to cryptographically sign their own data, broadcast it onto blockchains, and sell it directly to smart contracts. This makes it possible for data providers to earn additional revenue streams on their APIs by seamlessly expanding into blockchain ecosystems, as well as improve the reliability of their data through new data signing capabilities that definitively prove data came directly from a specific API. 

AWS Chainlink Quickstart also enables Chainlink’s existing pool of node operators as well as new enterprises and DevOps teams joining the Chainlink ecosystem to swiftly build support for new oracle jobs. Such a setup is particularly useful for node operators leveraging APIs within AWS, which already includes many premium data providers and API services. Chainlink nodes having fluid support for new APIs can bring many more premium data providers into the Chainlink ecosystem, whether by launching their own Chainlink nodes or through existing node operators.

AWS Chainlink Quickstart Architecture

AWS Quick Starts are automated reference deployments that use AWS CloudFormation templates to deploy key technologies on AWS according to AWS best practices. The AWS Chainlink Quickstart reference architecture is an implementation built on this framework and is designed to reduce manual procedures of deploying a Chainlink node instance into just a few steps. The result is a highly available and secure production Chainlink node environment that’s accessible in just a few minutes. 

Deploying the AWS Chainlink Quickstart with default parameters builds the following Chainlink node environment in the AWS Cloud:

• Highly available architecture that spans two Availability Zones.
• An Amazon virtual private cloud (VPC) configured with public and private subnets.
• In the public subnets:
  • An internet gateway to allow access to the internet.
  • Managed network address translation (NAT) gateways to allow outbound internet access for the Chainlink node instances in the private subnets.
  • A Linux bastion host in an Auto Scaling group to allow inbound Secure Shell (SSH) access to Amazon Elastic Compute Cloud (EC2) instances in public and private subnets.
• In the private subnets:
  • Two Chainlink nodes in an Auto Scaling group.
  • An Amazon Relational Database Service (Amazon RDS) PostgreSQL managed database instance.
• Security groups for fine-grained inbound access control.
• Elastic Load Balancing (ELB) to access the Chainlink node web graphical user interface.
• Amazon CloudWatch logging of Chainlink node activity.
• AWS Key Management Service (KMS) for AWS managed keys and customer managed keys for resources.
• AWS Secrets Manager to store and protect Chainlink node credentials.
• Amazon DevOps Guru tracking of the Chainlink node stack using Machine Learning.

Configuration Requirements

Before launching the AWS Chainlink Quickstart, sign into the AWS Management Console with the AWS Identity and Access Management (IAM) permissions for the resources that the templates deploy. Ensure your AWS account is configured correctly and has access to the required resources listed above. The AdministratorAccess managed policy within IAM provides sufficient permissions, although your organization may choose to use a custom policy with more restrictions.

Next, create a key pair and public certificate using Amazon Certificate Manager in your preferred AWS region. Run a blockchain client or use a third party client for the blockchain network you want your Chainlink node to support. In the case of a third party Ethereum client, you can use the websocket endpoint from Infura or Fiews. For other Ethereum clients, visit Run an Ethereum Client.

The AWS Chainlink Quickstart provides two deployment options for teams, including deploying a Chainlink node into a new VPC or deploying into an existing VPC. 

• Deploy Chainlink Node into a new VPC. This option builds a new AWS environment consisting of the VPC, subnets, NAT gateways, security groups, bastion hosts, and other infrastructure components. It then deploys Chainlink nodes into this new VPC.
• Deploy Chainlink Node into an existing VPC. This option provisions a Chainlink node in your existing AWS infrastructure. Make sure that your VPC has two private subnets in different Availability Zones. These subnets require NAT gateways in their route tables to allow instances to communicate with other Chainlink nodes without exposing them to the internet.

The AWS Chainlink Quickstart provides separate templates for these options. It also lets you configure Classless Inter-Domain Routing (CIDR) blocks, instance types, and Chainlink node settings. After these steps have been completed, you are now ready to begin the deployment process. 

Deployment Steps

Sign in to your AWS account and use the region selector in the navigation bar to choose the AWS Region where you want to deploy your Chainlink node. The template is launched in the us-east-1 Region by default, but this parameter can be changed to other regions as desired. Next, choose one of the following options to launch the AWS CloudFormation template.

Deploy Chainlink node into a new VPC on AWS

Deploy Chainlink node into an existing VPC on AWS

On the Create stack page, keep the default setting for the template URL, and then choose Next. On the Specify stack details page, change the stack name if needed, review the parameters for the template, and provide values for the parameters that require input. For all other parameters, review the default settings and customize them as necessary. Unless you are customizing the Quick Start templates for your own deployment projects, keep the default settings for the parameters labeled Quick Start S3 bucket name, Quick Start S3 bucket Region, and Quick Start S3 key prefix. When you finish reviewing and customizing the parameters, choose Next.

On the Configure stack options page, you can specify tags (key-value pairs) for resources in your stack and set advanced options, and then choose Next. On the Review page, review and confirm the template settings. Under Capabilities, select the two check boxes to acknowledge that the template creates IAM resources and might require the ability to automatically expand macros. Finally, Choose Create stack to deploy the stack and monitor its status. When the status is CREATE_COMPLETE, the Chainlink Node deployment is ready. To view the created resources, see the values displayed in the Outputs tab for the stack.

Post Deployment Steps

After the AWS Chainlink Quickstart deployment process is complete, you can access your Chainlink node through the web graphical user interface through two options:

1. If using a domain and SSL certificate created through AWS Certificate Manager, you can access the Chainlink node web GUI through the application load balancer endpoint or through a domain by adding the load balancer endpoint to your domain’s DNS record.
2. By enabling TCP port forwarding on your bastion host, you are able to port forward the Chainlink node web GUI to your machine. As the Chainlink node is accessed through a bastion, it is recommended to use SSH agent forwarding. Then connect to your bastion with SSH agent and TCP port forwarding. You can access the Chainlink node web GUI at http://localhost:6688/ using the following commands:

ssh-add <your_ssh_key>
ssh ec2-user@<bastion_host_public_ip> -A -L 6688:localhost:6688
ssh ec2-user@<chainlink_node_internal_ip> -L 6688:localhost:6688

If the Chainlink node instances are stopped, then new .env, .password, and .api files will need to be created to start the Chainlink node. An .env file can be created with the following command:

cd /home/ec2-user/.chainlink/ && ./create-env.sh \
${chainNetwork} \
${blockchainNodeUrl} \
${psqlUser} \
$(aws secretsmanager get-secret-value --secret-id DBSecret --query "SecretString" --output text) \
${psqlHostname} \
${psqlPort} \
${psqlDb} 

The .password file can be generated using the following command:

cd /home/ec2-user/.chainlink/ && ./create-password.sh \
$(aws secretsmanager get-secret-value --secret-id WalletSecret --query "SecretString" --output text)

The .api file can be generated using the following command:

cd /home/ec2-user/.chainlink/ && ./create-api.sh \
${apiUser} \
$(aws secretsmanager get-secret-value --secret-id ApiSecret --query "SecretString" --output text)

If the latest image of the Chainlink node Docker instance needs to be started, then the following command can be run:

latestimage=$(curl -s -S "https://registry.hub.docker.com/v2/repositories/smartcontract/chainlink/tags/" | jq -r '."results"[]["name"]' | head -n 1)
cd /home/ec2-user/.chainlink && docker run -d \
--log-driver=awslogs \
--log-opt awslogs-group=ChainlinkLogs \
--restart unless-stopped \
--name chainlink \
-p 6688:6688 \
-v /home/ec2-user/.chainlink:/chainlink \
--env-file=/home/ec2-user/.chainlink/.env  smartcontract/chainlink:$latestimage local n \
-p /chainlink/.password \
-a /chainlink/.api

After deploying a Chainlink node on AWS using this Quick Start template, it is important to follow best security and reliability practices to provide smart contracts a reliable source of external data. These approaches include:

• Failover Capabilities: To ensure there is very minimal downtime, failover capabilities are required on both the Chainlink and blockchain clients so that if any one server is disrupted, the service is still online. The Amazon EC2 Auto Scaling group has two Chainlink nodes: one active Chainlink node and one standby Chainlink node. Data from both the Chainlink and blockchain clients are stored in a PostgreSQL database which needs to be highly available.
• Disaster Recovery: To handle unexpected situations, the right processes need to be in place to ensure that as little downtime as possible occurs. Such incidents can occur due to a fully corrupted blockchain node that requires a re-sync. It is therefore recommended to perform:
  • Daily snapshots of the supported blockchain on a separate server than what the Chainlink node is connected to.
  • A blockchain client start-up process that pulls down the latest template of the chain and syncs it to the latest height.
• Active Monitoring: Issues must be detected before or when they occur, requiring active monitoring to be in place. The areas recommended to be monitored include:
  • Minimum required balance of the wallet address assigned to the node used to pay on-chain transaction fees.
  • Errored job runs.
  • Operator UI port to be open and responsive. (Usually: 6688)
  • Blockchain http and websocket ports to be open and responsive. (Usually: 8545 & 8546)
  • Blockchain client disk, RAM and CPU usage.
• Configuration Files: It is recommended to remove the .env, .password, and .api files once the Chainlink node instance is running to prevent potential exposure of sensitive passwords.

For more information on best practices, visit the Best Practices for Nodes on AWS Chainlink documentation.

Conclusion

Chainlink Labs is thrilled to be working with AWS to give data providers, enterprises, and DevOps an easy-to-integrate node infrastructure, and we are also exploring further integrations with Amazon Managed Blockchain to streamline deployment and management of Chainlink nodes. Doing so ensures data providers can raise the value of their datasets by bringing them into fast emerging blockchain markets without worrying about any technical challenges or prior expertise. Similarly, enterprises and DevOps teams can rapidly stand up new Chainlink nodes and connect to existing APIs in order to provision multi-chain oracle services. The end result is a plethora of important datasets made available across blockchains to support the next wave of hybrid smart contract applications. 

We see a large opportunity for data vendors and API providers to use Chainlink to expand their business model and play a major role in the future backend infrastructure of decentralized economic and social systems. To learn more about the AWS Chainlink Quickstart, refer to the following documentation: https://aws.amazon.com/quickstart/architecture/chainlink-node/

If you want to set up a call to discuss your integration in-depth, reach out to an expert. You can also reach out in the official Chainlink Discord with technical questions or follow our documentation for how to test and run Chainlink infrastructure. For more information, visit https://chain.link/.