Solana on Akashnet
This write-up will guide you to deploy Solana node on Akashnet
This write-up will guide you to deploy Solana node on Akashnet.
I'll try to make it short while keeping all necessary steps available for deployment from scratch.
I must point that this is not a production Solana node deployment, but rather the development deployment due to reasons which can be seen in the end of this write-up.
I used Ubuntu 20.04.2 LTS Linux distribution and the following software:
- akash: v0.12.1 (to work with the Akashnet);
- solana: v1.6.10 (to deploy it on the Akashnet);
Meet your new friends:
- Akashnet is a decentralized cloud of future;
- Solana is a high-performance blockchain supporting builders around the world creating crypto apps that scale today.
The idea is simple - leverage the Akashnet decloud in order to deploy Solana container.
It's up to 86% cheaper to run an instance on Akashnet than on Amazon AWS https://akashlytics.com/price-compare (at the time I wrote this write-up)
Ok, I'm not good at talking much, so to save your and my time let's jump straight to the CLI.
1. Install akash client
AKASH_NET="https://raw.githubusercontent.com/ovrclk/net/master/mainnet"
AKASH_VERSION="$(curl -s "$AKASH_NET/version.txt")"
curl https://raw.githubusercontent.com/ovrclk/akash/master/godownloader.sh | sh -s -- "v$AKASH_VERSION"Please also install jq tool as it will be used in this write-up.2. Create your akash wallet
akash keys add defaultThis command will output the mnemonic phrase, make sure to save it in a safe place.
It will also output your address, in my case the address was akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h.
export AKASH_ACCOUNT_ADDRESS=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h
More about the Akash wallet https://docs.akash.network/guides/wallet
3. Fund your akash wallet
You will need about 10 AKT (Akash Token) to get you started.
You can purchase them at one of the exchanges mentioned here https://akash.network/token/
You can query the balance of your wallet
export AKASH_NET="https://raw.githubusercontent.com/ovrclk/net/master/mainnet"
export AKASH_NODE="$(curl -s "$AKASH_NET/rpc-nodes.txt" | shuf -n 1)"
akash \
--node "$AKASH_NODE" \
query bank balances "$AKASH_ACCOUNT_ADDRESS"Denomination: 1 akt = 1000000 uakt (akt*10^6), or you can add -o json | jq -r '(.balances[0].amount | tonumber / pow(10; 6))' on the command line ;-)I'm using production akashnet-2 Akash network. For more on different Akash networks please refer to https://docs.akash.network/guides/version
4. Create certificate file
To create a deployment, a certificate must first be created. To do this, run:
export AKASH_CHAIN_ID="$(curl -s "$AKASH_NET/chain-id.txt")"
akash tx cert create client \
--from default \
--chain-id $AKASH_CHAIN_ID \
--node $AKASH_NODE \
--gas-prices="0.025uakt" --gas="auto" --gas-adjustment=1.15certificate needs to be created only once per account and can be used across all deployments. More about the certificate https://docs.akash.network/design/mtls
5. Prepare the deployment manifest file
Create a file named solana.yml with the following contents:
---
version: "2.0"
services:
solana:
image: solanalabs/solana:v1.6.10
expose:
- port: 8899
as: 8899
proto: tcp
to:
- global: true
profiles:
compute:
solana:
resources:
cpu:
units: 0.1
memory:
size: 512Mi
storage:
size: 512Mi
placement:
akash:
attributes:
host: akash
signedBy:
anyOf:
- "akash1365yvmc4s7awdyj3n2sav7xfx76adc6dnmlx63"
pricing:
solana:
denom: uakt
amount: 10
deployment:
solana:
akash:
profile: solana
count: 1Pricing - Note that there are bids from multiple different providers. In this case, both providers happen to be willing to accept a price of 1 uAKT. This means that the lease can be created using 1 uAKT or 0.000001 AKT per block to execute the container.
pricing sets the limits so providers know how to bid on your deployment. i.e. amount of uakt per block (each block happens every 6 seconds) you are willing to pay.
So in case with 1uakt, you are going to be paying ((1*((60/6)*60*24*30.436875))/10^6)*3.28 = $1.44 per month having 1akt worth $3.28 as of 31 May 2021.
signedBy must be akash1365yvmc4s7awdyj3n2sav7xfx76adc6dnmlx63 as in the doc https://docs.akash.network/guides/deploysignedBy is a "validation" thing, after a provider is approved by akash they get signed by akash to ensure security and reliability.
More on SDL (Stack Definition Language) used to write the Akash manifest https://docs.akash.network/documentation/sdl
6. Create your deployment
akash tx deployment create solana.yml \
--from default \
--node $AKASH_NODE \
--chain-id $AKASH_CHAIN_ID \
--gas-prices="0.025uakt" --gas="auto" --gas-adjustment=1.15If this command ends with Error: RPC error -32603 - Internal error: timed out waiting for tx to be included in a block then no worries, you can query the akash blockchain in order to find your deployment ID:
$ akash query txs \
--events "message.sender=$AKASH_ACCOUNT_ADDRESS&message.action=create-deployment" \
--page 1 --limit 999999 \
| jq -r '.txs[] | [ .tx.body.messages[].id[] ] | @csv' | \
tail -5
"akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h","1172777"Query the deployment:
I'm explicitly not "export"'ing the following 3 "SEQ" variables as to avoid 'failed to execute message; message index: 0: Deployment closed' error on 'akash tx deployment create' later. If you get this error, then simply unset AKASH_DSEQ.$ AKASH_DSEQ=1172777
$ AKASH_GSEQ=1
$ AKASH_OSEQ=1
$ akash query deployment get \
--owner $AKASH_ACCOUNT_ADDRESS \
--node $AKASH_NODE \
--dseq $AKASH_DSEQ
deployment:
created_at: "1172779"
deployment_id:
dseq: "1172777"
owner: akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h
state: active
version: R+bMsK3+N7Wtf8MWmUqoIJTgagXPunxFJqV4EXZahYE=
escrow_account:
balance:
amount: "5000000"
denom: uakt
id:
scope: deployment
xid: akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/1172777
owner: akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h
settled_at: "1172791"
state: open
transferred:
amount: "0"
denom: uakt
groups:
- created_at: "1172779"
group_id:
dseq: "1172777"
gseq: 1
owner: akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h
group_spec:
name: akash
requirements:
attributes:
- key: host
value: akash
signed_by:
all_of: []
any_of:
- akash1365yvmc4s7awdyj3n2sav7xfx76adc6dnmlx63
resources:
- count: 1
price:
amount: "10"
denom: uakt
resources:
cpu:
attributes: []
units:
val: "100"
endpoints:
- kind: RANDOM_PORT
memory:
attributes: []
quantity:
val: "536870912"
storage:
attributes: []
quantity:
val: "536870912"
state: openAs you may have noticed the 5 AKT are now under the escrow_account and your akash wallet balance was reduced by that value. These tokens are going to be used by this deployment as soon as you create a lease. And they will be returned as soon as you close the lease / deployment (minus few of them which were used for the lease itself).
More on deployment https://docs.akash.network/guides/deploy
7. Query the market for bids
Now that we want to actually run our deployment, we will need to see what market offers, accept the bid we feel comfortable with, create the lease and send manifest of our deployment to get it up and running.
$ akash query market bid list \
--owner=$AKASH_ACCOUNT_ADDRESS \
--node $AKASH_NODE \
--dseq $AKASH_DSEQ \
--state open
bids:
- bid:
bid_id:
dseq: "1172777"
gseq: 1
oseq: 1
owner: akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h
provider: akash10cl5rm0cqnpj45knzakpa4cnvn5amzwp4lhcal
created_at: "1172780"
price:
amount: "1"
denom: uakt
state: open
escrow_account:
balance:
amount: "50000000"
denom: uakt
id:
scope: bid
xid: akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/1172777/1/1/akash10cl5rm0cqnpj45knzakpa4cnvn5amzwp4lhcal
owner: akash10cl5rm0cqnpj45knzakpa4cnvn5amzwp4lhcal
settled_at: "1172780"
state: open
transferred:
amount: "0"
denom: uakt
- bid:
bid_id:
dseq: "1172777"
gseq: 1
oseq: 1
owner: akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h
provider: akash1f6gmtjpx4r8qda9nxjwq26fp5mcjyqmaq5m6j7
created_at: "1172780"
price:
amount: "1"
denom: uakt
state: open
escrow_account:
balance:
amount: "50000000"
denom: uakt
id:
scope: bid
xid: akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/1172777/1/1/akash1f6gmtjpx4r8qda9nxjwq26fp5mcjyqmaq5m6j7
owner: akash1f6gmtjpx4r8qda9nxjwq26fp5mcjyqmaq5m6j7
settled_at: "1172780"
state: open
transferred:
amount: "0"
denom: uakt
pagination:
next_key: null
total: "0"Set the AKASH_PROVIDER environment variable to the one you picked from the available market bids:
AKASH_PROVIDER=akash10cl5rm0cqnpj45knzakpa4cnvn5amzwp4lhcalIf akash query market bid list isn't showing any bids, make sure you unset AKASH_PROVIDER environment variable!
8. Create lease / Accept the bid
Create a lease for the bid from the chosen provider above by running:
akash tx market lease create \
--chain-id $AKASH_CHAIN_ID \
--node $AKASH_NODE \
--owner $AKASH_ACCOUNT_ADDRESS \
--dseq $AKASH_DSEQ \
--gseq $AKASH_GSEQ \
--oseq $AKASH_OSEQ \
--provider $AKASH_PROVIDER \
--from default \
--gas-prices="0.025uakt" --gas="auto" --gas-adjustment=1.15Please note that once the lease is created, the provider will begin debiting your deployment's escrow account, even if you have not completed the deployment process by uploading the manifest in the following step.
9. Send the manifest to deploy Solana node
Now, to get your Solana deployment started, you need to send your manifest to the provider:
akash provider send-manifest solana.yml \
--node $AKASH_NODE \
--dseq $AKASH_DSEQ \
--provider $AKASH_PROVIDER \
--from defaultAfter few moments, you will be able to see your deployment:
$ akash provider lease-status \
--node $AKASH_NODE \
--dseq $AKASH_DSEQ \
--provider $AKASH_PROVIDER \
--from default
{
"services": {
"solana": {
"name": "solana",
"available": 1,
"total": 1,
"uris": null,
"observed_generation": 1,
"replicas": 1,
"updated_replicas": 1,
"ready_replicas": 1,
"available_replicas": 1
}
},
"forwarded_ports": {
"solana": [
{
"host": "cluster.sjc1p0.mainnet.akashian.io",
"port": 8899,
"externalPort": 32509,
"proto": "TCP",
"available": 1,
"name": "solana"
}
]
}
}The endpoint URI is cluster.sjc1p0.mainnet.akashian.io:32509,
now why the port is 32509 rather than 8899 you will find out in the end of this write-up (see nodePort).
10. See the logs of your Solana container on Akashnet
You can see your Solana node's mnemonic seed phrase there.
$ akash \
--node "$AKASH_NODE" \
provider lease-logs \
--dseq "$AKASH_DSEQ" \
--gseq "$AKASH_GSEQ" \
--oseq "$AKASH_OSEQ" \
--provider "$AKASH_PROVIDER" \
--from default \
--follow
[solana-648bbc8979-hrjq6] solana-faucet 1.6.10 (src:5d4654d2; feat:3533521759)
[solana-648bbc8979-hrjq6] solana-genesis 1.6.10 (src:5d4654d2; feat:3533521759)
[solana-648bbc8979-hrjq6] solana-keygen 1.6.10 (src:5d4654d2; feat:3533521759)
[solana-648bbc8979-hrjq6] solana-validator 1.6.10 (src:5d4654d2; feat:3533521759)
[solana-648bbc8979-hrjq6] + solana address
[solana-648bbc8979-hrjq6] Error: No such file or directory (os error 2)
[solana-648bbc8979-hrjq6] + echo Generating default keypair
[solana-648bbc8979-hrjq6] Generating default keypair
[solana-648bbc8979-hrjq6] + solana-keygen new --no-passphrase
[solana-648bbc8979-hrjq6] Generating a new keypair
[solana-648bbc8979-hrjq6] Wrote new keypair to /root/.config/solana/id.json
[solana-648bbc8979-hrjq6] ==============================================================================
[solana-648bbc8979-hrjq6] pubkey: 2tpqeMBuqQNd2F6pmQmb2Dyj1A6k2yqqCnWXuWh2pX14
[solana-648bbc8979-hrjq6] ==============================================================================
[solana-648bbc8979-hrjq6] Save this seed phrase to recover your new keypair:
[solana-648bbc8979-hrjq6] wagon layer regret misery divorce wild noodle rent actress reflect sister lift
[solana-648bbc8979-hrjq6] ==============================================================================
...
[solana-648bbc8979-hrjq6] [2021-05-31T13:06:27.045978808Z INFO solana_core::cluster_info]
[solana-648bbc8979-hrjq6] IP Address |Age(ms)| Node identifier | Version |Gossip| TPU |TPUfwd| TVU |TVUfwd|Repair|ServeR|ShredVer
[solana-648bbc8979-hrjq6] ------------------+-------+----------------------------------------------+---------+------+------+------+------+------+------+------+--------
[solana-648bbc8979-hrjq6] 127.0.0.1 me| 4558 | 3QDTDmtqV3rLDdkLVPzP5RLg9TdXLFA7QPzv6Eeo72fi | 1.6.10 | 8001 | 8003 | 8004 | 8000 | 8002 | 8006 | 8007 | 48122
[solana-648bbc8979-hrjq6] Nodes: 1
[solana-648bbc8979-hrjq6]
[solana-648bbc8979-hrjq6] RPC Address |Age(ms)| Node identifier | Version | RPC |PubSub|ShredVer
[solana-648bbc8979-hrjq6] ------------------+-------+----------------------------------------------+---------+------+------+--------
[solana-648bbc8979-hrjq6] 127.0.0.1 me| 4558 | 3QDTDmtqV3rLDdkLVPzP5RLg9TdXLFA7QPzv6Eeo72fi | 1.6.10 | 8899 | 8900 | 48122
[solana-648bbc8979-hrjq6] RPC Enabled Nodes: 1
...11. Query your Solana node on Akashnet
Using Solana's json RPC API you can query your Solana node:
$ curl -s http://cluster.sjc1p0.mainnet.akashian.io:32509 -X POST -H "Content-Type: application/json" -d '
{"jsonrpc":"2.0","id":1, "method":"getBlockTime","params":[5]}
' | jq -r '.result | todate'
2021-05-31T13:05:39Z
$ curl -s http://cluster.sjc1p0.mainnet.akashian.io:32509 -X POST -H "Content-Type: application/json" -d '
{"jsonrpc":"2.0","id":1, "method":"getIdentity"}
'
{"jsonrpc":"2.0","result":{"identity":"3QDTDmtqV3rLDdkLVPzP5RLg9TdXLFA7QPzv6Eeo72fi"},"id":1}
$ curl -s http://cluster.sjc1p0.mainnet.akashian.io:32509 -X POST -H "Content-Type: application/json" -d '
{"jsonrpc":"2.0", "id":1, "method":"getClusterNodes"}
' | jq .
{
"jsonrpc": "2.0",
"result": [
{
"featureSet": 3533521759,
"gossip": "127.0.0.1:8001",
"pubkey": "3QDTDmtqV3rLDdkLVPzP5RLg9TdXLFA7QPzv6Eeo72fi",
"rpc": "127.0.0.1:8899",
"shredVersion": 48122,
"tpu": "127.0.0.1:8003",
"version": "1.6.10"
}
],
"id": 1
}12. Terminating the deployment
When you are done testing, you can terminate your deployment.
akash tx deployment close \
--node $AKASH_NODE \
--chain-id $AKASH_CHAIN_ID \
--dseq $AKASH_DSEQ \
--owner $AKASH_ACCOUNT_ADDRESS \
--from default \
--gas-prices="0.025uakt" --gas="auto" --gas-adjustment=1.1513. Depositing more AKT for the deployment
If you are happy with your deployment and would not want it to stop one day, you can deposit more AKT tokens for the escrow account:
akash tx deployment deposit \
--from default \
--chain-id $AKASH_CHAIN_ID \
--node $AKASH_NODE 1000000uakt \
--dseq $AKASH_DSEQ \
--gas-prices="0.025uakt" --gas="auto" --gas-adjustment=1.1514. Production Solana validator node
For production Solana validator node deployment, you will need to modify your manifest file as follows, likely also lifting the cpu, memory, storage limits as recommended.
---
version: "2.0"
services:
solana:
image: solanalabs/solana:v1.6.10
expose:
- port: 8899
as: 8899
proto: tcp
to:
- global: true
- port: 8001
as: 8001
proto: tcp
to:
- global: true
- port: 8900
as: 8900
proto: tcp
to:
- global: true
- port: 8010
as: 8010
proto: udp
to:
- global: true
- port: 8011
as: 8011
proto: udp
to:
- global: true
- port: 8012
as: 8012
proto: udp
to:
- global: true
- port: 8013
as: 8013
proto: udp
to:
- global: true
- port: 8014
as: 8014
proto: udp
to:
- global: true
- port: 8015
as: 8015
proto: udp
to:
- global: true
- port: 8016
as: 8016
proto: udp
to:
- global: true
- port: 8017
as: 8017
proto: udp
to:
- global: true
- port: 8018
as: 8018
proto: udp
to:
- global: true
- port: 8019
as: 8019
proto: udp
to:
- global: true
- port: 8020
as: 8020
proto: udp
to:
- global: true
env:
- SOLANA_RUN_SH_CLUSTER_TYPE=mainnet-beta
- |
SOLANA_RUN_SH_VALIDATOR_ARGS=
--trusted-validator 7Np41oeYqPefeNQEHSv1UDhYrehxin3NStELsSKCT4K2
--trusted-validator GdnSyH3YtwcxFvQrVVJMm1JhTS4QVX7MFsX56uJLUfiZ
--trusted-validator DE1bawNcRJB9rVm3buyMVfr8mBEoyyu73NBovf2oXJsJ
--trusted-validator CakcnaRDHka2gXyfbEd2d3xsvkJkqsLw2akB3zsN1D2S
--no-untrusted-rpc
--private-rpc
--entrypoint entrypoint.mainnet-beta.solana.com:8001
--entrypoint entrypoint2.mainnet-beta.solana.com:8001
--entrypoint entrypoint3.mainnet-beta.solana.com:8001
--entrypoint entrypoint4.mainnet-beta.solana.com:8001
--entrypoint entrypoint5.mainnet-beta.solana.com:8001
--expected-genesis-hash 5eykt4UsFv8P8NJdTREpY1vzqKqZKvdpKuc147dw2N9d
--wal-recovery-mode skip_any_corrupted_record
--limit-ledger-size
--dynamic-port-range 8010-8020
profiles:
compute:
solana:
resources:
cpu:
units: 0.1
memory:
size: 512Mi
storage:
size: 512Mi
placement:
akash:
attributes:
host: akash
signedBy:
anyOf:
- "akash1365yvmc4s7awdyj3n2sav7xfx76adc6dnmlx63"
pricing:
solana:
denom: uakt
amount: 10
deployment:
solana:
akash:
profile: solana
count: 1You will likely need to expose more ports, such as the ones specified in --dynamic-port-range 8010-8020 argument.
The only caveat is that it is not possible to export non-http/https (80/443) TCP ports, so-called nodePort ports such as 8899/tcp, 8010:8020/udp directly as-is.
The Akashnet providers use Kubernetes under the hood for actually running your containers. The Kubernetes control plane allocates a port from a range specified by --service-node-port-range flag (default: 30000-32767)(nodePort)
This means the operator will have to first deploy the Solana node and then check what ports the provider's Kubernetes assigned to your Akash deployment.
And then use some load balancer / reverse proxy (i.e. nginx / haprorxy / traefik) to forward to these ports.
You would also need to tell Solana validator to use load-balancer's hostname by specifying it via the --public-rpc-address <HOST:PORT> argument to Solana validator node.
However, the goal was to show the Akashnet rather than the complete production Solana node.
The official Solana documentation is not recommending Docker for the production deployment and even despite that the production Solana node should be running on a 12cores, 128Gi RAM, 500Gi+ SSD, Akashnet is not currently providing yet since it's a very new network and there aren't that many providers yet. Not an investment advice.
If you liked this article, please vote for it here https://forum.akash.network/t/twitter-deploy-challenge-solana-on-akash/160
P.S.
Few notes to myself (or to anyone else who wants to pick-up from here and push it further):
- solana docker container runs
solana-run.shhttps://github.com/solana-labs/solana/blob/v1.6.10/sdk/docker-solana/build.sh#L26, https://github.com/solana-labs/solana/blob/v1.6.10/run.sh#L41 - see how feasible to run
solana-sys-tunerbefore actually running the Solana validator https://docs.solana.com/running-validator/validator-start#system-tuning - set ulimit
nofile=700000(looks like akash SDL doesn't support that as of31 May 2021)