Building Docker images in Akash Network supercloud

Typically, running Docker within Docker (DinD) on Akash is not supported. Solutions such as DinD, DooD, and nestybox require your container to be executed in privileged mode, which Akash does not allow for security reasons.

However, there is a way - Kaniko. It lets you build and push Docker images without the need to be ran in a privileged container.

If you don't known Akash, follow through https://docs.akash.network

1. Deploy ssh container in Akash

Replace SSH_PUBKEY key value with your public SSH key so you can ssh to your deployment.

You can use the following deployment manifest (SDL):

---
version: "2.0"

services:
  ssh:
    image: alpine:3.18.4
    env:
      - 'SSH_PUBKEY=ssh-rsa AAAAB3NzaC1yc...' ## TODO: replace with your public SSH key
    command:
      - "sh"
      - "-c"
    args:
      - 'apk update;
      apk add openssh-server;
      ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -N "";
      ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N "";
      mkdir -m700 ~/.ssh;
      echo "$SSH_PUBKEY" | tee ~/.ssh/authorized_keys;
      chmod 0600 ~/.ssh/authorized_keys;
      exec /usr/sbin/sshd -D'
    expose:
      # HTTP
      - port: 80
        as: 80
        to:
          - global: true
      # SSH
      - port: 22
        as: 22
        to:
          - global: true


profiles:
  compute:
    ssh:
      resources:
        cpu:
          units: 1
        memory:
          size: 4Gi
        storage:
          size: 20Gi
  placement:
    akash:
      attributes:
        host: akash
      #signedBy:
      #  anyOf:
      #    - "akash1365yvmc4s7awdyj3n2sav7xfx76adc6dnmlx63"
      pricing:
        ssh:
          denom: uakt
          amount: 10000

deployment:
  ssh:
    akash:
      profile: ssh
      count: 1

2. Install prerequisites

apk --update add ca-certificates go git make bash

3. Create Kaniko dir tree

mkdir -p /kaniko/.docker /kaniko/ssl/certs
cp -pvi /etc/ssl/certs/ca-certificates.crt /kaniko/ssl/certs/
chmod 777 /kaniko
chmod 700 /kaniko/.docker

mkdir /workspace
chmod 0700 /workspace

4. Get Kaniko

cd
git clone --depth=1 -b v1.16.0 https://github.com/GoogleContainerTools/kaniko.git
cd kaniko
make
install out/executor /kaniko/executor

5. (Optional) Configure Docker Hub registry

read -p "Enter your Docker Hub Username: " USER
read -s -p "Enter your Docker Hub Password: " PASS

cat > /kaniko/.docker/config.json << EOF
{
    "auths": {
        "https://index.docker.io/v1/": {
            "auth": "$(echo -n "$USER:$PASS" | base64)"
        }
    }
}
EOF

You can also install the respective docker registry helper if you are using GCR, Amazon ECR or ACR docker registry:

# Get GCR credential helper
GOBIN=/kaniko go install github.com/GoogleCloudPlatform/docker-credential-gcr@latest

# Get Amazon ECR credential helper
GOBIN=/kaniko go install github.com/awslabs/amazon-ecr-credential-helper/ecr-login/cli/docker-credential-ecr-login@latest

# Get ACR docker env credential helper
GOBIN=/kaniko go install github.com/chrismellard/docker-credential-acr-env@latest

6. Build & push some image

Notes:
- replace andrey01/nginx with your Docker Hub <username>/<image>;
- replace index.docker.io with gcr.io or any other Docker Registry you want to use;
- for more Kaniko options please refer to https://github.com/GoogleContainerTools/kaniko;

cd /workspace

cat > Dockerfile <<EOF
FROM nginx:alpine
SHELL ["/bin/sh", "-c"]
RUN sed -i 's/^[ \t]*\(\#\+[ \t]*\)*worker_processes[ \t]\+[^;]\+;/worker_processes 1;/' /etc/nginx/nginx.conf && \\
  echo "test" > /root/file && \\
  echo "This image has been built on $(uname -n) instance." > /usr/share/nginx/html/index.html
EOF

export PATH="$PATH:/kaniko"
export SSL_CERT_DIR=/kaniko/ssl/certs
export DOCKER_CONFIG=/kaniko/.docker/
#export DOCKER_CREDENTIAL_GCR_CONFIG=/kaniko/.config/gcloud/docker_credential_gcr_config.json

executor --reproducible --destination=index.docker.io/andrey01/nginx:tag1 --tar-path=/tmp/image1.tar

# --no-push - skip pushing the image to the remote Docker registry
## executor --reproducible --destination=index.docker.io/andrey01/nginx:tag1 --no-push --tar-path=/tmp/image1.tar