Skip to content

Installing Nacos on the Kubernetes

In this post I will describe how to install Nacos on the Kubernetes environment. This post require you have some basic kubernetes knowledege and not suitable for very beginner.

What is Nacos

Nacos is an open source software provided by Alibaba, which can help you discover, configure and manage your microseries. it supports two major use cases, service registry and config service. Microservice can automactally and dynamically resigter in Nacos and get application configuration. for detail information, you can check out from Nacos Official website

To use Nacos, I would recommend to deploy Nacos on the kubernetes cluster environment. This will keep Nacos high availability. you can find the official deployment document, in their demo, they are using NFS as persistent storage.

I will install 3-Pod Nacos Cluster on a single node of Kubernetes and use local PV as persistent storage in this post. it sounds wired indeed, but this was one of our projects architecture.

Create Database and import SQL file.

You need to create Nacos database and import SQL file. there are many ways to do this. the simplest is using MySQL utilities. you can find the Nacos SQL file from GitHub

Create local folder

Create local folder on the host as persistent volume destination.

mkdir -p /data/gosysops/localpath-db/nacos/pv1
mkdir -p /data/gosysops/localpath-db/nacos/pv2
mkdir -p /data/gosysops/localpath-db/nacos/pv3

Create Storage Class

To create local volume, a StorageClass is needed. This kind of StorageClass don’t support dynamic provisioning. `WaitForFirstConsumer` is to delay volue binding until Pod scheduling.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: nacos-localpath-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

Note: Please ensure your kubernetes has the permission to access host. The kubernetes created by RKE tool, you need to add following into your `Cluster.yaml`

 kubelet:
   extra_binds:
     - "/data:/data"

Create PV

Create 3 Nacos PersistentVoume, – `nacos-local-pv1, nacos-local-pv1, nacos-local-pv3` ,change their name, path in the file respectively.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nacos-local-pv1
spec:
  capacity:
    storage: 5Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem
  storageClassName: nacos-localpath-storage
  local:
    path: /data/gosysops/localpath-db/nacos/pv1
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
                - master01

Create Nacos deployment

This Yaml consists of three part, Service, ConfigMap and Statefulset.

For Nacos Service, you will be able to define and update related ports.

ConfigMap: you can define database username, password, etc. this this yaml, I had added an extra field `mysql.host: mysql-0.mysql-ha`

Statefulset: there are 2 containers in StatefulSet file, one is init container, which is used to seek nacos other peers. the second pod is nacos server. the affinity is commented out because Nacos is deployed on a single node. at the end, we assign a StorageClassName bound to pod.

apiVersion: v1
kind: Service
metadata:
  name: nacos-headless
  namespace: gosysops-prod
  labels:
    app: nacos
  annotations:
    service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
  ports:
    - port: 8848
      name: server
      targetPort: 8848
    - port: 9848
      name: client-rpc
      targetPort: 9848
    - port: 9849
      name: raft-rpc
      targetPort: 9849
    ## 兼容1.4.x版本的选举端口
    - port: 7848
      name: old-raft-rpc
      targetPort: 7848
  clusterIP: None
  selector:
    app: nacos
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-cm
  namespace: gosysops-prod
data:
  mysql.host: "mysql-0.mysql-ha"
  mysql.db.name: "nacos"
  mysql.port: "3306"
  mysql.user: "nacos"
  mysql.password: "nacos"
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
  namespace: gosysops-prod
spec:
  selector:
    matchLabels:
      app: nacos
  serviceName: nacos-headless
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      #affinity:
      #  podAntiAffinity:
      #    requiredDuringSchedulingIgnoredDuringExecution:
      #      - labelSelector:
      #          matchExpressions:
      #            - key: "app"
      #              operator: In
      #              values:
      #                - nacos
      #        topologyKey: "kubernetes.io/hostname"
      initContainers:
        - name: peer-finder-plugin-install
          image: nacos/nacos-peer-finder-plugin:1.1
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - mountPath: /home/nacos/plugins/peer-finder
              name: data
              subPath: peer-finder
      containers:
        - name: nacos
          imagePullPolicy: IfNotPresent
          image: nacos/nacos-server:1.4.2
          resources:
            requests:
              memory: "3Gi"
              cpu: "500m"
          ports:
            - containerPort: 8848
              name: client-port
            - containerPort: 9848
              name: client-rpc
            - containerPort: 9849
              name: raft-rpc
            - containerPort: 7848
              name: old-raft-rpc
          env:
            - name: NACOS_REPLICAS
              value: "3"
            - name: SERVICE_NAME
              value: "nacos-headless"
            - name: DOMAIN_NAME
              value: "cluster.local"
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: MYSQL_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.host
            - name: MYSQL_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.db.name
            - name: MYSQL_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.port
            - name: MYSQL_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.user
            - name: MYSQL_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.password
            - name: NACOS_SERVER_PORT
              value: "8848"
            - name: NACOS_APPLICATION_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
          volumeMounts:
            - name: data
              mountPath: /home/nacos/plugins/peer-finder
              subPath: peer-finder
            - name: data
              mountPath: /home/nacos/data
              subPath: data
            - name: data
              mountPath: /home/nacos/logs
              subPath: logs
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        storageClassName: "nacos-localpath-storage"
        accessModes: [ "ReadWriteMany" ]
        resources:
          requests:
            storage: 5Gi

Deploy Nacos

After save and apply above files, you are able to see 3 nacos pods in kubenetes by kubectl get pod

Access Nacos

Two ways to access Nacos, Ingress and NodePort. In my internal environment, I created a new service called Nacos-headless-exposed and exposed port 32888 to access.

default username and password is nacos

Summary

You have learned how to deploy Nacos cluster on a single kubernetes node and how to create local valume as persistent storage. I do not usually recommend to use this way. if node fails, Nacos cluster is meaningless.