关于搭建Kubernetes环境可以参考我前面的公众号文章:Centos7下使用kubeadm搭建Kubernetes-v1.14.2,本篇文章主要实现Jenkins在k8s集群的安装、slave节点在k8s内自动创建销毁,通过pipeline实现java项目的持续集成发布。

安装Jenkins服务到K8S集群

使用Dockerfile制作Jenkins镜像

下载war包进行的安装,Dockerfile如下,war包下载地址:https://jenkins.io/zh/download

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
FROM java:8
RUN echo 'hello docker, start build image'

RUN mkdir -p /app
WORKDIR /app

RUN cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN echo "Asia/Shanghai" > /etc/timezone

COPY jenkins.war .

CMD ["java" ,"-Xms1024m","-Xmx1024m", "-jar","/app/jenkins.war"]

制作镜像

1
2
3
4
docker build -t registry.cn-hangzhou.aliyuncs.com/hiningmeng/jenkins:2.176.2  .

docker push registry.cn-hangzhou.aliyuncs.com/hiningmeng/jenkins:2.176.2

K8S安装Jenkins应用

在k8s集群内创建Jenkins工作的namespace,我这边统一放在devops这个ns底下;

1
kubectl create ns devops

我这里把Jenkins工作目录单独挂载到PVC,需要先创建pv-pvc,挂载点是使用的nfs服务,请先创建好服务,jenkins-pv-pvc.yaml如下:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-home-pv
spec:
  capacity:
    storage: 100Gi
  accessModes:
    - ReadWriteMany
  nfs:
    server: 192.168.1.100
    path: "/data/jenkins_home"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-home-pvc
  namespace: devops
spec:
  accessModes: ["ReadWriteMany"]
  resources:
    requests:
      storage: 100Gi

为Jenkins创建单独的ServiceAccount,这里的ClusterRole直接使用的cluster-admin,jenkins-serveraccount.yaml如下;

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: jenkins
  name: jenkins-admin
  namespace: devops
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: jenkins-admin
  labels:
    app: jenkins
subjects:
  - kind: ServiceAccount
    name: jenkins-admin
    namespace: devops
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

编写Deployment文件,我使用到了node标签apps.k8s.icjl/devops,打标签的命令如下:

1
kubectl label node your-node-name apps.k8s.icjl/devops=

jenkins-deployment.yaml如下:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins
  namespace: devops
  labels:
    app: jenkins
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      serviceAccountName: jenkins-admin
      imagePullSecrets:
        - name: ram-secret
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: apps.k8s.icjl/devops
                operator: Exists
      containers:
      - name: jenkins
        image: registry.cn-hangzhou.aliyuncs.com/hiningmeng/jenkins:2.176.2
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - name: jenkins-home
          mountPath: /root/.jenkins
          readOnly: false
        ports:
        - containerPort: 8080
        - containerPort: 50000
      volumes:
      - name: jenkins-home
        persistentVolumeClaim:
          claimName: jenkins-home-pvc

创建service,这边使用了NodePort,jenkins-service.yaml如下;

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: v1
kind: Service
metadata:
  labels:
    app: jenkins
  name: jenkins
  namespace: devops
  annotations:
    prometheus.io/scrape: 'true'
spec:
  type: NodePort
  ports:
  - name: jenkins-web
    port: 8080
    targetPort: 8080
    nodePort: 31442
  - name: jenkins-agent
    port: 50000
    targetPort: 50000
    nodePort: 30005
  selector:
    app: jenkins

也可以使用ingress暴露的方式,jenkins-ingress.yaml如下:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: jenkins
  labels:
    name: jenkins
  namespace: devops
spec:
  rules:
  - host: jenkins.hiningmeng.cn
    http:
      paths:
      - path: /
        backend:
          serviceName: jenkins
          servicePort: 8080

执行

1
2
3
4
5
 kubectl apply -f jenkins-pv-pvc.yaml
 kubectl apply -f jenkins-serveraccount.yaml
 kubectl apply -f jenkins-deployment.yaml
 kubectl apply -f jenkins-service.yaml
 kubectl apply -f jenkins-ingress.yaml

image-20190806145150888