'Tried deploying jenkins in a different namespace, getting issue with kubernetes plugin
Error testing connection https://10.10.5.20:6443: Failure executing: GET at: https://10.10.5.20:6443/api/v1/namespaces/java-app/pods. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods is forbidden: User "system:serviceaccount:java-app:default" cannot list resource "pods" in API group "" in the namespace "java-app".
I tried adding local kubernetes cluster url to the plugin and tested connection. I am getting the above error message.
This is my deployment file
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins
spec:
replicas: 1
selector:
matchLabels:
app: jenkins
template:
metadata:
labels:
app: jenkins
spec:
containers:
- name: jenkins
image: org/jenkins:v4
ports:
- containerPort: 8080
volumeMounts:
- mountPath: /var/run/docker.sock
name: docker-sock
- mountPath: /var/jenkins_home
name: jenkins-home
volumes:
- hostPath:
path: /var/run/docker.sock
name: docker-sock
- name: jenkins-home
emptyDir: { }
imagePullSecrets:
- name: jkdsecret
The same file worked in jenkins namespace but getting issue with org namespace. I modified the Dockerfile a little
FROM jenkins/jenkins:jdk11
USER root
RUN apt-get update && apt-get install -y make wget apt-utils
##Docker installation
RUN curl -fsSLO https://download.docker.com/linux/static/stable/x86_64/docker-19.03.9.tgz \
&& tar xzvf docker-19.03.9.tgz \
&& mv docker/docker /usr/local/bin \
&& rm -r docker docker-19.03.9.tgz
RUN dockerd &
## kubectl installation
RUN wget https://storage.googleapis.com/kubernetes-release/release/v1.20.5/bin/linux/amd64/kubectl
RUN chmod +x kubectl
RUN cp kubectl /usr/bin
## Jenkins plugin installation and setup
ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false
RUN wget https://github.com/jenkinsci/plugin-installation-manager-tool/releases/download/2.12.3/jenkins-plugin-manager-2.12.3.jar
RUN mv jenkins-plugin-manager-2.12.3.jar /usr/share/jenkins/ref/jenkins-plugin-manager.jar
WORKDIR /usr/share/jenkins/ref
COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN java -jar jenkins-plugin-manager.jar -f /usr/share/jenkins/ref/plugins.txt --verbose
Service account
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
namespace: org
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: jenkins
rules:
- apiGroups:
- '*'
resources:
- statefulsets
- services
- replicationcontrollers
- replicasets
- podtemplates
- podsecuritypolicies
- pods
- pods/log
- pods/exec
- podpreset
- poddisruptionbudget
- persistentvolumes
- persistentvolumeclaims
- jobs
- endpoints
- deployments
- deployments/scale
- daemonsets
- cronjobs
- configmaps
- namespaces
- events
- secrets
verbs:
- create
- get
- watch
- delete
- list
- patch
- update
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: jenkins
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: jenkins
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts:jenkins
These are the journal logs
Jan 25 01:27:10 kubemaster kubelet[948]: I0125 01:27:10.956688 948 operation_generator.go:797] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3c1af5c-d6f8-4ec1-83a9-4b96e115bf3a-jenkins-home" (OuterVolumeSpecName: "jenkins-home") pod "a3c1af5c-d6f8-4ec1-83a9-4b96e115bf3a" (UID: "a3c1af5c-d6f8-4ec1-83a9-4b96e115bf3a"). InnerVolumeSpecName "jenkins-home". PluginName "kubernetes.io/empty-dir", VolumeGidValue ""
Jan 25 01:27:10 kubemaster kubelet[948]: I0125 01:27:10.967093 948 reconciler.go:319] Volume detached for volume "jenkins-home" (UniqueName: "kubernetes.io/empty-dir/a3c1af5c-d6f8-4ec1-83a9-4b96e115bf3a-jenkins-home") on node "kubemaster" DevicePath ""
Jan 25 01:41:16 kubemaster dockerd[1234]: time="2022-01-25T01:41:16.795090772+05:30" level=info msg="ignoring event" container=b3b9e488ef1ebe60eca2eee507e263e7768727add66c0ee9aad0c5b11fec8360 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Jan 25 01:41:16 kubemaster containerd[954]: time="2022-01-25T01:41:16.795374828+05:30" level=info msg="shim disconnected" id=b3b9e488ef1ebe60eca2eee507e263e7768727add66c0ee9aad0c5b11fec8360
Jan 25 01:41:16 kubemaster containerd[954]: time="2022-01-25T01:41:16.795570895+05:30" level=error msg="copy shim log" error="read /proc/self/fd/29: file already closed"
Jan 25 01:41:18 kubemaster dockerd[1234]: time="2022-01-25T01:41:18.669727731+05:30" level=info msg="ignoring event" container=e20154e1230138204f9458cb66eb4bad4c6b64326400320fc5106b690a9db1f6 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Jan 25 01:41:18 kubemaster containerd[954]: time="2022-01-25T01:41:18.670178556+05:30" level=info msg="shim disconnected" id=e20154e1230138204f9458cb66eb4bad4c6b64326400320fc5106b690a9db1f6
Jan 25 01:41:18 kubemaster containerd[954]: time="2022-01-25T01:41:18.670342477+05:30" level=error msg="copy shim log" error="read /proc/self/fd/34: file already closed"
Jan 25 01:41:20 kubemaster kubelet[948]: I0125 01:41:20.117084 948 scope.go:95] [topologymanager] RemoveContainer - Container ID: 90575d042feefa9575bac56196a3f82a3591365822b4e5fd20fc94578cfeb312
Jan 25 01:41:20 kubemaster kubelet[948]: I0125 01:41:20.118045 948 scope.go:95] [topologymanager] RemoveContainer - Container ID: e20154e1230138204f9458cb66eb4bad4c6b64326400320fc5106b690a9db1f6
Jan 25 01:41:20 kubemaster kubelet[948]: I0125 01:41:20.143692 948 scope.go:95] [topologymanager] RemoveContainer - Container ID: b3b9e488ef1ebe60eca2eee507e263e7768727add66c0ee9aad0c5b11fec8360
Jan 25 01:41:21 kubemaster kubelet[948]: I0125 01:41:21.212728 948 scope.go:95] [topologymanager] RemoveContainer - Container ID: 4a258ab9424e09add0a690502dd5739756044b65f5b54663f495212ddd8113f2
Jan 25 01:41:23 kubemaster kubelet[948]: E0125 01:41:23.211123 948 remote_runtime.go:332] ContainerStatus "5dd7bb91234a4d9b2217da00556f2f5a1169eea07f9c236c05d1a53a1fd18d23" from runtime service failed: rpc error: code = Unknown desc = Error: No such container: 5dd7bb91234a4d9b2217da00556f2f5a1169eea07f9c236c05d1a53a1fd18d23
Jan 25 01:41:23 kubemaster kubelet[948]: E0125 01:41:23.211197 948 kuberuntime_manager.go:980] getPodContainerStatuses for pod "kube-controller-manager-kubemaster_kube-system(e40212d04c86d5dd84d91a4e84e76fdf)" failed: rpc error: code = Unknown desc = Error: No such container: 5dd7bb91234a4d9b2217da00556f2f5a1169eea07f9c236c05d1a53a1fd18d23
Jan 25 01:41:23 kubemaster kubelet[948]: E0125 01:41:23.218871 948 remote_runtime.go:332] ContainerStatus "bee8bffe366672c654921f8cff0450aaefa6764252843f54a0c209a0bcc29b2d" from runtime service failed: rpc error: code = Unknown desc = Error: No such container: bee8bffe366672c654921f8cff0450aaefa6764252843f54a0c209a0bcc29b2d
Jan 25 01:41:23 kubemaster kubelet[948]: E0125 01:41:23.218937 948 kuberuntime_manager.go:980] getPodContainerStatuses for pod "kube-scheduler-kubemaster_kube-system(0ae46508b5aeed56b7122644106323ce)" failed: rpc error: code = Unknown desc = Error: No such container: bee8bffe366672c654921f8cff0450aaefa6764252843f54a0c209a0bcc29b2d
Jan 25 01:41:28 kubemaster containerd[954]: time="2022-01-25T01:41:28.624948123+05:30" level=info msg="starting signal loop" namespace=moby path=/run/containerd/io.containerd.runtime.v2.task/moby/bee8bffe366672c654921f8cff0450aaefa6764252843f54a0c209a0bcc29b2d pid=29969
Jan 25 01:41:28 kubemaster containerd[954]: time="2022-01-25T01:41:28.883497501+05:30" level=info msg="starting signal loop" namespace=moby path=/run/containerd/io.containerd.runtime.v2.task/moby/5dd7bb91234a4d9b2217da00556f2f5a1169eea07f9c236c05d1a53a1fd18d23 pid=30023
~
Can someone please explain the above log? I have been stuck here. Any help would be very much appreciated. Thanks.
Solution 1:[1]
It seems the service account attached is for jenkins in the org namespace. kindly check the manifest to verify service account system:serviceaccount:java-app:default has permission to list pods in the java-app namespace
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Mekky_Mayata |