In the daily job routines, the developers are mainly focusing on gathering, implementing and testing the functional requirements so much needed by the business. However, there is a lot more to it, than the functionality, to make it a successfull production solution. One important aspect is the authentication side of the story.
In this article we’ll do a hands-on exercise on securing a simple Kubernetes application using a well known open source project, based on OAuth2 standard, namely oauth2-proxy.
- First we will install a k8s cluster on the local machine and configure it’s ingress controller,
- Then we are going to deploy a sample application in the cluster, making sure we publicly expose its UI,
- Next we will provide a small overview of the
oauth2-proxyproject and its critical configuration options,
- Following that, we will install and configure the
oauth2-proxycontainer in our Kubernetes cluster in order to secure the previously installed UI,
- Finally we will look at some advanced configuration options and their benefits.
Evidently since this exercise is leveraging a Kubernetes cluster, first we need to have access to an instance.
These days you have a tremendous variety of options like:
- running a small k8s instance on your local machine (through kind, k3s, Microk8s, Minikube or other options),
- using an as-a-service flavour from IBM, AWS, Azure, Google or any other cloud services vendors,
- leveraging an existing k8s instance on-prem or in cloud.
Note: For the scope of this exercise we don’t need a multi-node Kubernetes cluster, so even a single-node cluster is sufficient.
Installing a Kubernetes cluster on the local machine
Next, we will install a kubernetes cluster on the local machine using kind. You can follow these instructions:
- Install kind locally, instructions found here or just run:
curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.9.0/kind-darwin-amd64
2. Now that we have kind tool installed locally we need to create a kubernetes cluster. This will create a single node cluster and add
ingress-ready label, preparing it for the ingress-controller installation. Open a terminal and paste the following snippet (which should take 1–2 minutes to create and start the kubernetes cluster).
It takes about 2 minutes for all the cluster’s pods to start, so before moving to the next step make sure they are running:
3. Install ingress nginx in the cluster by running:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/kind/deploy.yaml
This will create a new namespace
ingress-nginx and then deploy the nginx ingress controller container. Make sure the nginx controller is running, you might wait a minute for the pod to be ready.
Updating the cluster’s core-dns
This step is necessary only for our local cluster use-case since we will define specific domains like
dex.localtest.me . When moving to a public accessible kubernetes cluster you will have a DNS that is created by the cloud provider.
This is done in two steps, first we update the core-dns configmap, by adding the two hosts, and then restarting the core-dns deployment to make sure the new configurations are loaded up. To change the host configuration, just run:
kubectl apply -f https://raw.githubusercontent.com/morarucostel/cloud-native-auth/main/custom-dns.yaml
and then force the core-dns deployment refresh by deleting the kube-dns pods:
kubectl delete pod -n kube-system -l k8s-app=kube-dns
Deploying a sample application
Next we want to install an application in our cluster which later we will secure the access to it. Clearly you can use any
hello-world applications out there, but for our exercise we’ll install a small http-echo container exposed through an ingress. The kubernetes resources for the sample application are: a
conservis/hello-world container that will echo the request body, a service and an ingress.
To install it just simply run the following command in the terminal:
kubectl apply -f https://raw.githubusercontent.com/morarucostel/cloud-native-auth/main/hello-world.yaml
hello-world application should start running immediately and be accessible from your local browser. You can now open your browser and just enter http://hello-world.localtest.me . If everything is OK you should see a similar response in your browser.
Now that we have our application running and accessible from the browser, we need to introduce a bit of context about
dex and how they will secure our application with the help of
An overview of oauth2-proxy and dex
OAuth2-Proxy is a open source project that provides authentication using various providers like Google, GitHub, Azure, Facebook and many others as well as any provider that implements OpenID Connect protocol which means … everyone. It is written in go, making it very fast in a kubernetes environment and has a mature community behind it.
Dex is a federated OpenID Connect provider which acts as a portal to other identity providers — IdP through connectors. In this exercise we are going to use it as an local identity provider, with a static user/password.
In a real scenario you will typically use dex with one or more of IBM, Google, GitHub, Azure, Facebook or your own company’s IdP. However, today since we are installing it locally on our machine, we can’t register a
callback url in a major IdP since they requires a public accessible DNS endpoint and we don’t have one on our local machine.
Note: In the last section of this article, I will add the link to another hands-on article that will walk you through the configuration using of a couple of major public IdPs.
It’s important to emphasize that in this exercise we are using only a very small part of dex’s capabilities and as such we are detailing only the important configuration values for a local identity provider.
staticClients section register the oauth2-proxy framework, that we’ll have a look imediatly after, as a oauth2 client. As in any public identity provider when you register an application, at a minimum you need to define a
client-secret and a
callback url. This is what is defined in the
staticClients section. In the
staticPasswords section we define a user/password entry as part of the dex local identity provider. Part of the authentication, we will validate our own credentials against the ones defined in this section. Just note that the hash corresponds to the
Next, lets have a look at the
The first section describe the details of the identity provider, that
oauth2-proxy needs to use for authentication process, in our case the local dex instance. The values from this section have to match the ones defined in the dex identity provider so that the two systems trust each other.
The second section describes the behaviour of the
oauth2-proxy framework in relation to the cookie it sets and the upstream systems.
Note: There are more advanced
oauth2-proxyconfiguration options which are going to be described later in this article.
Installing oauth2-proxy and dex
Now that we have a brief understanding of how this should work, lets install the two frameworks in our cluster. We have defined the necessary kubernetes resources like: ingress definitions, services, configmaps, secrets and deployments for the two frameworks to run, however the most important configuration was described earlier. To install it just simply run the following in the terminal:
kubectl apply -f https://raw.githubusercontent.com/morarucostel/cloud-native-auth/main/oauth2-proxy-dex.yaml
We’ll give it a minute for the containers to be downloaded and started after which you should see the following pods running:
Protecting our ingress using oauth2-proxy
We have our
hello-world application running in our cluster and we were able to access it through the UI. The only thing remaining is to instruct the ingress controller that we want to secure this resource with an external authentication provider. The ingress annotations
auth-signin allow us to use external authentication providers and to protect the ingress resources, also detailed here.
hello-world ingress is currently looking like:
and we need to add the
annotations section, to look like:
In order to update the ingress definition just run:
kubectl apply -f https://raw.githubusercontent.com/morarucostel/cloud-native-auth/main/hello-world-secured-ingress.yaml
Now that the ingress was updated, let’s try again the
hello-world application http://hello-world.localtest.me . Now that the ingress is configured to protect the access, it will redirect the browser to the identity provider, which in this case is
dex (check the url in the brower, is now
dex.localtest.me). Just use the credentials that we configured back at the An overview of oauth2-proxy and dex section which were:
After the successful authentication you will be redirected back to the initial requested page, in our case http://hello-world.localtest.me .
Note: If you want to invalidate the oauth2-proxy session, basically to sign-out from the application, you can invoke http://oauth2-proxy.localtest.me/oauth2/sign_out?rd=http://hello-world.localtest.me endpoint.
Additional configuration of oauth2-proxy
Let’s have a look at the
oauth2-proxy framework in more details and how it works.
Whenever you need to protect the access to a resourcevia the ingress, you annotate it with
auth-signin just like we did before.
The value of the annotation is the
oauth2-proxy‘s endpoint that start the authentication process called
/start. This will instruct the browser through an HTTP/302 to redirect it to the configured identity provider, in our case the local installed
dex or any configured public identity provider. After the end-user is authenticating successfully to the identity provider, the browser is being instructed again to do a redirect to the oauth2-proxy’s
/callback endpoint together with some parameters issued by the identity provider. Part of the last step of the authentication, the
oauth2-proxy has to validate that the information received is comming from the real identity provider and create a cookie acknowledging that the user was authenticated. This is very, very briefly how the OAuth2 protocol is being implemented by this
At the moment of this writing, the
oauth2-proxy is supporting the following identity providers:
Microsoft Azure AD,
Gitea and any generic
OpenID Connect provider.
In addition it brings additional capabilities like: persisting the client session in an internal redis database or in the cookie, basic authentication form via a
htpasswd file, setting and passing user headers, cookie configuration and others. If you want to check the details on how to configure a specific identity provider or to configure the
oauth2-proxy to a more detailed level, I suggest to go through their public documentation.
Introducing the oauth2-proxy helm chart
In our exercise we’ve installed the kubernetes resources using the
kubectl cli. If we would to write the resources by hand, it will be prone to errors due to the duplication of many kubernetes specific sections.
oauth2-proxy framework offers a helm chart to ease up with the process of installing and managing the kubernetes resources.
The helm chart is being maintained in the
oauth2-proxy github repository found here and it can be used as simple as:
helm install stable/oauth2-proxy --name my-release -f my-values.yaml
Conclusions and what’s next…
In this article we went through quite a few concepts,
i.we started by installing on our local machine a single-node kubernetes cluster,
ii.we then installed and exposed through an ingress a
iii.we introduced and installed
dex frameworks and then
iv.secured the access to
Note: Making an end-to-end exercise on a local environment is a very good start with the concepts and technologies. Stay tuned for another hands-on exercise using OpenShift on IBM Cloud cluster.
Big shout out to my colleagues from boomerang-io team for all their hard work and also to the OAuth2-Proxy maintainers for the doing an awsome work.
This one goes to nea Țuca.