What needs improvement with Kubernetes?

Although we face issues when migrating to new versions of Kubernetes, such as misunderstandings on using new features or integration with proxy services, these issues can be addressed with proper preparation.

Kubernetes is open source, which is both beneficial and negative depending on the responsibilities. Supported versions like Red Hat, Amazon, Microsoft, or Google are pricey. It's good for bigger organizations, but for smaller organizations or a few workloads, it may be too heavy, not easy to deploy, and the ROI may be less because it requires a control plane, worker nodes, and multiple VMs to run. It's good for bigger organizations where many applications are run, but overkill for handling one or two small applications.

Perhaps there are some aspects related to its operation that could be improved. One thing I noticed is when you have multiple deployments and your node count increases beyond eight or nine, the container creation process doesn't copy properties correctly. For instance, Kubectl wasn't working as expected with Docker Compose. I discovered that group secrets weren't being copied properly to all deployments. Permissions and other settings were fine, but this specific issue affected one group secret property. Similarly, when using Kafka for messaging, it would send traffic but not to the correct topics. With multiple topics, like contact topics, Kubernetes wouldn't handle the routing properly. These are the kinds of things I've observed where Kubernetes could improve. When sending messages to multiple friends using Kafka, Kubernetes doesn't provide direct access. You can use storage, but it's not ideal. That aspect definitely has room for improvement.

Currently, in Kubernetes, all of the health deployments or monitoring, and the discrete tools need to be configured. Changing this would make it much easier. Otherwise, we have to rely on a external tool to implement the monitoring.

One area where Kubernetes could improve is troubleshooting. The current process for troubleshooting and installation can be challenging, especially with a large ecosystem. Better tools and artificial intelligence capabilities developed to assist with troubleshooting, configuration, and support would be helpful. This improvement would be particularly beneficial for large enterprise customers.

The user-interface in regards to the other solution can be improved.

The platform could be more convenient to use. While the Kubernetes CLI is powerful, the interface needs to be improved. The users often navigate between various third-party IDEs. Thus, a more consolidated or standardized interface could streamline the user experience, allowing easier access without the need to balance between multiple tools.

For us, it's the shared AKS. It's really complex because each workstream has its own set of requirements that need to be satisfied within the shared AKS blueprint. But we need a starting point, so we began with basic Azure Active Directory role assignments and creating Kubernetes-native RBAC roles, like cluster-wide or namespace isolation. The fundamentals need to be there for workstreams to easily understand and adapt when they transition to the shared AKS. The most challenging aspect is cost tracking. How do you keep track of the cost per tenant within the AKS cluster, how much they consume in terms of resources? It's still a work in progress. For dedicated AKS, the difference is that if a workstream has a budget or compliance requirements, they can spin up a dedicated AKS for their applications only. We have a stable solution for that, but the hosting cost for a dedicated AKS, especially if running only a few applications, might not be as cost-effective as a shared AKS, where multiple workstreams can work on a single cluster.

The tool needs to improve its UI. The tool is very complex and basic.

There are several areas where Kubernetes could improve. For example, in one of our database projects, we needed a storage layer that would work on safer sites. Our application is a permanent one that requires low latency and is intensive in terms of networking. It works on every single URL and needs access to the database. After researching several solutions available in the market, we went with Portworx for the database back-end storage layer. However, we encountered an issue when we brought down one of the worker nodes in a cluster of three nodes. The pod that was hosted around that worker node was not responding on other worker nodes, even though it was responding. We found out that there was a feature in the alpha stages in the stable site that could have solved this issue, but we don't enable alpha features in our production environment. Therefore, we increased the replication factor in the storage layer from one to two to avoid this issue. Our application is latency-sensitive and demands low latency in terms of network and response time. So, increasing a replica of the storage level will also cause double the I/O, which has additional costs involved. We did extensive research on that and found that the feature needs to be stabilized; certain improvements are required.

The solution's learning courses for the new users and developers must be easier to understand. Presently, they are very abstract, and it is challenging for users to find data.

There is a large ecosystem of products surrounding Kubernetes, making it difficult to identify the right solution due to the vast number of options.

It would be great if Kubernetes could handle a level of data backup.

The lack of native support for billing and self-service capabilities is an area Kubernetes could improve. This requires the use of third-party integrations or managed services in order for customers to be able to deploy clusters on their own. It would be beneficial to have these features built-in into the Kubernetes platform.

The UI should be improved. It would be helpful if it was more graphical. Kubernetes currently runs perfectly with the Linux environment because it has Docker as a container runtime, and Docker works perfectly with the Linux operating system. It should also be able to run with the MacBook and Windows OS, similar to Linux and it would be helpful if they would include this in the next release.

Some of their services could be improved. Kubernetes deploys containers as ports, and there are some services required to communicate between the ports because communication isn't built into the ports by default. I would also like to have Spark as another distributor service on Kubernetes. Security could be improved. It would be helpful if there were other security modules built into Kubernetes. Security has to be implemented properly.

I'm a beginner, and I recently started working with Kubernetes. As of now, I don't see any bugs. However, it would be better if it could be deployed without coding.

Kubernetes is incredibly complicated, so one area of improvement is the ease of administration. I would like a user interface that you can run to help you debug and diagnose problems and suggest how to configure things.

The price is something they need to improve. I'm not a very technical guy. Graphically, the product could be more friendly for the users. We'd like it if they had some sort of web management tool, I don't know if there is already one out there, however, it would help a lot.

They have a very minimal interface to do certain things and that could be enhanced so that someone who is not as comfortable on CLI can also use the interface and play around with the cluster. Commercial offerings like Red Hat OpenShift offer it, but the open-source community edition from CNCF doesn't. I'd like to see an incubating project there. It's not one organization that is contributing to Kubernetes, it's a CNCF project, i.e. an open-source contributing forum. They could possibly promote some data APIs to the production stage. They have a lot of APIs which are in beta stage which they continue to test. Perhaps it's time to upgrade them to a more product-release stage. I think it would offer peace of mind to customers in terms of stability.

The solution can be improved by adding a management console that will allow the use of a graphical interface to do what is usually done using command line instructions. I would like to have a simplification of the update process, which is currently not straightforward and time-consuming. The security of the solution is in its infancy and needs a lot of work.

Kubernetes should improve its consistency across different types of deployments. My customers tell me that they get much better performance when Kubernetes is deployed on VM versus PaaS services from Azure.

Kubernetes could improve by having better integration with VMware solutions.

Kubernetes could improve security. The security is really hard to deploy with proxies and other elements. Additionally, We have had some issues downloading repos and libraries. In the next release, Kubernetes should develop a good interface for the administration and make the deployment of the solution easier.

Honestly, there is not much I like about Kubernetes. It's very complicated to deal with. I just do it because I have to. There is plenty of room for improvement with the configuration and runtime monitoring. That would make Kubernetes much easier to use.

Maybe it's not the scope of this product, however, some analytics information could be more available through this. Otherwise, we have to integrate Dynatrace or some kind of tool. When it has all the servers maybe it's a different scope and it wouldn't work. Some analytics would be so great, however. We'd like insights on the services and their uses, which are very limited. We have to use a third party and paid services like Dynatrace or AppDynamics. Sometimes what happens is, if we find, let's say, OutOfThread or OutOfMemory, where our threads are blocked. If you are doing real-time analysis, you can find them. However, if it's 24 hours after somebody reports, the product is already restarted. We don't have any information about that. Thread dump and memory dumps are not available. So then we have to wait for another crash to happen. There's a lack of backup storage. That's a daily problem. With Kubernetes, whenever we get this kind of production issue, we are clueless. We can see that time OutOfMemory happened, however, we don't have much information to work with. Therefore, having a thread dump and memory dump, and seeing how many objects were created would be useful. Sometimes we go to drill down. It says CPU utilization is very high. If you go inside, you'll see nothing, no information as to why. Similarly, when it says there were a lot of network errors, however, there is no information available on the network errors. It just says 10% network error, 20% network error. Yet if you drill down, there is no information available. You don't know whether it was a server that timed out, the port was not available, or some other network issue. We need more transparency in that regard. Sometimes the DNS Lookup service does not work very reliably unless you enable cache or something. Recently, I used the latest version of Kubernetes, and DNS cache was available, which was not available in the earlier version. Now we notice we're facing a lot of difficulties, like ENOENT errors, or "Host not found" exceptions. Every day they'd say it was an application problem, however, we ultimately figured out the DNS cache was not working properly. With the latest version, when we enabled it, things sorted out. However, when we were trying to drill down in the Kubernetes, it was not giving any information. There's no clear-cut information here as well as to why this was happening.

There are a couple of improvements there also need to be at play. For example, if we deploy without RBACs to the cluster, it can't be deployed. When there's like GCPW or AWS, there is room. Even though we have deployed the clusters without RBACs enabled, we can add the RBACs later. However, Microsoft has limitations. There are features in Google Cloud or AWS that aren't in Azure. They need to implement a couple more tools in Azure. The application gateway, app gateway, needs to be improved. It's not fully developed yet.

The setup process could be improved as it's quite complex, especially for newbies. In the next release, Kubernetes should include automatic deployment.

Kubernetes can improve by providing a service offering catalog that can be readily populated in Kubernetes. The service catalog, for example, could be a CRM application on Kubernetes or an eCommerce retail application packaged on Kubernetes and to be readily deployable. Instead of somebody trying to figure out all the configurations of hosting this on Kubernetes, if something was readily available, which the developers for these CRM or eCommerce products, they could partner with either AWS, Google, or Azure and make the deployment of such applications readily available on Kubernetes. This would allow very little work for a business to go live. The business can quickly straight away and subscribe, launch, and use. It is not difficult for an IT team to be involved to create an application environment to start up. It's would be much easier for businesses to use it directly and start off the applications.

More automation could improve the product - specifically, it would be useful to be able to shut down any IT machines that are currently not in use.

There are a lot of complexities. They're a lot of components that are working together internally. If you look into the installation methods nowadays, it's better, however, previously, it was a very complex process. It's improving. It could still be better. Currently, we do have a very simple method in order to install Kubernetes. They need to focus on more security internally. The majority of the security is coming from external frameworks which means I need to deploy a third-party framework to improve the security. For example, there's Notary, OPPI, or KubeCon. Basically, there are some areas where I need to take the help of a third party. The solution requires networking dependence. Kubernetes does not have its own networking component. Once again, I need to work with a third party. It is fully integrated, no doubt about that, however, I need to be dependent on third-party components to make it work. I want Kubernetes to improve security-wise and make their own stack available inside the core Kubernetes engine to make the secure implementation. If they can integrate the networking component inside the core component that would be best. With dependency removed it would give more choice to the customer. Currently, they're improving immutable structures and a lot of things. They're coming out with version 1.21 in order to reduce some security issues. They are removing the direct dependency from Docker. There are many areas they're working on. A policy enforcement engine is something people are really looking for, which could be part of the four component vertical port auto scaler. A horizontal port auto scaler is already available, however, a vertical port auto scaler should be available. If there was a built-in solution for login and a monitoring solution, if they can integrate some APIs or drivers where I can attach directly any monitoring tool, that would be great.

Scalability is good but I'd like to see it improved with more user-friendly operability.

Kubernetes could adopt UI-based approach. A UI-based approach would be really useful in the CI/CD pipeline. They should make everything a little bit more user-friendly. For example, when I'm deploying, it would be nice to load my code and be able to see which components need to be connected.

The configuration is a bit complicated. Because the platform provided is so simple, additional configuration is required to get your apps up and running. There are some issues with the upgrades. When updates are released, the older versions are decommissioned. The updates are quite frequent and are lengthy. It takes about an hour each time.

At this time, there is nothing that I would want to be added to a future release. The pricing could be improved. It would be ideal if it was a bit less.

The solution could be more stable.

The dashboard, monitoring, and login need improvements.

This solution is not very easy to use. We are looking also for some tools surrounding this solution to manage the environment and to secure it better. These two are areas that have caused some issues. We want to integrate it with what you call continuous integration and delivery. It must be scalable, cost-effective, more agile when it comes to developing and managing the environment for DevOps. All these things go together, it must be cured to allow better manageability. That is what we all are doing in most large companies. In a future release, the solution could become more like a core engine, in which tools like OpenShift are centered. You could see how all kinds of tools could help to better improve the management, security, or scalability of the product. Additionally, we will need more than the core in our organization, there needs to be more additional management tools moving forward.

They should update Kubernetes more regularly. Kubernetes is open-source and supported by cloud-native communities. But there are other proprietary versions of Kubernetes like VMware, which runs Tanzu with underlying Kubernetes architecture, or Red Hat, which runs OpenShift. These have priority over the open-source project over the last five years. The Cloud Native Foundation is currently out with version number two. The first version came out 14 years ago. We really don't know when we will see another version or improvement with this totally open-source project. Scalability can be improved. It should be flexible enough to run two instances that can be changed immediately to four, six, or eight swiftly. They could also simplify the logging process.

I think Kubernetes should simplify the setup and operation of the product. For now, we don't use many of the features that the solution provides, so there's nothing additional I'd like to see.

For improvements, I would say it's actually still evolving so they are already making a lot of improvements along the way. Each and every release comes with new features so I think they're doing well. One feature I would actually like to see is the network monitoring part. When we talk about communities, it's mostly the computer side. But it does have some enhancements on the networking side which they have recently released. I would like to see more enhancement where we can monitor the networks of the Kubernetes cluster or the Kubernetes workloads.

Some improvements that we would like to see are: * Have reacher built-in features and probably incorporate some features from the community toolset, like KEDA for pod scaling. * Even more tools from the community for monitoring, log collectors, authorization, and authentication. * Have some sort of simplifications for wider adoption. * This product should have a more advanced built-in scheduler that uses real application metrics in the scheduling strategy. * Wider integration with cloud providers in terms of volumes and key management services. * Add support of traffic encryption option from container to container, and Ingress to container.

It would be very interesting if they could introduce a template engine to set dynamic values in the deployment time. It would be ideal if it could be native in Kubernetes as it would be much easier.

That's a good question. I'm not that experienced but there are definitely challenges in Kubernetes, if you are managing the cluster yourself. So doing all the admin work, managing the masters, there are some learning curves involved. If some of those things could be simplified, that would be awesome.

The management needs to be improved.

If you're using the solution on the desktop, you eventually have to download the Azure package and install it before you can actually use the Azure commands in Kubernetes. There are more community packages that have been released, rather than releases by Kubernetes. I understand that it's an open server and people can contribute to it, that's how it works. However, sometimes people get misguided and that's where we need some support. It would make a difference.

It would be useful to have a basic and stable interface for monitoring and quick deployment purposes, especially when the deployments are big like a proof of concept or proof of technology. Currently, you need to use the Kubernetes console for all functionalities. It is not a quick-to-learn product if you are not from a Linux background. You need to be very skilled at Linux to learn it quickly. It took me two to three months because I mostly work with Microsoft products. For people who are not from a Linux background, the learning curve is a little bit longer.

The Kubernetes dashboard can be improved. It is currently a mess. We were using Rancher earlier, and everyone was happy with the dashboard. Right now, we are using Kubernetes, and it's not working with Microsoft workstations. Aks is using mcr.microsoft.com/oss/kubernetes/dashboard:v2.0.0-rc7 for dashboard. It has problems with auth. It constantly deletes tokens in kube/config file. And auth with kube/config file is not working on mac. It does not work on chrome in windows 10. It is still laggy and slow. Auto refresh function is not working correctly and you need to refresh your browser. Older versions have similar problems. There is no restart function such as in rancher. There is no possible to restart or scale more deployments at the same time. You need to write script for that. Graphics design is out of date. After a while of not clicking anywhere it give you 401 and you need to login again.

I would love to see a feature like VMware's vMotion, meaning a workload can be transferred from one host to another without being restarted. While true cloud native applications typically don't need such a feature, there is still a lot of single-container legacy applications out in the field. These applications get unavailable while being rescheduled to another node, for example when doing node maintenance.