Microsoft warned that hackers are hijacking Kubeflow, a machine learning (ML) toolkit for Kubernetes, and using the cloud computing resources running in Microsoft Azure to mine for cryptocurrency.
While Azure Security Center has detected “multiple” attack campaigns against Kubernetes clusters in the past, this is the first time it has seen an attack that specifically targets Kubeflow environments.
In April, Azure Security Center discovered a large-scale attack against Kubernetes clusters running on top of Azure that used exposed Kubernetes dashboards to deploy cryptocurrency miners.
This week Azure Security Center discovered a new campaign in which hackers use Kubeflow as an access vector deploy cryptominers.
Hackers Target Kubeflow in Azure
Kubeflow is an open source project that supports machine learning stacks on Kubernetes. “Nodes that are used for ML tasks are often relatively powerful, and in some cases include GPUs,” wrote Yossi Weizman, a security research software engineer for Azure Security Center, in a blog post. “This fact makes Kubernetes clusters that are used for ML tasks a perfect target for cryptomining campaigns, which was the aim of this attack.”
Because Kubeflow is a containerized service in which the various tasks run as containers in the cluster, if hackers can [gain] access to Kubeflow they have several ways to run a malicious image in the cluster,” Weizman explained.
Additionally, Kubeflow exposes its user interface functionality via a dashboard deployed in the cluster. The dashboard is exposed by Istio ingress gateway, which should only be accessible internally. However, some users modify the setting of the Istio Service to Load Balancer and this exposes it to the public internet.
“By exposing the Service to the internet, users can [gain] access to the dashboard directly,” Weizman wrote. “However, this operation enables insecure access to the Kubeflow dashboard, which allows anyone to perform operations in Kubeflow, including deploying new containers in the cluster.”
Once attackers have access to the dashboard they have multiple ways to deploy a backdoor container in the cluster and they can use this access to run cryptocurrency miners.
‘Any Image Can Be Used to Attack Kubernetes’
These kinds of attacks against container runtime environments aren’t new, said Tsvi Korren, field CTO at Aqua Security, in an email. “The fact that these environments blindly accept commands to pull (download) and run any publicly available image is giving a great incentive to attackers, because once they find an opening they can run basically anything they want,” Korren said. “In this case it was stealing computing resources, but it could have easily taken out data, explored more deeply into the network, or caused major disruption.”
In an earlier interview with SDxCentral, Rani Osnat, VP of strategy at Aqua Security, said Aqua’s threat hunting team started seeing an uptick in sophisticated malware attacks targeting containers a little over a year ago. “Their aim is to attack commercial applications, commercial infrastructure, for monetary gain,” he said. “And the way they do that, they try to run cryptocurrency miners, which is relatively easier to do with containers because the stakes are quite low.”
Korren noted that there are “hundreds if not thousands” of malicious images in public repositories including the Docker Hub. While hackers can use these during an attack, developers may also unknowingly bring these images into their company because they are looking to shortcut a piece of code.
“There needs to be broader awareness that any image out there could be embedded with code used to attack Kubernetes,” he wrote. “This is why at Aqua we recommend that organizations restrict the images that can run only to a pre-approved, scanned, and analyzed set of known images and reject anything else.”
Reference:https://www.sdxcentral.com/articles/news/hackers-cryptojack-microsoft-azure-ml-clusters/2020/06/
