DC/OS Enterprise offers a range of features that allow you to secure your cluster and prevent breaches and other attacks. This section provides an overview of the security features and recommendations for hardening your cluster.
The goals of DC/OS security are:
- Isolate the cluster perimeter with strong authentication and authorization across all interfaces.
- Secure and protect the internal cluster communication, containers, and sandboxes.
- Enhance cluster security with support for 3rd party security integrations.
DC/OS is based on a Linux kernel and userspace. The same best practices for securing any Linux system apply to securing DC/OS, including setting correct file permissions, restricting root and normal user accounts, protecting network interfaces with iptables or other firewalls, and regularly applying updates from the Linux distribution used with DC/OS to ensure that system libraries, utilities, and core services like systemd and OpenSSH are secure.
At the highest level we can distinguish three security zones in a DC/OS deployment, namely the admin, private, and public security zones.
The admin zone is accessible via HTTP/HTTPS and SSH connections, and provides access to the master nodes. It also provides reverse proxy access to the other nodes in the cluster via URL routing. For security, the DC/OS cloud template allows configuring a whitelist so that only specific IP address ranges are permitted to access the admin zone.
Access to the admin zone is controlled by the Admin Router.
HTTP requests incoming to your DC/OS cluster are proxied through the Admin Router (using Nginx with OpenResty at its core). The Admin Router denies access to most HTTP endpoints for unauthenticated requests. In order for a request to be authenticated, it needs to present a valid authentication token in its
Authorization header. A token can be obtained by going through the authentication flow.
The private zone is a non-routable network that is only accessible from the admin zone or through the edge router from the public zone. Deployed services are run in the private zone. This zone is where the majority of agent nodes are run.
The optional public zone is where publicly accessible applications are run. Generally, only a small number of agent nodes are run in this zone. The edge router forwards traffic to applications running in the private zone.
The agent nodes in the public zone are labeled with a special role so that only specific tasks can be scheduled here. These agent nodes have both public and private IP addresses and only specific ports should be open in their
A typical deployment, including load balancers is shown below:
You can control DC/OS Enterprise access by resource and operation (create, read, update, delete). The available security modes are disabled, permissive, and strict. Strict mode provides the finest-grained controls. The DC/OS permissions are enforced based on your security mode. The security mode is set during DC/OS installation and can only be changed by performing an upgrade.
|Admin Router permissions (
|Mesos permissions (
|Marathon and Metronome permissions (
|Secret store permissions (
See the permissions reference for a complete description.
This mode is designed to ensure smooth upgrades from earlier versions of DC/OS, but only provides minimal security features and is not intended for production environments. Disabled mode does not provide Marathon or Mesos permissions.
This mode provides some of the security features, but does not include the Mesos permissions.
This mode provides the most robust security posture and requires a significant amount of configuration.
Important: You can only move from
permissive, and from
strict during an upgrade.
You can use either of the following methods to determine the security mode of an existing cluster.
GETrequest to the following endpoint:
http[s]: /mesosphere/dcos//<cluster-url>/dcos-metadata/bootstrap-config.json. Requirements: Your user account must have either the
dcos:adminrouter:ops:metadata fullpermission or the
strict, you must use HTTPS. Review Securing your TLS communications to discover how to obtain the root certificate of your DC/OS CA and provision it to your preferred client.
SSH into your master and view the contents of
All requests from outside of the DC/OS cluster require an authentication token. Depending on your security mode, in-cluster authentication tokens may be required. For more information, see the Service Accounts documentation.
DC/OS provisions masters with ZooKeeper credentials during the bootstrap sequence. This allows the masters to nominate themselves as potential Mesos leaders.
Important: Each cluster will use the same default ZooKeeper credentials unless you change them during an install or upgrade (strongly recommended). See Hardening for more information.
Users can log in by using the DC/OS GUI, the DC/OS CLI, or a programmatic client.
- If you have configured an LDAP directory server, DC/OS will pass the user’s credentials to the LDAP directory server for verification.
- If you have configured a SAML or an OpenID Connect identity provider (IdP), the user passes their credentials directly to the IdP.
Tip: If the user is logging in with the DC/OS GUI, SAML and OpenID Connect providers may discover the necessary login details in a browser cookie. In this case, users will not need to pass their credentials.
The following diagram details the sequence.
When the authentication token expires, the user can re-authenticate to receive another.
When a user logs in with the DC/OS GUI, the Identity and Access Manager plants a cookie that contains the authentication token. While it is protected with an
HttpOnly flag, users should Sign Out at the end of their browser session to clear this cookie.
Note that clearing the cookie does not invalidate the authentication token. If sniffed over an unencrypted connection or extracted from the cookie, someone could use the authentication token to log into DC/OS. To mitigate this risk, we recommend setting the secure flag on the cookie in
strict modes, as discussed in Hardening.
Credentials for cluster-local user accounts (those not using LDAP, SAML, or OpenID Connect) consist of a user name and password that can be used to validate, but not reproduce, user passwords. Passwords are individually salted and cryptographically hashed using crypt(3) SHA-512. This results in one-way hashes that can be used to validate but not reproduce user passwords. To further impede brute force attacks and meet or exceed NIST FIPS security requirements, the hash function performs many iterations using a 128 bit salt length.
Once DC/OS IAM has validated user credentials, an authentication token is returned to the user. The authentication token is then used for further request authentication during the user session. This way the password does not need to be stored in the client and is only sent over the wire immediately after the user enters it. Over the wire, the authentication request is encrypted using TLS. TLS is required and enforced in strict mode, but optional in permissive mode. For more information, see Security Modes.
Service accounts provide an identity for services to authenticate with DC/OS. Service accounts control communication between services and DC/OS components. DC/OS services may require service accounts depending on your security mode.
In strict and permissive security modes, DC/OS automatically provisions DC/OS components (systemd services on the DC/OS nodes) with service accounts during the bootstrap sequence. Service accounts are not available in disabled security mode.
For example, the Mesos agents are provisioned with service accounts that they use to authenticate to the Mesos master. This ensures that only authorized agents can join the Mesos cluster, advertise resources, and get asked to launch tasks.
You can view the systemd service accounts from the Organization -> Service Accounts tab of the DC/OS GUI. These service accounts are prefixed with
Important: Modifying the permissions of any of the automatically provisioned service accounts may cause the service to fail.
In addition to authenticating requests, DC/OS also checks the permissions associated with the account to determine whether the requestor is authorized to access the requested resource.
The following diagram describes the authorization sequence.
OPT sequence in the diagram illustrates how permission enforcement varies by security mode.
The Admin Router and the Secret Store enforce their permissions in all security modes.
Metronome and Marathon enforce their permissions in
strictmodes. However, the enforcement in
permissivemode only occurs if the requestor presents an authentication token, which is optional in
permissivemode. If an in-cluster requestor does not present an authentication token, Metronome and Marathon will act as if the request was made by a user with the
The Mesos masters and agents enforce their permissions only in
The diagram does not show the Secret Store sequence. The Admin Router does not check the permissions on requests to the Secret Store. It routes these requests to the Secret Store, which enforces its own permissions on each request.
For more information about permissions, refer to Managing permissions.
The encryption of DC/OS communications varies according to your security mode.
|Security mode||External communications*||Internode communications|
|Disabled||Only HTTP is supported. HTTP connections are not redirected to HTTPS or vice versa. HTTPS connections will be rejected because the Admin Router is not configured to serve them. If you log in to DC/OS with a password, the password will be transmitted insecurely between the user agent (e.g. web browser or DC/OS CLI) and Admin Router.||Unencrypted|
|Permissive||HTTP and HTTPS are supported. HTTP requests to the root path (e.g.
|Strict||Only HTTPS is supported. All HTTP connections are redirected to HTTPS. If you log in to DC/OS with a password, the password is transmitted securely (requires proper certificate verification, including hostname verification on the client side). If one or more HTTP proxies or load balancers are between the user agent and Admin Router, then the secure password transmission applies to the final communication between Admin Router and the previous proxy or load balancer.||Encryption enforced**|
* Communications with clients outside of the cluster. For example, browsers and the DC/OS CLI.
** Except internode communications between instances of ZooKeeper, which are not encrypted in any security mode. Each master node has a ZooKeeper instance. These ZooKeeper instances communicate periodically to keep their in-memory databases in sync. You can use IPsec to manually encrypt these communications..
Not all existing user services support encryption at this time. If the service supports encryption, you can enable it in
permissive mode. In
strict mode, encryption of user service communications is enforced. As a result, only user services that support encryption can be deployed in
Internode communications occur over TLS 1.2. To ensure browser support, external communications currently accept TLS 1.0, 1.1, and 1.2. These settings are configurable.
For more information, see Securing communication with TLS.
Spaces allow you to:
At a minimum, we recommend using spaces to restrict service access to secrets.
One aspect of spaces involves service and job groups. You can put services and jobs into groups in any security mode. This can help users find the jobs or services that pertain to them.
permissive security modes, you can use permissions to restrict user’s access on a per service/job or service/job group basis.
The secret path controls which services can access it. If you do not specify a path when storing a secret, any service can access it.
Secret paths work in conjunction with service groups to control access. However, you do not need to have service groups to control access to secrets, you can also use the name of the service. The following table provides a few examples to show how it works.
|Secret||Service||Can service access secret?|
- If only a single service requires access to a secret, store the secret in a path that matches the name of the service (e.g.
hdfs/secret). This prevents it from being accessed by other services.
- Service groups begin with
/, while secret paths do not.
To secure sensitive values like private keys, API tokens, and database passwords, DC/OS provides:
DC/OS stores Secret Store data in ZooKeeper encrypted under an unseal key using the Advanced Encryption Standard (AES) algorithm in Galois Counter Mode (GCM). The Secret Store uses the unseal key to encrypt secrets before sending them to ZooKeeper and to decrypt secrets after receiving them from ZooKeeper. This ensures that secrets are encrypted both at rest and in transit. TLS provides an additional layer of encryption on the secrets in transit from ZooKeeper to the Secret Store.
The unseal key is encrypted under a public GPG key. Requests to the Secrets API return only the encrypted unseal key. When the Secret Store becomes sealed, either manually or due to a failure, the private GPG key must be used to decrypt the unseal key and unseal the Secret Store.
As a convenience, DC/OS automatically generates a new 4096-bit GPG keypair during the bootstrap sequence. It uses this keypair to initialize the Secret Store and stores the keypair in ZooKeeper.
The Secret Store is available in all security modes.
By default, you cannot store a secret larger than one megabyte. If you need to exceed this limit, contact Mesosphere support.
We do not support alternate or additional Secret Stores at this time. You should use only the
default Secret Store provided by Mesosphere.
DC/OS allows you to restrict:
User access to secrets: use permissions to control which users can access what secrets and what operations they can perform.
Application access to secrets: use spaces to control which applications can retrieve what secrets.
By default, all tasks will run inside of Docker containers. Please see Deploying a Docker-based Service to Marathon for an example.
The following table identifies the default Linux user in each situation.
|Mesos (UCR)||Task runs under
||Task runs under
||Task runs under
|Docker||Task runs under
||Task runs under
||Task runs under
Docker tasks run under
root by default, but Docker user privileges are confined to the Docker container. Should you wish to change the default task user, modify the Docker container. Please reference the Docker documentation for more information, as well as the user service documentation.
Note: If the fetched file is compressed, the individual files inside will retain the permissions and ownership assigned when the file was compressed and are unaffected by any other configurations or settings.
Managing users and groups
DC/OS Enterprise can manage two types of users:…Read More
Granting Access to the GUI
By default, a new user has no permissions and cannot view the DC/OS GUI. You must grant users and groups access to each tab of the GUI.…Read More
The DC/OS Identity and Access Management system is designed to protect resources via fine-grained authorization. Each protected resource has one associated ACL that declares which principals may perform which actions on a named resource. This is performed according to the whitelisting (deny-by-default) model.…Read More
You can control DC/OS access by resource and operation. See Permissions Management for details on how to control permissions.…Read More
Directory-based authentication via LDAP
If your organization has user records stored in a directory server supporting LDAP, you can configure DC/OS Enterprise to check user credentials against it. This allows you to avoid having to recreate your user accounts within DC/OS.…Read More
Use the DC/OS Enterprise Secret Store to secure sensitive information like database passwords, API tokens, and private keys. Storing secrets in secret paths allows you to restrict which services can retrieve the value.…Read More
Identity provider-based authentication
To provide Single Sign-On (SSO) in your organization, you can configure DC/OS Enterprise to authenticate users against one or more external user identity providers. In contrast to directory-based authentication, the identity provider-based authentication is not as rich (less information available) but more flexible for individual users.…Read More
Service accounts are used in conjunction with public-private key pairs, secrets, permissions, and authentication tokens to provide access for DC/OS services to DC/OS. Service accounts control the communications and DC/OS API actions that the services are permitted to make.…Read More
Tutorial – Restricting Access to DC/OS Service Groups
Demonstrates how to use the DC/OS web interface to achieve multi-tenancy in permissive mode. …Read More
Your cluster will become more secure as you move from disabled to permissive to strict security modes. However, there are a number of settings that you can modify independent of your security mode to increase the security of your cluster. …Read More
Identity and Access Management API
The Identity and Access Management API allows you to manage users, user groups, permissions, and LDAP configuration settings through a RESTful interface. It offers more functionality as the DC/OS GUI.…Read More
Securing communication with TLS
In permissive and strict security modes, your DC/OS certificate authority (CA) signs the TLS certificates and provisions them to systemd-started services during the bootstrap sequence. This accomplishes encrypted communications with no manual intervention. Each DC/OS cluster has its own DC/OS CA and a unique root certificate.…Read More