Cluster management is the set of tools and processes that Google uses to control the computing infrastructure in its data centers. It includes allocating resources to different applications on their fleet of computers, looking after software installations and hardware, monitoring, and many other things. John will provide an overview of Google's approach.

In this talk you will find out about the challenges the team faced while doing this, such as handling both non-Docker and Docker builds, image replication, integration with their deployment systems and other challenges when deploying Docker at scale.

Unikernels are complete standalone environments for your application, that do not require any other infrastructure, other than a hypervisor or a minimal boot loader (or a simple secure userspace loader).

Type safety can add a huge amount of security to systems software. As can removing the bits you don't need.

So we can build unikernels together, sharing configuration, code, formal methods, experience, and taking the pieces that are being built, and the pieces that are being adapted from existing systems, and build the future of the operating system.

Specifically, Shannon will discuss:

• How organizations are plugging existing devops pipelines into container orchestration tools to provide instant test environments, and manage releases.

• How scheduling tools, such as Docker Compose and Kubernetes are being adopted and used within organizations to define application stacks.

• How software-defined infrastructure services are making container networking and storage seamless, whether containers are deployed locally, to private clouds or to the public cloud.

• How service catalogs and are being used to define and extend application blueprints within organizations.

In the age of Containers you can take a radically new approach. Containers instantiate in seconds or sub seconds, which means that you can potentially scale in real time by re-purposing your existing capacity.

In this talk Liz will present the learnings from the Force12.io project on what you need to make microscaling work and how this approach performs in a range of container frameworks.

Nomad is a globally aware, distributed scheduler designed to handle any type of workload on any operating system. Developers specify jobs using a high-level HCL specification, and Nomad manages the placement, scheduling, auto-healing and scaling automatically.

In this talk you will explore the Nomad architecture and how it can be used to handle the challenges of scheduling in a modern datacenter.

In this session, you will discover some of the core architectural principles underlying Amazon ECS, a highly scalable, high performance service to run and manage distributed applications using the Docker container engine. You will walk through a number of patterns used by our customers to run their microservices platforms, to run batch jobs, and for deployments and continuous integration. You will explore the advanced scheduling capabilities of Amazon ECS and dive deep into the Amazon ECS Service Scheduler, which optimizes for long-running applications by monitoring container health, restarting failed containers, and load balancing across containers.

In this presentation, you'll learn about:

• Security vulnerability (CVEs) and Configuration issues (CCEs) notifications and checks

• Automating vulnerability management, security management, and compliance checking with OpenSCAP

• Scanning online and offline Containers and Virtual Machines with OpenSCAP

• Generating and understanding OpenSCAP audit reports

• Customizing OpenSCAP profiles

Sundial offers an easy to setup solution to job scheduling without the infrastructural overhead of Chronos. It does so by leveraging AWS ECS which offers a container service for Docker containers as tasks or services. Features include viewing live logs for running jobs and saved logs for finished jobs. It provides dependency management between jobs and visualization of dependency graph showing status of failed, succeeded and running jobs. The only requirements are AWS and Docker Registry. Infrastructure is provisioned entirely by Cloudformation and is automatically configured.

Find out more at https://github.com/gilt/sundial

This talk will showcase Weave Net, Weave Run and Weave Scope in Amazon's ECS, demoing a container-based application which uses Weave to provide Load Balancing, Service Discovery and Container Visibility out of the box.

http://weave.works/

Mesos has proven to be a powerful framework for orchestrating containerized applications in data centers with fixed pools of resources. In the cloud, however, pre-provisioning VMs to run Mesos workloads slows our ability to scale applications up and down with changing demand. Worse, VMs impose a performance penalty on application performance and risk becoming pets if not managed properly.

True container-native infrastructure can solve this problem by eliminating VMs and running Mesos workloads on bare metal across an entire cloud, rather than on a defined “cluster.” Eliminating the VM increases performance and speeds scaling, but how does this work and how can applications best take advantage of it?

Docker containers start and stop so quickly, and are so lightweight, that you could easily run a dozen of them on your developer work station (e.g. one for a front-end service, one for a back-end service, one for a database, and so on). But what makes Docker even more powerful is that a Docker image will run exactly the same way no matter where you run them. So once you’ve put in the time to make your code work in a Docker image on your local computer, you can ship that image to any other computer and you can be confident that your code will still work when it gets there.

Once you get your Docker image working locally, you can share it with others. You can run docker push to publish your Docker images to the public Docker registry or to a private registry within your company. Or better yet, you can check your Dockerfile into source control and let your continuous integration environment build, test, and push the images automatically. Once the image is published, you can use the docker run command to run that image on any computer, such as another developer’s workstation or in test or in production, and you can be sure that app will work exactly the same way everywhere without anyone having to fuss around with dependencies or configuration. Many hosting providers have first class support for Docker, such as Amazon’s EC2 Container Service and Google’s Container (GKE) Engine.

Once you start using Docker, it’s addictive — it’s liberating to be able to monkey around with different Linux flavors, dependencies, libraries, and configurations, all without leaving your development workstation in a messy state. You can quickly and easily switch from one Docker image to another (e.g. when switching from one project to another), throw an image away if it isn’t working, or use Docker Compose to work with multiple images at the same time (e.g. connect an image that contains a Go app to another image that contains a MySQL database). And you can leverage the thousands of open source images in the Docker Public Registry. For example, instead of building the my-go-app image from scratch and trying to figure out exactly which combination of libraries make Go happy, you could use the pre-built go image which is maintained and tested by the Docker community.

The tutorial serves two purposes. Once you are using Docker containers the next question is how to scale and start containers across multiple Docker hosts, balancing the containers across them. So enters Kubernetes it adds a higher level API to define how containers are logically grouped, allowing to define pools of containers, load balancing and affinity.

Kubernetes is an open source project to manage a cluster of Linux containers as a single system, managing and running Docker containers across multiple hosts, offering co-location of containers, service discovery and replication control. It was started by Google and now it is supported by Kismatic, Mesosphere, Microsoft, RedHat, IBM and Docker amongst many others.

Google has been using container technology for over ten years, starting over 2 billion containers per week. With Kubernetes it shares its container expertise creating an open platform to run containers at scale.

Kubernetes is an amazing project, and highly promising to manage Docker deployments across multiple servers and simplify the execution of long running and distributed Docker containers. By abstracting infrastructure concepts and working on states instead of processes, it provides easy definition of clusters, including self healing capabilities out of the box. In short, Kubernetes makes management of Docker fleets easier.

Patrick hopes that in the future, more and more companies will package their tech stacks as Docker images so that the on-boarding process for new-hires will be reduced to a single docker run or docker-compose up command. Similarly, he hopes that more and more open source projects will be packaged as Docker images so instead of a long series of install instructions in the README, you just use docker run, and have the code working in minutes.

In answer to this, Ben has spent the last year researching the most efficient way to run Containers directly on VMware's hypervisor and recently announced Project Bonneville. In this talk, Ben will take you under the covers of Bonneville to look at questions such as; how you can have a shared Linux kernel where every Container is privileged; how you can have containers without any Linux at all; and how VMware has brought dynamic resource constraints to the notion of a container host.

Whilst extending and rebuilding the company's core platforms using a microservice pattern would deliver their need for change, it also introduced complexity around deployment and development - hence their need for Docker. They truly believe that without containers they would have failed in their mission, and in fact, adopting Docker has given them so much more than the ability to change. It has given them the ability to scale, to geolocate, to develop in a way they never thought possible, and above all it has given them the ability to transform our business.

They went “All In”, every single line of code for our front and back end systems is running in a container and Nic would love to share the story of how they did it.

The talk will cover their overall approach to networking, why they think layer 3 networking (with an IP per container) is the way forward, how they achieve that and, how they differ from an overlay approach. It will also introduce the team's rich security policy engine (distributed firewall) and explain why they think it is so important for a microservice architecture.

To finish up, Spike will run down the current status of their integrations and give you an overview of what you can expect if you want to try Calico with Docker, Kubernetes, Mesos and, by the time of the talk, perhaps some additional platforms.

The project's website is http://www.projectcalico.org/.

rkt is a simple daemon-free tool that enables users to run containerized apps on their systems free of host dependencies. Containers running under rkt execute like regular processes and can be managed using existing process management tools like upstart, systemd, runit and etc.

This presentation will give you an overview of rkt and other essential components of infrastructure that utilizes containers like CoreOS, etcd and Kubernetes. Come and learn how to use these technologies to build performant, reliable, large distributed systems.

In this presentation, Gianluca Borello will cover the current state of the art for container monitoring and visibility, including real use-cases and pros / cons of each. He will then focus on advanced container visibility techniques, such as:

• visualizing a container’s network activity
• understanding detailed resource usage (CPU, memory and disk I/O) of containers and individual processes running inside containers
• following process and user activity inside containers
• collecting logs from multiple containers .

The presentation will include live interaction with container environments and aims to cover two of the potentially most popular subjects for attendees: containerization and performance monitoring. These container monitoring techniques will help DevOps engineers to deploy a containerized infrastructure in production with confidence and peace of mind. Special emphasis will be put on sysdig, an open source container and system troubleshooting tool that the presenter has helped author.

GitHub link: https://github.com/draios/sysdig

This talk will introduce you to Flocker, talk about Docker plugins and finally demonstrate the two working together to achieve the seamless scheduling and migration of stateful database containers using Marathon.

Firmament models the scheduling problem as an optimisation over a flow network. This approach considers all possible assignments concurrently and makes placement decisions that are optimal for the given scheduling policy. Ionel illustrates how this enables practical, useful scheduling policies to be expressed concisely. With the aid of two case studies, you will discover that Firmament makes high-quality decisions, reducing the per-task runtime by 2-4x over the widely-deployed Mesos cluster manager’s default decisions. In order to scale to large clusters, Firmament solves the optimisation problem incrementally. On a Google cluster of 12,500 machines, it makes decisions in 200ms on average, sufficiently fast for interactive applications.

Firmament is an Apache-licensed open-source project and includes both a cluster manager and several schedulers, and we are working on adding support for using Firmament atop Kubernetes and Mesos.

Find out more about Firmament