SlideShare a Scribd company logo
1 of 55
Download to read offline
Six Million Ways To Log In Docker
Dwayne Hoover, Senior Field Engineer
Christian Beedgen, Co-Founder & CTO
December 17th, 2014
Sumo Logic Confidential
Introduction
Sumo Logic Background
What Our Customers Are Telling Us
A Catalog Of Ways To Log In Docker
What We Would Like To Build
Agenda
Sumo Logic Confidential2
Señor Field Engineer at
Sumo Logic since 2013
Former developer and data
warehouse turned poly-
structured data junkie
Let’s Make This Personal - Who We Are
Co-Founder & CTO, Sumo
Logic since 2010
Server guy, Chief Architect,
ArcSight, 2001 – 2009
Dwayne Christian
The Machine Data Cloud
4
Search
Visualize
Predict
Sumo Logic Confidential
Sumo Logic is the only enterprise-grade 100% service-based offering
Sumo Logic Deployment “Architecture”
Sumo Logic Confidential5
Use Cases
Sumo Logic Confidential6
1. Availability &
Performance
2. Security and
Compliance
3. Customer
Analytics
Sumo Logic Confidential7
Container.
I Haz It.
We have one process per container
We like to log to stdout
We have multiple processes per container
We run the Sumo Logic collector on the host
We are looking into using Beanstalk with Docker
We are waiting for Amazon ECS
Everyone here loves Docker
We are logging straight from the application
We are using /dev/log for Syslog
What Our Customers Are Telling Us
Sumo Logic Confidential8
Sumo Logic Confidential9
One size doesn’t (yet?) fit all
It’s not our job to judge
What does the community say?
Let’s figure out how to collect them all!
What We Are Hearing
Sumo Logic Confidential10
Mailing list thread started in 2013
– https://groups.google.com/forum/#!searchin/docker-
dev/logging/docker-dev/3paGTWD6xyw/hvZlnFD5x5sJ
Superseded by Logging Drivers proposal mid-2014
– https://github.com/docker/docker/issues/7195
However, as of now no clear path
– Extension proposal as the way forward for integrating log forwarders?
What Does The Community Say
Sumo Logic Confidential11
Sumo Logic Confidential12
Let’s Jump Right In
Logs are…
– The actual message plus a bunch of meta data
– At scale, the meta data becomes very important
Timestamp
– With date, full year, down to at least milliseconds
– With time zone, ideally as an offset, or identifiable as straight UTC
Docker host info
– FQDN or IP address or both
– Correlate Docker daemon logs with container logs
Container ID
– Need a way to identify the unique instance of course
– With name if possible, sometimes we are just human…
Image ID
– To correlate, potentially, with logs from other containers from the same image
– Name would likely help the human operator as well
Process ID
– To correlate with logs from the process if there’s no other way to identify them
What Should Be In A Log
Sumo Logic Confidential13
Docker captures container stdout to file in JSON format
In /var/lib/docker/containers/[ID]/[ID]-json.log
The docker logs command can spit back the logs
Each invocation returns the full logs all over
But it can also be used to tail the logs
Careful! Stdout logs grow without bound on the host
Consider using logrotate on the Docker host
https://github.com/docker/docker/issues/7333
What Docker Provides
Sumo Logic Confidential14
docker logs –tf –-tail 0 [ID]
Sumo Logic Confidential15
A Catalog of Ways
to Log in Docker.
Log Directly From The Application
Sumo Logic Confidential16
1
Assuming you have control over the application
Use a library that can send Syslog
Or use a vendor library if HTTPS is required
This can work for other stack components as well
Apache can be coerced into sending Syslog
Nginx has an easy way to send error/access to Syslog
So does Postgres, and almost any Java-based app
Log Directly From The Application
Sumo Logic Confidential17
1
If you want to use Sumo Logic…
There’s an image to quickly set up a Syslog collector
Configure your applications to send to the host at 514
Log Directly From The Application
Sumo Logic Confidential18
docker run -d -p 514:514 -p 514:514/udp --name="sumo-logic-collector"
sumologic/collector:latest-syslog [Access ID] [Access key]
1
Pros
– Conceptually pretty straightforward
– Might not even have to change anything
– Syslog includes the container ID as the hostname
Cons
– Need control over the code or at least the configuration
– Every component might need different situps
– HTTPS straight from the app might not include the container ID
– Logging to service without a collector loses data if link is down
Log Directly From The Application
Sumo Logic Confidential19
1
Various application stacks
– http://help.papertrailapp.com/
Log4J
– https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/net/SyslogAppender.html
Apache Web Server
– http://httpd.apache.org/docs/trunk/mod/mod_syslog.html
– https://raymii.org/s/snippets/Apache_access_and_error_log_to_syslog.html
Nginx
– http://nginx.org/en/docs/syslog.html
Postgres
– http://www.postgresql.org/docs/9.1/static/runtime-config-logging.html
Sumo Logic blog on official syslog collector image
– http://www.sumologic.com/blog/company/an-official-docker-image-for-the-sumo-logic-collector
– https://github.com/SumoLogic/sumologic-collector-docker
Log Directly From The Application
Sumo Logic Confidential20
1
Install A File Collector In The Container
Sumo Logic Confidential21
2
It is not terribly uncommon that logs go to files
There’s many ways to tail logs and ship them off
Logstash, Rsyslog, Sumo Logic Collector, Splunk Forwarder, …
Log to volumes to bypass layered file system
Also, logs are not really container state?
Install A File Collector In The Container
Sumo Logic Confidential22
2
Pros
– Conceptually pretty straightforward
– If everything logs to files already, not a big change
– Collectors can be configured as part of building the image
Cons
– One collector per container could be unacceptable overhead
– No container ID included unless collector picks up hostname
Install A File Collector In The Container
Sumo Logic Confidential23
2
Install A File Collector As A Container
Sumo Logic Confidential24
3
Normalize the collector-per-container idea
Create a container that has only the collector
Mount a host directory into that container to collect from
Mount the same directory into each container
Configure the container to write log files to the mount
Configure the collector container to recursively collect
Could collector on the host, but not Docker-native
For example, using the Sumo Logic file collector image
Install A File Collector As A Container
Sumo Logic Confidential25
docker run -v /tmp/clogs:/tmp/clogs -d --name="sumo-logic-collector"
sumologic/collector:latest-file [Access ID] [Access key]
3
What about name clashes in the shared mounted directory?
Create a sub directory named after the container ID!
Assume the Dockerfile ends in:
Then do this in run.sh:
Install A File Collector As A Container
Sumo Logic Confidential26
ENTRYPOINT ["/bin/bash", "run.sh"]
# Create log directory
mkdir -p /tmp/clogs/$HOSTNAME
ln -s /tmp/clogs/$HOSTNAME /tmp/logs
# Do something
echo "ls -la /tmp/clogs"
ls -la /tmp/clogs
echo "ls -la /tmp/logs"
ls -la /tmp/logs
3
What about name clashes in the shared mounted directory?
Create a sub directory named after the container ID!
Assume the Dockerfile ends in:
Then do this in run.sh and observe:
Install A File Collector As A Container
Sumo Logic Confidential27
ENTRYPOINT ["/bin/bash", "run.sh"]
ls -la /tmp/clogs
total 16
drwxr-xr-x 4 root root 4096 Dec 15 23:51 .
drwxrwxrwt 3 root root 4096 Dec 15 23:51 ..
drwxr-xr-x 2 root root 4096 Dec 15 23:51 43da9cc4d050
drwxr-xr-x 2 root root 4096 Dec 15 23:51 7df836a68214
ls -la /tmp/logs
lrwxrwxrwx 1 root root 23 Dec 15 23:51 /tmp/logs -> /tmp/clogs/43da9cc4d050
3
Sumo Logic blog on official collector images
– http://www.sumologic.com/blog/company/an-official-docker-image-
for-the-sumo-logic-collector
– https://github.com/SumoLogic/sumologic-collector-docker
Rainer Gerhards on Rsyslog’s file input module
– http://www.slideshare.net/rainergerhards1/using-wildcards-with-
rsyslogs-file-monitor-imfile
OWASP Log Injection
– https://www.owasp.org/index.php/Log_injection
Install A File Collector As A Container
Sumo Logic Confidential28
3
Pros
– Not terribly hard to understand and setup
– File collection is very common collector functionality and can scale
Cons
– Have to expose a host directory to all containers
– Mounted directory might be considered an attack vector
– Unless performing described sit ups, name clashes likely
Install A File Collector As A Container
Sumo Logic Confidential29
3
Install A Syslog Collector As A Container
Sumo Logic Confidential30
4
If you want to use Syslog, and Sumo Logic…
There’s an image to quickly set up a Syslog collector
Use linking to configure the Syslog location in the containers
Easy to test with
Install A Syslog Collector As A Container
Sumo Logic Confidential31
docker run –d --name="sumo-logic-collector"
sumologic/collector:latest-syslog [Access ID] [Access key]
docker run -it --link sumo-logic-collector:sumo ubuntu /bin/bash
echo "I'm in ur linx" | nc -v -u -w 0 $SUMO_PORT_514_TCP_ADDR $SUMO_PORT_514_TCP_PORT
4
Pros
– Not terribly hard to understand and setup
– Will retain origin hostname and container ID
Cons
– Every component might need different situps for Syslog
Install A Syslog Collector As A Container
Sumo Logic Confidential32
4
Use Host Syslog For Local Syslog
Sumo Logic Confidential33
5
The process(es) in the container already do Syslog
There is some chance that the host is running Syslog daemon
Configure the host Syslog daemon to forward
Mount /dev/log from the host to /dev/log in the container
Now tail the host syslog
Run a container to test if it works
Should see something like this in the tail’ed file
Use Host Syslog For Local Syslog
Sumo Logic Confidential34
docker run -d -v /dev/log:/dev/log [image]
tail -F /var/log/syslog
docker run -v /dev/log:/dev/log ubuntu logger -t schnitzel Now!
Dec 14 16:33:49 ubuntu schnitzel: Now!
5
Pros
– Nothing extra to install if the host has Syslog already
– Host’s Syslog will be collected as well
Cons
– Hostname is set to the receivers hostname, no container ID in the logs
Use Host Syslog For Local Syslog
Sumo Logic Confidential35
5
Use A Syslog Container For Local Syslog
Sumo Logic Confidential36
6
From Jérôme Petazzoni’s blog – use a bind mount!
Create a simple Rsyslog container, claim /dev as a volume
Then run the Syslog container, capturing its /dev in /tmp/syslogdev
Finally, run the containers that log to local
Use A Syslog Container For Local Syslog
Sumo Logic Confidential37
docker run --name syslog -d -v /tmp/syslogdev:/dev [image]
FROM ubuntu:14.04
RUN apt-get update -q
RUN apt-get install rsyslog
CMD rsyslogd -n
VOLUME /dev
VOLUME /var/log
docker run --name [image-name] -d -v /tmp/syslogdev/log:/dev/log [image]
6
Jérôme Petazzoni’s Blog
– http://jpetazzo.github.io/2014/08/24/syslog-docker/
What is a bind mount?
– http://docs.1h.com/Bind_mounts
– http://man7.org/linux/man-pages/man8/mount.8.html
Use A Syslog Container For Local Syslog
Sumo Logic Confidential38
6
Pros
– Removes the need to have and configure Syslog on the host
– Encapsulates Syslog collection in a Docker-native way
Cons
– Hostname is set to the receivers hostname, no container ID in the logs
Use A Syslog Container For Local Syslog
Sumo Logic Confidential39
6
Containers model processes, not machines
Docker persists container stdout on the host
Simply point the collectors’s file collection mechanism to this path
Collector can also be a container, if the above path is mounted
For example, the Sumo file collector image expects logs in /tmp/clogs
Log To Stdout And Use A File Collector
Sumo Logic Confidential40
/var/lib/docker/containers/*/*-json.log
docker run -d -v /var/lib/docker/containers:/tmp/clogs
sumologic/collector:latest-file [Access ID] [Access Key]
7
Pros
– Relatively straightforward to set up
– Container ID available via filename
Cons
– Docker doesn’t bound the stdout logs on disk
– File collector needs to be able to deal with logrotate if used
– Must be willing to live with host directory mounted in container
Log To Stdout And Use A File Collector
Sumo Logic Confidential41
7
Rainer Gerhards on Rsyslog’s file input module
– http://www.slideshare.net/rainergerhards1/using-wildcards-with-
rsyslogs-file-monitor-imfile
Sumo Logic blog on official collector images and Github repo
– http://www.sumologic.com/blog/company/an-official-docker-image-
for-the-sumo-logic-collector
– https://github.com/SumoLogic/sumologic-collector-docker
On using Logrotate with Docker
– https://github.com/docker/docker/issues/7333
Log To Stdout And Use A File Collector
Sumo Logic Confidential42
7
Logspout is a very lightweight container that forwards stdout to syslog
Logspout uses the Docker Event API to track containers coming and going
For each container, Logspout gets the stdout from Docker via API
By default everything gets forwarded to the specified endpoint
Logspout supports routing to different endpoints
Routing rules can be expressed as filters on container name & ID
Logspout also exposes a little HTTP interface to bounce logs back live
We are hacking Logspout to forward to Sumo’s HTTP endpoint as well!
Log To Stdout And Use Logspout
Sumo Logic Confidential43
docker run –d –p 8000:8000 –v /var/run/docker.sock:/tmp/docker.sock
progrium/logspout syslog://[syslog-host]:[syslog-port]
curl localhost:8000/logs
8
Pros
– Trivial to set up and very lightweight
– Adds container ID and name to the logs
– Flexible, optionally persistent routing for complicated cases
Cons
– Docker doesn’t bound the stdout logs on disk
Log To Stdout And Use Logspout
Sumo Logic Confidential44
8
Logspout Github repository
– https://github.com/progrium/logspout
Various Articles
– http://stackengine.com/docker-logs-aggregating-ease/
– http://blog.froese.org/2014/05/15/docker-logspout-and-nginx/
On using Logrotate with Docker
– https://github.com/docker/docker/issues/7333
Log To Stdout And Use Logspout
Sumo Logic Confidential45
8
Collect From Docker Filesystems
Sumo Logic Confidential46
9
Ultimately, all files from container file systems end up on disk
One of my boxes is running AUFS and I can see all files in:
A simple test with tailing a file in a container from the host works…
Collect From Docker Filesystems
Sumo Logic Confidential47
9
/var/lib/docker/aufs/mnt/[Container ID]
Unfortunately, this doesn’t work with Devicemapper
Another box is using devicemapper and I can see all files in:
A simple test with tailing a file in a container from the host works
So now you can slab a file collector on the host and configure it…?
With devicemapper, stopping a container while tailing leads to error on start
This error will persist until the other process (tail) is stopped
And then, a manual umount is required before docker start
Collect From Docker Filesystems
Sumo Logic Confidential48
9
/var/lib/docker/devicemapper/mnt/[Container ID]/rootfs/
Error response from daemon: Cannot start container 6f62be47025d:
Error getting container 6f62be47025d... from driver devicemapper:
Error mounting '/dev/mapper/docker-202:1-277656-6f62be47025d....' on
'/var/lib/docker/devicemapper/mnt/6f62be47025d...': device or
resource busy
Pros
– If legal, it means a lot of existing file collection tools can just be used
Cons
– Could just be a batshit crazy idea and the universe collapses into itself
– Need to find a way to configure file collector per image
Collect From Docker Filesystems
Sumo Logic Confidential49
9
Inject Collector Via Docker Exec
Sumo Logic Confidential50
10
docker exec allows injection of a process into a container
A collector could live in a container, and talk to the Docker daemon
The collector could use the Event API to track containers come and go
Basically, just like Logspout… or put it on the host, I guess
When a container appears, the Exec API could be used to inject a process
The process could run the collection logic, starting with watching paths, etc.
The process could also actually tail the files and send logs to a service
Or, it could send logs back to the collector container via stdout or something
The collector in the container could then do caching, compression, …
Inject Collector Via Docker Exec
Sumo Logic Confidential51
10
Pros
– This could actually be a generic and non-crazy way to collect log files
– There’s a ton of tools that know how to collect from files
Cons
– In reality, will people accept/allow docker exec?
– It basically allows a container to access another container as root
Inject Collector Via Docker Exec
Sumo Logic Confidential52
10
Sumo Logic Confidential
What would Sumo Do?
Something that catches stdout from all containers…
– Logspout does this already!
…and that can tail files in containers in a clean way…
– Container can define which path(s)
…and forward messages via different protocols
– Logspout does Syslog, we are adding HTTP POST
We think the extensions discussion is very relevant!
– More realistic than adding to core Docker codebase?
What We Would Like To Build
Sumo Logic Confidential54
http://ecocatlady.blogspot.com/2012/08/tricks-for-
not-wasting-fresh-produce.html
http://up-ship.com/blog/?p=2456
http://videonem.com/lol-cat-get-now/
http://www.teefury.com/lolcat-taxonomy
Image References
Sumo Logic Confidential55

More Related Content

What's hot

Dockerizing Symfony Applications - Symfony Live Berlin 2014
Dockerizing Symfony Applications - Symfony Live Berlin 2014Dockerizing Symfony Applications - Symfony Live Berlin 2014
Dockerizing Symfony Applications - Symfony Live Berlin 2014D
 
Breaking the RpiDocker challenge
Breaking the RpiDocker challenge Breaking the RpiDocker challenge
Breaking the RpiDocker challenge Nicolas De Loof
 
Getting instantly up and running with Docker and Symfony
Getting instantly up and running with Docker and SymfonyGetting instantly up and running with Docker and Symfony
Getting instantly up and running with Docker and SymfonyAndré Rømcke
 
Dockerize your Symfony application - Symfony Live NYC 2014
Dockerize your Symfony application - Symfony Live NYC 2014Dockerize your Symfony application - Symfony Live NYC 2014
Dockerize your Symfony application - Symfony Live NYC 2014André Rømcke
 
Docker orchestration using core os and ansible - Ansible IL 2015
Docker orchestration using core os and ansible - Ansible IL 2015Docker orchestration using core os and ansible - Ansible IL 2015
Docker orchestration using core os and ansible - Ansible IL 2015Leonid Mirsky
 
Techtalks: taking docker to production
Techtalks: taking docker to productionTechtalks: taking docker to production
Techtalks: taking docker to productionmuayyad alsadi
 
Using docker to develop NAS applications
Using docker to develop NAS applicationsUsing docker to develop NAS applications
Using docker to develop NAS applicationsTerry Chen
 
Intro- Docker Native for OSX and Windows
Intro- Docker Native for OSX and WindowsIntro- Docker Native for OSX and Windows
Intro- Docker Native for OSX and WindowsThomas Chacko
 
Dockerize Me: Distributed PHP applications with Symfony, Docker, Consul and A...
Dockerize Me: Distributed PHP applications with Symfony, Docker, Consul and A...Dockerize Me: Distributed PHP applications with Symfony, Docker, Consul and A...
Dockerize Me: Distributed PHP applications with Symfony, Docker, Consul and A...Alexey Petrov
 
Plug-ins: Building, Shipping, Storing, and Running - Nandhini Santhanam and T...
Plug-ins: Building, Shipping, Storing, and Running - Nandhini Santhanam and T...Plug-ins: Building, Shipping, Storing, and Running - Nandhini Santhanam and T...
Plug-ins: Building, Shipping, Storing, and Running - Nandhini Santhanam and T...Docker, Inc.
 
Docker summit 2015: 以 Docker Swarm 打造多主機叢集環境
Docker summit 2015: 以 Docker Swarm 打造多主機叢集環境Docker summit 2015: 以 Docker Swarm 打造多主機叢集環境
Docker summit 2015: 以 Docker Swarm 打造多主機叢集環境謝 宗穎
 
Containerd: Building a Container Supervisor by Michael Crosby
Containerd: Building a Container Supervisor by Michael CrosbyContainerd: Building a Container Supervisor by Michael Crosby
Containerd: Building a Container Supervisor by Michael CrosbyDocker, Inc.
 
Using Docker in the Real World
Using Docker in the Real WorldUsing Docker in the Real World
Using Docker in the Real WorldTim Haak
 
Fluentd and PHP
Fluentd and PHPFluentd and PHP
Fluentd and PHPchobi e
 
Docker 原理與實作
Docker 原理與實作Docker 原理與實作
Docker 原理與實作kao kuo-tung
 
Small, Simple, and Secure: Alpine Linux under the Microscope
Small, Simple, and Secure: Alpine Linux under the MicroscopeSmall, Simple, and Secure: Alpine Linux under the Microscope
Small, Simple, and Secure: Alpine Linux under the MicroscopeDocker, Inc.
 
Under the Hood with Docker Swarm Mode - Drew Erny and Nishant Totla, Docker
Under the Hood with Docker Swarm Mode - Drew Erny and Nishant Totla, DockerUnder the Hood with Docker Swarm Mode - Drew Erny and Nishant Totla, Docker
Under the Hood with Docker Swarm Mode - Drew Erny and Nishant Totla, DockerDocker, Inc.
 
PHP development with Docker
PHP development with DockerPHP development with Docker
PHP development with DockerYosh de Vos
 
CoreOS Overview and Current Status
CoreOS Overview and Current StatusCoreOS Overview and Current Status
CoreOS Overview and Current StatusSreenivas Makam
 
What should be PID 1 in a container ? by Ranjith Rajaram for #rootConf 2017
What should be PID 1 in a container ? by Ranjith Rajaram for #rootConf 2017What should be PID 1 in a container ? by Ranjith Rajaram for #rootConf 2017
What should be PID 1 in a container ? by Ranjith Rajaram for #rootConf 2017Ranjith Rajaram
 

What's hot (20)

Dockerizing Symfony Applications - Symfony Live Berlin 2014
Dockerizing Symfony Applications - Symfony Live Berlin 2014Dockerizing Symfony Applications - Symfony Live Berlin 2014
Dockerizing Symfony Applications - Symfony Live Berlin 2014
 
Breaking the RpiDocker challenge
Breaking the RpiDocker challenge Breaking the RpiDocker challenge
Breaking the RpiDocker challenge
 
Getting instantly up and running with Docker and Symfony
Getting instantly up and running with Docker and SymfonyGetting instantly up and running with Docker and Symfony
Getting instantly up and running with Docker and Symfony
 
Dockerize your Symfony application - Symfony Live NYC 2014
Dockerize your Symfony application - Symfony Live NYC 2014Dockerize your Symfony application - Symfony Live NYC 2014
Dockerize your Symfony application - Symfony Live NYC 2014
 
Docker orchestration using core os and ansible - Ansible IL 2015
Docker orchestration using core os and ansible - Ansible IL 2015Docker orchestration using core os and ansible - Ansible IL 2015
Docker orchestration using core os and ansible - Ansible IL 2015
 
Techtalks: taking docker to production
Techtalks: taking docker to productionTechtalks: taking docker to production
Techtalks: taking docker to production
 
Using docker to develop NAS applications
Using docker to develop NAS applicationsUsing docker to develop NAS applications
Using docker to develop NAS applications
 
Intro- Docker Native for OSX and Windows
Intro- Docker Native for OSX and WindowsIntro- Docker Native for OSX and Windows
Intro- Docker Native for OSX and Windows
 
Dockerize Me: Distributed PHP applications with Symfony, Docker, Consul and A...
Dockerize Me: Distributed PHP applications with Symfony, Docker, Consul and A...Dockerize Me: Distributed PHP applications with Symfony, Docker, Consul and A...
Dockerize Me: Distributed PHP applications with Symfony, Docker, Consul and A...
 
Plug-ins: Building, Shipping, Storing, and Running - Nandhini Santhanam and T...
Plug-ins: Building, Shipping, Storing, and Running - Nandhini Santhanam and T...Plug-ins: Building, Shipping, Storing, and Running - Nandhini Santhanam and T...
Plug-ins: Building, Shipping, Storing, and Running - Nandhini Santhanam and T...
 
Docker summit 2015: 以 Docker Swarm 打造多主機叢集環境
Docker summit 2015: 以 Docker Swarm 打造多主機叢集環境Docker summit 2015: 以 Docker Swarm 打造多主機叢集環境
Docker summit 2015: 以 Docker Swarm 打造多主機叢集環境
 
Containerd: Building a Container Supervisor by Michael Crosby
Containerd: Building a Container Supervisor by Michael CrosbyContainerd: Building a Container Supervisor by Michael Crosby
Containerd: Building a Container Supervisor by Michael Crosby
 
Using Docker in the Real World
Using Docker in the Real WorldUsing Docker in the Real World
Using Docker in the Real World
 
Fluentd and PHP
Fluentd and PHPFluentd and PHP
Fluentd and PHP
 
Docker 原理與實作
Docker 原理與實作Docker 原理與實作
Docker 原理與實作
 
Small, Simple, and Secure: Alpine Linux under the Microscope
Small, Simple, and Secure: Alpine Linux under the MicroscopeSmall, Simple, and Secure: Alpine Linux under the Microscope
Small, Simple, and Secure: Alpine Linux under the Microscope
 
Under the Hood with Docker Swarm Mode - Drew Erny and Nishant Totla, Docker
Under the Hood with Docker Swarm Mode - Drew Erny and Nishant Totla, DockerUnder the Hood with Docker Swarm Mode - Drew Erny and Nishant Totla, Docker
Under the Hood with Docker Swarm Mode - Drew Erny and Nishant Totla, Docker
 
PHP development with Docker
PHP development with DockerPHP development with Docker
PHP development with Docker
 
CoreOS Overview and Current Status
CoreOS Overview and Current StatusCoreOS Overview and Current Status
CoreOS Overview and Current Status
 
What should be PID 1 in a container ? by Ranjith Rajaram for #rootConf 2017
What should be PID 1 in a container ? by Ranjith Rajaram for #rootConf 2017What should be PID 1 in a container ? by Ranjith Rajaram for #rootConf 2017
What should be PID 1 in a container ? by Ranjith Rajaram for #rootConf 2017
 

Viewers also liked

Retelling nonfiction
Retelling nonfictionRetelling nonfiction
Retelling nonfictionEmily Kissner
 
How to Build a High Performance Application Using Cloud Foundry and Redis (Cl...
How to Build a High Performance Application Using Cloud Foundry and Redis (Cl...How to Build a High Performance Application Using Cloud Foundry and Redis (Cl...
How to Build a High Performance Application Using Cloud Foundry and Redis (Cl...VMware Tanzu
 
IBM Bluemix Nice meetup #5 - 20170504 - Container Service based on Kubernetes
IBM Bluemix Nice meetup #5 - 20170504 - Container Service based on KubernetesIBM Bluemix Nice meetup #5 - 20170504 - Container Service based on Kubernetes
IBM Bluemix Nice meetup #5 - 20170504 - Container Service based on KubernetesIBM France Lab
 
Do we need a bigger dev data culture
Do we need a bigger dev data cultureDo we need a bigger dev data culture
Do we need a bigger dev data cultureSimon Dittlmann
 
Monitoring and tuning your chef server - chef conf talk
Monitoring and tuning your chef server - chef conf talk Monitoring and tuning your chef server - chef conf talk
Monitoring and tuning your chef server - chef conf talk Andrew DuFour
 
IoT and Big Data
IoT and Big DataIoT and Big Data
IoT and Big Datasabnees
 
Docker swarm-mike-goelzer-mv-meetup-45min-workshop 02242016 (1)
Docker swarm-mike-goelzer-mv-meetup-45min-workshop 02242016 (1)Docker swarm-mike-goelzer-mv-meetup-45min-workshop 02242016 (1)
Docker swarm-mike-goelzer-mv-meetup-45min-workshop 02242016 (1)Michelle Antebi
 
Docker security introduction-task-2016
Docker security introduction-task-2016Docker security introduction-task-2016
Docker security introduction-task-2016Ricardo Gerardi
 
Elks for analysing performance test results - Helsinki QA meetup
Elks for analysing performance test results - Helsinki QA meetupElks for analysing performance test results - Helsinki QA meetup
Elks for analysing performance test results - Helsinki QA meetupAnoop Vijayan
 
All you need to know about Orient Me
All you need to know about Orient MeAll you need to know about Orient Me
All you need to know about Orient MeLetsConnect
 
Complex realtime event analytics using BigQuery @Crunch Warmup
Complex realtime event analytics using BigQuery @Crunch WarmupComplex realtime event analytics using BigQuery @Crunch Warmup
Complex realtime event analytics using BigQuery @Crunch WarmupMárton Kodok
 
Cisco Network Functions Virtualization Infrastructure (NFVI)
Cisco Network Functions Virtualization Infrastructure (NFVI)Cisco Network Functions Virtualization Infrastructure (NFVI)
Cisco Network Functions Virtualization Infrastructure (NFVI)Cisco Russia
 
Introduction to Data Modeling in Cassandra
Introduction to Data Modeling in CassandraIntroduction to Data Modeling in Cassandra
Introduction to Data Modeling in CassandraJim Hatcher
 
What's new in oracle ORAchk & EXAchk 12.2.0.1.2
What's new in oracle ORAchk & EXAchk 12.2.0.1.2What's new in oracle ORAchk & EXAchk 12.2.0.1.2
What's new in oracle ORAchk & EXAchk 12.2.0.1.2Gareth Chapman
 
Fluentd v1.0 in a nutshell
Fluentd v1.0 in a nutshellFluentd v1.0 in a nutshell
Fluentd v1.0 in a nutshellN Masahiro
 
Microsoft Microservices
Microsoft MicroservicesMicrosoft Microservices
Microsoft MicroservicesChase Aucoin
 

Viewers also liked (20)

Retelling nonfiction
Retelling nonfictionRetelling nonfiction
Retelling nonfiction
 
Arquitecturas de microservicios - Medianet Software
Arquitecturas de microservicios   -  Medianet SoftwareArquitecturas de microservicios   -  Medianet Software
Arquitecturas de microservicios - Medianet Software
 
How to Build a High Performance Application Using Cloud Foundry and Redis (Cl...
How to Build a High Performance Application Using Cloud Foundry and Redis (Cl...How to Build a High Performance Application Using Cloud Foundry and Redis (Cl...
How to Build a High Performance Application Using Cloud Foundry and Redis (Cl...
 
IBM Bluemix Nice meetup #5 - 20170504 - Container Service based on Kubernetes
IBM Bluemix Nice meetup #5 - 20170504 - Container Service based on KubernetesIBM Bluemix Nice meetup #5 - 20170504 - Container Service based on Kubernetes
IBM Bluemix Nice meetup #5 - 20170504 - Container Service based on Kubernetes
 
Do we need a bigger dev data culture
Do we need a bigger dev data cultureDo we need a bigger dev data culture
Do we need a bigger dev data culture
 
Monitoring and tuning your chef server - chef conf talk
Monitoring and tuning your chef server - chef conf talk Monitoring and tuning your chef server - chef conf talk
Monitoring and tuning your chef server - chef conf talk
 
IoT and Big Data
IoT and Big DataIoT and Big Data
IoT and Big Data
 
Docker swarm-mike-goelzer-mv-meetup-45min-workshop 02242016 (1)
Docker swarm-mike-goelzer-mv-meetup-45min-workshop 02242016 (1)Docker swarm-mike-goelzer-mv-meetup-45min-workshop 02242016 (1)
Docker swarm-mike-goelzer-mv-meetup-45min-workshop 02242016 (1)
 
Docker security introduction-task-2016
Docker security introduction-task-2016Docker security introduction-task-2016
Docker security introduction-task-2016
 
Elks for analysing performance test results - Helsinki QA meetup
Elks for analysing performance test results - Helsinki QA meetupElks for analysing performance test results - Helsinki QA meetup
Elks for analysing performance test results - Helsinki QA meetup
 
All you need to know about Orient Me
All you need to know about Orient MeAll you need to know about Orient Me
All you need to know about Orient Me
 
Complex realtime event analytics using BigQuery @Crunch Warmup
Complex realtime event analytics using BigQuery @Crunch WarmupComplex realtime event analytics using BigQuery @Crunch Warmup
Complex realtime event analytics using BigQuery @Crunch Warmup
 
Cisco Network Functions Virtualization Infrastructure (NFVI)
Cisco Network Functions Virtualization Infrastructure (NFVI)Cisco Network Functions Virtualization Infrastructure (NFVI)
Cisco Network Functions Virtualization Infrastructure (NFVI)
 
Heelal
HeelalHeelal
Heelal
 
Introduction to Data Modeling in Cassandra
Introduction to Data Modeling in CassandraIntroduction to Data Modeling in Cassandra
Introduction to Data Modeling in Cassandra
 
Question 7
Question 7Question 7
Question 7
 
Spring Batch
Spring BatchSpring Batch
Spring Batch
 
What's new in oracle ORAchk & EXAchk 12.2.0.1.2
What's new in oracle ORAchk & EXAchk 12.2.0.1.2What's new in oracle ORAchk & EXAchk 12.2.0.1.2
What's new in oracle ORAchk & EXAchk 12.2.0.1.2
 
Fluentd v1.0 in a nutshell
Fluentd v1.0 in a nutshellFluentd v1.0 in a nutshell
Fluentd v1.0 in a nutshell
 
Microsoft Microservices
Microsoft MicroservicesMicrosoft Microservices
Microsoft Microservices
 

Similar to 6 Million Ways To Log In Docker - NYC Docker Meetup 12/17/2014

Comprehensive Monitoring for Docker
Comprehensive Monitoring for DockerComprehensive Monitoring for Docker
Comprehensive Monitoring for DockerChristian Beedgen
 
Securing Containers - Sathyajit Bhat - Adobe
Securing Containers - Sathyajit Bhat - AdobeSecuring Containers - Sathyajit Bhat - Adobe
Securing Containers - Sathyajit Bhat - AdobeCodeOps Technologies LLP
 
Docker Security Overview
Docker Security OverviewDocker Security Overview
Docker Security OverviewSreenivas Makam
 
Real World Experience of Running Docker in Development and Production
Real World Experience of Running Docker in Development and ProductionReal World Experience of Running Docker in Development and Production
Real World Experience of Running Docker in Development and ProductionBen Hall
 
Docker Runtime Security
Docker Runtime SecurityDocker Runtime Security
Docker Runtime SecuritySysdig
 
Docker London: Container Security
Docker London: Container SecurityDocker London: Container Security
Docker London: Container SecurityPhil Estes
 
Powercoders · Docker · Fall 2021.pptx
Powercoders · Docker · Fall 2021.pptxPowercoders · Docker · Fall 2021.pptx
Powercoders · Docker · Fall 2021.pptxIgnacioTamayo2
 
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...Mihai Criveti
 
Docker Essentials Workshop— Innovation Labs July 2020
Docker Essentials Workshop— Innovation Labs July 2020Docker Essentials Workshop— Innovation Labs July 2020
Docker Essentials Workshop— Innovation Labs July 2020CloudHero
 
Unraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production CloudUnraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production CloudSalman Baset
 
Tokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityTokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityPhil Estes
 
Jump into Squeak - Integrate Squeak projects with Docker & Github
Jump into Squeak - Integrate Squeak projects with Docker & GithubJump into Squeak - Integrate Squeak projects with Docker & Github
Jump into Squeak - Integrate Squeak projects with Docker & Githubhubx
 
Container Monitoring with Sysdig
Container Monitoring with SysdigContainer Monitoring with Sysdig
Container Monitoring with SysdigSreenivas Makam
 
Kubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with PodmanKubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with PodmanMihai Criveti
 
How Secure Is Your Container? ContainerCon Berlin 2016
How Secure Is Your Container? ContainerCon Berlin 2016How Secure Is Your Container? ContainerCon Berlin 2016
How Secure Is Your Container? ContainerCon Berlin 2016Phil Estes
 
Docker security: Rolling out Trust in your container
Docker security: Rolling out Trust in your containerDocker security: Rolling out Trust in your container
Docker security: Rolling out Trust in your containerRonak Kogta
 
Accelerate your development with Docker
Accelerate your development with DockerAccelerate your development with Docker
Accelerate your development with DockerAndrey Hristov
 
Accelerate your software development with Docker
Accelerate your software development with DockerAccelerate your software development with Docker
Accelerate your software development with DockerAndrey Hristov
 
Docker Introduction.pdf
Docker Introduction.pdfDocker Introduction.pdf
Docker Introduction.pdfOKLABS
 

Similar to 6 Million Ways To Log In Docker - NYC Docker Meetup 12/17/2014 (20)

Comprehensive Monitoring for Docker
Comprehensive Monitoring for DockerComprehensive Monitoring for Docker
Comprehensive Monitoring for Docker
 
Securing Containers - Sathyajit Bhat - Adobe
Securing Containers - Sathyajit Bhat - AdobeSecuring Containers - Sathyajit Bhat - Adobe
Securing Containers - Sathyajit Bhat - Adobe
 
Docker Security Overview
Docker Security OverviewDocker Security Overview
Docker Security Overview
 
Real World Experience of Running Docker in Development and Production
Real World Experience of Running Docker in Development and ProductionReal World Experience of Running Docker in Development and Production
Real World Experience of Running Docker in Development and Production
 
Docker Runtime Security
Docker Runtime SecurityDocker Runtime Security
Docker Runtime Security
 
Docker London: Container Security
Docker London: Container SecurityDocker London: Container Security
Docker London: Container Security
 
Powercoders · Docker · Fall 2021.pptx
Powercoders · Docker · Fall 2021.pptxPowercoders · Docker · Fall 2021.pptx
Powercoders · Docker · Fall 2021.pptx
 
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...
Kubernetes Story - Day 2: Quay.io Container Registry for Publishing, Building...
 
Docker Essentials Workshop— Innovation Labs July 2020
Docker Essentials Workshop— Innovation Labs July 2020Docker Essentials Workshop— Innovation Labs July 2020
Docker Essentials Workshop— Innovation Labs July 2020
 
Unraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production CloudUnraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production Cloud
 
Tokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityTokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker Security
 
Docker, c'est bonheur !
Docker, c'est bonheur !Docker, c'est bonheur !
Docker, c'est bonheur !
 
Jump into Squeak - Integrate Squeak projects with Docker & Github
Jump into Squeak - Integrate Squeak projects with Docker & GithubJump into Squeak - Integrate Squeak projects with Docker & Github
Jump into Squeak - Integrate Squeak projects with Docker & Github
 
Container Monitoring with Sysdig
Container Monitoring with SysdigContainer Monitoring with Sysdig
Container Monitoring with Sysdig
 
Kubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with PodmanKubernetes Story - Day 1: Build and Manage Containers with Podman
Kubernetes Story - Day 1: Build and Manage Containers with Podman
 
How Secure Is Your Container? ContainerCon Berlin 2016
How Secure Is Your Container? ContainerCon Berlin 2016How Secure Is Your Container? ContainerCon Berlin 2016
How Secure Is Your Container? ContainerCon Berlin 2016
 
Docker security: Rolling out Trust in your container
Docker security: Rolling out Trust in your containerDocker security: Rolling out Trust in your container
Docker security: Rolling out Trust in your container
 
Accelerate your development with Docker
Accelerate your development with DockerAccelerate your development with Docker
Accelerate your development with Docker
 
Accelerate your software development with Docker
Accelerate your software development with DockerAccelerate your software development with Docker
Accelerate your software development with Docker
 
Docker Introduction.pdf
Docker Introduction.pdfDocker Introduction.pdf
Docker Introduction.pdf
 

More from Christian Beedgen

Measuring Our Morals–Analog Thinking In A Digital World
Measuring Our Morals–Analog Thinking In A Digital WorldMeasuring Our Morals–Analog Thinking In A Digital World
Measuring Our Morals–Analog Thinking In A Digital WorldChristian Beedgen
 
Using AWS To Build A Scalable Machine Data Analytics Service
Using AWS To Build A Scalable Machine Data Analytics ServiceUsing AWS To Build A Scalable Machine Data Analytics Service
Using AWS To Build A Scalable Machine Data Analytics ServiceChristian Beedgen
 
How Sumo Logic And Anki Build Highly Resilient Services On AWS To Manage Mass...
How Sumo Logic And Anki Build Highly Resilient Services On AWS To Manage Mass...How Sumo Logic And Anki Build Highly Resilient Services On AWS To Manage Mass...
How Sumo Logic And Anki Build Highly Resilient Services On AWS To Manage Mass...Christian Beedgen
 
5 Years Of Building SaaS On AWS
5 Years Of Building SaaS On AWS5 Years Of Building SaaS On AWS
5 Years Of Building SaaS On AWSChristian Beedgen
 
Scaling A Start-up DevOps Team To 10x While Scaling The System 50x - DevOpsD...
Scaling A Start-up DevOps Team To 10x  While Scaling The System 50x - DevOpsD...Scaling A Start-up DevOps Team To 10x  While Scaling The System 50x - DevOpsD...
Scaling A Start-up DevOps Team To 10x While Scaling The System 50x - DevOpsD...Christian Beedgen
 
How to Meta-Sumo - Using Logs for Agile Monitoring of Production Services
How to Meta-Sumo - Using Logs for Agile Monitoring of Production ServicesHow to Meta-Sumo - Using Logs for Agile Monitoring of Production Services
How to Meta-Sumo - Using Logs for Agile Monitoring of Production ServicesChristian Beedgen
 

More from Christian Beedgen (7)

Measuring Our Morals–Analog Thinking In A Digital World
Measuring Our Morals–Analog Thinking In A Digital WorldMeasuring Our Morals–Analog Thinking In A Digital World
Measuring Our Morals–Analog Thinking In A Digital World
 
Machine Data for the Masses
Machine Data for the MassesMachine Data for the Masses
Machine Data for the Masses
 
Using AWS To Build A Scalable Machine Data Analytics Service
Using AWS To Build A Scalable Machine Data Analytics ServiceUsing AWS To Build A Scalable Machine Data Analytics Service
Using AWS To Build A Scalable Machine Data Analytics Service
 
How Sumo Logic And Anki Build Highly Resilient Services On AWS To Manage Mass...
How Sumo Logic And Anki Build Highly Resilient Services On AWS To Manage Mass...How Sumo Logic And Anki Build Highly Resilient Services On AWS To Manage Mass...
How Sumo Logic And Anki Build Highly Resilient Services On AWS To Manage Mass...
 
5 Years Of Building SaaS On AWS
5 Years Of Building SaaS On AWS5 Years Of Building SaaS On AWS
5 Years Of Building SaaS On AWS
 
Scaling A Start-up DevOps Team To 10x While Scaling The System 50x - DevOpsD...
Scaling A Start-up DevOps Team To 10x  While Scaling The System 50x - DevOpsD...Scaling A Start-up DevOps Team To 10x  While Scaling The System 50x - DevOpsD...
Scaling A Start-up DevOps Team To 10x While Scaling The System 50x - DevOpsD...
 
How to Meta-Sumo - Using Logs for Agile Monitoring of Production Services
How to Meta-Sumo - Using Logs for Agile Monitoring of Production ServicesHow to Meta-Sumo - Using Logs for Agile Monitoring of Production Services
How to Meta-Sumo - Using Logs for Agile Monitoring of Production Services
 

Recently uploaded

Kubernetes go-live checklist for your microservices.pptx
Kubernetes go-live checklist for your microservices.pptxKubernetes go-live checklist for your microservices.pptx
Kubernetes go-live checklist for your microservices.pptxPrakarsh -
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9Jürgen Gutsch
 
20240330_고급진 코드를 위한 exception 다루기
20240330_고급진 코드를 위한 exception 다루기20240330_고급진 코드를 위한 exception 다루기
20240330_고급진 코드를 위한 exception 다루기Chiwon Song
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadIvo Andreev
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmonyelliciumsolutionspun
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionsNirav Modi
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...OnePlan Solutions
 
Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!Neo4j
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Jaydeep Chhasatia
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilVICTOR MAESTRE RAMIREZ
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfBrain Inventory
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyRaymond Okyere-Forson
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLAlluxio, Inc.
 

Recently uploaded (20)

Kubernetes go-live checklist for your microservices.pptx
Kubernetes go-live checklist for your microservices.pptxKubernetes go-live checklist for your microservices.pptx
Kubernetes go-live checklist for your microservices.pptx
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9About .NET 8 and a first glimpse into .NET9
About .NET 8 and a first glimpse into .NET9
 
20240330_고급진 코드를 위한 exception 다루기
20240330_고급진 코드를 위한 exception 다루기20240330_고급진 코드를 위한 exception 다루기
20240330_고급진 코드를 위한 exception 다루기
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and Bad
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspections
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
 
Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!
 
Sustainable Web Design - Claire Thornewill
Sustainable Web Design - Claire ThornewillSustainable Web Design - Claire Thornewill
Sustainable Web Design - Claire Thornewill
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-Council
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdf
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human Beauty
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
 

6 Million Ways To Log In Docker - NYC Docker Meetup 12/17/2014

  • 1. Six Million Ways To Log In Docker Dwayne Hoover, Senior Field Engineer Christian Beedgen, Co-Founder & CTO December 17th, 2014 Sumo Logic Confidential
  • 2. Introduction Sumo Logic Background What Our Customers Are Telling Us A Catalog Of Ways To Log In Docker What We Would Like To Build Agenda Sumo Logic Confidential2
  • 3. Señor Field Engineer at Sumo Logic since 2013 Former developer and data warehouse turned poly- structured data junkie Let’s Make This Personal - Who We Are Co-Founder & CTO, Sumo Logic since 2010 Server guy, Chief Architect, ArcSight, 2001 – 2009 Dwayne Christian
  • 4. The Machine Data Cloud 4 Search Visualize Predict Sumo Logic Confidential
  • 5. Sumo Logic is the only enterprise-grade 100% service-based offering Sumo Logic Deployment “Architecture” Sumo Logic Confidential5
  • 6. Use Cases Sumo Logic Confidential6 1. Availability & Performance 2. Security and Compliance 3. Customer Analytics
  • 8. We have one process per container We like to log to stdout We have multiple processes per container We run the Sumo Logic collector on the host We are looking into using Beanstalk with Docker We are waiting for Amazon ECS Everyone here loves Docker We are logging straight from the application We are using /dev/log for Syslog What Our Customers Are Telling Us Sumo Logic Confidential8
  • 10. One size doesn’t (yet?) fit all It’s not our job to judge What does the community say? Let’s figure out how to collect them all! What We Are Hearing Sumo Logic Confidential10
  • 11. Mailing list thread started in 2013 – https://groups.google.com/forum/#!searchin/docker- dev/logging/docker-dev/3paGTWD6xyw/hvZlnFD5x5sJ Superseded by Logging Drivers proposal mid-2014 – https://github.com/docker/docker/issues/7195 However, as of now no clear path – Extension proposal as the way forward for integrating log forwarders? What Does The Community Say Sumo Logic Confidential11
  • 13. Logs are… – The actual message plus a bunch of meta data – At scale, the meta data becomes very important Timestamp – With date, full year, down to at least milliseconds – With time zone, ideally as an offset, or identifiable as straight UTC Docker host info – FQDN or IP address or both – Correlate Docker daemon logs with container logs Container ID – Need a way to identify the unique instance of course – With name if possible, sometimes we are just human… Image ID – To correlate, potentially, with logs from other containers from the same image – Name would likely help the human operator as well Process ID – To correlate with logs from the process if there’s no other way to identify them What Should Be In A Log Sumo Logic Confidential13
  • 14. Docker captures container stdout to file in JSON format In /var/lib/docker/containers/[ID]/[ID]-json.log The docker logs command can spit back the logs Each invocation returns the full logs all over But it can also be used to tail the logs Careful! Stdout logs grow without bound on the host Consider using logrotate on the Docker host https://github.com/docker/docker/issues/7333 What Docker Provides Sumo Logic Confidential14 docker logs –tf –-tail 0 [ID]
  • 15. Sumo Logic Confidential15 A Catalog of Ways to Log in Docker.
  • 16. Log Directly From The Application Sumo Logic Confidential16 1
  • 17. Assuming you have control over the application Use a library that can send Syslog Or use a vendor library if HTTPS is required This can work for other stack components as well Apache can be coerced into sending Syslog Nginx has an easy way to send error/access to Syslog So does Postgres, and almost any Java-based app Log Directly From The Application Sumo Logic Confidential17 1
  • 18. If you want to use Sumo Logic… There’s an image to quickly set up a Syslog collector Configure your applications to send to the host at 514 Log Directly From The Application Sumo Logic Confidential18 docker run -d -p 514:514 -p 514:514/udp --name="sumo-logic-collector" sumologic/collector:latest-syslog [Access ID] [Access key] 1
  • 19. Pros – Conceptually pretty straightforward – Might not even have to change anything – Syslog includes the container ID as the hostname Cons – Need control over the code or at least the configuration – Every component might need different situps – HTTPS straight from the app might not include the container ID – Logging to service without a collector loses data if link is down Log Directly From The Application Sumo Logic Confidential19 1
  • 20. Various application stacks – http://help.papertrailapp.com/ Log4J – https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/net/SyslogAppender.html Apache Web Server – http://httpd.apache.org/docs/trunk/mod/mod_syslog.html – https://raymii.org/s/snippets/Apache_access_and_error_log_to_syslog.html Nginx – http://nginx.org/en/docs/syslog.html Postgres – http://www.postgresql.org/docs/9.1/static/runtime-config-logging.html Sumo Logic blog on official syslog collector image – http://www.sumologic.com/blog/company/an-official-docker-image-for-the-sumo-logic-collector – https://github.com/SumoLogic/sumologic-collector-docker Log Directly From The Application Sumo Logic Confidential20 1
  • 21. Install A File Collector In The Container Sumo Logic Confidential21 2
  • 22. It is not terribly uncommon that logs go to files There’s many ways to tail logs and ship them off Logstash, Rsyslog, Sumo Logic Collector, Splunk Forwarder, … Log to volumes to bypass layered file system Also, logs are not really container state? Install A File Collector In The Container Sumo Logic Confidential22 2
  • 23. Pros – Conceptually pretty straightforward – If everything logs to files already, not a big change – Collectors can be configured as part of building the image Cons – One collector per container could be unacceptable overhead – No container ID included unless collector picks up hostname Install A File Collector In The Container Sumo Logic Confidential23 2
  • 24. Install A File Collector As A Container Sumo Logic Confidential24 3
  • 25. Normalize the collector-per-container idea Create a container that has only the collector Mount a host directory into that container to collect from Mount the same directory into each container Configure the container to write log files to the mount Configure the collector container to recursively collect Could collector on the host, but not Docker-native For example, using the Sumo Logic file collector image Install A File Collector As A Container Sumo Logic Confidential25 docker run -v /tmp/clogs:/tmp/clogs -d --name="sumo-logic-collector" sumologic/collector:latest-file [Access ID] [Access key] 3
  • 26. What about name clashes in the shared mounted directory? Create a sub directory named after the container ID! Assume the Dockerfile ends in: Then do this in run.sh: Install A File Collector As A Container Sumo Logic Confidential26 ENTRYPOINT ["/bin/bash", "run.sh"] # Create log directory mkdir -p /tmp/clogs/$HOSTNAME ln -s /tmp/clogs/$HOSTNAME /tmp/logs # Do something echo "ls -la /tmp/clogs" ls -la /tmp/clogs echo "ls -la /tmp/logs" ls -la /tmp/logs 3
  • 27. What about name clashes in the shared mounted directory? Create a sub directory named after the container ID! Assume the Dockerfile ends in: Then do this in run.sh and observe: Install A File Collector As A Container Sumo Logic Confidential27 ENTRYPOINT ["/bin/bash", "run.sh"] ls -la /tmp/clogs total 16 drwxr-xr-x 4 root root 4096 Dec 15 23:51 . drwxrwxrwt 3 root root 4096 Dec 15 23:51 .. drwxr-xr-x 2 root root 4096 Dec 15 23:51 43da9cc4d050 drwxr-xr-x 2 root root 4096 Dec 15 23:51 7df836a68214 ls -la /tmp/logs lrwxrwxrwx 1 root root 23 Dec 15 23:51 /tmp/logs -> /tmp/clogs/43da9cc4d050 3
  • 28. Sumo Logic blog on official collector images – http://www.sumologic.com/blog/company/an-official-docker-image- for-the-sumo-logic-collector – https://github.com/SumoLogic/sumologic-collector-docker Rainer Gerhards on Rsyslog’s file input module – http://www.slideshare.net/rainergerhards1/using-wildcards-with- rsyslogs-file-monitor-imfile OWASP Log Injection – https://www.owasp.org/index.php/Log_injection Install A File Collector As A Container Sumo Logic Confidential28 3
  • 29. Pros – Not terribly hard to understand and setup – File collection is very common collector functionality and can scale Cons – Have to expose a host directory to all containers – Mounted directory might be considered an attack vector – Unless performing described sit ups, name clashes likely Install A File Collector As A Container Sumo Logic Confidential29 3
  • 30. Install A Syslog Collector As A Container Sumo Logic Confidential30 4
  • 31. If you want to use Syslog, and Sumo Logic… There’s an image to quickly set up a Syslog collector Use linking to configure the Syslog location in the containers Easy to test with Install A Syslog Collector As A Container Sumo Logic Confidential31 docker run –d --name="sumo-logic-collector" sumologic/collector:latest-syslog [Access ID] [Access key] docker run -it --link sumo-logic-collector:sumo ubuntu /bin/bash echo "I'm in ur linx" | nc -v -u -w 0 $SUMO_PORT_514_TCP_ADDR $SUMO_PORT_514_TCP_PORT 4
  • 32. Pros – Not terribly hard to understand and setup – Will retain origin hostname and container ID Cons – Every component might need different situps for Syslog Install A Syslog Collector As A Container Sumo Logic Confidential32 4
  • 33. Use Host Syslog For Local Syslog Sumo Logic Confidential33 5
  • 34. The process(es) in the container already do Syslog There is some chance that the host is running Syslog daemon Configure the host Syslog daemon to forward Mount /dev/log from the host to /dev/log in the container Now tail the host syslog Run a container to test if it works Should see something like this in the tail’ed file Use Host Syslog For Local Syslog Sumo Logic Confidential34 docker run -d -v /dev/log:/dev/log [image] tail -F /var/log/syslog docker run -v /dev/log:/dev/log ubuntu logger -t schnitzel Now! Dec 14 16:33:49 ubuntu schnitzel: Now! 5
  • 35. Pros – Nothing extra to install if the host has Syslog already – Host’s Syslog will be collected as well Cons – Hostname is set to the receivers hostname, no container ID in the logs Use Host Syslog For Local Syslog Sumo Logic Confidential35 5
  • 36. Use A Syslog Container For Local Syslog Sumo Logic Confidential36 6
  • 37. From Jérôme Petazzoni’s blog – use a bind mount! Create a simple Rsyslog container, claim /dev as a volume Then run the Syslog container, capturing its /dev in /tmp/syslogdev Finally, run the containers that log to local Use A Syslog Container For Local Syslog Sumo Logic Confidential37 docker run --name syslog -d -v /tmp/syslogdev:/dev [image] FROM ubuntu:14.04 RUN apt-get update -q RUN apt-get install rsyslog CMD rsyslogd -n VOLUME /dev VOLUME /var/log docker run --name [image-name] -d -v /tmp/syslogdev/log:/dev/log [image] 6
  • 38. Jérôme Petazzoni’s Blog – http://jpetazzo.github.io/2014/08/24/syslog-docker/ What is a bind mount? – http://docs.1h.com/Bind_mounts – http://man7.org/linux/man-pages/man8/mount.8.html Use A Syslog Container For Local Syslog Sumo Logic Confidential38 6
  • 39. Pros – Removes the need to have and configure Syslog on the host – Encapsulates Syslog collection in a Docker-native way Cons – Hostname is set to the receivers hostname, no container ID in the logs Use A Syslog Container For Local Syslog Sumo Logic Confidential39 6
  • 40. Containers model processes, not machines Docker persists container stdout on the host Simply point the collectors’s file collection mechanism to this path Collector can also be a container, if the above path is mounted For example, the Sumo file collector image expects logs in /tmp/clogs Log To Stdout And Use A File Collector Sumo Logic Confidential40 /var/lib/docker/containers/*/*-json.log docker run -d -v /var/lib/docker/containers:/tmp/clogs sumologic/collector:latest-file [Access ID] [Access Key] 7
  • 41. Pros – Relatively straightforward to set up – Container ID available via filename Cons – Docker doesn’t bound the stdout logs on disk – File collector needs to be able to deal with logrotate if used – Must be willing to live with host directory mounted in container Log To Stdout And Use A File Collector Sumo Logic Confidential41 7
  • 42. Rainer Gerhards on Rsyslog’s file input module – http://www.slideshare.net/rainergerhards1/using-wildcards-with- rsyslogs-file-monitor-imfile Sumo Logic blog on official collector images and Github repo – http://www.sumologic.com/blog/company/an-official-docker-image- for-the-sumo-logic-collector – https://github.com/SumoLogic/sumologic-collector-docker On using Logrotate with Docker – https://github.com/docker/docker/issues/7333 Log To Stdout And Use A File Collector Sumo Logic Confidential42 7
  • 43. Logspout is a very lightweight container that forwards stdout to syslog Logspout uses the Docker Event API to track containers coming and going For each container, Logspout gets the stdout from Docker via API By default everything gets forwarded to the specified endpoint Logspout supports routing to different endpoints Routing rules can be expressed as filters on container name & ID Logspout also exposes a little HTTP interface to bounce logs back live We are hacking Logspout to forward to Sumo’s HTTP endpoint as well! Log To Stdout And Use Logspout Sumo Logic Confidential43 docker run –d –p 8000:8000 –v /var/run/docker.sock:/tmp/docker.sock progrium/logspout syslog://[syslog-host]:[syslog-port] curl localhost:8000/logs 8
  • 44. Pros – Trivial to set up and very lightweight – Adds container ID and name to the logs – Flexible, optionally persistent routing for complicated cases Cons – Docker doesn’t bound the stdout logs on disk Log To Stdout And Use Logspout Sumo Logic Confidential44 8
  • 45. Logspout Github repository – https://github.com/progrium/logspout Various Articles – http://stackengine.com/docker-logs-aggregating-ease/ – http://blog.froese.org/2014/05/15/docker-logspout-and-nginx/ On using Logrotate with Docker – https://github.com/docker/docker/issues/7333 Log To Stdout And Use Logspout Sumo Logic Confidential45 8
  • 46. Collect From Docker Filesystems Sumo Logic Confidential46 9
  • 47. Ultimately, all files from container file systems end up on disk One of my boxes is running AUFS and I can see all files in: A simple test with tailing a file in a container from the host works… Collect From Docker Filesystems Sumo Logic Confidential47 9 /var/lib/docker/aufs/mnt/[Container ID]
  • 48. Unfortunately, this doesn’t work with Devicemapper Another box is using devicemapper and I can see all files in: A simple test with tailing a file in a container from the host works So now you can slab a file collector on the host and configure it…? With devicemapper, stopping a container while tailing leads to error on start This error will persist until the other process (tail) is stopped And then, a manual umount is required before docker start Collect From Docker Filesystems Sumo Logic Confidential48 9 /var/lib/docker/devicemapper/mnt/[Container ID]/rootfs/ Error response from daemon: Cannot start container 6f62be47025d: Error getting container 6f62be47025d... from driver devicemapper: Error mounting '/dev/mapper/docker-202:1-277656-6f62be47025d....' on '/var/lib/docker/devicemapper/mnt/6f62be47025d...': device or resource busy
  • 49. Pros – If legal, it means a lot of existing file collection tools can just be used Cons – Could just be a batshit crazy idea and the universe collapses into itself – Need to find a way to configure file collector per image Collect From Docker Filesystems Sumo Logic Confidential49 9
  • 50. Inject Collector Via Docker Exec Sumo Logic Confidential50 10
  • 51. docker exec allows injection of a process into a container A collector could live in a container, and talk to the Docker daemon The collector could use the Event API to track containers come and go Basically, just like Logspout… or put it on the host, I guess When a container appears, the Exec API could be used to inject a process The process could run the collection logic, starting with watching paths, etc. The process could also actually tail the files and send logs to a service Or, it could send logs back to the collector container via stdout or something The collector in the container could then do caching, compression, … Inject Collector Via Docker Exec Sumo Logic Confidential51 10
  • 52. Pros – This could actually be a generic and non-crazy way to collect log files – There’s a ton of tools that know how to collect from files Cons – In reality, will people accept/allow docker exec? – It basically allows a container to access another container as root Inject Collector Via Docker Exec Sumo Logic Confidential52 10
  • 54. Something that catches stdout from all containers… – Logspout does this already! …and that can tail files in containers in a clean way… – Container can define which path(s) …and forward messages via different protocols – Logspout does Syslog, we are adding HTTP POST We think the extensions discussion is very relevant! – More realistic than adding to core Docker codebase? What We Would Like To Build Sumo Logic Confidential54

Editor's Notes

  1. INTROS For those of you new to Sumo Logic, we’re a Silicon Valley-based startup - founded by industry experts with strong backgrounds in Data Science, Enterprise Software & Internet Services and backed by some of the top VC firms in the Business Today. We were founded with a simple but far-reaching goal: To meet the challenge of the largest data explosion in history and help turn that data—whatever its type, location or volume—into actionable IT and business insights. It you are in IT today you have a choice in front of you. You can choose to to look at the machine data output of your infrastructure as just fumes and exhaust from your Apps, servers and Network OR you can look at it as the Life Blood of your Operation and Business. The Pulse. We are here to talk about how SumoLogic is disrupting the Status Quo and what that means to you. By Status Quo we are talking about the prior generation of On-Premise software, Home Grown solutions, and that one we all know…”Ignore and Wait.” Our intent is to break the barriers that were previously in front of you regarding Data Silos, inability to handle ever growing Volumes of data, Antiquated Architectures, and manual analytics. From Top to Bottom here we have a distinct focus on Customer Satisfaction and doing things the Right Way. We do this all as a Service – Secure, Reliable, Flexible with a ground breaking Time to Value. So let’s get started.
  2. Desired State: Turning a Chaotic Situation into your Benefit and Advantage Sumologic takes this chaos of information, 1000’s of sources every second of the day and makes it Human Readable for IT insights to make business decisions. How do we do it better than current solutions?