Friday, April 12, 2019

Terminologies to read about

BTRFS :

https://en.wikipedia.org/wiki/Btrfs
file system based on the Copy-on-Write (COW) principle.
Btrfs is intended to address the lack of pooling, snapshots, checksums and integral multi-device spanning in Linux file systems.
goal was " to let Linux scale for the storage that will be available. Scaling is not just about addressing the storage but also means being able to administer and to manage it with a clean interface that lets people see what's being used and makes it more reliable."

Tupperware :

http://www.adaltas.com/en/2017/11/03/lightweight-containerization-with-tupperware/

Tupperware Agents :
heart of Tupperware. They run on FB's hosts and manage every layer of the running application. They are composed of :
Task Manager
Package Manager
Volume Manager
Resource Manager
Scheduler heartbeat

Launching Containers:
-every container is launched the same way. At the start, they contain a btrfs image. They use ReadWrite Snapshots on a ReadOnly base. Everyone of the FB's packages and other common tools are pre-installed. They allow systemd-init using spawn. Containers also use croups v2.

Image Layering:
Every image on Tupperware is layered as follows:
- Running task
- Application image  - contains instructions required by the running task
- FB image  - like, custom repros, internal program & modules, network customizations
- BaseOS image

Why BTRFS?
Provides the following features :
- Copy on write
- Subvolumes
     - container can mount volumes
     - easy to manage
- Snapshots (RO and RW)
     - it allows going back in time easily
- Binary diffs
     - lower disk space usage
    - lower disk usage IO
     - improved disk data caching
    - independent version layers
- Quotas
    - use full to prevent container to take all disk space over other containers
- Cgroups IO Control
    - provides resource isolation
    - disk isolation
    - memory isolation
   - CPU isolation

Building images:
Images are build using Buick build. provides following features :
      - declarative image building
      - fast parallel builds
      - reproducible builds
      - incremental builds
      - separation of build and runtime
     - fully self contained
     - provides true FS isolation
     - testable

Systemd init:
systemd is container aware and allows SSH connection inside the container, which is useful for debugging or executing specific commands. It uses systemd-nspawn feature and also enables logging outside the container. Finally (but not advised), it can run containers at build time. (dockers doesn't allow it)


FIO (Flexible I/O )

https://github.com/axboe/fio