An Introduction to Fabric

Fabric is described as a simple, Pythonic tool for application deployment or systems administration tasks. The fab tool simply imports your fabfile (fabfile.py, by default) and executes the function or functions at your disposal. There is no magic about fabric – anything we can do in a Python script can be done in a fabfile. Here is a small but complete fabfile that defines one task:

def hello(name="World"):
    print("Hello %s!" % name)


Calling fab hello will display the familiar message "Hello World!" However, we can personalize it by issuing:

$fab hello:name=Jay

The message shown will be: "Hello Jay!"

Below is another small fabfile that allows us to find kernel release as well as machine hardware of a remote system by using SSH:

from fabric.api import run

def host_type():
    run("uname -srm")

Since there is no remote host defined in the fabfile, we need to specify it (hostname) using option -H:

$fab -H hostname host_type

Fabric provides many command-execution functions, the following five are frequently used:

• run(command) -- Run a shell command on a remote host.
• sudo(comand) -- Run (with superuser privileges) a shell command on a remote host.
• local(command) -- Run a command on the local host.
• get(remote_path, local_path) -- Download one or more files from a remote host.
• put(local_path, remote_path) -- Upload one or more files to a remote host.

Fabric also includes context managers for use with the with statement, the following three are commonly used:

• settings(*args, **kwargs) -- Nest context managers and/or override env variables.
• lcd(path) -- Update local current working directory.
• cd(path) -- Update remote current working directory. 

The third sample fabfile allows us to deploy a Django project to three production servers (serv1, serv2 and serv3). In fact, it can be extended to deploy various applications. The deployment consists of the following steps:

1. testing the project -- python manage.py test apps
2. packing the project  -- tar czf /tmp/project.tgz .
3. moving the packed file to server -- put("/tmp/project.tgz", "/path/to/serv/tmp/project.tgz") 
4. unpacking the file -- run("tar xzf /path/to/serv/tmp/project.tgz"), and 
5. deploying the project on server -- run("touch manage.py") 

To use it, we issue:
$fab install



from __future__ import with_statement
from fabric.api import *
from fabric.contrib.console import confirm

env.hosts = ['serv1', 'serv2', 'serv3']
env.user= "admin"

def test():
    """
    Run test;  if it fails prompt the user for action.
    """
    src_dir = "/path/to/local/src/directory"
    with settings(warn_only=True), lcd(src_dir):
        result = local("python manage.py test apps", capture=True)
    if result.failed and not confirm("Tests failed. Continue anyway?"):
        abort("Aborting at user request.")

def pack_move():
    """
    Archive our current code and upload to servers.
    """
    src_dir = "/path/to/local/src/directory"   
    with settings(warn_only=True), lcd(src_dir):
        local( "tar czf /tmp/project.tgz .")
    put( "/tmp/project.tgz", "/path/to/serv/tmp/project.tgz" )
    local( "rm /tmp/project.tgz" )

def install():
    """
    Deploy the project on servers.
    """
    test()
    pack_move()
    dst_dir = "/path/to/serv/dst/directory"
    with settings(hide("warnings"), warn_only=True):
        if run("test -d %s" % dst_dir).failed:
            run("mkdir -p %s" % dst_dir)   
    with cd(dst_dir):
        run("tar xzf /path/to/serv/tmp/project.tgz")
        run("touch manage.py")
    run("rm -f /path/to/serv/tmp/project.tgz")
      

Git -- Cheat Sheet

Here is a cheat sheet for git.

1. Create
git init                  # create a local repository
git clone <url>    # clone a repository from url

2. Commit
git commit -m "commit message"

3. Browse
git log                  # history of change
git status             # files changed in working directory
git diff                 # diff between working directory and the index
git diff HEAD       # diff between working directory and the most recent commit
git diff --cached   # diff between the index and the most recent commit
git show <object>   # show object
gitk                             # git repository (GUI) browser

4. Stage
git add <file>                  # add file to the index
git reset HEAD <file>    # unstage the staged file

5. Undo
git commit -a --amend          # fix the last commit
git reset --hard <commit>  # discard any changes and reset to the commit
git revert HEAD                      # revert the last commit
git revert <commit>             # revert the specific commit
git checkout -- <file>       # unmodify the modified file

6. Branch
git branch <new_branch>  # create a branch named new_branch based on HEAD
git branch -d <old_branch> # delete the branch named old_branch
git checkout <branch>           # switch to the branch
git checkout -b <branch>    # create a new branch then switch to it
git merge <branch>                # merge the specified branch into HEAD

7. Update
git fetch               # download latest changes from origin
git pull                 # fetch from and integrate (merge) with origin

8. Publish
git push                 # update origin

Git -- basics

Git is a distributed version control system (DVCS) designed to handle things from small to very large projects. DVCS features many advantages over the traditional centralized VCS, because users have the entire history of the project on their local disks (repository). Two of them are:

• It allows users to work productively without network connection, and
• it makes most operations much faster, since most operations are local.

Git employs snapshots instead of file diffs to track files. Every time we do a commit, it basically takes a picture of working directory at that moment and stores that snapshot. In addition to the working directory, Git has two main data structures:

• index (also called staging area or cache) -- a mutable file that caches information about the working directory, and 
• object database -- an immutable, append-only storage that stores files and meta-data for our project.

The object database contains four types of objects:

• blob (binary large object) -- each file that we add to the repository is turned into a blob object.
• tree -- each directory is turned into a tree object.
• commit -- a commit is a snapshot of the working directory at a point in time.
• tag -- a container that contains reference to another object.

Each object is identified by a SHA-1 hash of its contents. In general, git stores each object in a directory matching the first two characters of its hash; the file name used is the rest of the hash for that object. The command git cat-file or git show allows us to view content of objects.

The index (staging area) serves as a bridge between the working directory and the object database. Information stored in the index will go into the object database at our next commit. The following figure shows the normal work flow among them.

 Working        Index             Object
directory                             database      

     <--------checkout-----------|
     |-----add---->
                           |---commit--->


Each file in our working directory can be in one of two states: tracked or untracked. Tracked files are those that are in the last snapshot or in the index (staging area); they can be further classified as unmodified, modified, or staged. A staged file is one that has been modified/created and added to the index. Untracked files are everything else. The lifecycle of files in working directory is illustrated in the following figure:

 untracked         unmodified        modified        staged
                      
     |----------------------add------------------------->                  
     <---------------------reset------------------------|
                                  |-----edit------> 
                                                           |------add----->                                  
                                 <----------commit-------------|

The command git ls-files lists tracked files, whereas git ls-file -o (option o) lists untracked files.

SELINUX -- Bits and Pieces

Here are some bits and pieces on SELINUX:

How to view the current SELinux status?
$sestatus

Where is main configuration file?
/etc/selinux/config

How to set booleans?
$setsebool -P httpd_read_user_content 1
or,
$semanage boolean -m --on httpd_read_user_content

How to list booleans?
$getsebool httpd_read_user_content
or,
$semanage boolean -l |grep httpd_read_user_content

How to allow the Apache HTTP server to provide service on port 9876?
$semanage port -a -t http_port_t -p tcp 9876

How to allow the Apache HTTP server to connect to your database server?
$semanage boolean -m --on httpd_can_network_connect_db

How to allow the Apache HTTP server to send mail?
$semanage boolean -m --on httpd_can_sendmail

How to execute multiple commands within a single transaction?
$semanage -i command-file     

How to change the security context (temporarily) on a file/directory?
$chcon -t my_type_t /path/to/file                  # on single file
$chcon -R -t my_type_t /path/to/directory  # recursively on directory

How to change the security context (persistently) on a file/directory?
$semanage fcontext -a -t my_type_t /path/to/file
# this will add the specified rule to the local context file, then label it
$restorecon -v /path/to/myfile

How to check/correct the security context on filesystems?
$fixfiles -v check  /path/to/file_or_directory       # check only
$fixfiles -v restore  /path/to/file_or_directory   # restore/correct

How to restore default security contexts of a directory tree?
$restorecon -Rv /path/to/the/directory

How to relabel complete filesystem?
$touch /.autorelabel                                    # using init
$reboot
or,
$fixfiles restore                                          # using fixfiles

How to preserve file security contextx when copying?
$cp --preserve=context /path/to/src /path/to/dst

How to change file security contextx when copying?
$install --context=new_context /path/to/src /path/to/dst


How to create archives that retain security contexts?
$tar --selinux -cvzf archive.tgz /path/to/directory       # create archive
$tar --selinux -xvzf archive.tgz                            # extract files from archive
# star should be used, if option selinux is not supported in tar

How to mount a device with a specific security context?
$mount -o context=SELinux_user:role:type:level device dir

How to start SELINUX troubleshooting tool?
$sealert -b

Where is log file?
/var/log/audit/audit.log            #audit on
or,
/var/log/messages                          #audit off

How to add new rules regarding xxxx to policy?
$grep xxxx /var/log/audit/audit.log | audit2allow -M xxxxlocal
$semodule -i xxxxlocal.pp

Hot to start the SELinux management GUI tool?
$system-config-selinux
# we need to install package policycoreutils-gui first

SELINUX -- Concepts

Security-Enhanced Linux (SELinux) is an implementation of a mandatory access control (MAC) mechanism in the Linux kernel, which further enforces MAC after traditional discretionary access controls (DAC) are checked.

Processes and files are labeled with an SELinux context, which includes an SELinux user, role, type, and level. Within SELinux, all of this information are used to form the access control decisions. For performance reason, SELinux decisions are cached, and the cache is named the Access Vector Cache (AVC). In Fedora, SELinux provides a combination of Role-Based Access Control (RBAC), Type Enforcement (TE), and Multi-Level Security (MLS).


The command sestatus allows us to get the status of a system running SELinux. Here is an example output of command sestatus:

$setstatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      28

The output shows that the SELinux is enabled and is currently running in the enforcing mode the same as current configuration. It also tells us that the configuration root directory is /etc/selinux and a targeted policy is used for MAC. The status could be enabled, disabled, or permissive, where permissive means that SELinux should print warnings instead of enforcing. The command setenforce allows us to modify the running mode of SELinux. Changes should be made in the configuration file /etc/selinux/config if we want it be persistent.


All processes and files are labeled with a type (part of SELinux context). The option -Z allows us to find the type (security context) of a file/process. For examples:

$ls -Z /etc/shadow
----------. root root system_u:object_r:shadow_t:s0    /etc/shadow


It shows that the security context (user, role, type, level) of the file /etc/shadow is system_u:object_r:shadow_t:s0.

$ps -eZ|grep passwd
unconfined_u:unconfined_r:passwd_t:s0-s0:c0.c1023 2630 pts/0 00:00:00 passwd

It tells that the security context of the process passwd run by a regular user is unconfined_u:unconfined_r:passwd_t:s0-s0:c0.c1023.

The type of a process defines its domain. Processes are separated from each other by running in their own domains. The SELinux policy defines rules that determine how processes interact with files, and how processes interact with each other. Only what is specifically allowed by the rules is permitted. By default, every operation is denied and audited. The audited log will be saved in file /var/log/audit/audit.log or /var/log/messages depending on whether audit daemon (auditd) is running or not.

The command sesearch allows us to search the rules in a SELinux policy. The rules will be displayed in the following format:

allow <src_domain> <dst_type> : <class> { permission [ permission [ ... ] ] } ;

To verify that the process passwd is allowed to access the shadow password file /etc/shadow, we may issue:

$sesearch -s passwd_t  -t shadow_t -c file -p write -A

and its output will be something similar to:

allow passwd_t shadow_t : file { ioctl read write create ... } ;

Python Decompiler

Need a decompiler for your python byte-code?

Here is one: uncompyle2, which works for python 2.

Untangle Links

How to find the terminal pathname of a file/directory? For example, the file /usr/lib64/mozilla/plugins/libjavaplugin.so will be installed by IcedTea-web java plugin. It is a symbolic link to another link. The command namei enables us to untangle links by following each pathname until the endpoint is found.

Here is an example usage and its output:

%namei /usr/lib64/mozilla/plugins/libjavaplugin.so
f: /usr/lib64/mozilla/plugins/libjavaplugin.so
 d /
 d usr
 d lib64
 d mozilla
 d plugins
 l libjavaplugin.so -> /etc/alternatives/libjavaplugin.so.x86_64
   d /
   d etc
   d alternatives
   l libjavaplugin.so.x86_64 -> /usr/lib64/IcedTeaPlugin.so
     d /
     d usr
     d lib64
     - IcedTeaPlugin.so

which untangles /usr/lib64/mozilla/plugins/libjavaplugin.so by following links and gets its terminal pathname:
/usr/lib64/IcedTeaPlugin.so.