List All Installed Python Packages

There are times that we need a list of all installed python packages. One way to accomplish it is using

help('modules')

in python shell. It shows all installed packages including stdlib. If we are not interested in stdlib, command pip works:

pip freeze

It generates installed packages in requirements format.

Manage Autostart Applications in GNOME

After login in GNOME a lot of applications can be automatically started to make life easier. System-wide autostart applications can be found in /etc/xdg/autostart and in /usr/share/gnome/autostart. Users have choices to edit a autostart application by disabling it, editing its name, command or description; additionally, users may add their own autostart applications.

Gnome provides the tool gnome-session-properties, which allows us to add, modify and remove autostart applications. Once we are done with gnome-session-properties, new entries (files) are generated and saved in directory ~/.config/autostart/.

Here is an example file skype.desktop created and saved after adding skype as a new autostart application:

[Desktop Entry]
Type=Application
Exec=/usr/bin/skype
X-GNOME-Autostart-enabled=true
Name=Skype


A new file named gnome-keyring-ssh.desktop is created (and saved in directory ~/.config/autostart) after removing the default autostart application SSH key agent. The contents in it are the same as that in file /etc/xdg/autostart/gnome-keyring-ssh.desktop, except in one line:

X-GNOME-Autostart-enabled=false

which specifies that the autostart is disabled.

Some Usages of fuser on Linux Systems

Command fuser allows us to  identify processes using files or sockets. Here are some example usages of fuser.

1. Identify processes using a file or directory:

fuser -v /path/to/the/fileORdirectory

2. Identify processes using a particular TCP/UDP port#:

fuser -v -n tcp port#

where option -n enables us to specify an object in a different name space, tcp port# here.

3. Kill processes that are executing a particular command:

fuser -v -k -i /path/to/the/command

where option -i specifies asking the user for confirmation before killing a process.

4. Kill processes using a particular TCP/UDP port#:

fuser -v -k -i -n tcp port#

Load Python Module Dynamically

There are times that we need to import some python module and, for whatever reasons, we will not know the name of the module until run-time. This can be achieved by built-in function __import__().

Dynamic loading python module is useful, for example, if we plan to read configuration from a file. Here is an example that allows us to load the module specified in sys.argv[1]:

config_file = sys.argv[1]
config = __import__(config_file)

For newer versions (2.7 and up), there are convenience wrappers for __import__(). We may use module importlib:

import importlib
config_file = sys.argv[1]
config = importlib.import_module(config_file)

Both __import__() and importlib.import_module() search modules in certain locations (e.g., sys.modules, sys.meta_path, sys.path). There are times that modules being loaded are not in those default locations. This can be achieved by using imp.load_source(), which allows us to load and initialize  modules implemented as  Python source files:

import imp
pathname = sys.argv[1]
config = imp.load_source(name, pathname)

where pathname is the path name of the configuration file.

Some Usages of cpio

The command cpio allows us to copy files between archives and directories. There are three operation modes: output (-o), input (-i), and pass-through (-p).

To create an archive for directory tree src, we issue:


find src -print0 | cpio -ov0 > src.cpio


To extract files from its archive just created, we issue:

cpio -ivd < src.cpio


To copy files from directory tree src to directory des, we issue:

find src -print0 | cpio -pmd0 dest


To copy files from src to dest that are less than 2 days old and whose names contain 'txt', we issue:

find src -mtime -2 -print0 | grep txt | cpio -pmd0 dest


To copy files from src to dest that are less than 2 days old and which  contain the word 'txt', we issue:

find src -mtime -2 | xargs grep -l txt | cpio -pmd0 dest




Combining with ssh (or, natcat if you are not concerned with security),  cpio allows us to access remote hosts. Here are some example:

To backup the directory tree src on a remote host, we may issue:

find src -print0 | cpio -oaV0 -H tar -O user@remotehost:src.tar

which uses ssh to copy archive file from localhost to remote host.


To copy files from src to a remote host, we issue:

find src -print0 | cpio -oaV0 -H tar | ssh user@remotehost "cpio -imd"

Netcat for File Transfer

File transfer is one of the most practical usages of netcat.

To transfer a file named filename from client to server, we first issue the following command on server:

nc -l -p 1234 > filename

where 1234 is the port number used by the server. On the client side, we issue:

nc -q 10 server 1234 < filename

where '-q 10' specifies to wait 10 seconds and then quit after EOF on stdin. This would cause the server to quit.

To transfer a director tree named /path from client to server, we issue the following command on server:

nc -l -p 1234 | tar xvzf -

On the client side, we issue:

tar cvzf - /path | nc -q 10 server 1234



If we want to reverse the direction of file transfer, i.e., client pulls file from server, we use:

nc -q 10 -l -p 1234 < filename

on server, and

nc server 1234 > filename

on client. Similarly, to reverse the direction of directory tree transfer, we use:

tar cvzf - /path | nc -q 10 -l -p 1234

on server, and

nc server 1234 | tar xvzf -

on client.

How to test Django development server remotely?

Django development server is intended only for use while developing. There are times that we need to test Django applications remotely through development server. Thought we may bind the development server to some public IPs to achieve this, it is not a good practice in general for various reasons. Ssh port forwarding provides a much secure way for remote testing of Django development server. Let serv be the server where Django development server resides, test the remote host where you want to conduct the tests. Here is a how-to.

1. On host serv, start your development server:
python manage.py runserver 8080
where 8080 is any port number of your choice.

2. On host test, start ssh port forwarding:
ssh -L 9090:localhost:8080 user@serv
where 9090 is the port number providing remote service.

3. On host test, point your browser to localhost:9090 to perform tests.


If you want to test the development server through other hosts, say  third, you should enable gateway forwarding on host test by issuing:

ssh -g -L 9090:localhost:8080 user@serv

Then you may perform tests through host third by pointing your browser to test:9090.

Netcat tricks

Netcat is a utility which reads and writes data across network connections, using TCP or UDP. It is known as a "Swiss-army knife for TCP/IP". Here are some useful netcat tricks.



Simple Client

To connect to the SMTP server (srv), we type:

nc srv 25



One-shot Server

To provide a uptime service on a unix/linux system, we type:

nc -l -p Port# -e /usr/bin/uptime

where Port# is the service port number, uptime the utility reporting system uptime.



Simple Server

To provide a fixed web page (index.html) service on a unix/linux, we type:

while true; do {echo -ne "HTTP/1.0 200 OK\r\nContent-Length: $(wc -c < index.html)\r\n\r\n"; cat index.html; } | nc -l -p Port#; done

To implement a log server that logs the first 500 bytes data from any client, we type:

while true; do nc -l -p Port# | head -c 500 >> LogFile; date >> LogFile; done

where LogFile is the name of our log file, each log record is followed by a time stamp.

To set up a backdoor shell, we type:

nc -l -p Port# -e /bin/sh



File Transfer

To push a file (FileName) from host A to host B (hostB) , we type

nc -l -p Port# > FileName

on host B, and

nc -q 10 hostB Port# < FileName

on host A.


To pull a file (FileName) from host A (hostA) back to host B, we type

nc -q 10 -l -p Port# < FileName

on host A, and

nc hostA Port# > FileName

on host B.



Chat Service

To set up chat server (srv), we type:

nc -l -p Port#

on srv, and on chat client, we type:

nc srv Port#



Port Scanner

Netcat can be a port scanner. It does not have as many features as  nmap, but if we just want to see what ports numbered between 1 to 1024 are open on host with IP address 192.168.0.1, we type:

nc -z -v -n -w 1 192.168.0.1  1-1024

where -z specifies scanning, -v verbose, -n no DNS, -w 1 timeout after 1 second.



UDP mode

The option -u puts netcat to operate in UDP mode.