Mastering Linux: Command-line Interface Completion

File Utility Commands

sort: Arranges lines in text files in a specified manner.

Syntax: sort [OPTION]... [FILE]...

-r : Reverse the result of comparisons

-n : Sort numerically

-k : Sort via a key

-t : Specify the delimiter for -k

-o : Specify the output file

-b : Ignore leading blanks

-f : Ignore case

Question: Sort a file numerically on the second column and save result in another file

Command: sort -n -k 2 -t ',' -o sortedfile.txt file.txt
uniq: Removes duplicates from sorted lines in the file.

Syntax: uniq [OPTION]... [INPUT [OUTPUT]]

-d : Only print duplicate lines

-c : Display the number of occurrences along with lines

-i : Ignore differences in case when comparing

-u : Only print unique lines

-f : Avoid comparing the first N fields

-s : Avoid comparing the first N characters

-w : Compare no more than N characters in lines

Question: Remove duplicate lines from a sorted file and print the number of occurrences for each line

Command: uniq -c sortedfile.txt
diff: Compares files line by line.

Syntax: diff [OPTION]... FILES

-i : Ignore case differences

-w : Ignore all white space

-b : Ignore changes in the amount of white space

-B : Ignore changes whose lines are all blank

-u : Use unified diff format

-r : Recursively compare any subdirectories found

-y : Output in two columns

Question: Compare two files ignoring case, white space, and using the unified diff format

Command: diff -i -w -u file1.txt file2.txt
comm: Compares sorted files FILE1 and FILE2 line by line.

Syntax: comm [OPTION]... FILE1 FILE2

-1 : Suppress the output column of lines unique to FILE1

-2 : Suppress the output column of lines unique to FILE2

-3 : Suppress the output column of lines duplicated in FILE1 and FILE2

-i : Case-insensitive comparison

-z : Line delimiter is NUL, not newline

-u : Output lines are not sorted; compare all pairs

--check-order : Check that the input is correctly sorted

Question: Compare two sorted files and suppress the output column of lines unique to each file

Command: comm -1 -2 file1.txt file2.txt
cut: The cut command removes sections from each line of files.

Syntax: cut OPTION... [FILE]...

-d : Specify the delimiter

-f : Select the fields

-c : Select character ranges

-b : Select byte ranges

-s : Suppress lines with no field delimiter characters

--output-delimiter : Specify the output delimiter

Question: Remove the second and third fields from each line of a file using a colon as the delimiter

Command: cut -d ':' -f 2,3 file.txt
tr: Translates or deletes characters.

Syntax: tr [OPTION]... SET1 [SET2]

-d : Delete characters in SET1

-s : Squeeze repeats of input characters in SET1

-c : Complement SET1

-t : Truncate SET1 to length of SET2

--help : Display help and exit

--version : Output version information and exit

Question: Translate lowercase letters to uppercase and delete digits

Command: echo 'a1b2c3' | tr 'a-z' 'A-Z' | tr -d '0-9'
split: Splits a file into pieces.

Syntax: split [OPTION]... [FILE [PREFIX]]

-b : Specify the size of each piece in bytes (e.g., split -b 100M file prefix)

-l : Specify the number of lines per piece (e.g., split -l 1000 file prefix)

-d : Use numeric suffixes instead of alphabetic (e.g., split -d file prefix)

-a : Set the length of the suffixes (e.g., split -a 4 file prefix)

--additional-suffix : Append an additional SUFFIX to file names

-x : Use hex suffixes instead of decimal

Question: Split a file into pieces, each containing 1000 lines, and use numeric suffixes

Command: split -l 1000 -d file.txt prefix
join: Joins the lines of two files on a common field.

Syntax: join [OPTION]... FILE1 FILE2

-a : Also print unpairable lines from a file

-e : Replace missing input fields with empty string

-i : Ignore differences in case when comparing fields

-t : Use CHAR as input and output field separator

-v : Print only unpairable lines from a file

-1 : Join on field NUMBER in file 1

-2 : Join on field NUMBER in file 2

Question: Join two files on the first field in each, ignoring case and using a colon as the field separator

Command: join -i -t ':' file1.txt file2.txt

File Permissions (Octal/Numerical)

File permissions are broken down into three categories: read (r), write (w), and execute (x). Each category is assigned a numerical value, with 4 for read, 2 for write, and 1 for execute. These numbers are used in a three-digit sequence to form a permission code.

For example, a permission code of 755 typically means that the owner has full read, write, and execute permissions (7), while the group and others have read and execute permissions (5).

User/Owner (First digit): This represents what the owner of the file can do.
Group (Second digit): This symbolizes the permissions for users who are part of the file's group.
Others (Third digit): This stands for the permissions of all other users.

The code also includes specific numerical representations for combinations of these permissions:

0: No permission.
1: Execute only.
2: Write only.
3: Write and execute.
4: Read-only.
5: Read and execute.
6: Read and write.
7: Read, write, and execute.

vi editor

It is essential to thoroughly know at least one text editor

Inserting text

i - insert before cursor

Quitting

:wq - exit, saving changes

:q - exit, if no changes

:q! - exit, ignore changes

Motion

h - move left

j - move down

k - move up (escape mode)

l - move right

w - move to the next word

b - move to beginning of the word

e - move to end of word

( - move a sentence back

) - move a sentence forward

{ - move a paragraph back

} - move a paragraph forward

0 - move to beginning of line

$ - move to end of line

:n - move to nth line

:set number - Displays each line number

G - move to the last line of file

H - move to top of screen

M - move to middle of screen

L - move to bottom of screen

System Control

| (pipe): Takes output of command on the left and passes it as the input to the command on the right.
Tee: Creates a branch in a pipe, capturing the output in a file while also continuing to send it down the pipeline

-a, --append: Append to the given files instead of overwriting them.

-i, --ignore-interrupts: Ignore interrupt signals.

--help: Display a help message and exit.

--version: Output version information and exit.

Example: echo "Hello World" | tee -a hello.txt
sudo: To run programs with the security privileges of another user (normally the superuser, or root).

-l: List the allowed (and forbidden) commands for the invoking user (or the user specified by the -U option) on the current host.

-k: Invalidate the user's cached credentials.

-u: Run the command as a specified user.
nohup: To run a command or process even if the session is closed (eg: nohup command &)
[command] &: Running a command followed by & will run the process in the background.
\>>: Appends the output of a command to a file.
\>: Redirects and overwrites the output of a command to a file.
stdin: Expects input from user or from another command. By default, stdin is connected to the keyboard
stdout: Connected to the terminal, and output is displayed on the screen
stderr: Displays error messages or diagnositc information from a command. It is separate from stdout to distinguish and handle error-related output separately. It is connected to the terminal and displays output on screen.
echo: To display line of text/string that are passed as an argument.

-n: Do not output the trailing newline.

-e: Enable interpretation of backslash escapes.
xargs: Read items from standard input and execute a command using those items as arguments

-n: Number of arguments taken from the input for each invocation of command.

-p: Prompt the user about whether to run each command line.
1>2&: This is used to redirect the output of file descriptor 1 (STDOUT) to file descriptor 2 (STDERR).
2>: Redirects stderr to a file, overwriting its contents
2>>: Redirects stderr to a file, appending the output to the existing content
fg %N: This command brings job number N to the foreground.
jobs: This command shows the jobs that are currently running in the background
man: To display the user manual of any command that we can run on terminal.

-f: Displays a short description from the manual page if available.

-k: Search the short descriptions and manual page names for the keyword.

File Compression

tar : Creates or extracts archives but does not compress. Can be used in combination with compression tools like gzip and bzip2

-x : Extract files from an archive

-v : Verbosely list the files processed

-z : Filter the archive through gzip

-f : Use archive file or device ARCHIVE

-c : Create a new archive

-t : List the contents of an archive

-j : Filter archive through bzip2

Question: Extract a gzip-compressed tar file and Create gzip-compressed tar file

Command: tar -xvzf file.tar.gz and tar -cvzf output.tar.gz folder/file_to_archive

Question: Create a bzip2-compressed tar file

Command: tar -cvjf file.tar.bz2 folder_to_archive
gzip: To compress or expand files.

-v : Display verbose information

-d : Decompress

-r : Recursively compress files in directories

-k : Keep (don't delete) input files during compression or decompression

-l : List compressed file details

-f : Force compression, even if it does not reduce the file size

-c : Write to standard output

Question: Compress a file

Command: gzip -v file.txt

Question: Decompress a file

Command: gzip -d file.txt.gz
gunzip: To decompress files compressed by gzip.

-v : Display verbose information

-r : Recursively decompress files in directories

-k : Keep (don't delete) input files during compression or decompression

-l : List compressed file details

-f : Force decompression

-c : Write to standard output

-t : Test the compressed file integrity

Question: Decompress a file and keep the original

Command: gunzip -k file.txt.gz
zip: To package and compress (archive) files into a zip file.

-r : Recurse into directories

-m : Move specified files into the zip archive

-v : Verbose operation

-f : Freshen: only changed files

-u : Update: only changed or new files

-d : Delete entries in a zip archive

-j : Junk (don't record) directory

Question: Create a zip file, including all files within a directory

Command: zip -r archive.zip directory
unzip: It is used for listing, testing, or extracting files from a ZIP archive.

-v : List verbosely or show version info

-l : List archive files (short format)

-q : Perform operations quietly (-qq = even quieter)

-n : Never overwrite existing files (default)

-o : Overwrite files without prompting

-d : Specify directory to extract files

-p : Extract files to pipe (stdout)

Question: Extract ZIP file into a specific directory

Command: unzip -d destination_folder archive.zip
zcat: Concatenates and displays compressed files.

-f : Force

-t : Test the compressed file integrity

-c : Use archive file

-l : List compressed file details

-v : Display verbose information

-r : Recursively decompress files in directories

-k : Keep (don't delete) input files during decompression

Question: Display the contents of a gzipped file

Command: zcat file.txt.gz
bzip2: A block-sorting file compressor that uses Burrows-Wheeler block sorting text compression algorithm, and Huffman coding.

-d : Decompress

-z : Compress

-k : Keep (don't delete) input files during compression or decompression

-v : Verbose mode

-f : Force overwrite of output files

-t : Test integrity of specified file(s)

-c : Output to standard out (stdout)

Question: Compress a file

Command: bzip2 -z file.txt

Question: Decompress a file

Command: bzip2 -d file.txt.bz2
bunzip2: The decompression tool for bzip2.

-k : Keep (don't delete) input files during decompression

-v : Verbose mode

-f : Force overwrite of output files

-t : Test integrity of specified file(s)

-c : Output to standard out (stdout)

Question: Decompress a file

Command: bunzip2 file.txt.bz2

bzip2 can compress files to a smaller size than gzip and is slower than gzip and consumes more memory.

The choice between gzip and bzip2 often depends on the specific needs of the task. If you want a faster compression and decompression process, you might choose gzip, while if you are looking for the smallest possible file size and are willing to spend more time compressing, you might opt for bzip2. bzip2 is used for compression of large files.

scp (secure copy): For copying files between machines over a secure and encrypted ssh connection.

Syntax: scp [options] source destination

-r: Recursively copy entire directories.

-p: Preserves modification times, access times, and modes from the original file.

-C: Compression enable. Passes the -C flag to ssh(1) to enable compression.

-P port: Specifies the port to connect to on the remote host.

Example: scp -r /local/directory/ username@remote_host:/remote/directory/
rsync (remote sync): For copying and synchronizing files/directories remotely as well as locally.

Syntax: rsync [options] source destination

-r: Recursively copy entire directories.

-a: Archive mode, it is a quick way of saying you want recursion and want to preserve everything.

-v: Verbose mode, it provides more details about what the command is doing.

-z: Compress file data during the transfer.

--delete: Delete files that are not there in source directory.

-n: Perform a trial run that doesn't make any changes (also known as "dry run").

Example: rsync -avz /local/directory/ username@remote_host:/remote/directory/
ssh (Secure Shell): Used for remote command-line login and remote command execution.

Syntax: ssh [options] username@hostname

-p: Specifies the port on which the server is running.

Example: ssh -p 22 username@hostname

Disk Management

df: Reports the amount of available disk space being used by file systems.

-h : Displays sizes in human readable format (e.g., KB, MB, GB).

-T : Displays the file system type.

-a : Includes filesystems having 0 blocks.

-x : Excludes file systems of the specified type.
du: Estimates file and directory space usage.

-h : Displays sizes in human readable format (e.g., KB, MB, GB).

-a : Includes files in the output, not just directories.

-s : Displays only the total for each argument.

-c : Produces a grand total.

-d : Display depths up to N levels only.
free: Displays the total amount of free and used physical and swap memory in the system.

-b : Displays output in bytes

-k : Displays output in kilobytes.

-m : Displays output in megabytes.

-g : Displays output in gigabytes.

-h : Displays output in a human-readable format.
mount: Mounts a filesystem.

-t : Specifies the file system type.

-o : Specifies mount options.

-r : Mounts the filesystem as read-only.

-w : Mounts the filesystem as read-write.
umount: Unmounts file systems.

-a : Unmounts all file systems.

-r : Attempts a remount of the file system as read-only if the unmount operation fails.

-d : Free loop device if it has been used.

Process Management

ps: Reports a snapshot of the current processes.

-a : Shows all processes with a terminal, including other users' processes.

-u : Provides detailed information about each process.

-x : Includes processes without a controlling terminal.

-e : Select all processes.

-f : Full-format listing.
top: Displays a dynamic real-time view of the running system.

-d : Specifies the delay between screen updates.

-p : Monitors only processes with specified process IDs.

-u : Displays only processes of a specific user.

-c : Shows command line/path information.
bg: Continues suspended jobs in the background.

bg %n : Where n is the job number (use jobs command to see this).
fg: Brings a job to the foreground.

fg %n : Where n is the job number (use jobs command to see this).
kill pid: Sends a signal to a process, usually to terminate the process.

-SIGKILL or -9 : Kills the process.

-SIGSTOP or -19 : Pauses the process.

-SIGCONT : Continues a stopped process.
pkill process-name: Sends a signal to processes based on their names.

-SIGKILL or -9 : Kills the process.

-SIGSTOP or -19 : Pauses the process.

-SIGCONT : Continues a stopped process.

Network

wget file_name: Download files from the web.

-c : Continue getting a partially-downloaded file.

-r : Turn on recursive retrieving.

-O : Write documents to file.

-q : Turn off wget's output
ping: Sends ICMP ECHO_REQUEST to network hosts.

host : Send ICMP ECHO_REQUEST packets to network hosts.
ip addr show: Show IP addresses and details about the network interfaces.
ip address add: Add a new IP address to a network interface.
ifconfig: Display or configure a network interface.
ping host: Check network connectivity to host.
whois domain: Get whois information for a domain.
dig domain: Query DNS lookups.

+short : Display only the answer.

-x : Reverse DNS lookup.
host google.com: Performs a DNS lookup.
hostname -i: Display the network address(es) of the host name.

Hardware-based Commands

cat /proc/cpuinfo: Displays detailed information about the system's CPU.
cat /proc/meminfo: Shows the system’s memory usage details.
lshw: Displays information on hardware configuration of the system.

-C : Only shows a particular class of hardware.

-short : Displays the output in a brief format.

-businfo : Displays the bus information.
lsblk: Lists out all the storage blocks, which includes disk partitions and optical drives.

-a : Shows all block devices

-b : Shows the size of the block devices in bytes.

-l : Displays the output in a list format.

System-based Commands

uname: Show system information. The uname command without any options will display the operating system name.

-a : Prints all system information, including machine name, kernel name, version, etc.

-r : Prints the kernel release.

-v : Prints the kernel version.

-m : Displays the machine hardware name.
uptime: Show how long the system has been running.
hostname: Show or set the system's host name.

-i : Displays the network address of the host name.

-f : Displays the fully qualified domain name (FQDN).
last reboot: Show the system's last reboot.
w: Show who is logged on and what they are doing.
whoami: Displays the username of the current user.
who: Display who is logged on. The 'who' command in Linux is used to get information about currently logged-in user to the system.

-a : Shows all information

-q : Only show names and number of users currently logged in

-r : Show current run level

Users Management Commands

id: Displays real and effective user and group IDs.

-u : Print only the effective user ID

-g : Print only the effective group ID

-n : Print names instead of numeric IDs (can be used with -u or -g)
last: Displays a list of the last logged-in users.

-n : Number of lines to display

-f : Display the specified file instead of /var/log/wtmp

-R : Don't display the hostname in the last column
groupadd: Creates a new group in the system.

Syntax: groupadd [option] groupname

-g : Specify a unique group ID for the new group

-o : Allow creating a group with a non-unique GID

-r : Create a system group
adduser: The adduser and addgroup commands add users and groups to the system according to command-line options and configuration information in /etc/adduser.conf.

--home : To specify the new user's home directory

--shell : To specify the new user's login shell

--no-create-home : Do not create the home directory

--uid : Specify the new user's UID
userdel: Deletes a user account and related files.

Syntax: userdel [option] username

-r : Removes the user's home directory and its contents

-f : Force deletion of the user account even if the user is still logged in
usermod: Modify a user account.

Syntax: usermod [option] login

-a : Add the user to the supplementary group(s). Use only with -G option

-c : Add a comment (also called the GECOS field)

-d : New home directory for the user

-e : Account expiration date in YYYY-MM-DD format

Conclusion

Mastering these commands empowers users to tackle a wide range of tasks, from basic file manipulation to advanced tasks. By embracing the flexibility and power of these utilities, you can unlock new possibilities, streamline your workflows, and become more adept at interacting with your computer at a fundamental level. Whether you refer to this guide as a handy reference or study it in depth, the knowledge contained herein is an essential addition to any technology enthusiast's repertoire.