IBM Data Engineering: Hands-on Introduction to Linux Commands and Shell Scripting
This set of notes summarizes key concepts from the first three modules of IBM's "Hands-on Introduction to Linux Commands and Shell Scripting" course. The focus is on learning Linux from the terminal, working with commands, writing basic shell scripts, and automating tasks using cron. The learning flow moves from basic command-line usage into scripting concepts and job scheduling.
While much of the syntax is strict and context-dependent, the mental model built around piping, I/O redirection, and process execution proved consistent. These notes serve as a condensed reference archive and are written with accuracy and retention in mind rather than tutorial clarity.
Module 1 – Introduction to Linux
The first module focuses on foundational knowledge: Linux origins, structure, and terminal use. It introduces Linux as a Unix-like OS and outlines key differences in architecture, licensing, and use cases.
The system architecture is split into five layers:
Hardware
Kernel (manages memory, processes, devices)
OS services
Applications and shell
User interface
The filesystem uses a single-rooted tree structure with key directories like /home
, /usr
, /bin
, /boot
, and /media
Navigation commands are introduced:
pwd # Show current working directory ls # List files and directories cd .. # Move up one directory cd ~ # Go to home directory
This module also introduced the difference between shells and terminals and explained that most Linux distributions default to bash
, but other shells like sh
, zsh
, or fish
may be used.
Module 2 – Linux Commands and Operations
This module covers basic Linux CLI operations and the commands used to query system state, manage files, handle text content, and work with remote resources.
Informational commands:
whoami
,id
,uname
,df
,ps
,top
,date
for system and user info.echo $PATH
and other environment variables.
Manual pages are accessed with man
, and simplified documentation is available through tldr
.
File and directory management:
Includes creation, deletion, navigation, and copying/moving:
mkdir test # create directory rm file.txt # remove file cp file1 file2 # copy file mv file1 file2 # rename or move file touch file.txt # create empty file or update timestamp chmod +x script.sh # add executable permission
find
is introduced for recursive search
File permissions:
Ownership is defined by user and group.
Permissions follow the
rwx
bit pattern and differ for files vs directories.Use
ls -l
to view permissions; prefixd
indicates directory.
Text viewing and processing:
Basic tools:
cat
,more
,less
– view fileshead
,tail
– limit lineswc
– count lines, words, characters
Filters and processors:
sort
,uniq
,cut
,paste
,grep
,tr
– manipulate file content by lines, fields, or character patterns.
Redirection and pipes are implied, but more thoroughly covered in Module 3.
Networking:
Covers basic commands for connectivity and file transfer:
hostname
,ip addr
,ping
,curl
,wget
Concepts:
IP addressing and network nodes
URL structure: protocol, hostname, path
Curl outputs raw HTTP; wget focuses on download
Archiving and compression:
Linux uses .tar
and .gz
(often combined as .tar.gz
) for archiving and compression.
tar -czf archive.tar.gz folder/ # create compressed archive tar -xzf archive.tar.gz # extract zip -r archive.zip folder/ # zip format unzip archive.zip # extract zip
Notably, tar
archives first and compresses second, unlike zip
which does both simultaneously
Module 3 – Shell Scripting
This module focuses on writing and executing shell scripts, defining variables, working with pipelines, and automating via cron.
Scripting Basics:
Scripts start with a shebang directive:
#!/bin/bash
Scripts are made executable with chmod +x
and executed via ./script.sh
Variables and input:
name="Alice" echo $name read user_input
Shell variables have local scope. Environment variables are exported using export
Piping and redirection:
Introduced the use of pipes (|
) to chain commands:
cat data.txt | grep pattern | sort | uniq
Standard redirection:
>
overwrite>>
append2>
stderr<
input
Command substitution:
now=$(date)
Quoting rules:
'single quotes'
– literal"double quotes"
– interpolate variables\escape
– escape single characters
Conditionals and loops:
Introduces basic control flow using if
, else
, and fi
:
if [ "$name" == "Alice" ]; then echo "Hello" else echo "Bye" fi
Arrays:
my_array=(1 2 "three") echo ${my_array[0]} for item in "${my_array[@]}"; do echo $item done
Arithmetic:
echo $((3 + 2))
Job scheduling with cron
:
Crontab syntax:
m h dom mon dow command
Example:
30 15 * * 0 date >> Sundays.txt
Runs every Sunday at 15:30 and appends date to a file.
Other cron commands:
crontab -e
– edit job tablecrontab -l
– list jobscrontab -r
– remove all jobs
Final Thoughts
These three modules collectively establish a usable foundation for Linux command-line operations and shell scripting. Many commands build upon standard input/output streams and can be combined through piping, scripting, and scheduling. Syntax precision is critical, especially in conditionals, quoting, and environment variable handling. This material forms the basis for task automation, file manipulation, and light-weight data processing using Linux.
Course Certificate: View on Coursera
All notes and opinions are personal interpretations of the IBM Python for Data Science, AI & Development course on Coursera.