The Shell Program

Posted on Edited on In Views:

Sources:

  1. Machtelt Garrels. Chapter 1. Chapter 1. Bash and Bash scripts.
  2. 蔡德明. 第十一认识与学习Bash. 鸟哥的Linux私房菜.

The Shell Program

The concept of shell

At a high level, a "shell" program refers to any program that acts as an intermediary between the kernel and user applications. It operates at the lowest level of user-mode software, providing the primary interface for system interaction.

A shell is specifically a command interpreter - a program that takes user input (text commands or graphical actions), interprets and transfers them to the operating system to perform.

All shells provide a user interface, but the interface type varies:

  • Command line interfaces (CLI) accept text commands
  • Graphical interface (GUI) accept graphical actions (clicks, gestures)
1
2
3
4
5
6
7
Operating System Kernel
         ↑
    Shell Layer (Command Interpreter)
         ↑
    User Interface
    ├── Text Interface (CLI)
    └── Graphical Interface (GUI)

Both interface types perform the same core function - command interpretation:

1
2
3
4
5
6
7
8
9
Text-based Shell (bash/zsh):
User input: types "firefox"
Shell interprets: Launch Firefox browser
System action: Firefox process starts

Graphical Shell (GNOME Shell):
User input: clicks Firefox icon
Shell interprets: Launch Firefox browser  
System action: Firefox process starts

Command-Line Shells:

  • bash, zsh, fish (Linux/macOS)
  • PowerShell, cmd (Windows)

Graphical Shells:

  • GNOME Shell, Plasma Shell (Linux desktop environments)
  • Windows Explorer (Windows desktop)
  • Finder (macOS desktop)

This chapter focuses on Unix Command-Line shells (bash, zsh, etc.), though similar concepts apply to other operating systems like Microsoft Windows with PowerShell.

Different shell implementations create different command dialects and feature sets, but the fundamental concepts remain consistent across platforms. Once you understand one shell well, transitioning to others becomes much easier.

sh, bash and zsh

  • sh (Bourne Shell): The original UNIX shell, still available on every system for compatibility. It's a basic shell with minimal features but forms the foundation for other shells.
  • bash (Bourne Again Shell): The standard shell on most Linux distributions and the default for most users. It's a superset of the Bourne shell, meaning all sh commands work in bash, but with many additional features.
  • zsh (Z Shell): A modern shell with advanced features like better autocompletion, themes, and extensive customization options.

The sh command has been largely replaced by different implementations across modern systems:

  • Ubuntu/Debian: From Debian Squeeze onward, /bin/sh points to dash (Debian Almquist Shell) for better performance and POSIX compliance. Previously it was bash .
  • macOS: On macOS, /bin/sh is still interpreted by bash (version 3.2), though Apple has added dash and may eventually switch to it.
  • Other Linux Distributions: On many Linux systems, /bin/sh is still a symbolic link to bash, though this varies by distribution.

The /etc/shells file lists all available shells on the system (Linux and macOS):

1
2
3
4
5
6
❯ cat /etc/shells
/bin/sh
/bin/bash
/usr/bin/bash
/bin/rbash
/usr/bin/rbash

All examples in this post use bash since it's the most widely available and commonly used shell.

Shell session types

Shells operate in different modes based on how they're invoked:

Login vs Non-Login

  • Login Shell: Started when logging into the system from scratch
    • Examples: SSH sessions (ssh user@server), console login (Ctrl+Alt+F1), bash --login
    • Sets up complete user environment, reads login-specific config files
    • First character of $0, i.e., the name of the shell, is - (shows as -bash)
  • Non-Login Shell: Started when already logged into the system
    • Examples: New terminal windows in desktop, running scripts, VS Code terminal, bash from bash
    • Inherits environment from parent shell, reads session-specific files

Interactive vs Non-Interactive

  • Interactive Shell: Accepts user commands with prompt, provides editing/history/completion
  • Non-Interactive Shell: Executes scripts without user interaction, optimized for automation

Text-based vs Graphical Shells

  • Text-based shells (bash, zsh): Read text-shell-specific config files (~/.bashrc, ~/.zshrc)
  • Graphical shells: Read graphical-shell-specific config files

Common Combinations:

  • Interactive Login: SSH sessions, console login
  • Interactive Non-Login: Terminal applications in desktop environments
  • Non-Interactive Non-Login: Script execution, automated tasks

You can check your shell's mode:

1
2
3
4
5
# Check if login shell  
echo $0 # Shows "-bash" for login shell, "bash" for non-login shell 

# Check if interactive
[[ $- == *i* ]] && echo "Interactive" || echo "Non-interactive"

The Reality on macOS:

  1. Logging into macOS GUI: Graphical shell (doesn't read bash config, though)
  2. Opening iTerm2: Login command-line shell (reads shell config, i.e., ~/.bash_profile, ~/.zshrc, etc)
  3. Additional iTerm2 windows: Also login shells (by default). This is because macOS Terminal apps are configured to start login shells.

Configuration Files

Source: Shell initialization files

Bash initialization order

According to man bash,

  • Interactive login shell:
    1. /etc/profile (system-wide)
    2. First found: ~/.bash_profile, ~/.bash_login, ~/.profile (user-wide)
  • Interactive non-login shell:
    1. /etc/bash.bashrc (system-wide)
    2. ~/.bashrc (user-wide)

Key files:

  • /etc/profile: Sources /etc/bash.bashrc and executes all /etc/profile.d/*.sh scripts
  • /etc/bash.bashrc: System-wide interactive settings (prompt, aliases)
  • ~/.bashrc: User interactive settings (Ubuntu default, contains history settings, aliases)

Platform differences:

  • Ubuntu: Uses ~/.bashrc primarily
  • macOS: Uses ~/.bash_profile primarily
  • Missing files are skipped automatically

/etc/profile

Let's see it's content:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# In Ubuntu 22.04
# /etc/profile: system-wide .profile file for the Bourne shell (sh(1))
# and Bourne compatible shells (bash(1), ksh(1), ash(1), ...).

if [ "${PS1-}" ]; then
  if [ "${BASH-}" ] && [ "$BASH" != "/bin/sh" ]; then
    # The file bash.bashrc already sets the default PS1.
    # PS1='\h:\w\$ '
    if [ -f /etc/bash.bashrc ]; then
      . /etc/bash.bashrc
    fi
  else
    if [ "$(id -u)" -eq 0 ]; then
      PS1='# '
    else
      PS1='$ '
    fi
  fi
fi

if [ -d /etc/profile.d ]; then
  for i in /etc/profile.d/*.sh; do
    if [ -r $i ]; then
      . $i
    fi
  done
  unset i
fi

Basically, /etc/profile will do two things:

  1. Search if file /etc/bash.bashrc exists, if it does, execute it.

    1
    2
    3
    if [ -f /etc/bash.bashrc ]; then
          . /etc/bash.bashrc
    fi

  2. Search if dir /etc/profile.d exists, if it does, execute all the files (if readable) in it.

    1
    2
    3
    4
    5
    6
    7
    8
    if [ -d /etc/profile.d ]; then
      for i in /etc/profile.d/*.sh; do
        if [ -r $i ]; then
          . $i
        fi
      done
      unset i
    fi

/etc/bash.bashrc

We can see the content of /etc/bash.bashrc :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# System-wide .bashrc file for interactive bash(1) shells.

# To enable the settings / commands in this file for login shells as well,
# this file has to be sourced in /etc/profile.

# If not running interactively, don't do anything
[ -z "$PS1" ] && return

# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize

# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "${debian_chroot:-}" ] && [ -r /etc/debian_chroot ]; then
    debian_chroot=$(cat /etc/debian_chroot)
fi

# set a fancy prompt (non-color, overwrite the one in /etc/profile)
# but only if not SUDOing and have SUDO_PS1 set; then assume smart user.
if ! [ -n "${SUDO_USER}" -a -n "${SUDO_PS1}" ]; then
  PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi

/etc/profile.d

We can see the content of files in /etc/profile.d :

1
2
3
$ ls /etc/profile.d
01-locale-fix.sh  bash_completion.sh     debuginfod.csh  gnome-session_gnomerc.sh  vte-2.91.sh  xdg_dirs_desktop_session.sh
apps-bin-path.sh  cedilla-portuguese.sh  debuginfod.sh   im-config_wayland.sh      vte.csh

Every .sh file is just a simple shell script.

Zsh initialization order

--> Zsh manual

System-wide order: zshenv, zprofile, zshrc, zlogin User config: ~/.zshrc

Note: /etc/profile is only read by login shells, ~/.bashrc only by non-login shells. System creates the complete environment through this layered approach.

Hot to switch shell

Check availabe shells in the sysyem:

1
cat /etc/shells

Current shell:

1
echo $SHELL

set one shell as default for your user:

1
chsh -s <full-path-to-shell>

Shell and Syscall

由于系统调用fork()和exec()的分离，程序可以在fork()之后，exec()之前运行代码，方便了shell的工作

shell workflow

  • shell的workflow：
    1. 显示一个prompt,等待用户输入
    2. 用户进行输入，输入内容是一个命令，由一个可执行程序和若干参数组成
    3. shell找到该可执行程序。调用fork()创建新进程
    4. shell可能执行某些代码
    5. 调用exec()执行这个可执行程序
    6. 调用wait()等待该命令完成
    7. 子进程执行结束后，shell从wait()返回，继续步骤1

shell 重定向

shell实现重定向：对于wc 5_2.c > [filename].txt, wc的输出结果被重定向到[filename].txt

步骤解释：

  1. shell在fork()之后，exec()之前，会：

    1. 先用open打开文件[filename].txt, 给它分配一个文件描述符, 记为STDOUT_FILENO(一般是3，因为0，1，2都已被占用)
    2. 再关闭标准输出( fd = 1 )

  2. shell使用dup/dup2来分配一个STDOUT_FILENO的复制，由于dup默认是分配未使用的最小的fd），此时fd=1已经关闭，所以就又分配了fd=1，它是STDOUT_FILENO的复制

  3. shell执行指令，子进程准备输出，由于UNIX系统会从零开始寻找可用的fd(文件描述符)，因此STDOUT_FILENO会成为第一个可用的fd，作为子进程输出的目标

详见OS Persistence

  • UNIX pipe也用类似方式实现，但使用pipe()系统调用,将前一个进程的输入作为后一个进程的输出：grep -o foo fole | wc -l

Session 的创建和销毁

session的创建：

  • 通常，新的 session 由系统登录程序创建，session 中的领头进程是运行用户登录 shell 的进程。新创建的每个进程都会属于一个进程组，当创建一个进程时，它和父进程在同一个进程组、session 中。

将进程放入不同 session 的惟一方法是使用 setsid 函数使其成为新 session 的领头进程。这还会将 session 领头进程放入一个新的进程组

session的销毁：

  • 当 session 中的所有进程都结束时 session 也就消亡了。实际使用中比如网络断开了，session 肯定是要消亡的。
  • 让 session 的领头进程退出。一般情况下 session 的领头进程是 shell 进程，如果它处于前台，我们可以使用 exit 命令或者是 ctrl + d 让它退出。或者我们可以直接通过 kill 命令杀死 session 的领头进程。
    • 原理是：当系统检测到挂断(hangup)条件时，内核中的驱动会将 SIGHUP 信号发送到整个 session。通常情况下，这会杀死 session 中的所有进程

session 与终端的关系：

  • 如果 session 关联的是伪终端，这个伪终端本身就是随着 session 的建立而创建的，session 结束，那么这个伪终端也会被销毁。

    • 打开终端，会话开始；关闭终端，会话结束，会话内部的进程也会随之终止，不管有没有运行完。

      一个典型的例子就是，SSH 登录远程计算机，打开一个远程终端执行命令。这时，网络突然断线，再次登录的时候，是找不回上一次执行的命令的。因为上一次 SSH 会话已经终止了，里面的进程也随之消失了

      为了解决这个问题，会话与窗口可以"解绑"：窗口关闭时，会话并不终止，而是继续运行，等到以后需要的时候，再让会话"绑定"其他窗口（见“ Terminal Multiplexer”）

  • 如果 session 关联的是 tty1-6，tty 则不会被销毁。因为该终端设备是在系统初始化的时候创建的，并不是依赖该会话建立的，所以当 session 退出，tty 仍然存在。只是 init 系统在 session 结束后，会重启 getty 来监听这个 tty