Common Linux Commands

Posted on Edited on In Views:

Common Linux commands and operations.

Common Linux Commands

Data Compression

Archive a directory with tar + zstd

My default choice for large experiment outputs or checkpoints.

1
tar -I 'zstd -19 -T0' -cf all_ckpts.tar.zst all_ckpts

Flags:

  • -I 'zstd -19 -T0': use zstd as the compressor
  • -19: high compression level
  • -T0: use all CPU threads
  • -c: create archive
  • -f all_ckpts.tar.zst: output file

Notes:

  • all_ckpts is the input directory.
  • This preserves the directory structure.

Extract a .tar.zst archive

1
tar -I zstd -xf all_ckpts.tar.zst

Flags:

  • -I zstd: use zstd for decompression
  • -x: extract
  • -f all_ckpts.tar.zst: archive file

File Transfer Between Servers

Send a file with rsync

1
rsync -avP -e "ssh -p <PORT>" all_ckpts.tar.zst <USER>@<HOST>:/path/to/target/

Example:

1
rsync -avP -e "ssh -p 2222" all_ckpts.tar.zst [email protected]:/data/backups/

Flags:

  • -a: archive mode
  • -v: verbose
  • -P: show progress and keep partial transfers
  • -e "ssh -p <PORT>": use SSH with the given port

Notes:

  • Better than scp for large files or unstable connections.
  • Supports resume.

Pull a file with rsync

1
rsync -avP -e "ssh -p <PORT>" <USER>@<HOST>:/path/to/all_ckpts.tar.zst .

Notes:

  • Downloads the file into the current directory.

Disk Usage

Show all non-hidden files and directories in the current directory, sorted by size

1
du -sh -- * | sort -hr

Flags:

  • du: disk usage
  • -s: show only the total size for each argument
  • -h: human-readable sizes
  • --: stop option parsing
  • *: all non-hidden files and directories in the current directory
  • sort -h: sort by human-readable size
  • sort -r: reverse order (largest first)

Show all files and directories, including hidden ones, sorted by size

1
find . -maxdepth 1 -mindepth 1 -exec du -sh {} + | sort -hr

Flags:

  • find .: search from the current directory
  • -maxdepth 1: only the current level
  • -mindepth 1: exclude . itself
  • -exec du -sh {} +: run du -sh on the matched entries
  • sort -hr: sort from largest to smallest

Notes:

  • This includes hidden entries.
  • Unlike *, it will not skip names starting with ..

Show only directories, sorted by size

1
du -sh -- */ | sort -hr

User Config

Disk Config

fdisk：查看磁盘信息

1
sudo fdisk -l

卸载分区

1
2
#sudo umount  [磁盘路径]
sudo umount /dev/nvme0n1p11

将分区格式化为ext4类型

1
2
#（这里分区为/dev/nvme0n1p11）
sudo mkfs.ext4  /dev/sdb1

挂载分区（临时）

挂载分区（ 这里挂载到到/data目录下 ）

1
2
#sudo mount  /dev/sdb1  /[目录名] （目录当然是已存在的）
sudo mount /dev/sdb1 /data
  • 这个挂载是临时的，重新开机就会丢失。 如果要开机自动启动挂载， 需要编辑/etc/fstab

查看硬盘和挂载分区等信息

1
df  -h

这样就成功添加了一块硬盘并挂载到/data目录下了，

设置开机自动挂载

  1. 查询UUID
1
2
3
4
5
6
ls -al /dev/disk/by-uuid

#输出为：
...
... 88e7c2eb-82e6-48c2-a3d8-829c32468f1f -> ../../nvme0n1p11
...

可以查到对应分区nvme0n1p11的uuid为88e7c2eb-82e6-48c2-a3d8-829c32468f1f

  1. 编辑/etc/fstab(用来存放文件系统的静态信息的文件)
1
sudo vim /etc/fstab

末尾加上UUID=刚刚复制的UUID /data ext defaults 0 0

1
UUID=刚刚复制的UUID /data ext4 defaults 0 0

Common operations

clipboard

pbcopy and pbpasteon OSX can Copy&Paste data from stdin to the clipboard.

Open a terminal and run:

1
cat ~/Desktop/ded.html | pbcopy

The file is now in your clipboard.

To put it somewhere else (i.e. paste it) run:

1
pbpaste > ~/Documents/ded.html

Now you should have a copy of ded.html sitting in ~/Documents.

Data compression

xxd

xxd - make a hexdump or do the reverse.

查看键盘输入的字符对应的16进制表示:

1
xxd -ps

Example

输入 <Ctrl+b> + c，其会显示该输入的 hex codes 为：

1
2
^Bc
02630a

其中，02 代表 <Ctrl+b>，63 代表 c，而 0a 代表回车键

Aliases

1
alias alias_name="command_to_alias arg1 arg2"

alias是一个command，接受一个参数， 因此=左右不能有空格

example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# Make shorthands for common flags
alias ll="ls -lh"

# Save a lot of typing for common commands
alias gs="git status"
alias gc="git commit"
alias v="vim"

# Save you from mistyping
alias sl=ls

# Overwrite existing commands for better defaults
alias mv="mv -i"           # -i prompts before overwrite
alias mkdir="mkdir -p"     # -p make parent dirs as needed
alias df="df -h"           # -h prints human readable format

# Alias can be composed
alias la="ls -A"
alias lla="la -l"

# To ignore an alias run it prepended with \
\ls
# Or disable an alias altogether with unalias
unalias la

# To get an alias definition just call it with alias
alias ll
# Will print ll='ls -lh'

alias是面向session的，session关闭也使得alias失效， 要想使alias持久化，可以将其写在shell的启动文件，如 .bashrc or .zshrc

clear

清屏：

1
clear

切换tty

  1. CTRL + ALT + Fn

  2. ```shell sudo chvt N # N: tty number, 1 represents the main tty 

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13





    ## 下载

    ```shell
    # Download the contents of a URL to a file (named "foo" in this case):
    wget https://example.com/foo

    # Download the contents of a URL to a file (named "bar" in this case):
    wget -O bar https://example.com/foo

主机操作

查看主机名：

1
hostname

更改主机名：

1
vim /etc/hostname # 编辑该文件

添加域名映射：

1
vim /etc/hosts

时间

查看时间：

1
date

查看发行版

1
lsb_release -a