Back to Linux

After so many months, still have to go back to this…

It’s an interesting topic we are discussing today, and it’s LINUX!

Well, I remember the first week in my coding bootcamp when I was reading some learning materials and the horrendous vim. At that time, these materials were marked as “optional”. So I just “hard coded” a few common commands like mkdir and touch , rf -rf , ls into my brain and let the rest slipped away.

I thought I had an easy escape, until I went into the industry and started my first proper job as a software engineer. Well, realistically, I can still get away with that a few commands I know, and rely on stackoverflow to give me some quick fix when I had to use Linux commands from time to time.

It was not easy, more and more often I feel that. Especially when I found that I had to spend hours on debugging simply because I don’t know some simple concepts in Linux like PATH or commands like using pipe.

She was still too young to know that life never gives anything for nothing, and that a price is always exacted for what fate bestows. — Stefan Zweig

So it’s time to go back to basics, then.

Noted that this is still just a summary of what I learnt about Linux and I won’t go into all the details about the usage of different kernels (as I don’t know…).

So be patient, and let’s start!

Before we go into Linux, let’s take a look at Unix first…

The reason we want to know about Unix is because Linux is derived from Unix.

UNIX is an operating system which was first developed in the 1960s, and has been under constant development ever since. A big part of Unix software is the idea of the “Unix philosophy” which is to have many small and independent unit that we can use to compose to solve larger problems.

The UNIX operating system is made up of three parts; the kernel, the shell and the programs.

The kernel of UNIX is the hub of the operating system: it allocates time and memory to programs and handles the filestore and communications in response to system calls.

The shell acts as an interface between the user and the kernel. When a user logs in, the login program checks the username and password, and then starts another program called the shell. The shell is a command line interpreter (CLI). It interprets the commands the user types in and arranges for them to be carried out. The commands are themselves programs: when they terminate, the shell gives the user another prompt (% on our systems). Your shell is running inside some sort of emulator. That emulator could be Terminal.app or iTerm2 if you’re on macOS. This the window that’s containing the shell, and you can use that emulator to switch out what shell is running inside of it. The popular ones are bash, zsh, or fish shell.

UNIX systems also have a graphical user interface (GUI) similar to Microsoft Windows which provides an easy to use environment.

For Linux, it incorporates many of its ideas and interfaces from Unix. It was created in 1991 by Linus Torvalds. He created Linux because at the time there was no free, open-source implementation of the Unix operating system.

A map of all the kernels of Linux

Common Command and Tips

So once we have a basic concept of Unix/Linux, we are looking at some popular commands.

Tilda: ~. It represents your user's home directory. Try ls -a in home directory and you will see all the directories in your home d.

Slash: / represents root. If you cd / you'd be at the most root directory of your project.

.bash_history: This file is constantly being appended to when you run commands. Typetail ~/.bash_history you should see what the last few commands you've ran in your previous session of bash.

head / tail: Head will read the first lines of a file and tail will read the last lines of a file. By default both will read 10 lines. You can modify by passing -n like head -n 3 textfile.txt .

cat: read the entire file.

wilcard:

Let’s say what if you want to match file1.txt and file2.txt but not file.txt. file*.txt will match all three of those, but ? will only match one character. So you could match file1.txt as well by doing ls f?le1.txt

touch file1.txt file2.txt file.txt
ls file*.txt
ls file?.txt

Notice the first ls call does output file.txt and the second one doesn't.

Finally you can use [] to limit your characters . For examplels file[1-5].txt matches file1.txt to file5.txt. Not to match this pattern you can add ^characters by using ls file[^1-5].txt

Expansions

Let’s say you wanted touch file1.txt file2.txt file3.txt . It’s very repetitive, but you can use {} .

touch file{4,5,6}.txt
touch {A,B,C,D}-profile.txt
touch file{0..10}.txt

Kill to terminate a programme. kill -9 (or kill -SIGKILL) and it will send the SIGKILL which is a force shutdown without clean up.

Shortcuts (this saves me so much time!)

• CTRL + A — takes you to the beginning of the line
• CTRL + E — takes you to the end of the line
• CTRL + K — remove everything after the cursor
• CTRL + U — remove everything before the cursor
• CTRL + Y — “paste” everything you removed

Streams

stdout

Even we might already know it, but just want to rewind the concept about streams as interconnected input and output communication channels between a computer program and its environment when it begins execution. The three input/output (I/O) connections are called standard input (stdin), standard output (stdout) and standard error (stderr). So how are they working in Linux?

While 1> redirects stdout, 2> redirect stderr.

echo "this will get output to the file and not to stdout" 1> new-file.txt

However ls -lsah 2> error-log.txt will output the normal output of stdout to the terminal and in error-log.txt it’s empty. To redirect both streams:

ls -lsah 1> stdout.txt 2> stderr.txt

By default you don’t need to specify 1 or 2.

ls -lsah > ls.txt

This will have both streams outputed to ls.txt .

/dev/null: Anything that gets output to /dev/null is thrown away. Useful when you just want to run the programme.

echo "hello" > /dev/null # you won't get anything as it goes to abyss!

stdin

To redirect input into something, let's say we want to find one very specific line in a long file:

grep "specific info" < ls.txt

How about combine stdin and stdout together?

grep "info" < ls.txt 1> ls2.txt 2> /dev/null
//find "info" from ls.txt
// output finding with stand out to ls2
//output error to abyss

Pipes

You might find above command is not straight forward. That’s how pipes come into use:|. This takes what one outputs and puts it into the next one. To rewrite the same with pipe:

cat ls.txt | grep "info"

This is especially useful when you want to find a certain programme. As ps will output all the programmes, and as it’s very long, it’s useful to using pipe and grep.

ps aux | grep "ps aux"
//99306   0.0  0.0  4268056    808 s010  S+   10:05pm   0:00.00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn --exclude-dir=.idea --exclude-dir=.tox ps aux
root             99305   0.0  0.0  4281404   1236 s010  R+   10:05pm   0:00.00 ps aux

This should output two lines, the ps aux call and the grep we're running to find that ps aux.

Users groups and permissions

Run ls -l you will see the permissions for each file and directory in that folder.

-rw-r--r--    1  staff 89849 29 Jun 19:10 ES.txt
drwxr-xr-x    6   staff  192 20 Aug 17:02 week5

The d or - represents if it's a directory or a file.

The next 3 groups represent file permissions, each group with 3 characters. The first is the file permissions for the user that owns that file, the next three for the group that owns that file, the last three for everyone.

For each group, r is for read permission, w is for write permission, and x is for execute permission. Executable means an executable program. If you run ls -l /usr/bin, you'll see many of such programs.

rw-rw-r-- means read-and-write for the user and the group, and read for everyone.

rwx------ means read write executable for the user and none for others.

Apart from these, there is a command chmod which allows you to change the permissions of the file.

For example, if we sudo touch secret.txt to make a file as root, and then try to echo "new content" >> secret.txt you will have permission denied error as the file is by default -rw-r--r-- .

In order to change the permission, try run

sudo chmod u=rw,g=rw,o=rw secret.txt so anyone can read or write to the file. Now rerun the above echo command and it should be fine now.

The shortcut for doing so is using numbers. 4 is read, 2 is write, 1 is executable, and 0 if nothing. chmod 640 secret.txt would make it read+write for the user, read for the group, and no permission for anyone else.

You can also use + and -. chmod +x secret.txt and it'll add executable to each of the permissions, and chmod -x secret.txt is the opposite.

So that’s pretty much it for this blog.

Happy Reading!