Files, and their properties

2.5.1. Files, and their properties#

In Listing the content of a directory you learned the basic usage of the ls command to “list” all files in a directory. A very useful option to the ls command you saw before is -l, which tells ls to give more information on each file or directory:

$ ls -l /bin
total 6632
-rwxr-xr-x 1 root root  818232 2010-04-19 03:51 bash
-rwxr-xr-x 3 root root   30200 2010-02-08 11:54 bunzip2
-rwxr-xr-x 1 root root 1269432 2010-04-22 22:04 busybox
-rwxr-xr-x 3 root root   30200 2010-02-08 11:54 bzcat
lrwxrwxrwx 1 root root       6 2010-07-07 11:48 bzcmp -> bzdiff
-rwxr-xr-x 1 root root    2140 2010-02-08 11:54 bzdiff
lrwxrwxrwx 1 root root       6 2010-07-07 11:48 bzegrep -> bzgrep
-rwxr-xr-x 1 root root    4874 2010-02-08 11:54 bzexe
(etc.)

First of all, it shows you how many blocks the entire directory takes up on the filesystem. A block does not have the same size on all machines, but on Linux PCs it’s usually one kilobyte (1024 bytes). The directory /bin seems to take up 6632 blocks, or roughly 6.6 megabytes.

Next, for each file, ls -l now shows the file type, the first character of the string on the left. The file type indicates whether the file is a plain file (indicated by -), a link (l, see below) or a directory (d). There are some other file types, like b (block device) and c (character device), but you will not encounter these regularly, so don’t worry about them.

Some other important information given by ls -l is the user and group of the file (here: user root, group root); the date and time of creation; and, of course, the filename.

Exercise 2.65

A directory ~/project already exists. Change into it and run ls -l to see its contents for yourself.

{
  "filesystem": {
    "/home/student/project/README.md": "# Project notes\n",
    "/home/student/project/run.sh": "#!/bin/bash\necho hi\n",
    "/home/student/project/src/": null,
    "/home/student/project/data.txt/": null,
    "/home/student/project/logs": "No messages logged",
    "/home/student/project/src_link": { "symlink": "src" }
  }
}

Based on the first character shown by ls -l, select all of the following that are true:

2.5.1.1. Wildcards and glob patterns#

Above, you used ls to list files. However, often you want to limit the number of files you see, for example when you know that you’re looking for a filename containing a certain word. If this is the case, you can use wildcards: special characters that the shell expands into a list of matching filenames before it even runs the command. A command argument built out of ordinary characters plus one or more wildcards (like my*.txt below) is called a glob pattern (or just “a glob”), and the expansion itself is called globbing. (The name comes from an early Unix program literally called glob, short for “global”: it expanded these patterns into matching filenames before shells did that job themselves.)

Here are some of the most common wildcards, with some example glob patterns in action:

* will match any piece of filename. For example:
- ls *: list all files in the current directory;
- ls my*: list all file names starting with my;
- ls *.txt: list all file names ending with the .txt file extension (including the .);
- ls /etc/*.conf: list all files in the /etc directory ending with .conf.
- ls */*: list all files in all immediate subdirectories.
? will match a single character in a filename. For example:
- ls myfile?.txt: lists myfile0.txt, myfile1.txt, myfileq.txt, etc.
[abZ] will match the letter a, or the letter b, or the letter Z. For example:
- ls myfile[X_8b].txt: can list myfileX.txt, myfile_.txt, myfile8.txt and myfileb.txt.
[a-z] matches all letters in the range a - z, e.g.:
- ls myfile[A-K].txt: lists myfileA.txt … myfileK.txt.
- ls myfile[0-4].txt: lists myfile0.txt, myfile1.txt, myfile2.txt, myfile3.txt or myfile4.txt.
- ls myfile[a-zA-Z].txt: lists any variant of the same file, with either a lowercase (e.g., myfilej.txt) or an uppercase letter (e.g., myfileY.txt)

You can always use wildcards in the Unix shell when you have to specify a filename.

Exercise 2.66

List all files in /usr/bin starting with a z. Also list all files in /usr/bin that contain the string text in their name.

Exercise 2.67

cd into ~/photos, which already contains a mix of numbered and lettered files, and a couple of subdirectories. Using the wildcards from above:

List only the files named photo followed by a single digit and the extension .jpg (so photo1.jpg, but not photoA.jpg), using ?.
List only the log files report_a.log through report_e.log (not report_1.log), using a character range.
List every file with the .jpg extension, regardless of what comes before it, using *.
List every file inside all of ~/photos’s immediate subdirectories, in a single command, using a directory wildcard.

{
  "filesystem": {
    "/home/student/photos/photo1.jpg": "",
    "/home/student/photos/photo2.jpg": "",
    "/home/student/photos/photo3.jpg": "",
    "/home/student/photos/photoA.jpg": "",
    "/home/student/photos/report_a.log": "",
    "/home/student/photos/report_b.log": "",
    "/home/student/photos/report_c.log": "",
    "/home/student/photos/report_1.log": "",
    "/home/student/photos/vacation/beach.jpg": "",
    "/home/student/photos/work/slides.pdf": ""
  },
  "checks": [
    { "type": "commandEvent", "eventType": "coreutil.ls",
      "match": { "absPaths": { "$equals": ["/home/student/photos/photo1.jpg", "/home/student/photos/photo2.jpg", "/home/student/photos/photo3.jpg"] } },
      "desc": "should list exactly photo1.jpg, photo2.jpg and photo3.jpg, and nothing else" },
    { "type": "commandEvent", "eventType": "coreutil.ls",
      "match": { "absPaths": { "$equals": ["/home/student/photos/report_a.log", "/home/student/photos/report_b.log", "/home/student/photos/report_c.log"] } },
      "desc": "should list exactly report_a.log, report_b.log and report_c.log, and nothing else" },
    { "type": "commandEvent", "eventType": "coreutil.ls",
      "match": { "absPaths": { "$equals": ["/home/student/photos/photo1.jpg", "/home/student/photos/photo2.jpg", "/home/student/photos/photo3.jpg", "/home/student/photos/photoA.jpg"] } },
      "desc": "should list exactly the four .jpg files, and nothing else" },
    { "type": "commandEvent", "eventType": "coreutil.ls",
      "match": { "absPaths": { "$equals": ["/home/student/photos/vacation/beach.jpg", "/home/student/photos/work/slides.pdf"] } },
      "desc": "should list exactly the files in vacation/ and work/, and nothing else" }
  ]
}

The use of wildcards is not unique to ls, but works with any command that can take one or more file paths as arguments.

Exercise 2.68

A single cat command with multiple file names as arguments, like cat a.txt b.txt, would print the content of all the given files, one after another. In other words, it would “concatenate” the file contents of a.txt and b.txt (hence the name cat).

Various text files already exist in your home directory. Use a single cat command with a glob pattern to print all text files with the .txt extension, one after another, without writing out the full filenames as arguments.

{
  "filesystem": {
    "/home/student/myfile1.txt": "first file\n",
    "/home/student/myfile2.txt": "second file\n",
    "/home/student/myfile3.txt": "third file\n",
    "/home/student/myfile4.md": "markdown file\n",
    "/home/student/zzz.txt": "last file\n"
  },
  "checks": [
    { "type": "commandEvent", "eventType": "coreutil.cat",
      "match": { "absPaths": ["/home/student/myfile1.txt", "/home/student/myfile2.txt", "/home/student/myfile3.txt", "/home/student/zzz.txt"] },
      "desc": "should cat all .txt files in one command" },
    { "type": "commandSucceeded", "pattern": "^cat\\s+\\S*[*?]\\S*\\s*$",
      "desc": "should use an actual glob pattern (e.g. myfile*.txt), not just list both filenames literally" },
    { "type": "stdoutContains", "text": "first file" },
    { "type": "stdoutContains", "text": "second file" },
    { "type": "stdoutContains", "text": "third file" },
    { "type": "stdoutContains", "text": "last file" },
    { "type": "stdoutContains", "text": "markdown file", "negate": true }
  ]
}

2.5.1.2. Links#

A link (indicated by an l in the output of ls -l) is not a file, but a pointer to a file. It can also be a pointer to a directory. Links can be useful for when you want to have a file with changing content but a fixed name. For example, you could create files news_01_01_2010, news_02_01_2010 and so on, and then have a link news_today which points to the latest file. These links can be either hard or soft (also called symbolic). A hard link means that there is actually one physical file with two names; if you throw away the original, the other link still points to the same content. A soft link is different; you can remove the link without affecting the file itself, but if you remove the original, the link is left dangling. Soft links are probably the only ones you will be using.

In the directory listing of /usr/bin above, you can see that bzcmp is a link to bzdiff, as indicated by the arrow -> in the filename part (bzcmp -> bzdiff). That means that if you run the program bzcmp, really the program bzdiff is run. Note that you can only see the arrows with the linked names if you include the -l option of ls.

You can create links using the ln command (for link).

Exercise 2.69

Try:

$ cd
$ ln -s /etc/passwd passwords

This will create a symbolic link in your home directory which points to the password file in the directory /etc. You can now use this link as if it were the password file itself:

$ less passwords

After you’re done looking through the password file, remove the link:

$ rm passwords

Fortunately, this will only remove the link, not the real password file…

If you don’t specify a destination, ln will create a link in the current directory with the same name as the original file. Try:

$ ln -s /usr/share/common-licenses/GPL

Exercise 2.70

Two news files already exist in your home directory: news_01_01_2010.txt and news_02_01_2010.txt. As described above, create a symbolic link called news_today that always points to the most recent one (news_02_01_2010.txt), then read through the link with cat or less to check it works.

{
  "filesystem": {
    "/home/student/news_01_01_2010.txt": "Happy new year!\n",
    "/home/student/news_02_01_2010.txt": "Second day of the year, still quiet.\n"
  },
  "checks": [
    { "type": "commandEvent", "eventType": "coreutil.ln",
      "match": { "symbolic": true, "pairs": { "absTarget": { "$includes": "news_02_01_2010.txt" }, "absLinkName": { "$includes": "news_today" } } },
      "desc": "should create a symbolic link news_today -> news_02_01_2010.txt" },
    { "type": "fileContains", "path": "/home/student/news_today", "text": "Second day of the year",
      "desc": "reading news_today should show the newer file's content" },
    { "type": "tabCompletionHint" }
  ]
}

Exercise 2.71

A file original.txt already exists in your home directory. Create a symbolic link link.txt that points to it, then use cat on both original.txt and link.txt to confirm they show the same content. Now remove original.txt, and try cat link.txt again: what happens, and why?

{
  "filesystem": {
    "/home/student/original.txt": "Hello from the original file!\n"
  },
  "checks": [
    { "type": "commandEvent", "eventType": "coreutil.ln",
      "match": { "symbolic": true, "pairs": { "absTarget": { "$includes": "original.txt" }, "absLinkName": { "$includes": "link.txt" } } },
      "desc": "should create a symbolic link link.txt -> original.txt" },
    { "type": "stdoutContains", "text": "Hello from the original file!", "pattern": "original\\.txt",
      "desc": "cat original.txt should show its content" },
    { "type": "stdoutContains", "text": "Hello from the original file!", "pattern": "link\\.txt",
      "desc": "cat link.txt should show the same content, while original.txt still exists" },
    { "type": "commandEvent", "eventType": "coreutil.rm",
      "match": { "absPaths": ["/home/student/original.txt"] },
      "desc": "should remove original.txt" },
    { "type": "fileExists", "path": "/home/student/original.txt", "negate": true,
      "desc": "original.txt should no longer exist" },
    { "type": "stderrContains", "text": "No such file or directory", "pattern": "link\\.txt",
      "desc": "reading link.txt should fail" }
  ]
}

2.5.1.4. User groups#

As discussed above, you can assign permissions for the user of a file, the group of a file and others. The output of ls -l shows the user and group to which the files belong. In the listing of /usr/bin above, all files are owned by user root, who is in group root.

You might wonder what a group exactly is. Groups were made to make system administration easier. Say you run a company in which three project teams work on different projects. You would like the members of each team to be able to read and write the team’s files, but members of one team shouldn’t be able to read another team’s files. The group mechanism makes this very easy, by putting users into three different groups and setting the file permissions correctly.

To see what groups you belong to, you can (for example) use id:

Exercise 2.78

Try this:

$ id

It should print your user id (that is, your user number), your group id and all groups you belong to.

Exercise 2.79

You just used id to find your own numerical user id. What is the numerical user id of the root user? Find out for yourself.

{
  "checks": [
    { "type": "stdoutMatches", "pattern": "uid=0\\(root\\)", "cmdPattern": "id",
      "desc": "run a command that shows the root's uid" }
  ]
}

If you belong to more than one group, you can decide which group your file should have. The command chgrp can be used to assign a new group to the file, for example:

$ chgrp othergroup myfile.txt

You are often part of many groups, such as the people allowed to use the modem (dialout) or that can administer the printer (lpadmin). The role of groups for sharing files with different people has been largely replaced by other mechanisms.

Exercise 2.80

The file shared_notes.txt already exists in your home directory, currently owned by your own user and group. First run id to confirm which groups you belong to. Since root is not among them, change the file’s group to root anyway using sudo chgrp.

{
  "filesystem": {
    "/home/student/shared_notes.txt": "shared with the team\n"
  },
  "checks": [
    { "type": "commandEvent", "eventType": "coreutil.id", "desc": "should run id to check your groups" },
    { "type": "commandEvent", "eventType": "coreutil.chgrp",
      "match": { "absPaths": ["/home/student/shared_notes.txt"], "group": "root" },
      "desc": "should use sudo chgrp root to change shared_notes.txt's group" },
    { "type": "fileGroup", "path": "/home/student/shared_notes.txt", "group": "root" }
  ]
}

Files, and their properties

Contents

2.5.1. Files, and their properties#

2.5.1.1. Wildcards and glob patterns#

2.5.1.2. Links#

2.5.1.3. File permissions#

2.5.1.4. User groups#

2.5.1.5. Hidden files#