Category: Linux I : Linux Fundamentals

Running Windows and Macintosh Programs In Linux

Read Chapter 5, Accessing and Running Applications, Running Windows, DOS and Macintosh Applications, in the Red Hat Linux Bible.

Lots of software is available pre-compiled to run on Linux. Almost any UNIX-compatible program for which source code is available can be compiled, if packages aren’t already available. But what if you need to cross platforms?

• DOS
• Windows: older Win32
• Windows: NT and later
• Macintosh

 

DOS Emulators

Emulators “imitate” another environment. See http://rpmfind.net/ for an excellent resource for emulators (and a lot of other software). On that site, perform a search for these DOS-specific emulators:

• dosemu
• xdosemu

These packages are updated constantly; use the ones highest on the displayed lists. Keep in mind that you want the .i386 versions, not the .src versions (unless you want to compile them yourself). The dosemu package is more generally compatible; if you want to use xdosemu you’ll need to check your distro’s X library and glibc compiler for compatibility.

Download the .rpm package. Depending on system configuration, this file will arrive in your home directory or on your desktop if you’re running Gnome/KDE. The easiest way to perform this process is from the GUI; I’ll assume this is your working environment for this example.

Double-click the .rpm, and confirm all defaults as they’re offered.

When this process is complete (and it won’t give you much notice that it is), you can run:

whereis dosemu

If you installed as root, dosemu will be installed in /usr/bin/dosemu. Call it directly from that directory:

/usr/bin/dosemu

dosemu will be configured to run in /root/dosemu.

TAKE NOTE OF THE MESSAGE ABOUT SHUTTING DOWN DOSEMU:

YOU MUST COMMAND:

exitemu

TO LEAVE DOSEMU!

Now, let dosemu start. You can run:

dir

to see a listing of files in your “virtual” C: drive. Try:

edit config.sys

to see the familiar edit interface. Note that you have mouse support for the graphical menus.

Note that dosemu will often want access to your floppy drive. Normally only root and the floppy group can access it. Command:

chmod 666 /dev/fd0

to make it more widely available. Do be aware of the security considerations of doing this.

 

WINE (Wine Is Not an Emulator)

Be aware, this gets political: WINE truly is not an emulator. It is a Windows compatibility layer.

Go to http://winehq.org/ to learn more about WINE, particularly to check application compatibility. Follow the links to get pre-compiled packages for your distro.

Note that some packages are available in both i386 and i686 versions. Generally either will work for post-Pentium Pro-generation processors. See the WikiPedia entry for more discussion of i-numbering schemes.

You can get a commercial (and supported) version called WineX from Transgaming. If you’re specifically out to run Microsoft Office, you’ll need the commercial CrossOver Office package from CodeWeavers. But if your intent is to run games or mostly non-Microsoft applications for Windows, WINE by itself usually does the job.

You can install WINE from a Red Hat/Fedora GUI environment in the same way as dosemu: just double-click the .rpm and let it install.

Configuration files will be placed in /etc/wine/, and a pseudo-C: drive will be created in /usr/share/wine-c.

 

WINE Drive Mappings: Drive Letters to Linux Directories

You should know the default drive mappings WINE is going to use:

Name	Drive Letter	Linux Directory	Device
Floppy	A:	/mnt/floppy	/dev/fd0
Hard Disk 1	C:	/usr/share/wine-c
CD-ROM	D:	/mnt/cdrom	/dev/cdrom
Temp Dir	E:	/tmp
Home	F:	/home/username ( $HOME )
File system root	Z:	/

 

A Demo Installation Under WINE

This example assumes you have a connection to obtain two files: either a shared partition on a dual-boot machine, or a Windows share you can reach via Samba (smbclient), a floppy disk, a web site containing these files – any method available to you.

1. Get mspaint.exe.

2. Copy it to your /bin directory.

3. Run WINE. Command:

wine /bin/mspaint.exe

4. Note the message you receive.

5. Copy the DLL WINE tells you that you need. You’ll need it to be in the same place on your pseudo-C: drive that it would be under Windows: the C:/windows/system folder. Under WINE, this is at:

/usr/share/wine-c/windows/system

6. Now try to run Paint again.

 

Other Emulators

There are two other emulation environments of which you should be aware.

The first is the mtools. Generally, these are DOS-like tools and commands that have the letter “m” added to the front. See Chapter 5 of the Red Hat Linux Bible for details. The important thing to know as you get started with mtools is that they operate as if your current working directory is the floppy drive (/def/fd0 or “A:”.) If you don’t have a floppy drive, or don’t have a disk in that drive, or your user doesn’t have permissions to access that drive, you’re in difficulty immediately. They’re worth exploring, but generally are useful only for dealing with a floppy disk you’re going to pass back and forth from Windows to Linux.

The second is ARDI. This is called an “executor” rather than an emulator, strictly speaking, because like WINE it is actually an application compatibility layer. NOTE that for most terminal-based UNIX software that’s compiled for Mac, you can get source code and compile that software for Linux. You’ll do the usual compilation dance (get used to it):

./config

make

make install

However, if you want to run Mac GUI apps, ARDI is your tool. See http://www.ardi.com/executor.php for extensive details, including lists of compatible applications.

One of the most useful commands you will use is grep. Its syntax is:

grep [-b] [-c] [-E] [-F] [-h] [-i] [-l] [-n] [-s] [-v] [-w] search_text source_text

Think of this command as “get report.” The key is, report of what?

 

grep options

grep [-b] [-c] [-E] [-F] [-h] [-i] [-l] [-n] [-s] [-v] [-w] search_text source_text

 

 

Search patterns

grep uses regular expressions to perform character matching or string matching. The characters include:

.	Matches a single character.
*	Matches zero or more instances of the immediately preceding character. Example: C* if found would match C, CC or CCC … not to mention a blank string!
[]	Matches any character contained within the brackets.
^	Represents the beginning of the line, so if you specified ^T grep would search for any line starting with a T.
$	Represents the end of the line, so if you specified \.$ then grep would pull up any lines that ended with .
\	The escape character: it means to take the next character literally, so you can search for characters like * that have special meanings: \*

(See my Shell Scripting course page on Operators.)

 

grep: a caution about special characters

Note: Be careful using the characters $, *, [, ^, |, (, ), and \ in the pattern list because they are also meaningful to the shell. To be sure of your result, enclose the entire pattern list in single quotes (also called “literal quotes” or “hard quotes”): ‘… ‘.

grep ‘2 for $3’ specials.txt

 

Input and output

grep will begin to make sense if you’re really clear about what goes in and what comes out of this command by default.

The basic syntax of grep is:

grep string_to_match input

grep searches the file, and the output is a printout of lines that contain matches for string_to_match.

 

About text files

What’s the most relevant difference between text files and binary files?

Text files are composed of lines.

In other words, at the end of each line there is a carriage return and a line feed (cr/lf). (Yes, there are other differences.)

Here’s the critical point:
Commands that process files will process them one line at a time.

 

What’s the text source?

You could simply name a file:

grep chocolate myfile.txt

which will return any lines in the file myfile.txt that contain the string “chocolate.” What you get back is a printout, to the terminal screen, of the matching lines.

 

What can you do with the result?

You don’t have to just let the matching lines print to the screen, though.

Use the pipe character | to redirect the results to another command, or use the redirect character > to write the results to a file. Use your imagination to see how useful this could be:

grep Denmark Hamlet.txt>term_paper_quotations.txt

 

Using standard input

grep will also accept standard input instead of a file, provided you don’t supply a file. Remember, though, that you will have to supply a standard input:

cat myfile.txt | grep chocolate

I like chocolate
chocolate cookies

 

Avoid ambiguities: Use the dash to indicate standard input

Leaving the filename argument blank, as in the previous example, will work just fine a lot of the time.

It won’t work well at all if I’m trying to search the output of a cat command for the word Hamlet, for instance, and there happens to be a file named Hamlet. It’s too easy to become unclear whether Hamlet is the filename argument or the search string.

That’s why we use a dash to indicate standard input, just for clarity:

cat myfile.txt | grep chocolate –

I like chocolate
chocolate cookies

 

Why we love standard input

When you think about it, the fact that grep accepts standard input is pretty wonderful. How many ways can we generate standard input?

By reading a file:

cat myfile.txt | grep chocolate –

By listing files:

ls –la | grep txt –

By listing processes:

ps aux | grep tty –

By listing users:

who | grep studenth –

 

Matching strings

The fact that we can feed grep input from a variety of sources isn’t the only great thing about grep.

What’s even more wonderful is that the matching string doesn’t have to be just a simple string, like “chocolate.”

 

The escape character: \

Now, consider this: $ is a special character within regular expressions; it means “end of line.”

So how do you search for lines containing the dollar sign without grep thinking you’re looking for a line ending?

You have to “escape” the special meaning of $:

\$

 

The . wildcard

Try this test: first, create two files:

touch hello.gif hello1gif

Now do a long listing of your directory and pipe it to grep to look for hello.gif.

Go ahead, actually do it before reading on.

 

What command did you try ?

To search for a line containing the text hello.gif, the correct command is

ls | grep ‘hello\.gif’ –

since

ls | grep ‘hello.gif’ –

will match lines containing hello-gif , hello1gif , helloagif , etc.

Now try:

ls | grep hello\.gif

What results do you get? Why?

 

Matching pattern lists

How do you look for a list of possible matches, not just one?

grep –e “support\|help\|windows” myfile.txt

Would search for “support,” “help” and “windows” in the file myfile.txt.

ANY of these phrases would produce a match.

Note that grep –e is the same as egrep: either command searches using regular expressions.

 

This or That: matching one of two strings

The grep “or” operator consists of these two characters together:

\|

What’s really happening here is that we are escaping the “regular” meaning of the pipe character, in favor of a “special” meaning. Note that you MUST enclose this inside single or double quotes.

grep “cat\|dog” myfile

matches lines containing the word “cat” or the word “dog.”

grep “I am a \(cat\|dog\)” myfile

matches lines containing the string “I am a cat” or the string “I am a dog”.

 

Matching several characters

To match a selection of characters, use brackets: []

For example:

[HhJ]ello matches lines containing hello or Hello or Jello.

 

Ranges of characters are also permitted:

[0-3] is the same as [0123] 
[a-k] is the same as [abcdefghijk]
[A-C] is the same as [ABC]
[A-Ca-k] is the same as [ABCabcdefghijk]

 

There are also some alternate forms :

[[:alpha:]] is the same as [a-zA-Z]
[[:upper:]] is the same as [A-Z]
[[:lower:]] is the same as [a-z]
[[:digit:]] is the same as [0-9]
[[:alnum:]] is the same as [0-9a-zA-Z]
[[:space:]] matches any white space

 

Just the opposite: NOT matching

Trying to find lines that do NOT contain a string? Use the -v option:

grep -v Denmark Hamlet.txt

 

The \? Operator

An expression consisting of a character followed by an escaped question mark matches one or zero instances of that character.

bugg\?y

matches:

bugy

buggy

but not

bugggy

 

Grouping Expressions

Let’s say we want to find all references to Frederic or Fred, by making the string “eric” following “Fred” optional.

To do this we’ll make grep treat “eric” as a single letter.

An expression surrounded by “escaped” parentheses is treated like a single character:

Fred\(eric\)\? Smith

matches Fred Smith or Frederic Smith.

\(abc\)* matches abc , abcabcabc etc. (i.e. , any number of repetitions of the string abc , including the empty string)

Note that we have to be careful when our expressions contain white spaces or stars. When this happens, we need to enclose them in quotes so that the shell does not mis-interpret the command, because the shell will parse whitespace-separated strings as multiple arguments, and will expand an unquoted * to a glob pattern.

So to use our example above, we would need to type:

grep “Fred\(eric\)\? Smith” filename

 

Matching a Specific Number Of Repetitions of a Pattern

You can use escaped braces \{ \} to indicate a count of matches for the preceding item.

A good example is phone numbers. You could search for a 7 digit phone number like this:

grep “[[:digit:]]\{3\}[ -]\?[[:digit:]]\{4\}” filename

This matches phone numbers, possibly containing a dash or whitespace in the middle.

And all this is only the beginning…

 

Don’t try to digest all of this at once. Just carry away this:

grep matches strings in lines of text.

All text files consist of lines of text (with line returns at the end).

You can use wildcards, lists, or patterns to match.

If you need something fancy, find and modify an example!

 

Resouces

grep man page: http://linuxcommand.org/man_pages/grep1.html

GNU grep manual: https://www.gnu.org/software/grep/manual/grep.html

GNU grep texinfo: http://www.gnu.org/software/grep/doc/

SS64’s grep page: http://www.ss64.com/bash/grep.html

 

 

How do I search binary files?
The strings command
The od command

The strings command searches a binary file for text strings, primarily so you can figure out what it does:

strings /bin/bash | less

If you’re a real propeller-head, doing core-dump debugging for instance, you may use the od command. This command shows the contents of a binary file in octal (base 8) format:

od /bin/bash | less

For a real treat, render the file in hexadecimal (base 16) format:

od -x /bin/bash | less

First, about vi

The program called by the command vi on Linux systems is actually vim, or “vi improved.”

On other Unix systems, for instance an AIX system, you may be using either vi or vim (and won’t be able to tell).

To find out about vi/vim, on the command line type:

vi

Type :q to exit.

 

vimtutor

At the command line, type:

vimtutor

This gives you a nice tutorial you can review if you’re interested in using vi frequently.

 

The basics of working in vi

To open vi, just type:

vi

If you want to edit an existing document, type:

vi filename

You start in Command Mode. You can’t edit in this mode, just do things like save and quit. To get into Insert Mode, where you can edit text, issue the single-letter command:

i

Now you’re typing. Basic navigation is easiest with the arrows on your keyboard. Move around, type and delete just like you always would. But sooner or later you’ll have to leave Insert Mode. To do this, press the

<escape>

key. You’re back in Command Mode.

Need help? Type:

:help

To simply leave, type:

:q

That’s a colon and the letter “q”. If you’ve changed text, vi will complain that you haven’t saved. To quit without saving:

:q!

The exclamation point forces the issue. Or you can choose to write your changes (i.e. save them):

:wq

If you opened vi without specifying a file name, you’ll need to supply a file name to save:

:w filename

or

:wq filename

if you’re writing and quitting.

 

The SourceForge Beginner’s Guide to Vim: an excelent, extensive introduction with lots of examples

http://newbiedoc.sourceforge.net/text_editing/vim.html

A basic vi reference sheet:

The Vi Cheat Sheet at http://www.lagmonster.org/docs/vi.html

For advanced reference details:

See the Advanced Vi Cheat Sheet at http://www.lagmonster.org/docs/vi2.html.

And for really detailed information:

Jerry Peek’s The Joy of Vim series:

Part One, at http://www.linux-mag.com/id/2996/ has particularly good info on vim’s help system, and windowing and tabs in vim
Part Two, at http://www.linux-mag.com/id/2991/
Part Three, at http://www.linux-mag.com/id/3028/

Version 7 of Vim offers many new features:

See William Nagel’s “Vimming to New Heights” at
http://www.linux-mag.com/id/2906/

 

---

pico

If you find vi frustrating or just plain silly, you can use another text editor on UNM systems. (It’s not included in every Linux distribution, for instance the Fedora we’re using in this class.)

Command:

pico

 

pico, unlike vi, allows you to begin typing immediately. You’ll see a menu along the bottom of the terminal, which makes life much nicer.

What’s the command to save a file?

What’s the command to leave pico?

 

---

mcedit, nedit

Other terminal-based editors you may find on your system are mcedit (the Midnight Commander Editor) and nedit. Try:

mcedit filename

If you have mcedit, you’ll see a screen with function-key operations listed across the bottom of the screen.

nedit filename

will give you a menu-based editor.

 

---

gedit, kedit, nedit

If you have a graphical environment available, the next step up in text editors is XWindows-based text editors like gedit, kedit, or nedit.

All of these are clean, simple editors along the line of Microsoft WordPad. You’ll find the menus and command buttons very familiar.

Best of all, gedit is easy to invoke from a command prompt:

gedit filename

 

---

emacs

In the Unix world, emacs is the grand old text editor. Some people have accused emacs fanatics of trying to create a complete operating system from their word processor.

emacs operates in modes: text mode, Perl mode, C mode, and so forth.

You can open emacs with the simple command:

emacs

From here on, nothing is simple. emacs requires that you, the user, know a number of keyboard functions.

Some Common emacs Keyboard Functions
Key	Purpose
Ctl-a	Clear your terminal’s screen
Ctl-e	Reset your terminal to default settings
Ctl-h	Display a user’s information
Ctl-d	List all currently logged-in user
Ctl-k	List current users and their tasks
Esc-d	Show your own current logged-in name
Ctl-x + Ctl-c	Lists your UID and GIDs
Ctl-x + Ctl-s	Display today’s date
Ctl-x + Ctl-w	Display this month’s calendar
Ctl-x + u	Leave a sub-shell

Your system may have xemacs installed. On a Red Hat-based system, you can check by commanding:

rpm -q xemacs

This is an XWindows-based version of emacs that can give you some “training wheels.”

 

---

Finally, Arachnophilia and other Unix-compatible editors

There’s one very important point to keep in mind:

Line endings in Unix are different from line endings in Windows.

Thus it’s very important which text editor you use in Windows when you’re scripting for Unix.

Enter Arachnophilia.

On the home page for this class you’ll find a link to the Arachnophilia download page. The current version is written in Java. I personally like an older version, written in Visual Basic.

Download either or both and try them out. Find the menu item that deals with line endings!

And remember, Arachnophilia ain’t the only game in town for crossing the Unix/Windows border….

 

---

Converting Windows Files

If you find you’ve uploaded (or been sent) a Windows-based text file, you always have a quick fix:

dos2unix file1 file2 …

Be sure to check out man dos2unix for some interesting options.

 

---

Resources

Unix Manuals: vi Reference: http://www.unix-manuals.com/refs/vi-ref/vi-ref.htm

Building Shell Commands

Some Common Linux Commands
Command	Purpose	Usage
clear	Clear your terminal’s screen	clear
reset	Reset your terminal to default settings	reset
finger	Display a user’s information	finger; finger user
who	List all currently logged-in users	who
w	List current users and their tasks	w
whoami	Show your own current logged-in name	whoami
id	Lists your UID and GIDs	id
date	Display today’s date	date
cal	Display this month’s calendar	cal
exit	Leave a sub-shell	exit
logout	End your user session	logout
head	View the first ten lines of a file; or specify how many lines	head /var/log/messages head -3 /var/log/messages
tail	View the last ten lines of a file; or specify how many lines; optionally specify the line to start on. tail also can be used to continually track a file with the -f (follow) option	tail /var/log/messages tail -3 /var/log/messages tail +10 /var/log/messages tail -f /var/log/messages
touch	Refresh the last-modified date of a file to now; create the file if it doesn’t exist	touch ./myfile
chown	Change the Owner of a file Change the Owner of a directory, Recursively Note: You can change both owner and group with one command.	chown user filename chown -R user directory chown user:group filename chown user.group filename
chgrp	Change the Group ownership of a file Also Note: Regular users can only change a file’s group to a group of which they are already a member.	chgrp group filename

 

First, we need wild cards

Many functions in Unix deal with text strings like file names. You’ll need at least two file name wild cards (or shell wild cards) to make life easier.

The first wild card is “ * “, which means “match any character or string of characters.” It’s handy, for instance, for finding a file:

ls -l chap*

will get you a listing of all the files (in your current directory) with names beginning with “chap”.

It will also attempt to return listings from any subdirectory with a name beginning with “chap”.

 

A one-letter wild card

The second wild card character to know is “?”. It matches only a single letter.

ls chapter?

will return any file names with only a single letter after chapter.

 

BASH Shell Metacharacters
Metacharacter	Purpose	Usage
$	Marks a shell variable	echo $myvar
~	The current user’s Home directory	cd ~/bin
&	Execute a command in the background	gedit &
;	In-line return, used for multiple commands on one line	who; w
>	Output redirection	ls > listing.txt
>>	Output append	who; date >> users.txt
<	Input redirection	command < file.txt
<<	Input append
|	“Pipes” command output to the next command’s input	ls /bin | grep gzip
\	“Escapes” the special meaning of a special character	grep \$ my_accounts.txt
`	Command substitution	myvar=`ls`
( ) { }	Command grouping	echo (date; who)

 

Variables: symbolic names used to represent values stored in memory

There are three types of variables:

Configuration variables

Environment variables

Shell variables

 

BASH Quoting
Metacharacter	Purpose	Usage
” “	“Weak quotes” – These exist ONLY TO PRESERVE SPACES	echo “Hello, $USER” gets you: Hello, root
‘ ‘	‘Single-quotes’ are ‘strong quotes’ or ‘literal quotes.’ Code inside them won’t run. Variables inside them won’t be expanded (interpreted).	echo ‘Hello, $USER’ gets you: Hello, $USER

 

Configuration variables

Store information about the setup of the operating system

Cannot be changed

HOME

LOGNAME

TZ (Time Zone)

 

Environment variables

Determine the characteristics of your session

Can be changed by you or your sys admin

PATH

PS1 (Shell prompt)

PWD (Current Working Directory)

 

Shell Variables

Created at the command line or in a shell script

Useful in shell scripts to store temporary information

dog

memo

my_var

today

 

Assigning values to variables

~$: NAME=Becky

The string Becky is assigned to the shell variable NAME.

 

To display the value of NAME, it must be preceded with a dollar sign ($), for example

~$: echo NAME

displays NAME.

 

~$: echo $NAME

Displays Becky.

 

Assigning commands to variables

The backquote (`) operator can be used to store the output of a command in a variable:

~$: TODAY=`date`

~$: echo $TODAY

Displays today’s date.

 

~$: TODAY=date

~$: echo $TODAY

Displays the word “date.”

 

The pipe ( | )

Recall the pipe special character, and its use in feeding the stdout of one command into the stdin of another command:

ls –la /bin/ | less

Any command that can accept Standard Input and produce Standard Output is called a filter command.

 

The redirection character ( > )

Also recall the redirection character, which sends the output of a command to a text file:

(users) > /var/users.log

 

The append character ( >> )

If you want to append your command output to a file, rather than “clobbering” it, use the append character:

(users)>>/var/users.log

 

Sequential commands

Sometimes you need more than one command to do the job. Use the semicolon:

(date;users)>>/var/users.log

 

Running a command in the background ( & )

To execute a command in the background, follow the command with an &. When you run a command that you know is going to take a while to complete, running it in the background lets you continue working in the mean time. On many Linux distros, backgrounding a command gives it a nice value of 4 (see the nice section). Compare:

ls –laR /

with

ls –laR / >/files.txt &

The latter executes ls in the background and immediately gives you a new prompt to continue working.

Use the command:

jobs

to view background job IDs.

Add a ” % ” to kill a background job:

kill %1

Move a backgrounded job to the foreground:

fg %1

Move a foreground job to the background:

Ctrl-z #to assign a background ID to the job
bg %1 #to background that job; Use the ID supplied by the previous command

 

A word about arguments

In Unix, an argument isn’t something you get into with your sibling.

An argument is a parameter, like specifying which directory to list when you give an ls command:

ls –la /home/glenn

In this case, /home/glenn is the argument.

 

Expanding command output as an argument

Consider this chain of actions.

I want to “touch” every file in my home directory, so I could issue the commands:

touch /home/glenn/* ; touch home/glenn/ *.*; touch /home/glenn/.*

This will refresh the last-modified date of every file in my home. (Now, why would I want to do this?)

 

But, I decide I don’t want to touch all my files; only the .txt files. So instead I command:

find /home/glenn | grep \.txt

to find my .txt files.

 

Now I have the list of files I want, I can use that list (which is really the output of my last command) as the input for the touch command:

touch $(find /home/glenn | grep \.txt)

 

In this case, I’m indicating that I want to run the find command first by bracketing it with:

$( my_command )

But I could also use the format:

$‘my_command’

 

Expanding arithmetic

You can also bracket and expand math expressions, using slightly different notation:

echo “2 plus 2 equals $[2+2].”

 

Finding Files

locate

To use the locate command, you will need to have an slocate database set up on your system. On many systems it is updated periodically by the cron daemon. Try the locate command to see if it will work on your system:

locate hello

This should list every filename on your system that contains the string “hello.” If this doesn’t work, you can use the commands:

slocate -u

or

updatedb

to build the slocate database. This may take a few minutes. On most systems with slocate installed, slocate is run periodically by the cron facility.

 

find

The find command lets you search for files by criteria, for instance by user, size or date. The basic syntax is:

find directory_to_start_from [options_if_any] string_to_find

By default, the string_to_find is the file name, or some part of the file name. But you can search by criteria too, such as the owner of the file:

find / –user studenth

In a similar way, the -group option restricts searches to files owned by a group. Or search by the size of the file:

find / -size 100k

Another truly helpful option, if you have remote mounts on the workstation, is -mount:

find / -mount file_name

This restricts find to local disks only.

Note that a bare “find” command lists every file on the workstation.

 

whereis

This command only works for executables, and it will find their man pages and (particularly useful) their configuration files as well. For instance:

whereis bash

will return the full path to bash, bashrc and the man page bash.1.gz. This is handy for finding the true paths to programs that aren’t in your PATH, for instance.

 

which

which finds an executable file. Unlike whereis, it does not find man pages as well.

which bash

This command will show the full path to the bash executable.

 

Getting On-Line Help

Get a manual page for a command:

man command

man tracert

 

What? There’s no output? You need to build the man page database:

makewhatis

 

Search by keyword:

man –k keyword

man –k columns

apropos columns
(apropos gives the exact same output as man -k.)

 

Info pages:

info command

info coreutils utility_name

where utility_name is a utility like ls.

 

And finally, yes, there is a help command, not available for all commands but sometimes useful:

help echo

 

About those numbers following the command in man page output:

For instance WHOAMI(1).

Manual Page Sections
Section
1	Commands available to all users
2	System calls
3	Library routines
4	Special device files
5	File formats
6	Games
7	Macro packages
8	Commands available only to root
9	Kernel routines
n	New: not yet categorized

 

Resources

LinuxCommand.org: http://linuxcommand.org/

A good intro to commands and scripting.

SS64: Unix, Oracle, Windows and Mac commands: http://ss64.com/

A great site devoted solely to the ins and outs of the command line in various environments.

tar and zip

The zip utility has been around since long before the advent of WinZip. In fact it predates pkzip, the first popular DOS compression/archiving utility.

Unix, as usual, breaks things down to more elemental processes: zipping (compressing a file) is separate from archiving (enclosing a group of files in an “envelope”).

In the Linux world, there are several zip programs: gzip (GNU zip), bzip, zip and compress.

 

tar: tape archive

tar is used to pack the entire contents of a directory or directories into a single file called a tarball, which can then be backed up to tape or saved as a file.

tar preserves the entire directory organization including file ownership, permissions, links, and the directory structure.

Note that tar cannot archive files or directories with names longer than 255 characters.

 

tar functions

The most commonly used tar functions are:

c – create an archive

x – extract files from an archive

t – show the “table of contents” of an archive
(you must use -z as well, if the archive is compressed)

A – append one archive to another

r – append files to an existing archive

P – store files with absolute pathnames

u – appends files to an archive if they are newer than the same files already in the archive

W – verify contents of the archive after creation

 

tar options

Additionally, there are commonly used options:

v – verbose

f  <filename> – use the specified file

z – gzip/gunzip

–gzip – gzip

–gunzip – unzip

j – bzip/bunzip

Z – use the compress utility

 

gzip: Lempel-Ziv compression (LZ77)

The gzip utility compresses files like WinZip or pkzip. It can achieve up to 60-70% compression, though often this is much less. The command itself is used very simply:

gzip zipped_file.gz file_to_zip.txt

gzip -d zipped_file.gz #d for “decompress”

gzip -r zipped_file.gz /directory/to/zip

gunzip zipped_file.gz

zcat zipped_file.gz

zmore zipped_file.gz

The tricky part comes with the huge number of options. Particularly, you can control the level of compression, on a scale of 1 (fast) to 9 (small):

gzip -v -1 zipped_file.gz file_to_zip.txt

 

bzip: Burrows-Wheeler Block Sorting Huffman Coding Algorithm

bzip can’t compress a directory full of files, while other algorithms can. You also can’t zcat ar zmore a .bz file. But you often get higher compression: 50-75%.

bzip2 -v file1 file2

bzcat file1.bz2

bzmore file1.bz2

bzless file1.bz2

bunzip2 -v file1.bz2 file2.bz2

bzip -d file1.bz2

bzip -f file1.bz2 #force

bzip -k file1.txt #keep the original file; otherwise deleted

bzip -q file1.txt #quiet

 

Unpacking files

When you download a file, you often get an archive that contains several – perhaps many – files within it. Archive files can also contain directories, which can contain files and subdirectories. Generally, these archive files have names ending in .tar, .tgz, .tar.gz, tar.bz2 or .zip.

To get at the files contained in the archives, you must unpack them.

 

Unpacking .tar files

If the file is a .tar file, open a terminal window, move to the directory containing the .tar file, and issue the command:

tar xvf filename.tar

The command will unpack the contained files to your current directory. You can use the ls command to view the results of your work.

(Quick: what do the three options mean?)

 

Unpacking .tgz and .tar.gz files

If the packed file is a .tgz or .tar.gz file, it has been archived and then compressed. So, the previous command, which merely unarchives the file, won’t work. To unpack a .tgz or .tar.gz file, open a terminal window, move to the directory containing the .tgz or .tar.gz file, and type:

tar zxvf <filename>

The command will expand the compressed file and then unpack the archived files to your current directory.

 

Unpacking .bz2 files

To unpack a .bz2 (bzip) file, open a terminal window, move to the directory containing the . bz2 file, and type:

tar jxfv file.tar.bz2

Notice that the only difference is the “j” option replacing the “z” option.

 

Unzipping .zip files

If the packed file is a .zip file, you shouldn’t issue a tar command at all. Instead, open a terminal window, move to the directory containing the compressed file, and type:

unzip filename.zip

Like the previous command, this command also unpacks the contained files to your current directory.

 

Archiving files: Making a tarball

Why do we do this? For one, it’s easier to send someone a single file than a whole mess of them. But for another, it’s often important to send files with permissions and file attributes intact, something that never happens with most forms of copying.

To create a .tar file, open a terminal window, move to the directory containing the files or directories you want to pack, and type:

tar cvf mynewfile.tar file1 file2 …

You can list as many files and directories as you want; just separate each entry from the previous entry with a space.

 

Compressing your tarball

If you want to compress the packed file, simply include the z flag on the command line:

tar zcvf mynewfile.tgz file1 file2 …

Be very aware that tar syntax gets quite tricky. Resort to man tar often, and/or follow instructions diligently.

 

Creating a .zip file

A potential advantage of a .zip file is that familiar Windows programs use the .zip format. To create a .zip file, at a command line and in the directory containing the files you want to zip, type:

zip myfile.zip file1 file2 …

where myfile.zip is the name of the .zip file you want to create and file1 and file2 are files or directories you want to pack. As in the .tar command, the ellipses (…) indicate that your list can go on forever.

 

The compress utility: LZW compression

compress -v file1 file2

compress -f smallfile1 symlink1

uncompress file1.Z file2.Z

zcat file1.Z

zmore file1.Z

zless file1.Z

 

Building blocks

The area of archives is a great place to look at how Unix commands can be assembled.

Imagine almost any scenario in which you wanted to zip up a group of files. On the command line, the most heinous chore is listing those files.

How about using the find command to do the listing for you?

find . –name “*.[ch]” –print | zip newfile.zip -@

Which means:

“Find, in this very directory ( . ), the files with names like anything dot c or h, print that list, and pipe it to zip, which will create a new zip file, and put the files listed by find into it.”

Check out man zip to find out about using -@.

 

Getting regular

find . –name “*.[ch]” –print | zip new.zip -@

Notice that text string in weak quotes? The square brackets are part of a whole other phenomenon in Unix: Regular Expressions.

Suffice it to say for now that in this case, anything.c or anything.h would match the expression. Just be aware that these things exist and that the letters inside the brackets describe a list or range of matches.

Tarring what you find

Want to find files, tar them up, maybe even remove the originals once you’ve created your tarball? Consider this:

find * -type f -mtime +90 -print > list.txt

tar -create -gzip -file my.tar.gz -files-from list.txt -remove-files

What Is Mounting?

Mounting is simply the process of “plugging” a directory into your file system tree. You can mount a whole partition, for instance when you add a hard drive. Or you can mount a shared network directory into your local file system tree.

There are a couple of critical principles to know. First, you will use a directory that already exists on your file system tree as a point to which you’ll mount. Normally you’d create an empty directory for just this purpose, but in the event that you use a directory that already has contents, those contents won’t be available until you dismount what you’ve mounted over them.

Second, file system type matters. Your system may or may not have support for various file systems, and in the case of some file systems like NTFS, you may have read-only access. You’ll be expected to know, in most cases, what type of file system you’re mounting as well.

 

The mount Command

mount is, more or less, a simple command. Its basic syntax is:

mount [options] device_to_mount mount_point

 

For example, to mount a partition to a directory:

mount -t ext3 /dev/hdc5 /projects

This assumes that you’ve created a directory, /projects, where you’ll access the contents of the new hard drive.

 

Mounting A Floppy Disk

Use the full and formal:

mount -t ext2 /dev/fd0 /mnt/floppy

If the floppy disk is already listed in /etc/fstab (it ought to be) the mount command can consult that file if you supply only the barest information:

mount /dev/fd0

or

mount /mnt/floppy

 

Mounting a CD

CD-ROMs generally (once they’re created) use the iso9660 filesystem, and are (generally) read-only:

mount -r -t iso9660 /dev/cdrom /mnt/cdrom

 

Mounting A Partition

Command:

mount -t ext3 /dev/hdc1 /newdirectory

ss

How can I mount everything that’s supposed to be mounted at boot time?

mount -a

 

The /etc/fstab File

The File System Table (fstab) file holds a list of mountable drives. Not everything in this list mounts at boot; some entries are there to allow easy mounting of removable drives, for instance.

This the fstab file has two roles:

1. It mounts partitions at boot time, and
2. it holds information that allows users to mount partitions/drives with minimal information.

The file has six columns or fields:

1. The device to mount, e.g. /dev/hda1
2. The mount point, e.g. /home
3. The filesystem type – ext3, swap, proc, udf, auto, etc.
4. Mount options – defaults, ro, rw, noauto, owner, GID and more.
5. dump# – 1=Back it up, 2=No backup
6. fsck# – 0=No check, 1=Check first, 2=Check later

Filesystems with the noauto mount option are not mounted at boot time.

 

What’s Currently Mounted?

Simply command:

mount

and you’ll get a list of currently mounted filesystems. Note that this command abstracts this information from the file /etc/mtab, which you can view directly yourself:

cat /etc/mtab

 

Who’s Using It?

This can be a critical question when you’re performing an unmount. Command:

fuser -u /mnt/floppy

The -u option specifically checks for users. You can kill the processes accessing files on the filesystem with a -k:

fuser -k /mnt/floppy

Or be nice and ask first:

fuser -ik /mnt/floppy

 

Unmounting

Use the umount command, with either the device file or the mount point as the argument:

umount /mnt/floppy

umount /newdirectory

Note that floppies, CDs and other removable media must be unmounted before they are removed.

 

Resources

LinuxPlanet: Adding Additional Hard Drives In Linux, at http://www.linuxplanet.com/linuxplanet/tutorials/4232/1/

A Safe Environment for Multiple Users

Unix systems have multiple users – and a permissions system that lets each user share or protect their files.

This unique system has some odd twists. For example, did you know that, to rename or remove a file, you need write permission for the directory that the file is in?

 

Users and Groups

In Unix, every user has a username (like zoe) and a corresponding user ID number (or UID) like 1324. Every user also belongs to one or more groups, where each group also has its own name and ID number (the GID). You can run the id command to see your own user and group information:

id

uid=1324(zoe) gid=1324(zoe) groups=1324(zoe),111(staff)

 

Home Owners

Every file and directory has a user owner and a group owner, as well as a set of permissions that dictates what access is allowed for the file’s user, group, and others. Use pwd to check your location, and ls –dl to get information about the directory itself:

pwd /home/zoe

ls –dl

drwxr-x–x 5 zoe zoe 4096 Nov 2 10:33

 

A quick review of ls

The first character on each ls -l output line shows the file’s type. Here, d is a directory file. This file’s user owner is zoe, and its group owner is the group named zoe.

The next characters are the types of access allowed for the user owner (here, rwx), for members of the group (r-x), and for all others.

The meaning of the rwx permissions for files is simple: r permits reading from a file, w lets you modify it, and x lets you execute it (as a program). However, how rwx applies to directories isn’t so obvious.

 

Directory magic

Read (r) access to a directory lets you list its contents using ls or with a GUI file browser.

A directory is a special type of file that contains (among other things) the names of files that are stored “in” the directory. So, being able to read a directory file lets you see the names stored in it.

 

Writing to directories

In the same way, write (w) access to a directory lets you modify the directory’s contents – to change the names of the files stored in that directory.

To add, remove, or rename a file, you need write permission on the directory that contains the file’s name. If a directory is write-protected, the list of entries in that directory cannot be changed – accidentally or on purpose.

 

Directory x permission

Execute (x) permission on a directory lets you access entries in that directory. Even if you have read permission on a directory (which lets you see what’s in it), you can’t use any of its entries unless you also have x permission.

For example, even if an individual file’s access permissions allow you to read and write to a file, you still can’t read or write the file unless you have execute permission on its directory.

 

Strict parents

In fact, you can’t access a directory unless you also have execute permission on all of its parent directories, all the way up to the root directory! Similarly, without execute permission on a directory, you can’t browse its subdirectories.

But you don’t need read (r) permission on a directory to access an entry in it if you know the entry’s exact name.

 

Catch-22: Graphical Interfaces

Graphical applications may not handle Unix permissions very well. Because they usually show dialogs with icons for the files in a directory, they have to be able to read the directory. So, if users try to reach zoe’s xfotos subdirectory by clicking through /home to /home/zoe, they’ll get some sort of error on her home directory because the GUI app can’t read it.

 

Can’t get it write

Some applications – graphical and otherwise – also aren’t sophisticated about write permission: they assume that, if they can read a directory, they also can write to it. Writes can fail in strange ways because, although a particular file in a directory may be writable, the directory itself isn’t.

 

Bypassing the unreadable

GUI file-open dialogs will generally let you type (or paste) an absolute pathname (like /home/zoe/xfotos) into their “location” box.

This lets you bypass unreadable intermediate directories.

 

Granting Permission

Unix permissions let groups of users share files – even modify or remove other users’ files – if they’re given permission. Directory and file access permissions are often set to allow general file sharing, so let’s see a specialized example.

Let’s say that Zoe’s home directory is on a networked filesystem shared between all computers in her office. She could tell her friends to point their photo viewers at her directory, ~zoe/xfotos.

The shell expands ~zoe to the absolute pathname of zoe’s home directory; Zoe herself can use just ~.

 

Just Enough Slack

Her permissions give everyone just enough access to read the photos, but not enough access to add, delete, or modify files:

ls -dl ~/ ~/xfotos

drwxr-x–x zoe zoe…/home/zoe
drwxr-xr-x zoe zoe…/home/zoe/xfotos

ls -l ~/xfotos

-rw-r–r– zoe zoe … alex.jpg
-rw-r–r– zoe zoe … boss.jpg

 

Easy Sharing

This example shows one of the strengths of a Unix filesystem: it’s easy to share when you want to. Zoe didn’t need to fill friends’ mailboxes with individual copies of her photos, or configure the office Web server so her friends (but no one else) could see those files. Instead, she simply “opened” her directory and her files just enough to let people read them.

 

Private Areas

An easy way to keep your private files private is to make specific directories in your home directory with owner-only access. For instance, make a directory named personal and type:

chmod 700 personal

to give yourself all (rwx) permissions, but no access to members of your group or to others. All files and subdirectories under personal – no matter what their permissions – are protected from everyone except the system superuser.

Give your private directories a less obvious name than personal or private, which could be a red flag for crackers and others who want to get into your account somehow.

 

Default Permissions: umask

If you’re concerned about security but also need to share files, check the permissions of every file – including “hidden” files like .bashrc – before you start sharing.

You should also set the correct umask. You’ll recall from the previous unit that the umask defines the default permissions of new files and directories. The umask is a bit mask that specifies which permissions to deny when creating a file. You can see the current umask setting by typing umask at a shell prompt. To set it, use umask nnn, where nnn is the new umask.

 

Groups

Recall the output of id at the start of this presentation. zoe belongs to two groups: zoe and staff. Unix groups can have many members, and each user can belong to many groups.

For instance, all system administrators and staff might belong to the staff group. They have access to private, group-only directories and files that all non-members (non-staff users) can’t access. In the same way, zoe’s manager and the office manager might be members of the zoe group – to have group-mode access to some or all of her files and directories.

 

Changing group ownership

If you belong to a group (and you always belong to at least one group: yours), you can change the group owner of any file or directory you own with the chgrp command.

For instance, zoe might keep confidential notes on system users in a file named users.xml. She’d use:

chgrp staff users.xml

to set the file’s group owner to staff. Then she’d use:

chmod 750 users.xml

(or chmod u=rw,g=r,o= users. xml) to give herself read and write access, give read access to members of the group staff, and no access to other users. Any staff member can open the file, but other users have no access.

 

But which group?

If you belong to several groups, which group owns files you create? By default, it’s your primary group – the group listed first in the id output, after gid=. That’s usually not what you want in a directory shared by all users from a particular group.

For example, the directory /projectdir/4staff might have the permissions drwxrwx— so all members of the staff group can create, remove, and rename files in that directory. But if zoe creates a file in that directory with her default group of zoe, other members of the staff group probably can’t read it (unless they’re also members of the zoe group).

 

Set Group ID

The answer is to set the directory’s setgid bit, like the example below. (This bit is shown by s in the permission string drwxrws—.)

Files (and subdirectories) created in a directory with its setgid bit on will have the same group owner as their directory. So, with the following settings, any file created in /projectdir/4staff will automatically have the group owner staff:

chmod g+s /projectdir/4staff/

ls -ld /projectdir/4staff/

drwxrws— 5 zoe staff /projectdir/4staff/

 

Self-protection

If you’re the only user of your Unix system, and your filesystems aren’t networked to (or from) other computers, you may wonder why you need to understand permissions. One good reason is to prevent accidents – unintended deletes or changes to your own files and directories. Mistakes happen, so protecting yourself makes sense.

 

Unwritable

If report.doc is an important file, you can make it unwritable by everyone – including yourself – with:

chmod a-w report.doc

But setting permissions on the file’s contents don’t control whether the file can be removed or replaced!

For instance, if you run gedit report.doc, gedit might rename report.doc to report.doc.bak, then create a new report.doc file. That’s perfectly legal.

To protect against all changes, the file’s directory also needs to be unwritable.

 

Attributes

The permissions scheme we’ve seen works on Unix systems, in general, as well as on Linux. Some Unix filesystems – the widely-used ext2 and ext3 types, for instance – also have other file attributes.

The manual pages for lsattr and chattr explain how to list and change file attributes.

 

Here’s a quick example. The superuser can use chattr to set the a attribute, which makes a file append-only: the file can’t be modified, only added to. Running lsattr shows this attribute, but ls -l doesn’t.

# chattr +a log.txt
# exit
$ ls -l log.txt
-rw-rw-r– 1 zoe staff 1168 Oct 30 12:29 log
$ lsattr log.txt
—–a– log.txt
$ cp /var/log/dmesg log.txt
cp: overwrite `log’? y
cp: cannot create regular file `log’: Operation not permitted
$ cat /var/log/dmesg >> log.txt
$ ls -l log.txt
-rw-rw-r– 1 zoe staff 3109 Oct 30 12:33 log.txt

What are files?

As we’ve discussed before, files are globs of data, in either text or binary form.

File names can be up to 256 characters long with “-“, “_”, and “.” characters along with letters and numbers.

When you do a long file listing, you see 10 characters on the left that indicate file type and permissions:

drwxrwxrwx

What are permissions?

File and directory permissions define your access to those files or directories.

There are actually three sets of permissions:

User/owner/author’s

Group’s

Other users’

And there are three distinct permissions for each user or group:

Read

Write

Execute

which determine what level of access they have.

Default permissions

Every user’s profile defines what permissions will be given to files and folders created by that user.

When the user is created, a home directory for that user is created at /home/username, and the initial contents are copied from /etc/skel. (You can change these content files to change new user defaults.)

Triple Triplets: User, Group, Others

When you run an ls command, user permissions for directories look like:

drwxrwxrwx

User permissions for files look like:

-rwxrwxrwx

The first character in the field indicates a file type of one of the following:

d = directory

l = symbolic link

s = socket (a connection to send data from a process on one computer to a process on another)

p = named pipe (a connection to send data from one command or process to another; represented as a file)

– = regular file

c= character (unbuffered) device file

b=block (buffered) device file

Types of permissions

There are 5 possible characters in the permission fields. They are:

r = read – This is only found in the read field.

w = write – This is only found in the write field.

x = execute – This is only found in the execute field.

s = setgid/setuid – This is only found in the user or group execute field

t = set text attribute, which shows up in the others execute field

If there is a “-” in a particular location, there is no permission given. This may be found in any field whether read, write, or execute field.

Owner permissions

The first rwx refers to the owner’s permissions.

By default, the owner gets full permissions: read, write and execute:

rwx

Group permissions

The second triplet describes the permissions set for the default group of the owner, for instance:

r-x

which allows group members to read and execute the file, but not write to it.

(Generally a new group is created for every new user, but you can override this behavior and place your new user in another group.)

Others’ permissions

The last triplet describes the permissions granted to any other user. You might, for instance, only allow them to read your files, but deny them write or execute access:

r–

Permissions are additive

If a user is both a member of the Group-permission group, and is (always, of course) a member of Others, the permissions that user will have are additive, which is to say they add together, rather than subtracting from each other.

Note the permissions of this file:

-rwxr–r-x root wheel … myfile.txt

Root owns the file; the wheel group has read permissions; Others have execute permissions. A member of the wheel group will get both the read permission from the Group triplet, and the execute permission from the Others triplet.

Changing permissions

The shell command chmod changes file and directory permissions.

Generally you’ll use numeric permissions, though you can use alphabetic mode to modify permissions also (see man chmod.)

chmod 755 ~/myfile.sh

Numeric permissions

Numeric permissions are calculated by adding individual permissions.

4 means “read”

2 means “write”

1 means “execute” or “browse” (depending on whether you’re changing a file or a directory).

It all adds up

Add 1, 2 and 4 to describe a permission set.

1+4 = 5 (read and execute/browse)

1+2 = 3 (read and write)

1+2+4 = 7 (read, write, execute/browse)

Three triplets, three numbers

Remember, you can set groups for the owner, their group, and other users. This means you’ll use three numbers to describe a full set of permissions:

chmod 755 ~/myfile.txt

sets permissions to:

-rwxr-xr-x

Careful with that directory!

The “x” permission has special consequences for directories. You might think you should eliminate “execute” permissions for directories that don’t contain executables.

That would be wrong. If you deny “x” permissions on a directory, users will not be able to browse to that directory.

Directory Permissions

Directory permissions don’t work the same way file permissions do. In particular, the execute permission is very different.

Permission

Meaning for files

Meaning for directories

read

User can open and read a file

User can list contents of a directory – only if user also has execute permission!

write

User can open, read and edit file

User can add files to or remove files from the directory

execute

User can execute the file (if it’s executable)

User can enter directory and work with its contents. This is sometimes called the search permission.

Important Note: A user’s permissions on a directory do not transfer to the contents of the directory! Individual file permissions still prevail.

Some common permissions

chmod 777 ~/myfile.txt (= rwxrwxrwx)

chmod 755 ~/myfile.txt (= rwxr-xr-x)

chmod 644 ~/myfile.txt (= rw-r–r–)

chmod 000 ~/myfile.txt (= ———)

When should you allow execute permissions?

There are two times you’ll want to give users an “x” permission:

When you want to let them enter and browse directories.

When a file is a shared executable, for instance a Perl file used in a web site.

Alphabetic Permissions

So far, we’ve described permissions using numbers, which works in most cases.

Be aware that you can modify permissions by “adding” or “subtracting” individual permissions.

For instance, if you wanted to remove execute permissions (for yourself) from a file named myfile, you could issue the command:

chmod u–x myfile

Effectively, you just “subtracted” execute permission.

Notice that in that last command,

chmod u–x myfile

we didn’t specify a user name. That’s because we just modified the owner’s permissions.

If we want to modify the group’s permissions, the command becomes:

chmod g-x myfile

Even more, if a category of user is not specified, ALL users are assumed:

chmod -x myfile

removes everyone’s execute permission.

Why the heck do we want to do this?

You can use this feature for quick modifications of permissions, but its real use is in setting the

User Identification Attribute

(Let that one sink in.)

Let’s get abstract. Under Unix, every file (which means everything) is owned by someone.

Further, every process that’s executed is also owned by someone.

The trick question is, who “are” you as you run an executable file that belongs to someone else?

The answer is, it depends.

Under most circumstances, you execute executables as yourself, with all the permissions (and restrictions) you have.

However, there are some programs that have to be run as someone else. Consider services, for instance.

Daemons

What is a service? A service is a utilitarian program that is run by a daemon, which is the process that actually sits and waits for a signal. The daemon provides the service.

So what? Here’s what: daemons run even when no one is logged in. They’re running in kernel mode as opposed to user mode.

And what does “daemon” mean? Disk Access and Execution MONitor

Set the User ID Attribute

Recall that the file permissions bits include an execute permission bit for file owner, group and other.

When the execute bit for the owner is set to “s” the set-user-ID bit is set. This causes any persons or processes that run the file to have access to system resources as though they are the owner of the file. When the execute bit for the group is set to “s”, the set-group-ID bit is set and the user running the program is given access based on access permission for the group the file belongs to. The following command:

chmod u+s myfile

sets the user ID bit on the file “myfile.”

The ping command is an example of a command with SUID. ping can be called by anyone, but it runs as root – it has to, for some of its functionality.

NOTE: You can only SUID compiled binary programs on some platforms. Test this on yours.

Set the Group ID Attribute

The command:

chmod g+s /somedir

sets the group ID bit on the directory “/somedir”.

These two items have the group or user ID bit set:

-rws–x–x 1 root root 14024 Sep 9 1999 chfn

-rwxr-sr-x 1 root mail 12072 Aug 16 1999 /locks

Set UID/GID Examples

-rws–x–x 1 root fred 14024 Sep 9 1999 chfn

-rwxr-sr-x 1 root mail 12072 Aug 16 1999 /locks

The “s” takes the place of the normal location of the execute bit in the file listings above.

This special permission mode has no meaning unless the file has execute permission set for either the group or other as well. This means that in the case of the lockfile, if the other users (world execute) bit is not set with permission to execute, then the user ID bit set would be meaningless since only that same group could run the program anyhow.

In both files, everyone can execute the binary. The first program, when run is executed as though the program is the root user. The second program is run as though the group “mail” is the user’s group.

So Please Note: SUID/SGID will only work if execute permissions on lower triplets are set as well.

Careful with that axe

For system security reasons it is not a good idea to set many program’s set user or group ID bits any more than necessary, since this can allow an unauthorized user privileges in sensitive system areas. If the program has a flaw that allows the user to break out of the intended use of the program, then the system can be compromised.

Special Directory Permissions

There are two special bits in the permissions field of directories. They are:

s – Set group or user ID (SGID/SUID)

t – Save text attribute – The user may delete or modify only those files in the directory that they own or for which they have write permission.

Directory Set Group ID

If the setgid bit on a directory entry is set, files in that directory will have the group ownership of the directory, instead of the group of the user who created the file.

Special Directory Permissions

This attribute is priceless when people are working on projects.

If users work in a directory with the setgid attribute set, any files created in the directory by any of the users will have the permission of the directory group, rather than the user’s default (primary) group.

For example, the administrator creates a group called project and adds the users Kathy and Mark to the group project. The directory projectdir is created with the set GID bit set. Kathy and Mark, although in different primary groups, can work in the directory and have full access to all files in that directory – but still not be able to access files in each other’s primary group.

Setting the GID bit

Issue this command to set the GID bit on a directory:

chmod g+s /projectdir

Resulting in this directory listing of the directory projectdir:

drwxrwsr-x 2 kathy project 1674 Sep 17 1999 projectdir

The “s” in place of the execute bit in the group permissions causes all files written to the directory projectdir to belong to the group “project.”

Save text attribute (the “sticky bit”)

The /tmp directory is typically world-writable and looks like this in a listing:

drwxrwxrwt 13 kathy project 4096 Apr 15 08:05 tmp

Everyone can read, write, and access the directory. The “t” indicates that only the user who created a file in this directory can delete or modify that file. Root always can overrule this, of course.

Setting a sticky bit

To set the sticky bit in a directory, issue the following command:

chmod +t directory_name

So why would you do this?

This setup is one step “tighter” than the set GID setup.

If you didn’t create the file, you can’t change or delete the file. But otherwise you can work freely in the directory.

If you set the GID bit instead, anyone with access to the directory can modify anyone else’s files. However, in many programming situations, this is a good thing.

Using numerical values for special permissions

You can use the numerical values of chmod to set special permissions:

4 = SUID
2 = SGID
1 = Sticky Bit

So, for example, you could set the sticky bit using:

chmod 1755

simply by placing an additional number to the left of the permission set.

Examples of making changes to permissions

chmod u+x myfile

Gives the user (owner) execute permission on myfile.

chmod ugo+x myfile

Gives everyone execute permission on myfile.

chmod 400 myfile

Gives the user read permission, and removes all other permission.

chmod 764 myfile

Gives user full access, group read and write access, and other read access.

chmod 751 myfile

Gives user full access, group read and execute permission, and other, execute permission.

chmod u+s myfile

Set the setuid bit.

chmod go-rx myfile

Remove read and execute permissions for the group and other.

umask

Quick: when you create a file, who owns the file?

What group “owns” the file?

What are its default permissions?

We’re into the realm of a concept called the umask: the “user mask.”

The umask is a numerical value that describes the default permissions any file you create will have.

You can see your own current umask with the command:

umask

The numbers look similar to file permission numbers, like 421. However, they function differently.

About masks in general

A user mask is just one kind of mask you’ll find in the world of Unix.

Another common one is the subnet mask, a number like:

255.255.255.0

Here’s the thing to know:

Masks are subtractive, not additive.

The subnet mask 255.255.255.0 actually means (in the case of an IP address), “subtract this value from the maximum possible values.”

In this case it means, “mask the first three octets (triplets) of local IP addresses.” In other words, if my IP is 192.168.0.5, and I’m trying to reach 192.168.0.7, the address I’ll really look for is simply 0.0.0.7.

If you’re not familiar with this concept in IP addressing, don’t worry about it.

Instead, consider the highest possible level of permissions a file can have, expressed as numbers:

777 for executable files

666 for text files

Note that contemporary Linux distros won’t even allow you to create a file as executable by default. This is a good idea.

 

Better than 666

777, as a file permissions descriptor, means the user, group and others all have read, write and execure permissions.

I don’t want everyone to have full permissions to my files. I want them to have less. In fact, for most cases, I want them to have:

700

However, this may be too draconian, so I’ll settle for:

755 (for executables)

644 (for text files)

umask: the numerical value

So, if I want to “lower” the default permissions that a file I create will have, I have to subtract from 777 (for executables) or 666 (for text files.)

If I’m keeping full permissions for myself, the first number of my umask will be 0.

If I’m restricting my group members to reading and executing, but not writing to my files, the second number of my umask will be 2 (thus subtracting 2, or the “write” value) for any file I create.

And if I’m willing to give the same read/execute permissions to others, the third number of my umask will also be 2 (thus subtracting write permissions).

Setting my umask

You can set the value of your shell’s umask from any shell prompt.

umask is a built-in bash shell command. The preferred way to set it is to add a umask command to the .profile or .bashrc file in your home directory.

To change it system wide there is a file in the /etc directory hierarchy where the default values for user shells are set.

System default umask

The system default umask is set in the file /etc/bashrc, which is generally modifiable only by system admins.

Its current default value is probably the best one for system wide use. For specific groups of users you may want a slightly more open value, but it shouldn’t be needed most of the time.

Your own umask

You can override the system default umask by setting your own in your home directory.

Modify either ~/.bashrc or ~/.bash_profile (preferably the second) to change your own umask permanently.

Use the umask command to change it only for the current session:

umask 022

Think like chmod

The three digits of a umask are written in the same order as you’d specify with chmod: first, the user mask; then, the group; finally, all others.

So, a umask of 002 denies no (0) permission for the user owner and group owner, and denies write (2) permission for others. A umask of 027 would deny no (0) permission for user, write (2) permission for group, and all (7, or 4+2+1) permissions for other users.

umask by example

An easy way to see what a umask does is to set it, create an empty file and directory (with touch and mkdir), and then use ls -l to see the permissions (here we’re using the shell’s semicolon (;) operator to run two commands from a single prompt):

umask 027

touch myfile; mkdir ymdir

ls -l myfile; ls -ld mydir

-rw-r—– glenn glenn … myfile
drwxr-x— glenn glenn … mydir

rm myfile; rmdir mydir

The Filesystem Tree

The Unix and Linux filesystem is based on the Filesystem Heirarchy Standard (FHS). (You can research it at http://www.pathname.com/fhs/ .)

There are no C: drives, D: drives or any other lettered drives. Instead, all disks are mounted into a common filesystem tree. You can even mount other computers into this tree. Everyone shares the same filesystem (which is why permissions are so important).

Common Unix/Linux Directories
Directory	Description
/	Call this “the root of the filesystem” (which it is), not the “root directory” (which you’ll see listed below: /root).
/bin	“Binaries” – actually both binaries and scripts – very basic commands available to all users
/boot	Kernel and boot loader files
/dev	Device files
/etc	Configuration files
/home	Users’ homes
/lib	Shared libraries, called shared objects in Unix, with a .so extension
/mnt	An empty directory, allowing creation of subdirectories for mounting CDs, floppies, etc.
/opt	Optional, additional software (usually on the server in NFS networks)
/proc	A virtual directory that holds process and kernel information
/root	root’s home
/sbin	System binary commands, used by administrators
/tmp	Temporary files
/usr	The vast majority of commands/utilities available to all users, often divided into: /usr/bin /usr/games /usr/include /usr/lib /usr/local /usr/sbin /usr/share /usr/src /usr/X11R6
/usr/local	Optional, additional software (usually installed on a local workstation in an NFS network)
/var	Log files, spoolers, email

 

See the Partitioning section of the Installation page for information about setting up partitions.

 

Filesystem Types

See the excellent Novell page, File System Primer, at http://wiki.novell.com/index.php/File_System_Primer#File_System_Comparison

Review the list on the page above, then note these filesystems not included in the Novell list:

Additional Unix/Linux Filesystems
Type	Description
bfs	Boot filesystem – used at startup on most Unix systems
cdfs	Compact Disk filesystem – one way to access CD data as normal files
iso9660	“The” CD-ROM filesystem, based on an ISO standard
udf	Universal Disk Format filesystem – used during the creation and burning of CDs
minix	The original proto-Linux filesystem
msdos	FAT 16-bit filesystem (not FAT32)
vfat	FAT compatible filesystem with long filename support
VxFS	HP-UX filesystem

 

For a more comprehensive list of filesystem types, see Guide to Linux Filesystem Mastery by Sheryl Calish at http://www.oracle.com/technology/pub/articles/calish_filesys.html

Learn about Journaling File Systems from Steve Best’s article at
http://www.linux-mag.com/content/view/1167

Tuning Filesystems for Performance

Journaling File Systems in particular benefit from performance tuning: See Steve Best’s article “Tuning Journaling File Systems” at
http://www.linux-mag.com/id/2927/

 

Formatting (Making A Filesystem)

Formatting a Floppy Disk

You’re likely to need to make a file system on (format) a floppy, if it’s new (is there still such a thing?). Use the mkfs command. For a “Linux-only” floppy, you could use ext2 as your file system:

mkfs -t ext2 /dev/fd0

If it needs to be usable under Windows, use FAT instead:

mkfs -t msdos /dev/fd0

If you don’t specify, you’ll get ext2:

mkfs /dev/fd0

 

Formatting a Partition (creating a filesystem)

Filesystem Creation Commands
Command	Purpose	Usage
mkfs	Makes filesystems of most types	mkfs -t ext2
mkfs.msdos mkdosfs mkfs.vfat	Create a DOS FAT filesystem	mkdosfs /dev/fd0
mkfs.ext2 mke2fs	Creates an ext2 filesystem	mkfs.ext2 /dev/hda2
mkfs.ext3 mke2fs -j	Create an ext3 (journaled) filesystem	mkfs.ext3 /dev/hda2
mkisofs	Creates a CD-ROM filesystem	mkisofs -o iso_file starting_directory
mkfs.reiserfs mkreiserfs	Creata a REISERFS filesystem	mkreiserfs /dev/hda3

 

Tuning Filesystems

ext2 and ext3

Use tune2fs to “adjust tunable filesystem parameters” on ext2 and ext3 filesystems. These include:

• The number of mounts between filesystem checks
• The maximum time between filesystem checks
• Set the volume label
• Reserve blocks for the filesystem or by group
• Add journaling, which turns an ext2 filesystem into an ext3 filesystem.

  See man tune2fs for details before using this command.

 

Checking Filesystems

fsck

fsck is the generic filesystem check utility. There are actually whole families of filesystem check utilities, specific to certain filesystems. fsck is actually a “pass-through” utility that invokes the filesystem-specific checkers, which have names like fsck.fstype.

See a typically excellent page at SS64.com:

http://ss64.com/bash/fsck.html

for a simple rundown. Of course see man fsck for full details (which will refer you through to filesystem-specific details).

 

fsck wants you to provide a filesystem type. If you don’t it assumes ext2.

fsck -t ext3 /dev/hda1

Filesystem-specific Check Commands
Command	Purpose	Usage
e2fsck	Checks ext2 and ext3 filesystems	e2fsck /dev/fd0
badblocks e2fsck -c	Checks ext2 and ext3 filesystems for bad blocks	badblocks /dev/fd0 e2fsck -c /dev/fd0
reiserfsck	Checks REISERFS filesystems	reiserfsck /dev/hdc1
reiserfsck -c	Checks REISERFS filesystems for bad blocks	reiserfsck -c /dev/hdc1
mkisofs	Creates a CD-ROM filesystem	mkisofs -o iso_file starting_directory
mkfs.reiserfs mkreiserfs	Creata a REISERFS filesystem	mkreiserfs /dev/hda3

 

fsck only performs a quick check unless you specify the -f “full” option:

fsck -f -t ext3 /dev/hda1

Common fsck Options
Option	Description
-f	Full check
-a	Automatically repair errors
-A	Check all filesystems marked with a 1 or 2 in /etc/fstab
-Cf	Do a full check and display progress line
-AR	Check all filesystems marked with a 1 or 2 in /etc/fstab, except the / filesystem
-V	Verbose output

 

Note that you can only check unmounted filesystems. Be particularly clear that you can’t fsck the root filesystem while it’s mounted (that is, while your system’s running). (See http://www.cs.uu.nl/wais/html/na-dir/unix-faq/unixware/qt-faq.html – search for fsck when you arrive.)

While the defaults can vary, an ext2 or ext3 filesystem will force a check every 20-40 mounts (which usually means reboots), or every 180 days, whichever comes first. This is one place you can use tune2fs:

tune2fs -i 90d /dev/fd0

The command above, for instance, changes the check interval to 90 days (90d). You can use seconds (the default), days (d), weeks (w) or months (m). A setting of 0 disables interval-based testing.

 

If fsck can’t repair a file, it places it in the lost+found directory, giving the file its inode number for its name.

 

Monitoring Filesystems

Disk Usage

The df (disk free space) command is the first tool to know. It gives you a fast audit of free space. Use it by itself:

df

or with human-readable output:

df -h

in HP-UX:

ddf

 

Perhaps more useful is the du (directory usage) command. Specify a directory to analyse:

du /usr

Or just get a summary:

du -s /usr

Make it human-readable:

du -hs /usr

 

Check used and available inodes on ext2 and ext3 filesystems with dumpe2fs. To keep it readable, always use the -h option:

dumpe2fs -h /dev/hda1