5.1.4.1. String comparison

Table 5-1. String comparison operators

Operator	True if...
str1 = str2[4]	str1 matches str2
str1 != str2	str1 does not match str2
str1 < str2	str1 is less than str2
str1 > str2	str1 is greater than str2
-n str1	str1 is not null (has length greater than 0)
-z str1	str1 is null (has length 0)

5.1.4.2. File attribute checking

Table 5-2. File attribute operators

Operator	True if...
-a file	file exists
-d file	file exists and is a directory
-e file	file exists; same as - a
-f file	file exists and is a regular file (i.e., not a directory or other special type of file)
-r file	You have read permission on file
-s file	file exists and is not empty
-w file	You have write permission on file
-x file	You have execute permission on file, or directory search permission if it is a directory
-N file	file was modified since it was last read
-O file	You own file
-G file	file 's group ID matches yours (or one of yours, if you are in multiple groups)
file1 -nt file2	file1 is newer than file2 [6]
file1 -ot file2	file1 is older than file2

^[6] Specifically, the -nt and -ot operators compare modification times of two files.

$* and $@

"$*" is a single string that consists of all of the positional parameters, separated by the first character in the value of the environment variable IFS (internal field separator), which is a space, TAB, and NEWLINE by default. On the other hand, "$@" is equal to "$1" "$2"... "$ N", where N is the number of positional parameters. That is, it's equal to N separate double-quoted strings, which are separated by spaces. If there are no positional parameters, "$@" expands to nothing. We'll explore the ramifications of this difference in a little while.

5.2. for

for x := 1 to 10 do
begin
    statements...
end

for name  [in list ]
do
    statements that can use  
    $name... 
done 

5.1.5. Integer Conditionals

Table 5-3. Arithmetic test operators

Test	Comparison
-lt	Less than
-le	Less than or equal
-eq	Equal
-ge	Greater than or equal
-gt	Greater than
-ne	Not equal

Task 4-2

Task 4-2

You are writing a graphics file conversion utility for use in creating a web page. You want to be able to take a PCX file and convert it to a JPEG file for use on the web page.^[7]

${...}, $(...)

${...}:  String Operator

• String Substitution
•  
• String Pattern Matching
• String Length

$(...):  Command Substitution

Here are some simple examples:

• The value of $(pwd) is the current directory (same as the environment variable $PWD).
• The value of $(ls $HOME) is the names of all files in your home directory.
• The value of $(ls $(pwd)) is the names of all files in the current directory.
• The value of $(< alice) is the contents of the file alice with any trailing newlines removed.^[10]

  ^[10] Not available in versions of bash prior to 2.02.

• To find out detailed information about a command if you don't know where its file resides, type ls -l $(type -path -all command-name). The -all option forces type to do a pathname look-up and -path causes it to ignore keywords, built-ins, etc.
• If you want to edit (with vi) every chapter of your book on bash that has the phrase "command substitution," assuming that your chapter files all begin with ch, you could type:

        vi $(grep -l 'command substitution' ch*)

  
  

• The -l option to grep prints only the names of files that contain matches.

Ch. 5 Flow Control

5.1.  if/else

The if construct has the following syntax:

if condition
then
    statements
[elif condition
    then statements...]
[else 
 statements]
fi

5.1.1.  Exit Status

5.1.2.  Return

5.1.3.  Combination of Exit Status

5.1.4.  Condition Tests

5.1.5.  Integer Conditional

4.4. Command Substitutions

4.4.1. 

4.3. String Operators

4.3.1.  Syntax of String Operators

Table 4-1. Substitution operators

Operator	Substitution
${ varname :- word }	If varname exists and isn't null, return its value; otherwise return word. Purpose: Returning a default value if the variable is undefined. Example: ${count:-0} evaluates to 0 if count is undefined.
${ varname := word}	If varname exists and isn't null, return its value; otherwise set it to word and then return its value. Positional and special parameters cannot be assigned this way. Purpose: Setting a variable to a default value if it is undefined. Example: ${count:=0} sets count to 0 if it is undefined.
${ varname :? message }	If varname exists and isn't null, return its value; otherwise print varname: followed by message, and abort the current command or script (non-interactive shells only). Omitting message produces the default message parameter null or not set. Purpose: Catching errors that result from variables being undefined. Example: {count:?"undefined!"} prints "count: undefined!" and exits if count is undefined.
${ varname:+word }	If varname exists and isn't null, return word; otherwise return null. Purpose: Testing for the existence of a variable. Example: ${count:+1} returns 1 (which could mean "true") if count is defined.
${ varname:offset:length }	Performs substring expansion.[5] It returns the substring of $varname starting at offset and up to length characters. The first character in $varname is position 0. If length is omitted, the substring starts at offset and continues to the end of $varname. If offset is less than 0 then the position is taken from the end of $varname. If varname is @, the length is the number of positional parameters starting at parameter offset. Purpose: Returning parts of a string (substrings or slices). Example: If count is set to frogfootman, ${count:4} returns footman. ${count:4:4} returns foot.

4.3.2.  Patterns and Pattern Matching

Table 4-2. Pattern-matching operators

Operator	Meaning
${variable #pattern}	If the pattern matches the beginning of the variable's value, delete the shortest part that matches and return the rest.
${variable ##pattern}	If the pattern matches the beginning of the variable's value, delete the longest part that matches and return the rest.
${variable %pattern}	If the pattern matches the end of the variable's value, delete the shortest part that matches and return the rest.
${variable %%pattern}	If the pattern matches the end of the variable's value, delete the longest part that matches and return the rest.
${variable/ pattern/ string}${variable// pattern/ string}	The longest match to pattern in variable is replaced by string. In the first form, only the first match is replaced. In the second form, all matches are replaced. If the pattern begins with a #, it must match at the start of the variable. If it begins with a %, it must match with the end of the variable. If string is null, the matches are deleted. If variable is @ or *, the operation is applied to each positional parameter in turn and the expansion is the resultant list.[6]

4.3.3.  Length Operator

${# varname }

4.3.4.  Extended Pattern Matching

Table 4-3. Pattern-matching operators

Operator	Meaning
*(patternlist)	Matches zero or more occurrences of the given patterns.
+(patternlist)	Matches one or more occurrences of the given patterns.
?(patternlist)	Matches zero or one occurrences of the given patterns.
@(patternlist)	Matches exactly one of the given patterns.
!(patternlist)	Matches anything except one of the given patterns.

 

How to show .bash_profile and .bash_history

Go to home directory

cd ~

and list files and directory include start with . and .. with option -A

ls -A

3.4.2.4 Command search path

Why you should care about path?

1. Once you have read the later chapter of this book and  you try writing your own shell program, you will want to test them and eventually set aside a directory for them.
2. Your system may be set up so that certain restricted commands' executable files are kept in directories that are not listed in PATH

Add directory to PATH:put this line to .bash_profile
PATH=$PATH":/home/you/bin/"

3.4.2.2. Mail Variables

3.4.2.2. Mail Variables

Since the mail program is not running all the time, there is no way for it to inform you when you get new mail; therefore the shell does this instead.^[11] The shell can't actually check for incoming mail, but it can look at your mail file periodically and determine whether the file has been modified since the last check. The variables listed in Table 3-6 let you control how this works.

Table 3-6. Mail variables

Variable	Meaning
MAIL	Name of file to check for incoming mail
MAILCHECK	How often, in seconds, to check for new mail (default 60 seconds)
MAILPATH	List of filenames, separated by colons (:), to check for incoming mail

Under the simplest scenario, you use the standard UNIX mail program, and your mail file is /usr/mail/yourname or something similar. In this case, you would just set the variable MAIL to this filename if you want your mail checked:

MAIL=/usr/mail/yourname

If your system administrator hasn't already done it for you, put a line like this in your .bash_profile.

However, some people use nonstandard mailers that use multiple mail files; MAILPATH was designed to accommodate this. bash will use the value of MAIL as the name of the file to check, unless MAILPATH is set; in which case, the shell will check each file in the MAILPATH list for new mail. You can use this mechanism to have the shell print a different message for each mail file: for each mail filename in MAILPATH, append a question mark followed by the message you want printed.

For example, let's say you have a mail system that automatically sorts your mail into files according to the username of the sender. You have mail files called /usr/mail/you/martin, /usr/mail/you/geoffm, /usr/mail/you/paulr, etc. You define your MAILPATH as follows:

MAILPATH=/usr/mail/you/martin:/usr/mail/you/geoffm:\
/usr/mail/you/paulr

If you get mail from Martin Lee, the file /usr/mail/you/martin will change. bash will notice the change within one minute and print the message:

You have new mail in /usr/mail/you/martin

If you are in the middle of running a command, the shell will wait until the command finishes (or is suspended) to print the message. To customize this further, you could define MAILPATH to be:

MAILPATH="\
/usr/mail/you/martin?You have mail from Martin.:\
/usr/mail/you/geoffm?Mail from Geoff has arrived.:\
/usr/mail/you/paulr?There is new mail from Paul."

The backslashes at the end of each line allow you to continue your command on the next line. But be careful: you can't indent subsequent lines. Now, if you get mail from Martin, the shell will print:

You have mail from Martin.

You can also use the variable $_ in the message to print the name of the current mail file. For example:

MAILPATH='/usr/mail/you?You have some new mail in $_'

When new mail arrives, this will print the line:

You have some new mail in /usr/mail/you

The ability to receive notification of mail can be switched on and off by using the mailwarn option to the shopt command.

HISTTIMEFORMAT

3.4.2. Build-in Variables
3.4.2.1. Editing mode variables

HISTTIMEFORMAT="%y/%m/%d %T "

...
78 04/11/26 17:14:05 HISTTIMEFORMAT="%y/%m/%d %T "
79 04/11/26 17:14:08 ls -l
80 04/11/26 17:14:09 history

echo "$HISTTIMEFORMAT"
%y/%m/%d %T

Learning the bash

Ch. 1 bash Basics

Ch. 2 Command-Line Editing

• emacs, vi editer
• 2.2. The History List
• emacs Editing Mode
• vi Editing Mode
• The fc Command
  

  fc is a built-in shell command that provides a superset of the C shell history mechanism. You can use it to examine the most recent commands you entered, to edit one or more commands with your favorite "real" editor, and to run old commands with changes without having to type the entire command in again.

Ch. 3 Customizing Your Environment

• Special files: .bash_profile, .bash_logout, and bashrc
• Aliases:
  aliase name='command'
  aliase search=grep
  aliase cdvoy='cd sipp/demo/animation/voyager'
  * just alias show all aliases
  *unalias remove alias

  Aliases are very handy for creating a comfortable environment, but they have essentially been superseded by shell scripts and functions, which we will look at in the next chapter. These give you everything aliases do plus much more, so if you become proficient at them, you may find that you don't need aliases anymore. However, aliases are ideal for novices who find UNIX to be a rather forbidding place, full of terseness and devoid of good mnemonics. Chapter 4 shows the order of precedence when, for example, an alias and a function have the same name. 

• Options:
  shopt -o :
• Variables: Shell variables are characteristics which cannot be expressed as an on/off choise.
  varname=value
• 3.4. built-in variables
  HISTCMD
  HISTFILESIZE
  HISTSIZE
• 3.4.2.2. Mail variables

  Since the mail program is not running all the time, there is no way for it to inform you when you get new mail; therefore the shell does this instead.

• 3.4.2.3 Prompting variables
  PS1, PS2, PS3, PS4
• 3.4.2.4 Command search path
  

Ch. 4 Basic shell Programming

• Shell Scripts and Shell Functions
• 
  

Ch. 5 Flow Control
Ch. 6 Command-Line Options and Typed Variables
Ch. 7 Input/Output and Command-Line Processing
Ch. 8 Process Handling
Ch. 9 Debugging Shell Programs
Ch. 10 bash Administration
Ch. 11 Shell Scripting
Ch. 12 bash for Your System

Appx. A. Related Shells
Appx. B. Reference Lists
Appx. C. Loadable Built-Ins
Appx. D. Programmable Completion

PATH about shell script

You can run a script by typing its name only if the directory where the script is located is in your command search path, or . (the current directory) is part of your command search path, i.e., the script's directory path (as discussed in Chapter 3). If these aren't in your path, you must type ./scriptname, which is really the same thing as typing the script's absolute pathname (see Chapter 1).

Acronym of grep

grep, the UNIX file-searching utility, was named as an acronym for something like "Generalized Regular Expression Parser."

LTB 1.7. Input and Output

The UNIX I/O scheme is based on two dazzlingly simple ideas.
First, UNIX file I/O takes the form of arbitrarily long sequences of characters (bytes). In contrast, file systems of older vintage have more complicated I/O schemes (e.g., "block,""record,""card image," etc.).
Second, everything on the system that produces or accepts data is treated as a file; this includes hardware devices like disk drives and terminals.

1.7.1. Standard I/O

1.7.2. I/O Redirection

1.7.3. Pipelines

Summary of Bash features

Summary of Bash features

bash is a backward-compatible evolutionary successor to the Bourne shell that includes most of the C shell's major advantages as well as features from the Korn shell and a few new features of its own. Features appropriated from the C shell include:

• Directory manipulation, with the pushd, popd, and dirs commands.
• Job control, including the fg and bg commands and the ability to stop jobs with CTRL-Z.
• Brace expansion, for generating arbitrary strings.
• Tilde expansion, a shorthand way to refer to directories.
• Aliases, which allow you to define shorthand names for commands or command lines.
• Command history, which lets you recall previously entered commands.

bash's major new features include:

• Command-line editing, allowing you to use vi- or emacs-style editing commands on your command lines.
• Key bindings that allow you to set up customized editing key sequences.
• Integrated programming features: the functionality of several external UNIX commands, including test, expr, getopt, and echo, has been integrated into the shell itself, enabling common programming tasks to be done more cleanly and efficiently.
• Control structures, especially the select construct, which enables easy menu generation.
• New options and variables that give you more ways to customize your environment.
• One dimensional arrays that allow easy referencing and manipulation of lists of data.
• Dynamic loading of built-ins, plus the ability to write your own and load them into the running shell.

IPython Basics

Tab completion
Introspection

Python for Data Analysis (Rough Cuts)

Why Python for data analysis?

Python as glue

Part of Python's success as a scientific computing platform is the ease of integrating C, C++, and FORTRAN code. Most modern computing environments share a similar set of legacy FORTRAN and C libraries for doing linear algebra, optimization, integration, fast fourier transforms, and other such algorithms. The same story has held true for many companies and national labs that have used Python to glue together 30 years' worth of legacy software.

 In the last few years, the Cython project (http://cython.org) has become one of the preferred ways of both creating fast compiled extensions for Python also interfacing with C and C++ code.

Solving the two language problem

In many organizations, it is common to research, prototype, and test new ideas using a more domain-specific computing language like MATLAB or R then later port those ideas to be part of a larger production system written in, say, Java, C#, or C++. What people are increasingly finding is that Python is a suitable language not only for doing research and prototyping but also building the production systems too. I strongly believe that more and more companies will go down this path as there are often significant organizational benefits to having both scientists and technologists using the same set of programmatic tools.

Downside of Python

Execution speed

After using it for 17 years and teaching it for 12, the only downside to Python I’ve found is that, as currently implemented, its execution speed may not always be as fast as that of compiled languages such as C and C++.
We’ll talk about implementation concepts in detail later in this book. In short, the standard implementations of Python today compile (i.e., translate) source code statements to an intermediate format known as byte code and then interpret the byte code. Byte code provides portability, as it is a platform-independent format. However, because Python is not compiled all the way down to binary machine code (e.g., instructions for an Intel chip), some programs will run more slowly in Python than in a fully compiled language like C.

Managing multiple Python installation on Mac OS X

Managing multiple Python installation on Mac OS X

Step 2: Storing Record Persistently

To make our people persistent, they need to be stored in a file or some sort.

Using formatted files

One way to keep our data around between our program runs is to write all the data out to a simple text file, in a formatted way.

Data format script

Step 1: Representing Data

Step 1: Representing Data


Using Lists
A database list
Field labels

Using Dictionaries
Other ways to make dictionaries
Lists of dictionaries
Nested Structures
Dictionaries of dictionaries


dict()

Programming python Part 2: System Programming

Part 2. System Programming
Chapter 2: System Tools
Exploring system application domain: script that deal with files, programs and general environment surrounding a programs.

Programming Python

The task
You need track the information of people.
You want write programs that keeps track of details about people.

Step 1: Representing record
What records in a data base look like?

built-in objects: lists, dictionaries


Step 2: Storing Records Persistently
They need to be stored to file or some sort.


Step 3: Stepping up OOP
Structure, encapsulation and customization.


Step 4:  Adding Console Interaction


Step 5: Adding GUI


Step 6: Adding Web Interface
Basic CGI script

First post

This is the first post.