Introduction

Octave is a free, both free as in beer and free as in speech, MATLAB compatible numerical programming tool available under the GNU General Public License. MATLAB, an abbreviation for “Matrix Laboratory”, is currently the most widely used commercial, proprietary numerical programming tool. Even a single license for the MATLAB software is a substantial cost. MATLAB is essentially a scripting language similar to Perl, Python, or PHP with a comprehensive, highly integrated set of numerical, statistical, mathematical, and graphical functions including, for example, the Fourier transform, matrix inversion, and histograms. In part because MATLAB has become the de facto industry standard for numerical programming, Octave is of particular interest to individuals, companies, and organizations engaged in numerical and mathematical programming and research and development.

At this time (2011), there is considerable interest in using machine learning and other advanced mathematical techniques to increase sales by detecting and predicting the buying patterns of end users of mobile devices such as iPhones and Android phones, social networks such as Facebook and LinkedIn, and search engines such as Google and Yahoo. Octave has long been used for data analysis, modeling, and machine learning. In addition to its extensive built-in features, Octave has add on packages for optimization and mathematical model fitting, econometrics, signal processing, image processing, neural networks, and a number of other areas used in the “mobile social search” space. Octave is also well suited for the development and implementation of algorithms for image and video compression, audio compression, image and video processing, audio processing, speech recognition, pattern recognition, artificial intelligence, design and simulation of machines, and many other practical uses.

In general, it is faster and easier to develop mathematical algorithms in Octave and similar tools than compiled languages such as C, C++, and Fortran. Because research and development usually involves large amounts of trial and error, Octave and similar tools have been used traditionally to prototype algorithms, analyze data from experiments, and so forth, and only occasionally for “production code” or commercial products. Traditionally, once a proof of concept existed in Octave or a similar tool, it was often translated to a faster compiled language such as C, C++, or Fortran, often a time consuming, tedious, and costly process. Octave today is quite fast and modern computers are extremely fast and inexpensive, so that there are many projects, such as data mining in the backend of a web site, that can probably skip the time consuming and costly step of conversion to a compiled language. In the future, just in time (JIT) compilers and similar capabilities will probably be added to Octave and similar tools so that the remaining speed difference between Octave and traditional compiled languages such as C, C++, and Fortran will vanish.

A significant problem with using Octave and many other free programs is making Octave work cost-effectively with other programs. It is rarely possible to use Octave for real-world projects as a purely standalone program. It is often necessary to share data and transfer the control flow between Octave and other programs including operating systems, databases, spreadsheets, web sites, and so forth. Many programs, especially proprietary programs, achieve customer lock-in through custom, sometimes secret, data formats and so forth. Microsoft Office is notorious for introducing new Office file formats with each “upgrade” that are not backward compatible with earlier versions of Office. Thus, users find themselves receiving spreadsheets, documents, and so forth that they cannot read with their old version of Office, forcing an “upgrade” at substantial cost.

This proliferation of ever-changing data formats, API’s, programming languages, and miscellaneous other standards, presents a continual problem for using free software. End users often spend hours, even days or weeks, figuring out how to convert from one format to another. Software developers often spend hours, days, weeks, and even months writing functions to read and write unsupported formats, whether new or old. This makes it difficult for end users or software developers to add real value by inventing substantive new features, capabilities, and algorithms.

This article covers a number of methods, specific commands and specific programs to make Octave work cost-effectively with other programs. There are many excellent on-line and published documents on the syntax of Octave and its add-on packages. Readers with specific questions of this type are referred to the excellent existing documentation. Rather, this article discusses some of the issues with making Octave work with other programs that are often not covered explicitly or well in the standard Octave or free software literature.

Using Simple Human Readable Files to Reduce Costs

There are a number of advantages to using simple human readable files (typically ASCII or UTF8) such as standard tab delimited files or comma separated values (CSV) files to export data, import data, and store data. One can view the files in any text editor or word processor. Most programs such as spreadsheets, databases, and many others can easily read and write tab-delimited and comma separated values (CSV) files. Many scripting languages have direct support for tab-delimited and sometimes comma separated values files. Essentially all computer programming languages can easily read and write tab-delimited files; it is just a few lines of code. Essentially all computer programming languages can read and write comma separated values files; it requires more programming, more lines of code, but it can be done. Octave can read and write tab delimited files using its built-in dlmread and dlmwrite functions. Octave can also read and write comma separated values files using its built-in csvread and csvwrite functions.

Historically, disk drives were small. Networks had very limited bandwidth. Computers had limited speed to read and process files. Hence, binary file formats had very clear advantages over bulkier human readable formats such as tab delimited files. With terabyte drives, 100 Megabit per second networks, and inexpensive 3 GHz multi-core computers, this is much less true today. Thus, there is a strong case for using simple human readable file formats such as tab delimited files to reduce high end user and development costs, avoid customer lock-in (a costly proposition if you are the customer), and limit the costly perpetual upgrade cycle to genuine improvements in the products and services.

Simple human readable files, notably tab delimited files, enable users and developers to share data between Octave and other programs quickly and easily. There are a few caveats to be aware of.

First, the basic data type in Octave is a “matrix,” originally a double precision two-dimensional array. The MATLAB/Octave “matrix” has been extended to multiple dimensions and also to support characters and strings. The character/string support in MATLAB and Octave is something of a patch. Thus, the dlmread and dlmwrite commands in Octave handle matrices or arrays of numbers fine but map character strings in tab-delimited files to zero (0). This is often not an issue for the numerical programming and analysis that Octave is usually used for. The string and text manipulation capabilities in Octave are limited, so one should consider other tools such as Perl, Python, or Ruby for heavily text or string oriented work.

The precision of floating point numbers can be an issue using human readable files such as tab delimited files that represent floating point numbers as character strings such as “1.123456″ or “1.1234567890123456″. Octave, MATLAB, and most computer programs use the IEEE-754 standard for floating point numbers, or extensions of this standard. In this standard, a single precision floating point number (e.g. a float in the C family of programming languages) requires six decimal digits after the decimal point to represent the floating point number as a character string such as “1.123456″. An IEEE-754 double precision floating point number requires 16 decimal digits after the decimal point to represent the double as a character string such “1.1234567890123456″.

Many programs and programming languages default to writing floating point numbers as character strings with six decimal digits such as “1.123456″. For example, the standard printf() function in the C programming language defaults to six decimal digits of precisoin when its %f format specifier for floating point numbers is used:

double value = 1.12345678901234456;
print("my number: %f\n", value); // defaults to six decimal digits of precision

usually prints:

my number: 1.123456

Thus, if a mathematical algorithm in Octave depends on double precision, it may be necessary to prepare a human readable input data file with 16 decimal digit character strings for the floating point numbers.

In Octave, one can use the command format long to display numbers at full precision:

Octave>format long

Exchanging Binary Files Between Octave and Other Programs

Octave supports a relatively small set of binary file formats directly. This is a common problem with free software. Both users and software developers can find themselves spending hours, days, even weeks on occasion (time is money) converting to and from other binary formats used by other programs. There are however a number of free programs that can quickly convert between a wide range of binary formats, thus greatly reducing or eliminating this overhead.

The free GIMP (The GNU Image Manipulation Program) program can read and write nearly all widely used still image file formats and many obscure still image formats. GIMP is available as both pre-compiled binaries and source code for all three major computing platforms: Microsoft Windows, Linux/Unix, and Mac OS X.

The widely used, free Audacity audio-editing program can read and write raw audio files, Microsoft uncompressed WAV files, and the open Ogg Vorbis audio file format. By default Audacity can read MP3 audio files but it cannot write them due to the onerous MP3 licensing restrictions. Users can install the LAME MP3 encoding library on all three major computing platforms — Microsoft Windows, Linux/Unix, and Mac OS X — and then Audacity can use this add on library to encode audio as MP3. Audacity is available as source code and precompiled binaries for all three major computing platforms.

The widely used, free sox (Sound Exchange) audio conversion utility can read and write a broad range of audio formats. Like Audacity, it needs the LAME encoding library to be separately installed to encode MP3 files due to the MP3 licensing restrictions. Users should consider using the free ogg vorbis audio format to avoid the MP3 licensing issues altogether. sox is available as both source code and precompiled binaries for all three major computing platforms.

The widely used, (mostly) free ffmpeg video and image sequence conversion utility can read and write a broad range of video file formats. Like Audacity and sox, certain video encoding formats have licensing restrictions. Again, users may wish to consider the free ogg theora video format to avoid some of the licensing issues with better known video formats such as h.264.

Installing Octave Add-On Packages

Octave has extensive built-in capabilities. It also has a large number of add-on packages available through the Octave Forge project and web site. These include an extensive library of standard optimization functions such as the Nelder-Mead method used in model fitting, statistics, signal processing, econometrics, and so forth. Some of the Octave Forge packages are rather limited, but a number are very extensive and a number extend Octave’s compatibility with MATLAB.

For example, MATLAB has a function xcorr which computes the cross-correlation of a data series or data set. Octave does not have xcorr in its base set of built-in functions. There are a number of implementations of xcorr on the web as xcorr.m files, but xcorr is also found in the Octave Forge signal package which provides a large number of standard signal processing functions.

The author found the explanations of the Octave pkg install procedure on the Octave and Octave Forge web sites a bit confusing. Here is a fuller explanation. Octave packages are downloaded as standard *.tar.gz gzipped (compressed) Unix tar (tape archive) files. The Unix tar (tape archive) command combines a group of files and/or folders and their contents into a single huge file without compressing the files. They are loosely concatenated together. Historically, the tar command was used to store files, folders, even entire file systems on backup tapes. The Unix gzip utility compresses files. On Unix, a *.tar.gz or *.tgz file is a file that has been tarred and then compressed using gzip. The Octave packages are distributed as *.tar.gz files.

Octave has a built in command pkg install which installs the packages. NOTE: pkg install works directly on *.tar.gz or *.tar files. There is no need to uncompress or unpack the *.tar or *tar.gz files. On the Macintosh, the Mac OS may automatically unzip the file when downloaded, creating a *.tar file.

In Octave, simply switch the folder where the *.tar.gz or *.tar file has been downloaded, then type

Octave> pkg install package-name.tar.gz

or

Octave> pkg install pakcage-name.tar

if the operating system has automatically unzipped the package.

The Octave pkg install command will automatically create the folders for the new package in the Octave installation and copy the package files to these folders and perform all other setup.

Many Octave packages contain C, C++, or Fortran source code. Octave automatically installs a compiler when it is installed. It can compile these files. There is no need for the Octave user to compile them by hand, run the make utility or anything like that. Octave has a built-in mkoctfile command to compile C, C++, or Fortran extensions to Octave and add them to the built-in commands of Octave. The Octave pkg install command will run mkoctfile if needed to build and install the package. On the Macintosh (see below) it may be necessary to modify the mkoctfile command to install many of the Octave packages.

Extending Octave with Fast, Compiled Languages

Developers may want to extend Octave by adding C, C++, or Fortran programs to the built-in functions of Octave. This may be done because the compiled programs will be faster to execute than Octave or to add existing C, C++, or Fortran code to Octave.

mkoctfile [-options] file

The mkoctfile function compiles source code written in C, C++, or Fortran. Depending on the options used with mkoctfile, the compiled code can be called within Octave or can be used as a stand-alone application. mkoctfile can be called from the shell prompt or from the Octave prompt.

Making Octave Work with Emacs

Emacs is a widely used free source-code and text editor. Emacs is available on all three major computing platforms: Microsoft Windows, Unix/Linux, and Mac OS. The author has had good experiences using the free Aquamacs version of Emacs on the Macintosh. Emacs has an “everything including the kitchen-sink” philosophy. Emacs has extensive built-in capabilities. There is extensive and excellent on-line and published documentation on Emacs. Again, this article will only discuss some key gotchas that are likely to arise using Emacs in combination with Octave.

Emacs has a MATLAB mode for editing MATLAB or Octave source code. The MATLAB code editing mode handles indenting, highlights keywords, and so forth. Emacs also has an Objective C mode. Since Objective C “method” or “message” source files also use the .m file extension used by MATLAB and Octave, this can cause problems. There is also an Objective C mode for Emacs. The author has encountered versions of Emacs configured to use the Objective C mode for .m files and other versions of Emacs configured to use the MATLAB mode for .m files.

The Emacs user can explicitly switch to the MATLAB mode by typing Meta-X where the mysterious Emacs Meta key is usually the Escape (Esc) key on most keyboards and then typing matlab-mode at the Emacs command prompt.

If the matlab mode is not preinstalled in the user’s version of Emacs, one can get an Emacs MATLAB mode add on package from the web, for example at http://www.mathworks.com/matlabcentral/fileexchange/104

Emacs can be reconfigured to use the MATLAB mode automatically for .m files instead of the Objective C mode if needed.

As discussed, in addition to source code, it is often necessary to share data between Octave and other programs. Simple human readable file formats such as the common tab delimited file format or simple uncompressed binary file formats such as Microsoft WAV audio files are often the easiest and most cost-effective way to do this. Working with files of this type sometimes requires examining the files, for example to detect or rule out errors such as non-printing characters or incorrect header information. Emacs can be used to do this.

Emacs has a couple of special modes that are useful for examining data files and sometimes source code. Emacs has a hexadecimal viewer/editor mode that can be invoked by typing hexl-mode at the Emacs command prompt. The Emacs hexl-mode is useful for examining binary file formats and occasionally human readable formats. Emacs has a special whitespace mode that displays whitespace characters such as tab, carriage return, newline, and so forth. Type whitespace-mode at the Emacs command prompt. This can useful for examining human readable data files and also source code files where the whitespace characters may be incorrect or confusing.

Making Octave Work on the Macintosh

There are a couple of gotchas to making Octave work on the Macintosh platform. By default, on MacOS X, Octave is not set up to display plots through the X server on the Macintosh. It is necessary to set an environment variable so that Octave can display plots. This is done by entering the following command at the Octave prompt.

Octave> setenv GNUTERM 'x11'

By default, Octave uses the gnuplot package, which it installs if needed, to display plots. This environment variable tells gnuplot to use ‘x11′, the X Windows server on the Macintosh, to display plots.

As discussed above, Octave has a large number of add-on packages available through the Octave Forge web site. By default, a number of these packages do not build and install correctly on MacOS. One gets a cryptic error. This is because the mkoctfile command in Octave defaults to building 64 bit versions on the Macintosh. This can be fixed by adding the lines:

CFLAGS="-m32 ${CFLAGS}"
FFLAGS="-m32 ${FFLAGS}"
CPPFLAGS="-m32 ${CPPFLAGS}"
CXXFLAGS="-m32 ${CXXFLAGS}"
LDFLAGS="-m32 ${LDFLAGS}"

in the file Octave.app/Contents/Resources/bin/mkoctfile-3.2.3 just after the “set -e” line. The problem is that in Snow Leopard (code name for a recent version of Mac OS X) compilers try always to build in 64bit, while the libraries shipped with Octave are 32 bit. The -m32 flag forces the compiler to build 32 bit programs.

Screen Capture with Octave

Using Octave or similar programs often involves making plots, histograms, and other graphics that the user wants or needs to share with others, attach to e-mails, embed in publications, and so forth. One quick way to do this on Microsoft Windows and Macintosh platforms is through the built-in screen capture capabilities of the operating system and the gnuplot package used by Octave.

On Microsoft Windows, there is a “Print Screen” button on keyboards that will capture the entire screen to the MS Windows clipboard. One can then use GIMP or many other image editors to create a file from the contents of the clipboard. One can then edit the image file to select desired regions and so forth. On MS Windows, the gnuplot display window has a window capture feature. One can type Control-C in the plot window to capture the plot window to the Windows clipboard. One can also select the plot window menu by clicking the icon in the upper left corner of the plot window, then “Options” from the menu, and then “Copy to Clipboard” from the Options menu.

The Macintosh has a number of keyboard shortcuts to capture either the entire screen or a region of the screen to the clipboard or to a file on the Macintosh desktop. These keyboard shortcuts are (Apple Command)-Shift-3 to capture the entire screen to a file on the Desktop, (Apple Command Key)-Shift-4 to capture a region to a file on the Desktop, (Apple Command Key)-Control-Shift-3 to capture the entire screen to the Macintosh clipboard, and (Apple Command Key)-Control-Shift-4 to capture a region to the Macintosh clipboard. The region commands turn the mouse cursor into a cross-hairs icon; the user can select a region to capture by clicking and dragging on the screen with the cross hairs.

Conclusion

Octave is free, both free as in beer and free as in speech. Octave has many virtues, notably that it is mostly compatible with MATLAB, which is currently the de facto standard for numerical and mathematical programming. Octave can be used to quickly develop software for a wide range of practical problems including the prediction of buying patterns, image and signal processing, pure and applied scientific research, and the invention and design of new machines, to name only a few. With the tools, methods, and commands discussed above, it is possible to make Octave work quickly and cost-effectively with a wide range of other programs on all three major computing platforms: MS Windows, Linux/Unix, and Macintosh.

© 2011 John F. McGowan

About the Author

John F. McGowan, Ph.D. is a software developer, research scientist, and consultant. He works primarily in the area of complex algorithms that embody advanced mathematical and logical concepts, including speech recognition and video compression technologies. He has extensive experience developing software in C, C++, Visual Basic, Mathematica, MATLAB, and many other programming languages. He is probably best known for his AVI Overview, an Internet FAQ (Frequently Asked Questions) on the Microsoft AVI (Audio Video Interleave) file format. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech). He can be reached at jmcgowan11@earthlink.net.

Sponsor’s message: Check out Math Better Explained, an insightful ebook and screencast series that will help you see math in a new light and experience more of those awesome “aha!” moments when ideas suddenly click.

Possibly related articles:

1. Octave: An Alternative to the High Cost of MATLAB
2. 3 awesome free Math programs
3. Free Kindle Math Books

Today Kalid Azad released a premium edition of his well-received Math Better Explained ebook.

This brand new edition (available for a limited time for just $47) boasts a screencast for each of the 12 chapters in the book, as well as a 3-part interview about developing your mathematical intuition with Scott H Young, who has written extensively about study techniques. The package also includes a PDF version of the book, a worksheet, and slides/image as a zip file.

We were provided early access to the material, and have found the ebook + screencasts combo to be highly effective in conveying fundamental mathematical concepts that will serve you well for years to come.

To celebrate the announcement, and as a Math Blog exclusive, today we are publishing an excerpt from the book here. It covers how to develop math intuition and the great importance this skill plays in the field of mathematics. Enjoy!

Developing Math Intuition

Our initial exposure to an idea shapes our intuition. And our intuition impacts how much we enjoy a subject. What do I mean?
Suppose we want to define a “cat”:

• Caveman definition: A furry animal with claws, teeth, a tail, 4 legs, that purrs when happy and hisses when angry. . .
• Evolutionary definition: Mammalian descendants of a certain species (F. catus), sharing certain characteristics. . .
• Modern definition: You call those definitions? Cats are animals sharing the following DNA: ACATACATACATACAT. . .

The modern definition is precise, sure. But is it the best? Is it what you’d teach a child learning the word? Does it give better insight into the “catness” of the animal? Not really. The modern definition is useful, but after getting an understanding of what a cat is. It shouldn’t be our starting point.

Unfortunately, math understanding seems to follow the DNA pattern. We’re taught the modern, rigorous definition and not the insights that led up to it. We’re left with arcane formulas (DNA) but little understanding of what the idea is.

Let’s approach ideas from a different angle. I imagine a circle: the center is the idea you’re studying, and along the outside are the facts describing it. We start in one corner, with one fact or insight, and work our way around to develop our understanding. Cats have common physical traits leads to Cats have a common ancestor leads to A species can be identified by certain portions of DNA. Aha! I can see how the modern definition evolved from the caveman one.

But not all starting points are equal. The right perspective makes math click — and the mathematical “cavemen” who first found an idea often had an enlightening viewpoint. Let’s learn how to build our intuition.

1.1 What is a Circle?

Time for a math example: How do you define a circle?

There are seemingly countless definitions. Here’s a few:

• The most symmetric 2-d shape possible
• The shape that gets the most area for the least perimeter (the isoperimeter property)
• All points in a plane the same distance from a given point (drawn with a compass, or a pencil on a string)
• The points (x,y) in the equation x² + y² = r² (analytic version of the geometric definition above)
• The points in the equation r · sin(t),r · cos(t), for all t (really analytic version)
• The shape whose tangent line is always perpendicular to the position vector (physical interpretation)

The list goes on, but here’s the key: the facts all describe the same idea! It’s like saying 1, one, uno, eins, “the solution to 2x + 3 = 5” or “the number of noses on your face” — just different names for the idea of unity.

But these initial descriptions are important — they shape our intuition. Because we see circles in the real world before the classroom, we understand their “roundness”. No matter what fancy equation we see (x² + y² = r²), we know deep inside that a circle is round. If we graphed that equation and it appeared square, or lopsided, we’d know there was a mistake.

As children, we learn the caveman definition of a circle (a really round thing), which gives us a comfortable intuition. We can see that every point on our “round thing” is the same distance from the center. x² + y² = r² is the analytic way of expressing that fact (using the Pythagorean theorem for distance). We started in one corner, with our intuition, and worked our way around to the formal definition.

Other ideas aren’t so lucky. Do we instinctively see the growth of e, or is it an abstract definition? Do we realize the rotation of i, or is it an artificial, useless idea?

1.2 A Strategy For Developing Insight

I still have to remind myself about the deeper meaning of e and i — which seems as absurd as “remembering” that a circle is round or what a cat looks like! It should be the natural insight we start with.

Missing the big picture drives me crazy: math is about ideas — formulas are just a way to express them. Once the central concept is clear, the equations snap into place. Here’s a strategy that has helped me:

• Step 1: Find the central theme of a math concept. This can be difficult, but try starting with its history. Where was the idea first used? What was the discoverer doing? This use may be different from our modern interpretation and application.
• Step 2: Explain a property/fact using the theme. Use the theme to make an analogy to the formal definition. If you’re lucky, you can translate the math equation (x² + y² = r²) into a plain-english statement (“All points the same distance from the center”).
• Step 3: Explore related properties using the same theme. Once you have an analogy or interpretation that works, see if it applies to other properties. Sometimes it will, sometimes it won’t (and you’ll need a new insight), but you’d be surprised what you can discover.

Let’s try it out.

1.3 A Real Example: Understanding e

Understanding the number e has been a major battle. e appears everywhere in science, and has numerous definitions, yet rarely clicks in a natural way. Let’s build some insight around this idea. The following section has several equations, which are simply ways to describe ideas. Even if the equation is gibberish, there’s a plain-english concept behind it. Here’s a few common definitions of e:

The first step is to find a theme. Looking at e’s history, it seems it has some- thing to do with growth or interest rates. e was discovered when performing business calculations (not abstract mathematical conjectures) so “interest” (growth) is a possible theme.

Let’s look at the first definition, in the upper left. The key jump, for me, was to realize how much this looked like the formula for compound interest. In fact, it is the interest formula when you compound 100% interest for 1 unit of time, compounding as fast as possible. The chapter on e describes this interpretation.

Definition 1: Define e as 100% compound growth at the smallest increment possible.

Let’s look at the second definition: an infinite series of terms, getting smaller and smaller. What could this be?

After noodling this over using the theme of “interest” we see this definition shows the components of compound interest. Now, insights don’t come instantly — this insight might strike after brainstorming “What could 1 + 1 + 1/2 + 1/6 + . . . ” represent when talking about growth?”

Well, the first term (1 = 1/0!, remembering that 0! is 1) is your principal, the original amount. The next term (1 = 1/1!) is the “direct” interest you earned — 100% of 1. The next term (0.5 = 1/2!) is the amount of money your interest made (“2nd-level interest”). The following term (.1666 = 1/3!) is your “3rd-level interest” — how much money your interest’s interest earned!

Money earns money, which earns money, which earns money, and so on — the sequence separates out these contributions (the chapter on e describes how Mr. Blue, Mr. Green & Mr. Red grow independently). There’s much more to say, but that’s the “growth-focused” understanding of that idea.

Definition 2: Define e by the contributions each piece of interest makes.

Neato. Now to the third, shortest definition. What does it mean? Instead of thinking “derivative” (which turns your brain into equation-crunching mode), think about what it means. The feeling of the equation. Make it your friend.

It’s the calculus way of saying “Your rate of growth is equal to your current amount”. Well, growing at your current amount would be a 100% interest rate, right? And by always growing it means you are always calculating interest – it’s another way of describing continuously compound interest!

Definition 3: Define e as always growing by 100% of your current value.

Nice — e is the number where you’re always growing by exactly your current amount (100%), not 1% or 200%.

Time for the last definition — it’s a tricky one. Here’s my interpretation: Instead of describing how much you grew, why not say how long it took?

If you’re at 1 and growing at 100%, it takes 1 unit of time to get from 1 to 2. But once you’re at 2, and growing 100%, it means you’re growing at 2 units per unit time! So it only takes 1/2 unit of time to go from 2 to 3. Going from 3 to 4 only takes 1/3 unit of time, and so on.

The time needed to grom from 1 to A is the time from 1 to 2, 2 to 3, 3 to 4. . . and so on, until you get to A. The first definition defines the natural log (ln) as shorthand for this “time to grow” computation.

ln(a) is simply the time to grow from 1 to a. We then say that e is the number that takes exactly 1 unit of time to grow to. Said another way, e is is the amount of growth after waiting exactly 1 unit of time!

Definition 4: Define the time needed to grow continuously from 1 to as ln(a). e is the amount of growth you have after 1 unit of time.

Whablamo! These are four different ways to describe the mysterious e. Once we have the core idea (“e is about 100% continuous growth”), the crazy equations snap into place — it’s possible to translate calculus into English. Math is about ideas!

1.4 What’s the Moral?

In math class, we often start with the last, most complex idea. It’s no wonder we’re confused: we’re showing students DNA and expecting them to see a cat. I’ve learned a few lessons from this approach, and it underlies how I understand and explain math:

• Search for insights and apply them. That first intuitive insight can help everything else snap into place. Start with a definition that makes sense and “walk around the circle” to find others.
• Be resourceful. Banging your head against an idea is no fun. If it doesn’t click, come at it from different angles. There’s another book, another article, another person who explains it in a way that makes sense to you.
• It’s ok to be visual. We think of math as rigid and analytic — but visual interpretations are ok! Do what develops your understanding. Imaginary numbers were puzzling until their geometric interpretation came to light, decades after their initial discovery. Looking at equations all day didn’t help mathematicians “get” what they were about.

Math becomes difficult and discouraging when we focus on definitions over understanding. Remember that the modern definition is the most advanced step of thought, not necessarily the starting point. Don’t be afraid to approach a concept from a funny angle — figure out the plain-English sentence behind the equation. Happy math.

Order the book and screencasts.

Disclaimer: The links to Math Better Explained contain our referrer id.

Possibly related articles:

1. What kind of Math did they teach you?
2. Math Books reviewed and classified by area of mathematics
3. The nicest math book I own

Cheat sheets can be very useful and make for great posters around your room. The following is a collection of 13 cheat sheets for several mathematical topics and programs:

1. General Math Cheat Sheet (iPaper and other formats)
2. Elementary Algebra Cheat Sheet (PDF)
3. Trigonometry Cheat Sheet (PDF)
4. Calculus Cheat Sheet (PDF)
5. Derivatives and Integrals Cheat Sheet (PDF)
6. Laplace Transforms Cheat Sheet (PDF)
7. Abstract Algebra Cheat Sheet (PDF)
8. Probability Theory Cheat Sheet (PDF)
9. Matlab Cheat Sheet (PDF)
10. Mathematica Cheat Sheet (PDF)
11. Maple Cheat Sheet (PDF)
12. Maxima Cheat Sheet (HTML web page)
13. LaTeX Cheat Sheet (several formats)

And since most of us like to show our math pride off when out and about as well, Amazon sells this awesome Math Cheat Sheet T-shirt with formulas on both sides (Also available for Science and Engineering). How awesome is this?

Sponsor’s message: Looking for online algebra homework solutions? Algebra help at AssignmentExpert.com! Solve your problems right now.

Possibly related articles:

1. Where Math-Blog is headed
2. A 10 minute tutorial for solving Math problems with Maxima
3. 3 awesome free Math programs

gnuplot is an excellent scientific package for visualizing data and plotting functions. Despite its name, it has nothing to do with the GNU project, even though it’s Open Source and entirely free. This tool is very handy whenever you need to produce production quality graphics from a given data-set (or function). It is no wonder that in its 20+ years of existence, it’s been employed in all sorts of industries.

Flexible, powerful, and easy to use, gnuplot is not only handy as a stand-alone program but can be used successfully by a variety of programming languages, including but not limited to Ruby, Python and Perl. It has also been adopted as a plotting engine by Open Source programs like Maxima and GNU Octave. The minimal effort required to learn gnuplot is therefore a very worthwhile endeavor. Being able to visualize things as you analyze data and explore mathematics, is a very useful aid.

gnuplot is cross-platform, and you can download it from SourceForge.

Once you have installed gnuplot, you can start the tool by simply running gnuplot from the shell. Getting started with the program is straightforward, given that the most basic functionalities are fairly intuitive. For example, if you wanted to plot the sin(x) function, you could run:

gnuplot> plot sin(x)

gnuplot will automatically decide for you what portions of the function should be visualized (in our case between -10 and 10 on the x axis). If you want to overwrite this, you can specify otherwise, as shown below:

gnuplot> plot [-pi:pi] sin(x)

Similarly, you can customize the plotting, by specifying a range on the y axis:

gnuplot> plot [] [-0.5:0.5] sin(x)

It is also possible to plot several functions at once:

gnuplot> plot [-2:2] x, x**2, x**3

gnuplot can draw all sorts of functions and graphics. For example, the following plots two surfaces:

gnuplot> splot x**2+y**2, x**2-y**2

What you see plotted above, is based on a sensible set of defaults, but gnuplot offers countless options to customize the appearance of your graphics.

gnuplot isn’t just useful for plotting mathematical functions. I often use it for plotting data that is stored in tabular format in simple text formats. For example, assume that you have collected the following data on Deaths by Major Causes in the US, from a website:

stats.txt

That’s a big chunk of data, but it’s not very easy to analyze. It’d be nice to be able to visualize it. Assuming you saved it in a stats.txt file (you can include or remove the first few lines of comments) from gnuplot you can run the following:

gnuplot> plot "stats.txt"

That’s a meaningless mess. But gnuplot is very flexible, so we can easily do much better. Let’s use the first column for x values, and each of the remaining columns as a curve of its own by running:

gnuplot> plot "stats.txt" using 1:2 title "Heart disease" with lines, "stats.txt" using 1:3 title "Cancer" with lines, "stats.txt" using 1:4 title "Cerebro-vascular diseases" with lines, "stats.txt" using 1:5 title "Lower respiratory diseases" with lines, "stats.txt" using 1:6 title "Diabetes mellitus" with lines, "stats.txt" using 1:7 title "Influenza and pneumonia" with lines, "stats.txt" using 1:8 title "Chronic liver disease" with lines, "stats.txt" using 1:9 title "Accidents" with lines, "stats.txt" using 1:10 title "Suicide" with lines, "stats.txt" using 1:11 title "Homicide" with lines

Or it’s abbreviated version:

gnuplot> plot "stats.txt" u 1:2 title "Heart disease" w l, "" u 1:3 title "Cancer" w l, "" u 1:4 title "Cerebro-vascular diseases" w l, "" u 1:5 title "Lower respiratory diseases" w l, "" u 1:6 title "Diabetes mellitus" w l, "" u 1:7 title "Influenza and pneumonia" w l, "" u 1:8 title "Chronic liver disease" w l, "" u 1:9 title "Accidents" w l, "" u 1:10 title "Suicide" w l, "" u 1:11 title "Homicide" w l

Much better! Of course, the command above has a bit of repetition in it, but it’s justified by the flexibility of being able to pull data from different files if required, and plot some data with lines and others without (or in different styles) if we want to. To improve the appearance and legibility of the chart, let’s add a grid:

gnuplot> set grid
gnuplot> replot

Nice! And notice how the replot command runs the last plotting directive for us, saving us some typing (or scrolling through the previous commands). To improve this further, we can also set the labels for the axis and a title for the whole chart:

gnuplot> set xlabel "Years"
gnuplot> set ylabel "Death rates per 100,000 people"
gnuplot> set title "Deaths by Major Causes, 1960–2005"
gnuplot> replot

(Click here for a larger version)

Effortlessly, we went from having some data in a text file, to a meaningful, and professional looking chart that could be used in an article or a book. Not bad. You can save it in your favorite format, by setting the output file name and the terminal type. In the example below, I specify the font type, size and its location on my Mac OS X system, but it’s entirely optional. If missing, the default fonts apply.

gnuplot> set output "mychart.png"
gnuplot> set terminal png font "/Library/Fonts/Times New Roman.ttf, 11"
gnuplot> replot

This very short introduction should you give you a glimpse into what gnuplot can do for you, and even get you started with the tool. But gnuplot is much more than this and infinitely customizable, so you really may want to consider learning more about it. Unfortunately, despite being widely used for more than a decade, gnuplot has never had a book published about it.

Thankfully, Manning Publications will be putting out a book called Gnuplot in Action in October. The good news is that you don’t have to wait for the dead tree version, you can purchase it today through the Manning Early Access Program (MEAP). 12 out of the 15 chapters are already available, so the book is pretty much complete. If you’re interested in making the best out of gnuplot, this book is a must have. You don’t have to be a mathematician or a programmer to follow along, and it’s so much more than a bunch of options for customizing your graphics. The book guides you through the best techniques for taking advantage of the tool in several common scenarios. By reading, this easy to follow book, you’ll be able to generate colorful, nifty plots and have good mastery of the tool in a very short amount of time.

No related posts.

The Jenkins-Traub Algorithm is a standard in the field of numerical computation of polynomial roots, fundamentally developed as a numerical algorithm specifically for the task of computing polynomial roots. In other words, (i) because it was planned from the outset for numerical purposes rather than being simply an adaptation of an analytic formula, it is extremely robust, effectively minimizing the effects of computer round-off error, while (ii) also being extremely efficient compared to more general methods not written specifically for the task of computing polynomial roots; in fact, the algorithm converges to polynomial roots at a rate better than quadratic. Furthermore, since being introduced over thirty years ago, the algorithm has had time to be rigorously tested and has successfully proven its quality; as a result, it has gained a wide distribution as evidenced by its incorporation in commercial software products and the posting on the NETLIB website of source code for programs based on the algorithm.

An Introduction to the Field of Root-finding

Given a function which is defined in terms of one or more independent variables, the roots (also called zeros) of the equation are the values of the independent variable(s) for which the function equals :

Note that, in general, the z values are complex numbers, comprised of real and imaginary components (indicated by the ): .

Consider the following equation:

The values of and which satisfy this equation may not be possible by analytical methods, so the equation would be rearranged into the following form:

which is of the form

Once in this form (the standard form), solving the original equation becomes a matter of finding the and values for which f equals , and well-developed fields of mathematics and computer science provide several root-finding techniques for solving such a problem. Note that since the theme of this article is polynomial root-finding, further examples will focus on single-variable equations, specifically, polynomials.

Analytical Root-finding Techniques

Consider the following quadratic equation:

This equation is a second-degree polynomial–the highest power applied to the independent variable is 2. Consequently, this equation has two roots; an n-degree polynomial equation has n roots.

Because the equation is a simple polynomial, a first approach to finding its zeros might be to put the equation into the form and make a few educated guesses; a person could quickly determine that (a strictly real result) is one root of this equation. This root could then be divided out of the original polynomial

to yield the second root: .

In this simple example, the two roots were found easily; the second root was found immediately after the first one was known. However, in most real-world problems the roots of a quadratic are not found so easily. Furthermore, in problems where the original polynomial is of degree greater than two, when one root is found, the other roots do not follow immediately — more sophisticated techniques are required.

Among the techniques available are analytical ones, in other words, techniques that yield explicit, algebraic results. For example, in the case of a quadratic equation, explicit expressions for the two roots are possible via the Quadratic Formula: once a quadratic equation is arranged into the standard form

(a, b and c are known constants), the two roots are found via the Quadratic Formula:

The quantity within the square root sign is called the discriminant and determines whether the solution is complex or strictly real. If the discriminant is less than , the numerator contains an imaginary component and the roots are, therefore, complex. If the discriminant is greater than or equal to , the solutions are real. In fact, if the discriminant is , the two roots are real and equal:

In addition to the Quadratic Formula for quadratic equations, analogous formulae exist for polynomials of degree three and four. However, for polynomials of degree five and higher, analytical solutions are not possible (except in special cases), only numerical solutions are possible.

Numerical Root-finding Techniques

For the multitude of real-world problems that are not amenable to an analytical solution, numerical root-finding is a well-developed field offering a wide variety of tools from which to choose; many computer programs written for the computation of roots are based on algorithms that make them applicable to the computation of roots of functions in addition to polynomials, and virtually all of them employ an iterative approach that terminates once the desired degree of tolerance has been achieved. For example, the bisection method is a simple and reliable method for computing roots of a function when they are known ahead of time to be real only. The algorithm starts with the assumption that a zero is somewhere on a user-supplied interval [a,b]. In other words, the function value f(a) is of the opposite sign of f(b) – in going from a to b, f either goes from being positive to being negative, or it goes from being negative to being positive. A point, m, in the middle of the interval [a,b] is then selected. The function is evaluated at point m: f(m). If the sign of f(m) is the same as the sign of f(a), the desired zero is not in the interval [a,m]. In this case, the interval is cut in half and the new interval becomes [m,b]. On the other hand, if the sign of f(m) is not the same as the sign of f(a), the desired zero is in the interval [a,m]. In this case, the interval is cut in half and the new interval becomes [a,m]. This process is repeated until either the interval converges to a desired tolerance or a value of f is found that is within an acceptable tolerance, in which case the value of z at this point is the value returned by the program as the root.

In addition to the bisection method, more elegant–and faster converging–techniques are available: the False Position method, Brent’s method, the Newton-Raphson method, the secant method, and others. The Newton-Raphson method uses the function and its derivative to quickly converge to a solution. This method is good for situations in which the derivatives are either known or can be calculated with a low computational cost. In other situations, the cost of computing derivatives may be too high. The secant method, by comparison, does not require the use of derivatives, but the calculation of each iterate requires the use of the two previous iterates and does not converge to a solution as quickly as the Newton-Raphson method. In some situations, this method, too, may be deemed impractical. Usually, when some thought is given to a problem, a particular technique can be applied to it that is more appropriate than another. In fact, for some problems, one technique may fail to find a solution at all, whereas another technique will succeed. For other problems, several techniques may, indeed, be able to solve the problem and the numerical analyst may select the one that is simply more computationally efficient.

The Jenkins-Traub Algorithm

The field of numerical root-finding is so well-developed that the use of a good-quality numerical program is recommended — when numerical results are sought — even when an analytical solution to a problem is known. In other words, even though an analytical solution to, say, the quartic equation exists, writing a computer program that simply implements the textbook formula is not recommended; computer round-off errors often render results of such programs meaningless. The use of a robust numerical program, based upon sound theory and an excellent algorithm, and coded to thoroughly deal with computer round-off errors, is the recommended action. The Jenkins-Traub Algorithm is such an algorithm; it is a three-stage, extremely effective, globally convergent algorithm designed specifically for computing the roots of polynomials.

Stage One is the “No Shift” stage; the main purpose of this stage is to accentuate the smaller zeros. The search for a zero is started by taking an initial guess of for a fixed number, M, of iterations (M is usually assigned the value 5 on the basis of numerical experience⁽¹⁾).

Stage Two is the “Fixed Shift” stage, the purpose of this stage is to separate zeros of equal or almost equal magnitude. As a starting point in this stage, the following value is used:

is a lower bound on the magnitudes of the probable zeros in the cluster. could be taken at random, since the cluster could be anywhere in the complex plane; however, in practice is usually initialized to 49°, putting s near the middle of the first quadrant of the complex plane. After a certain number of iterations, if s does not converge to a root, s is assigned a new value by increasing by 94°. Repeated attempts would have the search for a root start with points in all four quadrants of the complex plane until the search is returned to the first quadrant. Should the search, indeed, return to the first quadrant, successive cycles start at points 16° away from the starting point of the preceding cycle.

Stage Three is the “Variable Shift” stage, which is terminated when the computed value of the polynomial at a possible zero is less than or equal to a specified bound.

In addition to the three fundamental stages around which it was developed, the Jenkins-Traub Algorithm incorporates several other techniques for making it as effective as possible. One of those techniques is deflation of the polynomial by synthetic division each time a root is found. Consider the following monic polynomial:

The are known constants and, in general, are complex. Now say the root () has been found. Synthetic division would be employed to divide that root out of the original polynomial:

to yield

The are new constants. The root-finding process is then repeated on this new–simpler–polynomial. As each root is found, the polynomial becomes successively simpler and each successive iteration of the algorithm involves, in general, fewer computations.

For polynomials whose coefficients are real only, when a complex root is found (), an additional benefit arises: that root’s complex conjugate is also a root (). In other words, two roots are computed — and the polynomial can be deflated by two degrees — in a single iteration of the algorithm. Furthermore, this deflation involves real only, rather than complex, operations; the product of these two roots is a real quadratic equation:

In this case, synthetic division is employed on the polynomial as follows:

In fact, for computing roots of polynomials which have only real coefficients, a modified version of the Jenkins-Traub Algorithm has been written which incorporates several features that take advantage of the characteristics of real-only polynomials to yield significant decreases in program execution time; “If the complex and real algorithms are applied to the same real polynomial, the real algorithm is about four times as fast.”⁽⁴⁾

The Jenkins-Traub Algorithm not only deflates the polynomial as roots are computed, it computes the roots roughly in order of increasing magnitude. This approach is taken because deflation of a polynomial can be unstable unless done by factors in order of increasing magnitude. By starting the search for roots with an initial guess of , as is done in Stage One, roots are, indeed, computed roughly in order of increasing magnitude, the factors by which the polynomial is successively deflated are roughly in order of increasing magnitude — the deflation of the polynomial is made quite stable.

As a quick check of the effectiveness of the Jenkins-Traub Algorithm, consider a contrived numerical example:

The roots of this polynomial are very close and should test how effectively the algorithm discerns near-multiple roots:

1.001
0.998
1.00002
0.99999

In addition, the polynomial is real-only so that readers may confirm the results with a free, ready-to-use, online Javascript program (URL given below).

Expanded, the test polynomial becomes

and the solutions computed by the Javascript root-finder follow:

0.9980051486669109
0.9998618452719573
1.0001539207016332
1.0009890853594987

The program, in fact, did a very good job:

1. no errors were returned,
2. no erroneous imaginary components were found,
3. four distinct roots were computed, and
4. the computed roots are extremely close to the exact roots (round-off errors are unavoidable).

Conclusion

Since its introduction in 1969 (in Michael Jenkins’ PhD thesis) the Jenkins-Traub Algorithm has been rigorously tested and has proven itself as an excellent algorithm for the computation of the roots of polynomials. It is extremely robust, is globally convergent for any distribution of zeros, and converges at a better than quadratic rate. Attesting to its high quality is the fact that source code for programs (written in FORTRAN) based on the algorithm have been posted on the NETLIB site, a repository of high-quality numerical programs; the complex coefficient version is posted as CPOLY (http://www.netlib.org/toms/419) and a real coefficient only version has been posted as RPOLY (http://www.netlib.org/toms/493). Translations into C++ and Javascript are also available; for example, Polynomial Root-finder (http://www.akiti.ca/PolyRootRe.html) is a Javascript translation of RPOLY and has been successfully tested with data published in Jenkins’ PhD thesis. In addition, the algorithm is incorporated in commercial software products (i.e. Mathematica). The proven effectiveness of the Jenkins-Traub Algorithm and its wide-spread distribution make it an extremely important tool in the toolbox of numerical analysts.

References

1. “A First Course in Numerical Analysis: Second Edition”
   Anthony Ralston and Philip Rabinowitz
   McGraw-Hill Book Company
   New York
   1978
2. “Numerical Methods and Software”
   David Kahaner, Cleve Moler, Stephen Nash
   Prentice-Hall, Inc.
   Englewood Cliffs, New Jersey
   1989
3. “Numerical Recipes. The Art of Scientific Computing.”
   William H. Press, Brian P. Flannery, Saul A. Teukolsky, and William T. Vetterling
   Cambridge University Press
   New York
   1986
   (3rd edition now available.)
4. Wikipedia
   Jenkins-Traub Algorithm for Polynomial Zeros
5. Jenkins, M.A. Three-stage variable-shift iterations for the solution of polynomial equations with a posteriori error bounds for the zeros. Diss., Rep. CS 138, Comput. Sci.
   Dep., Stanford U., Stanford, Cal., 1969.

This article was written by David Binner. You can visit his math site at http://akiti.ca. If you’d like to write for Math-Blog.com, please email us at .

No related posts.