The author of the site, describes the project as follows:

Project Euler is a series of challenging mathematical/computer programming problems that will require more than just mathematical insights to solve. Although mathematics will help you arrive at elegant and efficient methods, the use of a computer and programming skills will be required to solve most problems.

When you first register, you will be presented with 252 problems available to be solved. This number changes frequently, however, as new problems are routinely created and added to the list. The solutions to some problems veer towards the obvious, while others require advanced mathematics to tackle. Likewise, a few can be solved with paper and pencil and a good dose of insight. Most however, will require the aid of a computer program written in the language of your choice.

A sample problem (187)

Each program you execute should output your result within a minute. That’s part of the challenge, even though there is no way for the site to enforce this rule.

When you think you’ve found the correct result for a given problem, you can fill in the result field for the problem, and see if you got it right. If you have, your answer will be added to the list of problems you solved and will count towards your overall score, while bumping up your position on the participant scoreboards. There are several scoreboards, depending on what level you are at, which in turn is determined by the number of problems you have solved so far.

More importantly, solving a problem will gain you access to the respective forum (for that problem), where you’ll be able to see solutions and discussions from other participants, who may have used different languages and solution strategies.

Project Euler is appealing for a variety of reasons. In particular:

• It allows you to explore areas of math that you might not be familiar with. Solving certain problems may require you do a fair amount of research and in turn help you to learn more about certain branches of mathematics;
• If you are a math major who is trying to hone their programming skills, you’ll find Project Euler to be challenging, fun, and an excellent opportunity for improvement. Having to implement several small programs and being able to compare your style with those of other people in the forum, will definitely end up improving your knowledge of many algorithms and your skills as a programmer;
• The site’s one minute rule has an important consequence. The computational complexity of your programs really ends up mattering. While some simple problems can in fact be brute forced, the majority of them require you to write faster algorithms and take advantage of mathematical insight so as to improve the performance of your program. For certain problems a naive approach would take more than a lifetime on current PCs;
• Finally, if you are a programmer who’s scoping out a new programming language, be it Python, Ruby, Scala, Haskell or Erlang, you’ll find a great ally in Project Euler. Having to write hundreds of programs in a given language, will naturally increase your familiarity with that language. And again, comparing your newcomer coding style with those of more experience participants, will no doubt contribute to your advancement within the given language you’re focusing on.

Project Euler truly deserves five stars. I recommend it highly to anyone who is unafraid of writing a few lines of code in order to solve interesting math problems.

Complex algorithms are typically implemented as either computer software or in custom VLSI chips (e.g. Application Specific Integrated Circuits or ASICs). Research and development of complex algorithms is a specialized area and differs in a number of ways from most software and hardware development. Remarkably, many computer software business and engineering professionals often underestimate or do not understand the difficulties and scope of complex algorithm projects.

Complex algorithms are already in widespread use in commercial applications. Prominent examples include the video compression algorithms that enable BluRay, DVD Video, YouTube, and many other modern digital video systems. The US DVD Video market is around $25 billion per year (2007). Although limited, speech recognition such as now frequently encountered in telephone help and customer service systems is another example. Other examples include encryption, seismic modeling used in oil and gas exploration, sophisticated financial models, traffic models, and many others.

Complex algorithms may solve a range of major problems confronting the human race including major diseases such as cancer, the need for more and cheaper energy, and so forth. Molecular modeling may enable the design of drugs or systems of drugs that can selectively target and destroy cancer cells based on the identifying characteristics of cancer cells such as chromosomal anomalies, something currently impossible. Electromagnetic modeling software may enable the successful design and fabrication of working commercial fusion power sources. These are potential trillion-dollar markets. The global annual energy market is over $1 Trillion.

Note: Since I am a software developer, I will focus primarily on complex algorithms developed and implemented as computer software. Many of my comments apply equally well to hardware implementation. Where I have some knowledge and experience, I will make some comments on specific hardware issues.

Complex Algorithms Are Difficult

Complex algorithms are usually quite difficult to develop and often take longer than experience with other software projects would indicate. Although there are exceptions, complex algorithm projects usually take between four (4) months and several years. True research projects in which new mathematical or logical methods are developed are extremely unpredictable and typically take years. Most major scientific discoveries and inventions have taken at least five (5) years.

Complex algorithms frequently involve a tight coupling between different parts of the algorithm. All parts must work together within tight tolerances for the entire algorithm to work. This is similar to mechanical systems such as automobile engines or mechanical clocks. Indeed, implementations of complex algorithms are often referred to as “engines”, partly for this reason. Complex algorithms are often very unforgiving. Even very small errors, getting a single bit wrong, results in the implementation failing. This occurs frequently with encryption where usually every bit must be correct and video compression where even small errors often result in unacceptable “artifacts” in the decoded video. In practical terms, this means that the amount of time spent per line of working code is often significantly larger for complex algorithm projects than other software projects such as web sites, user interfaces, database reporting systems, and so forth.

Most commercial software projects involve at most mathematics taught in early high school (9th, 10th grade) in the US. Even advanced high school mathematics such as the solution to quadratic equations is uncommon outside of computer graphics. Complex algorithms in widespread use today typically involve mathematics that is taught in the first and second year of college at a good college or university in the US. A few complex algorithms involve more advanced mathematics. For example, the Global Positioning System (GPS) uses General Relativity, advanced undergraduate or graduate level mathematics, to determine the location and time correctly. In the future, more advanced mathematics may be needed for pattern recognition and other advanced tasks. Most commercial software developers do not have much experience with mathematical software at the level found in complex algorithms. Research and development of complex algorithms often requires a range of mathematical and logical skills that are not common.

Return on Investment

The return on investment for a successful complex algorithm project can be very high. Complex algorithm research and development is typically done by small teams or individuals. Small teams are the most common. Even a multi-year project, for example five years, with a ten person team (a large team) has a total cost of about $7.5 million (using a total cost per full time employee of $150K/year). A home run can solve a billion dollar or larger problem, bringing in hundreds of millions or even billions of dollars.

• Return = $100 M / $7.5 M = 13.3 (small home run)
• Return = $1 B / $7.5 M = 133 (big home run)
• Return = $1 T / $7.5 M = 133,000 (off the charts)

The greatest opportunities and the greatest risks lie in areas that require development of new mathematical or logical methods; that is true research. New complex algorithms can be converted very rapidly to commercial software products, even in a matter of months, as happened with new video compression algorithms in 2003.

Research and Development

The commercial software industry focuses overwhelmingly on “technically feasible” projects. Many venture capital firms explicitly claim to only invest in proven, technically feasible projects. Similar thinking pervades the commercial software industry. Where complex algorithms are concerned, technically feasible means proven algorithms for which working prototypes exist somewhere. The working prototypes are usually computer programs, often slow, that successfully implement the algorithm. These are frequently prototypes in the C or C++ programming languages, although Java is becoming more common (see the discussion of software engineering below). Thus, most commercial projects in the complex algorithms arena involve such tasks as porting algorithms to a different platform (for example, Unix to Windows), optimizing the algorithms for a new platform, integrating the algorithms into an application program such as a media player, converting a prototype into a production system, and so forth. Most research scientists would call these activities “Development” and not “Research” or “Research and Development”.

The commercial software industry follows a widely accepted rule to avoid projects that are not technically feasible, meaning true research projects. Nonetheless, the rhetoric of the commercial software industry, both aimed at unsophisticated investors and customers is the opposite. Terms like research, science, and research and development are used routinely to describe commercial software development activities. Many companies make statements that either explicitly claim or imply that the company has a large R&D group engaged in true research. Note that rhetoric aimed at sophisticated investors such as venture capitalists is often the opposite, which can be quite confusing.

Historically, the commercial software industry has relied heavily on government sponsored research programs such as the Defense Advanced Research Projects Agency (DARPA) and the National Aeronautics and Space Administration (NASA) for the true research in software. Many types of software and specific software products can be traced back to government sponsored research programs. Some well known examples include the Internet, originally a DARPA project, and the World Wide Web, which grew out of research projects at CERN and NCSA. Many other examples exist. Essentially all speech recognition software is derived from research sponsored by DARPA, especially projects at Carnegie-Mellon University. Nonetheless, industry rhetoric often invokes the image of private inventors in garages, the Wright brothers, and similar images of “free enterprise” and individual initiative. Typically, the putative inventor such as Tim Berners-Lee or Marc Andreessen is emphasized and the relevant government research program ignored or downplayed. Often there is little or no progress in commercial software if the relevant government research program is unable to make progress. This is most evident in pattern recognition and artificial intelligence, where progress has been very slow or non-existent.

Many government research programs are afflicted by a single “right way” that is pursued to the exclusion of all others. If this right way is good, then there is steady progress in the associated commercial software field. DARPA in particular relies upon periodic contests pitting different methods against one another. This has repeatedly resulted in a single approach that showed early promise taking over a field. A contest of this type during the 1970’s resulted in the so-called Hidden Markov Model (HMM) based speech recognition approach replacing essentially all speech recognition research on a global scale. Essentially all major speech recognition research groups, many directly funded by DARPA, pursue some variant of the HMM algorithm. Yet the performance of the HMM algorithm continues to be quite limited after 30+ years.

Most major commercial opportunities in complex algorithms require a company to fund and undertake genuine research, a difficult task that few companies understand. Artificial intelligence, speech recognition, cures for major diseases such as cancer or working fusion energy sources require substantial research.

Evaluating Technical Feasibility

In complex algorithms, technical feasibility generally means that a working prototype of the complex algorithm exists. In practice, working prototypes are often implemented in computer programming languages such as ANSI C, C++, or Java. Special algorithm research and development tools such as Matlab and Mathematica are also used. The working prototype may be slow, inefficient, or have other limitations, but it is or should be a proof of concept.

Seemingly, it should be easy to evaluate technical feasibility. Simply acquire and test the working prototype. Nonetheless, there is a long history of ostensibly sophisticated organizations and investors investing in complex algorithm projects that are not technically feasible, often apparently in the belief that the relevant problem had been solved. This has happened repeatedly with pattern recognition and artificial intelligence. It has also happened with various data compression algorithms.

For some reason, some organizations and investors do not even acquire and test the working prototypes. This is absolutely essential. It is often said that one should invest in people, not in ideas. Venture capitalists and other sophisticated investors often quote this platitude. However, the history of complex algorithms is filled with companies and projects with officers, directors, managers, chief scientists, and so forth with gold-plated resumes, impressive credentials, top degrees from top universities, and so forth that have flopped. The bottom line in complex algorithms is that the ideas count. If the complex algorithm doesn’t work, it doesn’t work. The company or project will fail for purely technical reasons. This is the lesson, for example, of the Pen Computing fad of the early 1990’s, GO, and Lernout and Hauspie (see below). This is also one of the lessons of the complex financial models involved in the current global financial crisis (2008). Due diligence requires a thorough, well-considered independent test and evaluation of the working prototype.

It can be difficult to evaluate technology demonstrations of complex algorithms. Often there are subtle and not so subtle ways to manipulate a technology demonstration. For example, in the early days (1995) of MPEG digital video on personal computers, MPEG software video player companies gave technology demonstrations on high-end personal computers with high-end video acceleration graphics cards. In pattern recognition, many present-day (2008) pattern recognition algorithms can achieve recognition rates in the range of 80-95% at least under certain controlled conditions. This seems high. In schools, 80-95% is usually a B or an A, a good grade. However, practical real-world pattern recognition often requires a recognition rate of 98 or 99% or even better under field conditions. In both examples, the technology demonstration can be misleading without demonstrably false statements.

Actual fraud can be difficult to detect and difficult to prove. It is important to conduct tests and evaluations of working prototypes independent of the purveyor or other interested parties who may have a reason to perpetrate a fraud. Complex algorithms can involve the potential for very large returns on investment—as noted above. Consequently, there is considerable motivation for fraud in some cases.

How Does Research Differ from Development?

Commercial software development is usually unpredictable. Software projects frequently involve unexpected problems and usually take substantially longer than planned. Nonetheless, technically feasible commercial software development projects are more predictable than true research projects. Often if one takes a conservative cost and schedule estimate and multiplies this by a factor of three to four, one gets the actual cost and schedule of the project. A common joke with a great deal of truth is: to get the real schedule multiply the official schedule by PI (3.14) for running around in a circle. Because the project is technically feasible it can certainly be completed. Massive cost and schedule overruns (such as factors of ten) can usually be explained by incompetence or severe political problems. True research is extremely unpredictable. Many true research projects simply fail. The researchers are unable to find the solution. For example, to date (2008), essentially all attempts to decipher human speech have failed in close to a century of attempts. Substantial research efforts at Bell Labs, MIT, and other institutions have failed to determine why certain sound spectra correspond to the different sounds in English and other languages. Even in successful research, estimates are often way off. For example, the mathematician Johannes Kepler made a bet in 1600 that he could determine the orbit of Mars in eight (8) days. His discovery of the elliptical orbit of Mars and other planets, one of the most important and difficult discoveries in scientific history, took five frustrating years in which every attempt to solve the problem failed until he found the answer in just a few days in 1605. This process in which long periods of little or no progress are punctuated by sudden unpredictable leaps forward is typical of true research, especially major scientific discoveries or technological inventions.

True research, especially major scientific discoveries or inventions, usually involves a very large amount of trial and error. Often, after many failures, there is a leap or leaps in which a new approach or concept is tried which unexpectedly solves the problem. Most major scientific discoveries or inventions took somewhere between five (5) and twenty (20) years. This is significantly longer than the time frame of typical commercial software industry and venture capital funded projects. In many cases, one is talking about five to twenty years of failure followed by a “breakthrough”, as in Kepler’s case.

In my experience, people involved in commercial software development are often unaware that they have little or no experience with true research. The misleading rhetoric of the computer software industry often leads people involved in commercial software development to think that they are engaged in the sort of true research conjured up by iconic names like Einstein or the Wright brothers. This undoubtedly leads to many bad decisions and frustrating experiences.

The Importance of Rapid Prototyping

History records many remarkable instances when an individual or small team succeeded in making a major scientific discovery or invention on a very small budget, sometimes beating far better funded competitors. Major scientific discoveries or inventions almost always involve a large amount of trial and error. Discoverers or inventors who managed to make a major discovery or invention on a shoe-string budget usually found a very fast, inexpensive way to perform the many trials and errors required to make a major discovery or invention.

For example, James Watt is remembered for inventing the separate condenser steam engine, a major conceptual leap that turned the steam engine from a niche device used in coal mining to a major driver of the industrial revolution. The Newcomen steam engines of Watt’s time were huge expensive house-sized engines. Watt however built and experimented with tiny scale models built from inexpensive wood, copper, and other materials. This enabled him to perform hundreds of trials and errors that led to the breakthrough concept of the separate condenser that radically improved the Newcomen steam engine.

Octave Chanute and the Wright brothers, his proteges, conducted research and development of gliders constructed of cheap wood and canvas. The gliders were flown at low altitude on soft sand beaches, first in Gary, Indiana near Chicago where Chanute lived, and later at Kitty Hawk in North Carolina. This meant that the inevitable damage from crashes was limited and easily repaired. The pilots did not die from the crashes as was common with other early would-be aviators. By delaying work on the expensive engines until last (they planned to buy a commercial off-the-shelf engine), they avoided the enormous cost involved in repairing or replacing an engine after each crash. This enabled Chanute and the Wright brothers to eventually succeed where better funded efforts such as Hiram Maxim and Samuel Langley failed.

Progress in aviation and rocketry today is quite slow, almost flat-lined since 1970, in part because the cost of a single trial, especially a new high performance engine, has become extremely high, easily in the millions if not billions of dollars per prototype engine and vehicle. In rocketry and other high performance engines, the prototype engine and vehicle are often destroyed during each trial. Internet entrepreneur Elon Musk of PayPal fame encountered this problem with his SpaceX startup as have many other Internet and software entrepreneurs attracted by the dream of space travel.

In algorithm research and development today, rapid prototyping tools such as Matlab and Mathematica (see below) speed up and reduce the cost of the many trials and errors required in true research. This is very important because the number of trials and errors is usually very large.

The Importance of Conceptual Analysis

Most major scientific discoveries and inventions usually involve a large amount of conceptual analysis expressed in words and pictures (often hundreds of thousands of words). It is common to find lengthy verbal discussions of the issues combined with rough sketches or drawings of concepts. For example, Octave Chanute wrote an entire book Progress in Flying Machines containing his lengthy verbal analysis of the problem of flight. This book outlines his successful research plan to develop working powered flight. It contains several rough drawings, as is common in major breakthroughs, and only a few brief calculations. The mathematician Johannes Kepler devoted much of his book, now known as New Astronomy, to a lengthy conceptual analysis of the problem of planetary orbits which was critical to his resolution of the problem.

At some point, these verbal analyses are refined into precise technical drawings in the case of mechanical inventions and specific mathematical expressions in the case of mathematical discoveries like Kepler’s. However, the verbal and visual analysis appears to be critical in many discoveries and inventions and usually comes first. It is likely that this sort of verbal and visual analysis will be essential to solve many problems such as artificial intelligence and pattern recognition.

Historically, this conceptual analysis was considered a part of philosophy. Much of the classical training in Greek philosophy and mathematics probably provided important training in this conceptual analysis. The discovery of new mathematical expressions of practical use strictly by the symbolic manipulation and the highly abstract thought favored by the famous mathematician David Hilbert and his school at the University of Gottingen in the early 20th century seems to be rare. This is specifically mentioned because Hilbert’s extremely abstract approach to higher mathematics has come to dominate mathematics and theoretical physics in the 20th century.

Some Famous Flops

Complex algorithm research and development is a treacherous area. There have been numerous flops and fiascoes over the years. It is easy to misjudge the technical feasibility of projects. There is a long history of exaggerated claims for complex algorithms that emulate aspects of human intelligence such as speech recognition. There has been enormous success in data compression over the last few decades. Nonetheless, there is a long history of exaggerated claims for advanced in data compression. Video and other data compression involves complex algorithms that are difficult to evaluate. Caveat emptor!

The Pen computing fad of the early 1990’s is an example of a famous flop. The most prominent of these firms was Jerry Kaplan’s GO, described in his book Startup. GO and similar firms’ business plans hinged on handwriting recognition, an unsolved problem in pattern recognition. Kaplan actually devotes only a few pages of his book to the handwriting recognition problem.

Another notorious example is the speech recognition firm Lernout and Hauspie. Lernout and Hauspie collapsed in a major financial scandal with court cases and allegations of massive fraud. Again, the success of Lernout and Hauspie’s business depended on the solution of the speech recognition problem, which remains largely unsolved even today.

Note that many apparently sophisticated investors invested many millions of dollars in both GO and Lernout and Hauspie, even though a modicum of research would have revealed the poor state of handwriting and speech recognition technology at the time.

Some Famous Successes

Video and audio compression is one of the most successful areas in complex algorithms. Technologies such as VideoCD, DVD, MP3, and BluRay all incorporate sophisticated audio and video compression algorithms.

A major breakthrough in video compression reached the market in 2003, embodied in H.264, Windows Media 10, Flash Video, and other video standards and products. Prior to 2003 the bitrate for usable, loosely VHS quality video was about one (1) megabit per second. In 2003, the new video technologies achieved a bit rate of around 275 Kilobits/second, often with close to DVD quality with proper tuning of the compression. This was a truly major advance, a rare technological leap forward. This enabled YouTube and other forms of Internet/web video over DSL connections.

The bottom line is that complex algorithm research and development can be done, but it is difficult.

Software Engineering

There are significant differences between software engineering for complex algorithm research and development and mainstream software development. As mentioned above, complex algorithms often involve a tight coupling between parts of the algorithm that makes development more difficult and tedious than most software development.

It is often easier to research and develop complex algorithms using tools such as Matlab, Mathematica, AXIOM, or Maxima (formerly known as MACSYMA). These are scripting languages similar to Python or PHP. They usually have implicit variable declaration and/or conversion. They are usually “weakly typed” languages and break many textbook rules of “good” software engineering. They include comprehensive, well-integrated libraries of mathematical, numerical, and statistical functions. They usually have a data type known variously as a list, vector, or matrix that represents sequences of numerical or symbolic data in a flexible way. These tools are sometimes referred to as computer algebra systems (CAS), although this is really only one subset of their features.

Adding Two Vectors in Mathematica

A = {1.0, 2.0, 3.0}; (* A is a Mathematica list *)
B = {1.1, 0.0, 4.0};
C = A + B
Out[1]={2.1, 2.0, 7.0}

Adding Two Vectors in C/C++

#include <iostream.h>
double A[3] = {1.0, 2.0, 3.0}; // A is a C++ array
double B[3] = {1.1, 0.0, 4.0};
double C[3];
int index;

for(index = 0; index <3; index++)
   C[index] = A[index] + B[index];

cout << “{“ << C[1] << “,” <<
C[2] << “,” << C[3] << “}” <<
endl;

Note that there is a vector class template in the C++ Standard Template Library (STL) with somewhat similar properties to the lists in Mathematica. The above comparison is for illustrative purposes. Even using the STL classes, it is usually much easier to research, develop, and test algorithms in these tools than using traditional compiled, strongly typed languages such as C/C++, Java, or <insert your favorite programming language here>. However, these tools are slow and require large amounts of memory. This is a significant drawback. Once an algorithm is developed, it is often necessary to convert the algorithm to a fast compiled language for performance reasons. This is easier if the target fast language has good libraries of mathematical, numerical, and statistical functions.

One can also research and develop algorithms directly in a fast programming language such as C/C++ or Java. This avoids conversion costs, speed, and memory issues. However, it is often much easier to do research and development using a tool such as Matlab or Mathematica.

The leading algorithm research and development tools are:

• MATLAB: Matlab is widely used in the commercial world, especially in digital signal processing.
• MATHEMATICA: Mathematica is widely used in government sponsored research and development and academic research. It has a following in Wall Street finance and economics.
• AXIOM: Axiom is free, open source, with a Berkeley style license. AXIOM was started in 1971 and has over 300 man years of work integrated into it.
• MAXIMA: Maxima is free, open-source, with a GNU license.

Many fast programming languages have been used for complex algorithms. The most popular are probably:

• ANSI C: ANSI C is almost universally available for all processors. It is simple, efficient, with small memory needs and high speed.
• C++: C++ is object-oriented. It often has larger memory needs than C and can be slower.
• Java: Java is compiled to byte-codes, but is approaching C/C++ in speed. It can be slower and less efficient. It can be easier to reverse engineer.

The dream algorithm R&D tool would be similar to Matlab or Mathematica but could be compiled to fast, efficient binaries similar to ANSI C and would be available for all platforms. An integrated GUI builder similar to Visual Basic and integrated network support would be helpful. The biggest single weakness of all kinds of scripting languages is that they are slow and cannot be compiled. For compute- intensive complex algorithms this can be a very significant problem. Of scripting languages, only Visual Basic 6 appears to have solved the problem of producing a compiler that can produce binary executables with similar performance to C/C++.

These algorithm research and development tools are not, of course, a substitute for thought, creativity, and the extensive conceptual analysis frequently required for major advances. Trial and error alone, without insight, rarely succeeds.

Conclusion

Complex algorithm research and development can be done successfully. Some great successes exist. Nonetheless, it is not easy and many things can go wrong. The project scope is significant. Project feasibility is difficult to assess. Genuine breakthroughs are unpredictable and take time. The return on investment for a home run can be five to thousands of times the original investment. There are great unrealized opportunities, most of which require genuine research.

About the Author

John F. McGowan, Ph.D. is a software developer, research scientist, and consultant. He works primarily in the area of complex algorithms that embody advanced mathematical and logical concepts, including speech recognition and video compression technologies. He has many years of experience developing software in Visual Basic, C++, and many other programming languages and environments. He has a Ph.D. in Physics from the University of Illinois at Urbana-Champaign and a B.S.in Physics from the California Institute of Technology (Caltech). He can be reached at jmcgowan11 (at) earthlink.net.

References

Some Complex Algorithms

• MPEG compression, one of the great success stories: http://www.chiariglione.org/mpeg/
• x264 is a free, opensource h.264 video encoder: http://www.videolan.org/developers/x264.html
• The Carnegie Mellon Sphinx Project, an open-source speech recognition engine: http://cmusphinx.sourceforge.net
• National Library of Medicine Insight Image Registration and Segmentation Toolkit: http://www.itk.org

Books and Articles

1. Startup: A Silicon Valley Adventure, by Jerry Kaplan, Houghton Mifflin Co, Boston, 1995, ISBN 0-140-25731-4
2. “How High-Tech Dream Shattered in Scandal at Lernout & Hauspie”, by Mark Maremont, Jesse Eisinger, and John Carreyrou, Wall Street Journal, December 7, 2000
3. New Astronomy (Nova Astronomia), by Johannes Kepler, Translated from the Latin original by William H. Donahue, Cambridge University Press, Cambridge, UK, 1992, ISBN 0-521-30131-9

This article was written by John F. McGowan, PhD. If you’d like to write for Math-Blog.com, please email us at submissions (at) math-blog.com.

At first glance we’re peering at a service that is able to provide objective information about a wide subset of human knowledge. But the idea behind Wolfram|Alpha is much more ambitious, as demonstrated by the first screencast to have been released (narrated by Stephen Wolfram).

Taking advantage of the popular and powerful mathematical software Wolfram Mathematica, this new engine is able to perform calculations on information requested by its users. Therefore there are three components at work: the ability to correctly interpret queries (in English) from the users, the ability to elaborate on the data source so as to reply with coherent results to somewhat complex questions, and finally the not-so-simple task of maintaining an up-to-date and accurate knowledge base for a very wide spectrum of human knowledge.

This is the theory behind – or at least the ultimate aim of – the service. And as such, unlike what has been reported elsewhere, Wolfram|Alpha should immediately be viewed as an addition to what Google already offers (not as a replacement for it).

As expected, the service is extremely good at mathematical calculations. In this case the only limit is the timeout imposed on each query to exclude those that require far too many resources to complete. An example of a calculation that is executed is “integrate e^-x^2”.

For the record, this service is currently under a fair bit of stress because of the initial curiosity of many worldwide, therefore the calculation of the integral above has shown an error message a couple of times (a tribute to HAL 9000):

All sorts of disciplines are represented by the examples on the site, with a particular focus on scientific and engineering ones. The results are elegantly presented and well organized, often illustrated and shown in table form, particularly when the user is asking for a comparison between different data sets. There are also fun tributes, including those to Monty Python and Douglas Adams. But how well does this system work when we step outside of the predefined examples provided by Stephen Wolfram’s team? The results range from exceptionally good to downright disappointing. Let’s see a few examples.

Let’s compare IBM, Google and Sun. As you can see, the results are definitely excellent. The comparison is almost exclusively numeric in nature, but the answer provided by WA jives with what I was hoping for. The same is true if you look for statistical information on a single large company like Telecom Italia.

Since this is a “computational engine” we can even try to perform a few calculations starting with the data that we found to be available in the comparison tables for the companies above. For example, “employees IBM/Google” will show us the ratio over the last few years between the number of IBM and Google employees (IBM currently hires almost 20 times as many people). We can calculate the revenue for each IBM employee by running “(market cap / employees IBM)”, but the engine fails to compare this parameter between companies: “(market cap / employees IBM) / (market cap / employees Google)”, despite the fact that the data for such a calculation is all there in the knowledge base (and as you can see there was already a revenue/employee row in the initial comparison table).

Moving on to something else, we can try to compare two cities like Toronto and Milan, and obtain very useful details. It’s also possible to calculate only certain attributes by running, for example, “population Toronto/Milan” or “distance Toronto Milan”. In the first case the data was updated to 2004, and is perhaps a little outdated now despite the fact that we are talking about demographic data. But the real problem arises when we try to compare the regions of Ontario and Lombardy. We’d expect to see nice geographical, demographical and economical comparisons between the two, yet instead we get nothing. Wolfram|Alpha will report information about little towns like Unionville (NC), but totally ignores a region like Lombardy or a Canadian province like Ontario (with a population of 13 million people). It would seem that this is a big hole in the knowledge of this service. Sure it’s fairly easy to fix, but it’s a symptomatic weakness nevertheless.

If we consider for a moment searches about famous people, we’ll find an excess of zealousness in trying to be concise and objective. In fact, when searching for Barack Obama, the results are limited to his place and date of birth, and stating that he is a head of state. We may be spoiled by Wikipedia, but a photo or a few more details at least were to be expected. For example, at a bare minimum, it could be indicated that we are reading the page of the 44th President of the United States of America. But Wolfram|Alpha reserves this type of treatment for all famous people, including Stephen Wolfram himself.

In the comparison between “Paul Erdos vs Euler” one would expect a nice parallel between these two great, prolific mathematicians (for example, you might expect comparisons on the number of publications, total number of pages, the most significant discoveries, and so on). Instead, the results are limited to a comparison of births and deaths. This is quite disappointing no matter how you look at it. On a side note however, it must be said that WA is quite good at interpreting misspelled names (e.g., Paul Erddsos).

The Natural Language Processing (NLP) capabilities of Wolfram|Alpha are good enough to use the service without encountering any major frustrations, but it doesn’t appear to be a particular revolution or advancement in the field of artificial intelligence either. It is also clear that we are not dealing with a Google-killer nor a Wikipedia-killer, but rather with an innovative new tool that can be used in addition to the existing ones. That said, elsewhere I mentioned that I personally think that this is a step forward for humanity. It may be a small step, but I stand behind that bold statement.

We are at the forefront of a service that will be useful for research and to anyone with a need for correct data as quickly as possible. The knowledge base will have to grow, some data will have to be updated, and the engine will need to permit more freedom in the kind of calculations that are allowed to be performed. There can be little doubt thought that we are witnesses to the birth of something ambitious that has the potential to accelerate the advancement of our civilization.

An Italian translation of this article is available on Stacktrace.it.

It was “buzz worthy” for sure, and all the attention quickly attracted the interest of numerous scientists from many disciplines. As soon as the mathematicians, and particularly computer scientists, managed to get through its 1000+ pages, the first negative reviews began to pour in. Though, in all fairness, a few scientists had a little too much fun with this book and managed to showcase their comedic abilities by writing some of the most hilarious reviews known to humankind.

In this controversial best-seller, Stephen Wolfram primarily dissects the subject of cellular automata and its relevance to other scientific disciplines, in a systematic manner. It’s a book that covers a lot of ground and is arguably a remarkable piece of writing. Yet, the scientific community greeted the book with a fair dose of criticism.

So what went wrong? The main problem with A New Kind of Science is that it set very high expectations due to its author, title, and the numerous reminders of how important this material is, throughout the book.

The main accusations ranged from the book being called a display of Wolfram’s ego, to having very little “new” content, all the way to the more severe claims of not crediting other people’s work. For example, the idea of the universe as a cellular automaton was first presented by Konrad Zuse, so Wolfram’s “new” idea of a discrete, computable universe was anything but groundbreaking. On top of that, the most remarkable technical achievement revealed in this book was arguably the proof that the rule 110 cellular automaton is Turing complete. While this was conjectured by Wolfram, it was actually proven by his assistant Matthew Cook, who was refrained from publishing his results elsewhere by Wolfram’s lawyers.

It’s important to understand that, while perhaps not accepted as the breakthrough that Wolfram had hoped for, this book – and the methods for studying computational systems illustrated within it – is far from gibberish. Wolfram’s ambitious project failed in the eyes of the community due to the extremely high expectations that were set for this book. When you claim to have something radically new, you must be able to back that claim up in a convincing enough manner or else you’re bound to end up with egg on your face.

To be fair to Wolfram (for the few who are not familiar with his work) NKS is a controversial project, but he was already famous for having created the excellent program Mathematica (whose 7th version was recently released), one of the world’s most complete and advanced mathematical software.

Now Wolfram is at it again. According to his recent announcement, he is about to unleash something called WolframAlpha to the world, which combines both his work with Mathematica and NKS. In Wolfram’s own words:

I had two crucial ingredients: Mathematica and NKS. With Mathematica, I had a symbolic language to represent anything—as well as the algorithmic power to do any kind of computation. And with NKS, I had a paradigm for understanding how all sorts of complexity could arise from simple rules.

The project has been kept on the down-low for the past few years, while some of the brightest mathematicians and engineers employed by Wolfram Research, Inc. worked on it. It’s currently in private beta, but will go live in May of this year. From an initial glance, it would seem to be just another search engine a la Google.com. But is it? Not quite. It’s labeled as a “computational knowledge engine”, whose aim is to compute answers from the human knowledge available on the web. Whereas on Google you can search for strings and the results will be a series of relevant links, WolframAlpha will supposedly be able to parse and “understand” a query that’s inputted in English, and compute an answer based on the extensive knowledge stored in its system (assuming that a univocal answer exists). Conceptually speaking, it’s leaps and bounds more complex to get right than Google, which simply looks for matching strings and orders the results based on the popularity of the given keywords (For more information about the mathematics behind Google, read this book).

According to Nova Spivack, who had a chance to try out WolframAlpha, the service is able to compute factual answers to questions such as “What is the location of Timbuktu?”, “How many protons are in a hydrogen atom?,” “What was the average rainfall in Boston last year?,” “What is the 307th digit of Pi?,” “where is the ISS?” or “When was GOOG worth more than $300?”. This project has the potential to change the world as we know it, just like Google did. Several years ago Altavista was fine for most people’s search needs – or so we thought. It took Google to show us how much better off we could be search-wise, how much we needed Google, and ultimately how inadequate Altavista was. Unlike the case of Google and Altavista though, WolframAlpha would not replace Google, since the two services cover complimentary needs. Having access to a service that’s able to compute answers out of the chaos of the factual information that’s available to man would be a major breakthrough for humanity and computer science. And if an API (Application Programming Interface) were to become available, other developers would be able to tap into that with their applications.

Bold claims, high expectations. You understand why, two months away from experiencing something so potentially revolutionary, there is a lot of hype surrounding this project – but also major skepticism. For many this is A New Kind of Science all over again, especially since natural language processing and “computing knowledge” are extremely ambitious challenges in a realm where many have failed before. Pulling this one off would be a major accomplishment (that would dwarf Wolfram’s past achievements, including Mathematica), and, at long last, it would be the hard earned, practical validation of some of the methods and philosophies expressed in NKS by Wolfram.

I fully expect people to find bugs and have many simple questions, for which we will see bizarre answers. We’ll read blog posts about the whole thing and perhaps have a good laugh. But what interests me the most is whether, as Google did in the past, this new engine will be able to be practical and useful on an everyday level. Bugs are fair play and expected, but what we’re looking for here is a spark of true innovation thanks to the mathematical modelling of human knowledge.

I suspect that this engine will either have us in awe like Mathematica did, or leave us with mixed feelings – if not downright disappointment, like A New Kind of Science did for many. I can’t help but hope for the former, as I wait for my chance to try it out.

1. General Math Cheat Sheet (iPaper and other formats)
2. Elementary Algebra Cheat Sheet (PDF)
3. Trigonometry Cheat Sheet (PDF)
4. Calculus Cheat Sheet (PDF)
5. Derivatives and Integrals Cheat Sheet (PDF)
6. Laplace Transforms Cheat Sheet (PDF)
7. Abstract Algebra Cheat Sheet (PDF)
8. Probability Theory Cheat Sheet (PDF)
9. Matlab Cheat Sheet (PDF)
10. Mathematica Cheat Sheet (PDF)
11. Maple Cheat Sheet (PDF)
12. Maxima Cheat Sheet (HTML web page)
13. LaTeX Cheat Sheet (several formats)

And since most of us like to show our math pride off when out and about as well, Amazon sells this awesome Math Cheat Sheet T-shirt with formulas on both sides (Also available for Science and Engineering). How awesome is this?

Flexible, powerful, and easy to use, gnuplot is not only handy as a stand-alone program but can be used successfully by a variety of programming languages, including but not limited to Ruby, Python and Perl. It has also been adopted as a plotting engine by Open Source programs like Maxima and GNU Octave. The minimal effort required to learn gnuplot is therefore a very worthwhile endeavor. Being able to visualize things as you analyze data and explore mathematics, is a very useful aid.

gnuplot is cross-platform, and you can download it from SourceForge.

Once you have installed gnuplot, you can start the tool by simply running gnuplot from the shell. Getting started with the program is straightforward, given that the most basic functionalities are fairly intuitive. For example, if you wanted to plot the sin(x) function, you could run:

gnuplot> plot sin(x)

gnuplot will automatically decide for you what portions of the function should be visualized (in our case between -10 and 10 on the x axis). If you want to overwrite this, you can specify otherwise, as shown below:

gnuplot> plot [-pi:pi] sin(x)

Similarly, you can customize the plotting, by specifying a range on the y axis:

gnuplot> plot [] [-0.5:0.5] sin(x)

It is also possible to plot several functions at once:

gnuplot> plot [-2:2] x, x**2, x**3

gnuplot can draw all sorts of functions and graphics. For example, the following plots two surfaces:

gnuplot> splot x**2+y**2, x**2-y**2

What you see plotted above, is based on a sensible set of defaults, but gnuplot offers countless options to customize the appearance of your graphics.

gnuplot isn’t just useful for plotting mathematical functions. I often use it for plotting data that is stored in tabular format in simple text formats. For example, assume that you have collected the following data on Deaths by Major Causes in the US, from a website:

stats.txt

That’s a big chunk of data, but it’s not very easy to analyze. It’d be nice to be able to visualize it. Assuming you saved it in a stats.txt file (you can include or remove the first few lines of comments) from gnuplot you can run the following:

gnuplot> plot "stats.txt"

That’s a meaningless mess. But gnuplot is very flexible, so we can easily do much better. Let’s use the first column for x values, and each of the remaining columns as a curve of its own by running:

gnuplot> plot "stats.txt" using 1:2 title "Heart disease" with lines, "stats.txt" using 1:3 title "Cancer" with lines, "stats.txt" using 1:4 title "Cerebro-vascular diseases" with lines, "stats.txt" using 1:5 title "Lower respiratory diseases" with lines, "stats.txt" using 1:6 title "Diabetes mellitus" with lines, "stats.txt" using 1:7 title "Influenza and pneumonia" with lines, "stats.txt" using 1:8 title "Chronic liver disease" with lines, "stats.txt" using 1:9 title "Accidents" with lines, "stats.txt" using 1:10 title "Suicide" with lines, "stats.txt" using 1:11 title "Homicide" with lines

Or it’s abbreviated version:

gnuplot> plot "stats.txt" u 1:2 title "Heart disease" w l, "" u 1:3 title "Cancer" w l, "" u 1:4 title "Cerebro-vascular diseases" w l, "" u 1:5 title "Lower respiratory diseases" w l, "" u 1:6 title "Diabetes mellitus" w l, "" u 1:7 title "Influenza and pneumonia" w l, "" u 1:8 title "Chronic liver disease" w l, "" u 1:9 title "Accidents" w l, "" u 1:10 title "Suicide" w l, "" u 1:11 title "Homicide" w l

Much better! Of course, the command above has a bit of repetition in it, but it’s justified by the flexibility of being able to pull data from different files if required, and plot some data with lines and others without (or in different styles) if we want to. To improve the appearance and legibility of the chart, let’s add a grid:

gnuplot> set grid
gnuplot> replot

Nice! And notice how the replot command runs the last plotting directive for us, saving us some typing (or scrolling through the previous commands). To improve this further, we can also set the labels for the axis and a title for the whole chart:

gnuplot> set xlabel "Years"
gnuplot> set ylabel "Death rates per 100,000 people"
gnuplot> set title "Deaths by Major Causes, 1960–2005"
gnuplot> replot

(Click here for a larger version)

Effortlessly, we went from having some data in a text file, to a meaningful, and professional looking chart that could be used in an article or a book. Not bad. You can save it in your favorite format, by setting the output file name and the terminal type. In the example below, I specify the font type, size and its location on my Mac OS X system, but it’s entirely optional. If missing, the default fonts apply.

gnuplot> set output "mychart.png"
gnuplot> set terminal png font "/Library/Fonts/Times New Roman.ttf, 11"
gnuplot> replot

This very short introduction should you give you a glimpse into what gnuplot can do for you, and even get you started with the tool. But gnuplot is much more than this and infinitely customizable, so you really may want to consider learning more about it. Unfortunately, despite being widely used for more than a decade, gnuplot has never had a book published about it.

Thankfully, Manning Publications will be putting out a book called Gnuplot in Action in October. The good news is that you don’t have to wait for the dead tree version, you can purchase it today through the Manning Early Access Program (MEAP). 12 out of the 15 chapters are already available, so the book is pretty much complete. If you’re interested in making the best out of gnuplot, this book is a must have. You don’t have to be a mathematician or a programmer to follow along, and it’s so much more than a bunch of options for customizing your graphics. The book guides you through the best techniques for taking advantage of the tool in several common scenarios. By reading, this easy to follow book, you’ll be able to generate colorful, nifty plots and have good mastery of the tool in a very short amount of time.

Maxima as a calculator

You can use Maxima as a fast and reliable calculator whose precision is arbitrary within the limits of your PC’s hardware. Maxima expects you to enter one or more commands and expressions separated by a semicolon character (;), just like you would do in many programming languages.

(%i1) 9+7;
(%o1) 
(%i2) -17*19;
(%o2) 
(%i3) 10/2;
(%o3) 

Maxima allows you to refer to the latest result through the % character, and to any previous input or output by its respective prompted %i (input) or %o (output). For example:

(%i4) % - 10;
(%o4) 
(%i5) %o1 * 3;
(%o5) 

For the sake of simplicity, from now on we will omit the numbered input and output prompts produced by Maxima’s console, and indicate the output with a => sign. When the numerator and denominator are both integers, a reduced fraction or an integer value is returned. These can be evaluated in floating point by using the float function (or bfloat for big floating point numbers):

8/2;
=> 
8/2.0;
=> 
2/6;
=> 
float(1/3);
=> 
1/3.0;
=> 
26/4;
=> 
float(26/4);
=> 

As mentioned above, big numbers are not an issue:

13^26;
=> 
13.0^26
=> 
30!;
=> 
float((7/3)^35);
=> 

Constants and common functions

Here is a list of common constants in Maxima, which you should be aware of:

• %e – Euler’s Number
• %pi –
• %phi – the golden mean ()
• %i – the imaginary unit ()
• inf – real positive infinity ()
• minf – real minus infinity ()
• infinity – complex infinity

We can use some of these along with common functions:

sin(%pi/2) + cos(%pi/3);
=> 
tan(%pi/3) * cot(%pi/3);
=> 
float(sec(%pi/3) + csc(%pi/3));
=> 
sqrt(81);
=> 
log(%e);
=> 

Defining functions and variables

Variables can be assigned through a colon ‘:’ and functions through ‘:=’. The following code shows how to use them:

a:7; b:8;
=> 
=> 
sqrt(a^2+b^2);
=> 
f(x):= x^2 -x + 1;
=> 
f(3);
=> 
f(a);
=> 
f(b);
=> 

Please note that Maxima only offers the natural logarithm function log. log10 is not available by default but you can define it yourself as shown below:

log10(x):= log(x)/log(10);
=> 
log10(10)
=> 

Symbolic Calculations

factor enables us to find the prime factorization of a number:

factor(30!);
=> 

We can also factor polynomials:

factor(x^2 + x -6);
=> 

And expand them:

expand((x+3)^4);
=> 

Simplify rational expressions:

ratsimp((x^2-1)/(x+1));
=> 

And simplify trigonometric expressions:

trigsimp(2*cos(x)^2 + sin(x)^2);
=> 

Similarly, we can expand trigonometric expressions:

trigexpand(sin(2*x)+cos(2*x));
=> 

Please note that Maxima won’t accept 2x as a product, it requires you to explicitly specify 2*x. If you wish to obtain the TeX representation of a given expression, you can use the tex function:

tex(%);
=> $$-\sin ^2x+2\,\cos x\,\sin x+\cos ^2x$$

Solving Equations and Systems

We can easily solve equations and systems of equations through the function solve:

solve(x^2-4,x);
=> 
%[2]
=> 
solve(x^3=1,x);
=> 
trigsimp(solve([cos(x)^2-x=2-sin(x)^2], [x]));
=> 
solve([x - 2*y = 14,  x + 3*y = 9],[x,y]);
=> 

2D and 3D Plotting

Maxima enables us to plot 2D and 3D graphics, and even multiple functions in the same chart. The functions plot2d and plot3d are quite straightforward as you can see below. The second (and in the case of plot3d, the third) parameter, is just the range of values for x (and y) that define what portion of the chart gets plotted.

plot2d(x^2-x+3,[x,-10,10]);

plot2d([x^2, x^3, x^4 -x +1] ,[x,-10,10]);

f(x,y):= sin(x) + cos(y);
plot3d(f(x,y), [x,-5,5], [y,-5,5]);

Limits

limit((1+1/x)^x,x,inf);
=> %
limit(sin(x)/x,x,0);
=> 
limit(2*(x^2-4)/(x-2),x,2);
=> 
limit(log(x),x,0,plus);
=> 
limit(sqrt(-x)/x,x,0,minus);
=> 

Differentiation

diff(sin(x), x);
=> 
diff(x^x, x);
=> 

We can calculate higher order derivatives by passing the order as an optional number to the diff function:

diff(tan(x), x, 4);
=> 

Integration

Maxima offers several types of integration. To symbolically solve indefinite integrals use integrate:

integrate(1/x, x);
=> 

For definite integration, just specify the limits of integrations as the two last parameters:

integrate(x+2/(x -3), x, 0,1);
=> 
integrate(%e^(-x^2),x,minf,inf);
=> 

If the function integrate is unable to calculate an integral, you can do a numerical approximation through one of the methods available (e.g. romberg):

romberg(cos(sin(x+1)), x, 0, 1);
=> 0.57591750059682

Sums and Products

sum and product are two functions for summation and product calculation. The simpsum option simplifies the sum whenever possible. Notice how the product can be use to define your own version of the factorial function as well.

sum(k, k, 1, n);
=> 
sum(k, k, 1, n), simpsum;
=> 
sum(1/k^4, k, 1, inf), simpsum;
=> 
fact(n):=product(k, k, 1, n);
=> 
fact(10);
=>  

Series Expansions

Series expansions can be calculated through the taylor method (the last parameter specifies the depth), or through the method powerseries:

niceindices(powerseries(%e^x, x, 0));
=> 
taylor(%e^x, x, 0, 5);
=> 

The trunc method along with plot2d is used when taylor’s output needs to be plotted (to deal with the in taylor’s output):

plot2d([trunc(%), %e^x], [x,-5,5]);

I hope you’ll find this useful and that it will help you get started with Maxima. CAS can be powerful tools and if you are willing to learn how to use them properly, you will soon discover that it was time well invested.

1. Maxima

A general purpose CAS (Computer Algebra System) is a program that’s able to perform symbolic manipulation for the resolution of common problems. As a matter of fact, modern CAS covers an extremely wide range of functionalities. Maple is a prime example of a commercial CAS software. In this category I’d say that Maxima is an excellent open source cross platform tool. wxMaxima is a good GUI version (possibly the best one on Windows), but other front-ends exist. The 3D plot below was generated by Maxima.

Valuable mentions are:

• Axiom
• Sage
• Yacas

2. Scilab

Matlab is the standard for numerical computing, but there are a few clones and valid alternatives that are entirely free. Scilab is the closest that you can get to Matlab without spending a penny. It’s very compatible with Matlab (albeit not 100%) but it’s really flexible, powerful and comes with a Matlab converter and Scicos which is a block diagram modeler and simulator.

Valid alternatives are:

• GNU Octave
• Euler

3. R

For statistical computing and analysis in the Open Source world, it doesn’t get any better than R. It is a programming language and environment that enables you to do pretty much anything that the commercial software (S-Plus) does. It is so widely adopted that it can be considered a standard in the field.

As usual, please feel free to share your experiences and add your suggestions to enrich the discussion.