Greener Software is Better Software

CPU Load and Green Software
CPU Load Correlates with Power Consumption

Faster Software is Greener Software

Simply put, when one software product is more efficient than another, it runs faster and takes less time to solve the same problem.  The less time software takes to run, the less power is consumed.

By way of illustration, consider the efficiency of a steam ship going from New York to San Fransisco before and after the Panama Canal was built.  The canal was a technological marvel of its time, and it cut the journey distance from 13,000 miles to 5,000.  It cut travel time by (more than) half, and reduced the journey’s coal consumption by 50%.  The same work was performed, with the same “hardware” (the steamer), but in just 30 days rather than 60, and using half the fuel.

Faster run time is the most significant and most visible component of green software, but it is not the only significant factor.  Other factors affecting how much power software consumes include:

  • Cache miss rate
  • Streamlined versus bloated, crufty software
  • Use of best-suited hardware resources
  • Algorithm scalability

Without getting too technical, I’ll briefly touch on each bullet point.  A cache hit is when a CPU finds the information it needs in its internal cache memory, while a cache miss is when the CPU must send an off-chip request to the computer’s RAM to get the required data.  A cache miss is about 100x slower than a cache hit, in part because the data has to travel about 10cm for a cache miss, versus about 5mm for a cache hit.  The difference in power consumption between a cache hit and a cache miss easily be 20x to 100X, or more.

Most software starts out reasonably streamlined.  Later, if the software is popular, comes a time when enhancement requests and bug reports come in faster than developers can implement them in a streamlined manner.  Consequently many developers implement quick but inefficient fixes.  Often this behavior is encouraged by managers trying to hit aggressive schedule commitments.  The developers have intentions to come back and improve the code, but frequently their workload doesn’t permit that.  After a while developers forget where the software “kludges” or hack are.  Even worse, the initial developers either get reassigned to other projects or leave for other jobs.  The new software developers are challenged learn the unfamiliar code and implement fixes and enhancements — adding their own cruft.   This is how crufty, bloated software emerges:  overworked developers, focused on schedule over software efficiency, and developer turnover.

Modern CPUs have specialized instructions and hardware for different compute operations.  One example is Intel SSE technology which features a variety of Single-Instruction, Multiple-Data (SIMD) extensions.  For example, SSE4 (and AVX) can add 4 or more pairs of numbers (2 4-number vectors) in one operation, rather than 4 separate ADD operations.  This reduces CPU instruction traffic and saves power and time.

Finally algorithm scalability is increasingly important to modern computing and compute efficiency.  Scalability has many meanings, but I will focus on the ability of software to use multiple compute resources in parallel.  [Also known as parallel computing.]   Unfortunately most software in use today has limited or no compute-resource scalability.  This means that this software can only use 1 core of a modern 4-core CPU.  In contrast, linearly-scalable software could run 3x faster by using 3 of the 4 cores at full speed.  Even better, it could run 3x faster on 4 cores running at 75% speed, and consume about 30% less power.  [I’ll spare you the math, but if you are curious this link will get you started.]

“Distributed Software” is Greener

Distributed computing is technology that allows compute jobs to be distributed into the “cloud” or data center queue.  Rather than having desktop workstations sitting idle much of the day, a data center is a room full of computers that direct compute jobs to the least busy computers.  Jobs can be directed to the computers best-suited to a particular compute request. Intelligent data centers can even put unused computers into “deep sleep” mode that uses very little power.

I use the term distributed software to mean software that is easily integrated with a job-submission or queuing software infrastructure.  [Short for distributed-computing-capable software.]  Clearly distributed software benefits directly from the efficiencies of a given data center.  Distributed software can also benefit from the ability to run in parallel on multiple machines.  The more tightly-coupled with the capabilities and status of the data center, the more efficiently distributed software can adapt to dynamic changes.

Sigma1 Software is Green

Sigma1 financial software (code-named HAL0) has been designed from the ground up to be lean and green.  First and foremost, HAL0 (code-named in honor of Arthur C. Clarke’s HAL 9000 — “H-A-L is derived from Heuristic ALgorithmic (computer)”) is architected to scale near-linearly to tens or hundreds of cores, “sockets”, or distributed machines.  Second, the central kernel or engine is designed to be as light-weight and streamlined as possible — helping to reduce expensive cache misses.  Third, HAL0 uses Heuristic Algorithms and other “AI” features to efficiently navigate astronomically-large search spaces (10^18 and higher).  Fourth, HAL0 uses an innovative computation cache system that allows repeated complex computations to be looked up in the cache, rather than recomputed.  In alpha testing, this feature alone accounted for a 3X run-time improvement.  Finally, HAL0 portfolio software incorporates a number of more modest run-time and power-saving features such as coding vector operations explicitly as vector operations, thus allowing easier use of SIMD and possibly GPGPU instructions and hardware.

Some financial planners still use Microsoft Excel to construct and optimize portfolios.  This is slow and inefficient — to say the least.  Other portfolio software I have read about is an improvement over Excel, but doesn’t mention scalability nor heuristic algorithms.  It is possible, perhaps likely, that other financial software with some the capabilities of HAL0 exists.  I suspect, however, that if it does, it is proprietary, in-house software that is not for sale.

A Plea for Better, Greener Software

In closing, I’d like the software community to consider how the efficiency (or inefficiency) of their current software products contribute to world-wide power consumption.  Computer hardware has made tremendous strides to improving performance/power in the last ten years, and continues to do so.   IT and data-center technology is also becoming more power efficient.  Unfortunately, most software has been trending in the opposite direction — becoming more bloated and less efficient.  I urge software developers and software managers to consider the impact of the software they are developing.  I challenge you to consider, probably for the first time, how many kilowatt- or megawatt-hours your current software is likely to consume.  Then ask yourself, “How can I reduce that power?”

Financial Software: Heuristics Explained

Software visualization
Abstract Visual of Software

A Baseball Analogy

Imagine you’re the general manager of a Major League ball club.  Your primary job is to construct (and maintain) a team of players  that will win lots of games, while keeping the total player payroll as low as possible.  When considering a hypothetical roster a baseball GM has two primary objectives in mind:

  1. Total annual payroll (plus any associated “luxury tax”)
  2. Expected season wins (and post-season wins)

These objectives can also be called heuristics — rules of thumb to help find solutions to complex problems.   These heuristics can be turned into numbers (quantified) by creating cost functions or utility functions.  Please don’t let all of this jargon disembolden you; we are merely talking a little baseball here.

The cost function function for payroll is just that… the total annual salaries for a proposed roster.  It is called a cost function because cost is something we are trying to minimize.  Expected wins is called a utility function, because utility is good, and we want to maximize it.

Now, accurately predicting number of wins for a hypothetical (or real) roster of players is a real challenge.  Every scout and adviser is going to have his or her own ideas or heuristics.  Just watch Moneyball to see what I mean.  To turn any given roster into a utility score a GM could write a proposed roster on a whiteboard and point-blank ask each advisory “How many wins will this team produce?”  The GM could average these predictions and, boom!, that’s an utility function.  The GM could also hire a computer scientist and statistician to code up a utility function for any proposed roster relying on a chosen set of stats.

Either way, now the GM has can evaluate any proposed roster based on two metrics: cost and wins.  These data can be plotted, and quickly patterns will emerge.  Some proposed rosters will be both more expensive and less “winning” than others.  These rosters are said to be dominated, and they can be removed from consideration.  Once all the dominated rosters are eliminated, what remains is a series of dots that form a curve.  As one moves up that curve, one finds more winning, but more expensive rosters.  Moving the other way, the payroll cost is less, but the expected wins decrease.  This curve resembles what financial folks call an efficient frontier — the expected risk/reward tradeoff for an optimized portfolio selected from a basket of securities.

Back to Portfolio Optimization Software

The baseball analogy above tries to explain mathematical concepts without resorting to math.   OK, I did use a few math words, but no equations!

There are several differences between a baseball roster and an investment portfolio.  Key differences from an investment portfolio are:  1) You can own multiple shares of a stock or ETF (but have only 1 of any player),  2) You can trade stocks/ETFs virtually whenever you want.

Nonetheless, the baseball analogy is useful in illustrating what Sigma1 Software will be able to do for fund managers and investors.  Instead of building a baseball roster, you are building an investment portfolio.  In the classic “CAPM” investing model, the cost function is standard deviation (risk), and the utility function is expected returnHistorical standard deviation is easy to compute, but expected return is much harder to accurately compute.

Now, if you are an active fund manager, you probably have in-house analysts paid to help you pick stocks (just like GM’s have scouts).  But scouting reports from analysts do not a portfolio make… even if your analysts are giving you positive-alpha stock picks. A robust asset allocation strategy is necessary to build a robust portfolio out of your chosen list of securities.

The Vision for Sigma1 Portfolio Software

A Vision for Financial Professionals

It started with the desire to create software that would allow me to build a better portfolio for my proprietary trading fund — Software that could optimize portfolios using heuristics, cost functions, and utility functions of my own choosing.   I wanted to create portfolio software for investment managers that:

  • Allows them to select their own list of securities (or chosen dynamically from all investable securities)
  • Takes advantage of one or more “seed portfolios” if desired
  • Allows proprietary heuristics, cost functions, market models, etc. to plug seamlessly into the optimization engine
  • Isn’t limited to linear or Gaussian risk-analysis measures
  • Runs in minutes or hours, not days
  • Is capable of efficiently utilizing distributed and parallel computing resources — Scalability

A Vision for “Retail” Investors

For retail investors, the general investing public, I envision scaled-down versions of the professional portfolio optimization software.  The retail investor software will run as an application on a web server.  A free version will provide portfolio optimization for a small basket of user-chosen securities, perhaps limiting portfolio size to 10.   A paid-subscription plan will offer more features and allow retail users to build larger portfolios.

To keep the software easy to use, a variety of ready-to-use heuristics will be available.  These are likely to include:

  • Standard deviation
  • Historic best-year and worst-year analysis
  • Beta (versus common indices)
  • Diversification measures (e.g. sector, market-cap)
  • Price-to-earnings
  • Proprietary expected-return predictors