ALL ABOUT OFFICE

EXCEL

HOME

ACCESS

EXCEL

WORD

OUTLOOK

POWERPOINT

Numeric Precision  

Microsoft Excel was developed to compete in the business software market where precision is probably not important for simple calculations.  However, engineering and other scientific calculations require many digits of precision to the right of the decimal point.

Excel maintains an internal numeric precision of 15 digits.   By comparison, a typical scientific calculator displays 10 digits but probably stores 12 digits.   Excel stores 15 digits internally, but rounds the value for the screen display according to the format of the cell.

You should be aware that when Excel sums numbers that have been rounded to display a certain number of decimal places (e.g. currency values) it will total any fractional parts of those numbers that are not displayed, this may mean that the total does not add up to the same value as you would get if you added the values with a calculator.  See the page on precision of results for a solution to this.

Click here to return to the Excel index.

webmaster@allaboutoffice.co.uk