CISC 121 Lecture Notes - Stabilisation Force In Bosnia And Herzegovina, Approximation Error, Roundoff

• School: Queen's University
• Department: Computing
• Course: CISC 121
• Professor: Robin W Dawes

Document Summary

Good news: computers are really fast at arithmetic. Good enough much of the time, but not always. Must understand shortcomings and how to cope with them. Every time the computer stores something, it boils down to lots and lots of bits (binary items, base 2) We have in any device sub-divided into billions of microscopic on/off switches, 1/0. Almost always sorted into groups of 8, called a byte. Most numeric types have set numbers of bits used. If that"s not enough, we have a problem. A sequence of bits, interpreted as a binary number (base 2) In binary: 1x2^4(16) + 0x2^3(8) + 1x2^2(4) + 0x2^1(2) + 1x2^0(1) Python ints are usually 32 bits long, one bit used for sign, so range is -2^31 to 2^31-1. Range is not symmetric because 0 is in the middle. Trying to store a value that"s too big: overflow. Python long type: integer values with no restrictions on size.