EECS 1520 Midterm: EECS 1520 Midterm #2 Notes

8 Pages
Unlock Document

Electrical Engineering and Computer Science
EECS 1520
John Hofbauer

2.4 Digital media Audio - Audio as a sound wave, microphone is a transducer, converting audio (sound waves) to electrical signals. Sampling Rate and Bit Rate - Digitizing sound involves taking samples (measurements) at fixed rate (sampling rate) - Too fast  large file. Too slow  inaccurate - Low sampling rate reproduces a sound that does not match the original frequency - Each sample I a fixed size, called bit depth - Higher bit depth means a more accurate sample. Red: original, Black: sampling, Green: difference. - Combining bit depth and sampling rate gives a bit rate - Higher bit rate = better quality = larger file size - Range of human hearing: 20 Hz – 20 kHz, Nyquist-Shannon theorem recommends sampling rate approx. 2x max frequency for accurate reproduction - CD-quality audio (industry standard): - 16 bits bit depth (per channel, stereo has 2 channels), 44.1 kHz sampling rate, 1.4 M bits/second bit rate - Uncompressed audio typically saved in a .wav format Audio Compression Techniques - Lossless audio compression, like text compression techniques - Lossy audio compression, remove imperceptible sounds (using psychoacoustics models), Reduce bit rate (less accurate reproduction of the original) - Codecs, Compression and decompression algorithms for audio (and video), Portmanteau (word combination) of “coder” and “decoder” or (as in this case) “compressor” and “decompressor” Example codecs - FLAC (Free Lossless Audio Codec) - Lossless, using a combination of run-length and Huffman encoding, Compression ratio around 62%, Typically used for archiving high quality audio - MP3 (Moving Pictures Experts Group, Audio Layer 3), Lossy, with a compression ratio around 13%, Uses psychoacoustics, Huffman encoding, and lower bit rates, Popular on most mobile devices - AAC (Advanced Audio Coding), Lossy, with a compression ratio around 14%, Uses psychoacoustics, Huffman encoding, and lower bit rates, Better perceived quality than MP3, Popular on Apple devices and most smartphones Graphics (Images) Representing Colour - Colour is typically represented using the RGB colour model, Three integers (in the range 0..255) describe the amount of Red, Green, and Blue required to reproduce the colour - 255, 255, 255 White, 255,0,0 Red, 0,0,255 Blue, 255,165,0 Orange, 0,0,0 Black, 0,255,0 Green, 255,255,0 Yellow, 128,0,128 Purple - Other colour models include: Cyan, Magenta, Yellow, and black (CMYK), Hue, Saturation, and Luminance (HSL), Luminance, Blue, Red (YUV) Representing Colour with Hexadecimal - Each R, G, and B value is in the range 0..255 - Convert the decimal value to hexadecimal, 255 values = 1 byte = 2 nibbles = 2 hexadecimal digits, 6 hexadecimal digits used to represent the entire colour: RRGGBB - Example this colour is RGB = (178,102,255)  RR = 17810= B216, GG = 10210= 6616, BB = 25510= FF16, So using hexadecimal, it is written: #B266FF (# indicates hexadecimal) Digitized Images and Graphics - Pixel (picture element), Dots of colour in an image (or on a display) - Resolution, Number of pixels in and image (or on a display), Size of an image, measured in pixels, sometimes refers to the pixel density, the number of pixels per unit of distance (e.g., pixels per inch or ppi) Vector Graphics - An image that is defined using mathematical equations representing lines, curves, and polygons - Can be enlarged without loss of detail, or change in file size - Images created using drawing applications or text editors SVG (Scalable Vector Graphics), Popular vector graphics format, Files are text-based and can be compressed accordingly Raster Graphics, an image that is comprised of a matrix of pixels, Images created/edited using painting applications, very good for photographs, but suffers from pixilation, when enlarged, the rectangular shape of pixels becomes apparent Indexed Colour, Popular technique for compressing images, instead of representing all colours, save only those used, Like compressing text with keyword encoding, but with colours instead Example Image Formats GIF (Graphics Interchange Format, pronounced “Jiff”), Lossless, indexed colours (maximum of 256), run-length encoding, Compression ratios of 10%, Allows transparency and animation PNG (Portable Network Graphics), Lossless compression using indexed colours, run-length encoding Compression ratios of 7%, Allows transparency PNG and GIF best used for line drawings, logos, or diagrams JPEG (Joint Photographic Experts Group), Ideal format for photos, Compression ratios of 1-10%, depending on quality, Lossy; compression technique: Divide the entire image into blocks of 8x8 pixels For each block: Store the average intensity, For each pixel in the block: Store the difference between each pixel’s intensity and the average intensity of the block, Differences smaller than a threshold are ignored (this is the lossy part of the compression), The threshold can be influenced by the quality setting, chosen by the user Guide to Choosing Image Format, Google has an excellent guide, Aimed at developers, “TL;DR” summaries for each section Video - Comprised of frames of still images (combined with audio) - When viewed in rapid succession, the images give the appearance of motion - Typical frame rates are 24-30 frames per second or even 60 fps - Typical frame sizes (resolutions): - 4096x2160 pixels, 4K - 1920x1080, Full HD - 1280x720, HD - 853x480, SD - Uncompressed video occupies vast storage space (more than 100 MB per second for Full HD) - Compression techniques are often used Techniques for Video Compression - Typically, little change between successive video frames, use techniques to remove that redundant information - Spatial(Intra-frame) techniques, use information from within the same frame to reduce file size - Temporal(Inter-frame) techniques, use data from nearby frames (before or after) to reduce the file size - Keyframes (I-frames): typically compressed using only spatial techniques can be reproduced independently, inserted automatically at scene changes and/or at regular intervals to preserve quality of streaming or playback - Other (non-keyframe) frames are encoded by saving the difference between it and the previous keyframe - If a keyframe is lost (e.g., during streaming) or skipped (e.g., when skipping forwards or backwards) the displayed video can be distorted Effect of Bitrate - To further reduce file size, video files (like audio files) can limit their bitrate - Constant bitrate (CBR): same bit rate throughout the video - Variable bitrate (VBR): can vary (up to a maximum), Allows higher bitrates during fast motion scenes, and lower bitrates when there is little motion - If the bitrate is too low the video becomes pixelated Example Codecs HuffYUV (Huffman, YUV colour space), Lossless, Huffman encoding, compression ratios around 47% MPEG-2, Lossy, temporal and spatial compression, Huffman encoding, Compression ratios around 3.3% Highly dependent on bitrate and other encoding options H.264 - Lossy, temporal and spatial compression, frame prediction, Predicts the next frame using spatial and temporal information, compares with the actual frame, saves only the difference - Compression ratios around 1.6%, Highly dependent on bitrate and other encoding options, Perceived higher quality than MPEG-2 at same bitrate File Containers - Combine a video stream and an audio stream into one file - Does not necessarily indicate the codecs used - Some allow for additional data: Additional video stream representing different viewing angles - Multiple language audio tracks - Optional subtitles - Containers are indicated by the file extension.avi (Audio Video Interleave): not good for streaming - .mp4(MPEG-4): supports multiple audio/video streams and subtitles, but limited codecs, is popular on mobile devices - .mkv (Matroska): like mp4, but supports unlimited streams and unlimited codecs, poor support on mobile devices 3 Computer components Displays Size, measured diagonally, reported in (approximate) inches Resolution, Dimensions of the display, reported as “(number of pixels wide) x (number of pixels tall)” Typical resolutions: 4096x2160 (“4K”), 1920x1080 (“Full HD”), 1280x720 (“HD”) Density, Sometimes, size and resolution share combined in a single “pixels per inch” (ppi) measure Display Technology - Most modern displays use a liquid crystal display (LCD) panel and light emitting diodes (LEDs) to produce the image on screen - Panel technology characteristics: Twisted Nematic (TN), Accurate reproduction of fast moving images (e.g., gaming), Colours appear distorted when viewed from off-centre angles - In Plane Switching (IPS), Better reproduction of colours, especially when viewed from angles, Used in most contemporary displays Units of Speed and Space - Processing speed, Computation occurs in “cycles”, Base unit: Hz= cycles per second - Storage space, Data is represented in “bits”, Base unit: Byte = 8 bits - Measuring in base units are impractical in contemporary computing, so they are combined with metric prefixes to represent larger magnitudes, E.g., gigahertz (GHz) or gigabytes (GB) Central Processing Unit (CPU) - “Brains” of computer, Decodes instructions and carries out the corresponding arithmetic, logic, or control operations Components of the CPU - Control unit (CU): Coordinates the flow of instructions and data between the other components of the CPU - Registers: Stores values, such as the currently executing instruction, operands and results of operations, and memory addresses to access - Arithmetic logic unit (ALU): Performs mathematical, logical, and bit manipulation operations Fetch- Execute Multiple Processing Cores - Up until about 2005, most CPUs had only one set of “core” components (i.e., CU, registers, ALU) - Now, CPUs contain multiple cores, each with their own set of CUs, registers, and ALU - Each core can run a separate thread of program execution - Manufacturers now focus on designing CPUs with multiple modestly fast cores, rather than a single fast core: Smartphones (2 –8 cores), Laptops (2 –4 cores), Desktops (2 –8 cores), Servers (4 –22 cores) Other Features: Cache: temporary storage for recently accessed data; faster to access than main memory “Turbo Boost”: increasing the processing speed of one or more cores; typically requires disabling other cores to ensure continued stable operation. “Hyper-Threading”: processing two unrelated instructions simultaneously in the same core; yields better performance than one ordinary core, but not as much as two cores (due to requirement of instructions being unrelated). Graphics Processing Unit (GPU) A processor specializing in rendering 3D graphics, I.e., performing repeated math operations on matrices and vectors. These operations do not require sophisticated computing cores, so 100s –1000s of cores can exist in a single GPU. Not just for video games, also used for, 3D effects in operating systems, Image and video editing and encoding, Computer aided design (CAD) for architecture and engineering, visualizing large data sets, Machine learning, Medical imaging, Modelling weather, protein folding, molecular dynamics, fluid dynamics, aerodynamics, etc. Main Memory - Also, known as dynamic random access memory (DRAM or just RAM) - A collection of temporary storage locations, each with a unique physical address - Used to store instructions and data for currently running programs - Contents erased once powered off (i.e., contents are “volatile”) - Cell at address 11111110 contains 10101010 The von Neumann Architecture - John von Neumann created a simplified computing model, Comprised of a CPU, memory, (secondary) storage, and input/output devices Buses - Like a data highway that connects computer components CPU-RAM bus - Bus width is typically of multiple of a word length (64-bits), plus additional bits for error detection, Bus speed 2.4 GHz –4.8 GHz - PCI-express bus connects CPU to other components (e.g., graphics, storage, network) - Bus width varies from “x1” (8-bits) to “x32” (256-bits) - Bus speed varies from 2.5 GHz –8 GHz Disk-Based Storage Hard disk drive (HDD): Mechanical in nature, Contains one or more spinning magnetic disks (platters) A read/write head, supported by an articulating arm, swings across the disk and hovers over its surface The head can read magnetically encoded information, and encode information on the disk with incredible precision, Fixed storage Blu-Ray, DVD, and CD: Optical in nature, Lasers are used to read the disks, Disks have microscopic “lands” and “pits” on their surface, Lands reflect light (binary 1), pits disperse light (binary 0), Removable storage Flash-Based Storage: Uses solid state (i.e., no moving parts) memory chips, Non-volatile, so they retain state without a power source,
More Less

Related notes for EECS 1520

Log In


Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.