PARALLEL PROCESSING

An Introduction

COURSE
ILP Pipelined VLIW DSP Architectures Superscalar DLP SIMD Associative & Neural Architectures Systolic Architectures Vector Architectures
ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA 2

COURSE
MIMD Multi-threaded Distributed memory MIMD Shared memory MIMD

Case Studies Simulation based Performance Evaluation Studies MIPSit, Simple Scalar

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

EVALUATION
EC No. 1 2 3 Evaluation Component Test I Test II Assignment & Case Studies, Class Room Interactions Comprehensive Duration Weightage (min) (%) 60 60 ----20 20 20 Nature of Component Closed Book Closed Book

180

40

Closed/Open Book

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

SPEED UP
Deeply pipelined machines

Many instructions/cycle
Out-of-order execution of instructions Aggressive branch prediction techniques 2010 1 billion transistors clock frequencies >10GHz

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

ARCH, IMPLEMENTATION & REALIZATION

Architecture ISA Functional level behavior of processor Implementation Micro-architecture Logic structure that implements the arch Realization Physical Implementation

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

ISA
Contract between software and hardware

Multiple machines can implement ISA

Advantage program portability Microprocessor design starts with ISA ISA produces micro architecture Micro architecture has to be rigorously verified

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

ISA
Development is very slow

ISAs varied No. of operands Implied operands Operands may be stored in stack

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

CONTRACT BETWEEN H/W & S/W SPECIFICATIONS OF MICROPROCESSOR DESIGN

ISA functions
ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA 9

DYNAMIC STATIC INTERFACE

Separates what is done

Compile Time Statically

At run time Dynamically

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

10

DSI
Program (Software) Compiler Complexity Exposed to software

Static

Architecture
Hardware Complexity Exposed to hardware Dynamic

Machine (Hardware)

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

11

DSI
DEL CISC VLIW RISC HLL

DSI1 DSI2 DSI3

Hardware

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

12

WHAT IS PARALLEL COMPUTING ? SERIAL COMPUTING

Traditionally, software has been written for serial computation: To be run on a single computer having a single Central Processing Unit (CPU) A problem is broken into a discrete series of instructions Instructions are executed one after another Only one instruction may execute at any moment in time

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

13

WHAT IS PARALLEL COMPUTING ? SERIAL COMPUTING

Problem

CPU
TN T3 T2 T1

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

14

WHAT IS PARALLEL COMPUTING

In the simplest sense - parallel computing is the simultaneous use of multiple compute resources to solve a computational problem: To be run using multiple CPUs A problem is broken into discrete parts that can be solved concurrently Each part is further broken down to a series of insts Insts from each part execute simultaneously on different CPUs

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

15

WHAT IS PARALLEL COMPUTING

Problem 1

CPU 1 CPU 2 CPU 3 CPU 4

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA 16

Problem 2

Problem 3

Problem 4

PARALLEL COMPUTING
The compute resources might be: A single computer with multiple processors An arbitrary number of computers connected by a network A combination of both

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

17

PARALLEL COMPUTING
The computational problem should be able to: Be broken apart into discrete pieces of work that can be solved simultaneously Execute multiple program instructions at any moment in time Be solved in less time with multiple compute resources than with a single compute resource.

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

18

AMDAHLS LAW
Ttotal = Timproved 1 [ Ttotal - Tcomponent ]+ Tcomponent n

ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA

19

APPLNS THAT USE PARALLEL PROCESSING

Databases, data mining Oil exploration Web search engines, web based business services Medical imaging and diagnosis Pharmaceutical design Management of national and multi-national corporations Financial and economic modeling Advanced graphics and virtual reality, particularly in the entertainment industry Networked video and multi-media technologies Collaborative work environments
ADAVNCED VLSI AECHITECTURES K.R.ANUPAMA 20