Parallel computing is a form of computation in which
many calculations are carried out simultaneously, operating on the principle
that large problems can often be divided into smaller ones, which are then
solved concurrently ("in parallel"). There are several different
forms of parallel computing: bit-level, instruction level, data, and task
parallelism. Parallelism has been employed for many years, mainly in high-performance
computing, but interest in it has grown lately due to the physical constraints
preventing frequency scaling. As power consumption (and consequently heat
generation) by computers has become a concern in recent years, parallel computing has become the dominant paradigm in computer
architecture, mainly in the form of multi-core processors.
Parallelism concept : Multiple copies of hardware unit used, All
copies can operate simultaneously, Occurs at many levels of architecture, Term parallel
computer applied when parallelism dominates entire architecture. Characterizations
Of Parallelism : Microscopic
vs. macroscopic, Symmetric vs. asymmetric, Fine-grain vs. coarse-grain, Explicit
vs. implicit
Distributed processing refers to any of a variety of computer
systems that use more than one computer, or processor, to run an application. Distributed
processing is the type of processing whereby processing occurs on more than one
processor in order for a transaction to be completed. In other words,
processing is distributed across two or more machines and the processes are
most likely not running at the same time.
Architectural parallel computer : Design in which computer
has reasonably large number of
Processors. Intended
for scaling. Example: computer with thirty-two processors
Not generally
classified as parallel computer
– Dual processor
computer
–
Quad processor computer
Programming with
threads introduces new difficulties even for experienced programmers.
Concurrent programming has techniques and pitfalls that do not occur in sequential
programming. Many of the techniques are obvious, but some are obvious only with
hindsight. Some of the pitfalls are comfortable (for example, deadlock is a
pleasant sort of bug—your program stops with all the evidence intact), but some
take the form of insidious performance penalties.
A “thread” is a
straightforward concept: a single sequential flow of control. In a highlevel language
you normally program a thread using procedures, where the procedure calls follow
the traditional stack discipline. Within a single thread, there is at any
instant a single point of execution. The programmer need learn nothing new to
use a single thread.
Having “multiple
threads” in a program means that at any instant the program has multiple points
of execution, one in each of its threads. The programmer can mostly view the
threads as executing simultaneously, as if the computer were endowed with as
many processors as there are threads. The programmer is required to decide when
and where to create multiple threads, or to accept such decisions made for him
by implementers of existing library packages or runtime systems. Additionally,
the programmer must occasionally be aware that the computer might not in fact
execute all his threads simultaneously.
Having the threads
execute within a “single address space” means that the computer’s addressing hardware is configured so as
to permit the threads to read and write the same memory locations. In a
high-level language, this usually corresponds to the fact that the off-stack
(global) variables are shared among all the threads of the program. Each thread
executes on a separate call stack with its own separate local variables. The
programmer is responsible for using the synchronization mechanisms of the
thread facility to ensure that the shared memory is accessed in a manner that
will give the correct answer. Thread facilities are always advertised as being
“lightweight”. This means that thread creation, existence, destruction and
synchronization primitives are cheap enough that the programmer will use them
for all his concurrency needs.
Pemrograman
CUDA GPU. What is CUDA?
CUDA
Architecture
Expose
general-purpose GPU computing as first-class capability
Retain
traditional DirectX/OpenGL graphics performance
CUDA
C
Based
on industry-standard C
A
handful of language extensions to allow heterogeneous programs
Straightforward
APIs to manage devices, memory, etc.
Related Link :
https://www.tacc.utexas.edu/c/document_library/get_file?uuid=e05d457a-0fbf-424b-87ce-c96 http://web.eecs.umich.edu/~qstout/parallel.htmlfc0077099
https://computing.llnl.gov/tutorials/parallel_comp/
http://en.wikipedia.org/wiki/Parallel_computing
http://www.csse.monash.edu.au/~rdp/research/Papers/Parallelism_in_a_computer_architecture_to_support_orientation_changes_in_virtual_reality.pdf
http://www.nvidia.com/content/GTC-2010/pdfs/2131_GTC2010.pdf
