What are the fundamental differences among Parallel, Distributed and Concurrent Computing ?
Answers
Concurrent: Happening over the same time interval. In the “olden days” when Unix was young (and so was I…) there was one CPU and all processes that were running at any given time were given “slices” of processor time. They were concurrent but not parallel. There is no communication between nor coordination of these procresses. Parallel: Executing at the same time. Parallel computation by definition requires multiple CPUs (please disregard the “core” vs. CPU nomenclature). In this situation more than one concurrent process can be simultaneously executing. As with concurrent processing, there is no communication between nor coordination of these procresses. Distributed: Multiple programs execute concurrently and communicate with each other to collectively perform a computation. The essence of distributed computing is that an overall computation is “distributed” over multiple processes (usually—amost always but not necessarily—using multiple processors) that perform the overall computational task using communication between the processes. Today (2015) there are typically many (8, 16, 24, 32) processors in a single computer. Often multiple computers (each with multiple CPUs) participate in a distributed algorithm. The participants in a distributed system may each be executing the same code or (typically) distinct code.