Download Full Version ebook - Visit ebooknice.

com

(Ebook) MATLAB Parallel Computing Toolbox™ User's

Guide by The MathWorks, Inc.

https://ebooknice.com/product/matlab-parallel-computing-
toolboxtm-user-s-guide-11236254

Click the button below to download

DOWLOAD EBOOK

Discover More Ebook - Explore Now at ebooknice.com

 Instant digital products (PDF, ePub, MOBI) ready for you
Download now and discover formats that fit your needs...

Start reading on any device today!

(Ebook) MATLAB Econometrics Toolbox™ User's Guide by The

MathWorks, Inc.

https://ebooknice.com/product/matlab-econometrics-toolboxtm-user-s-
guide-11236174

ebooknice.com

(Ebook) MATLAB Bioinformatics Toolbox™ User's Guide by The

MathWorks, Inc.

https://ebooknice.com/product/matlab-bioinformatics-toolboxtm-user-s-
guide-11236232

ebooknice.com

(Ebook) MATLAB Mapping Toolbox™ User's Guide by The

MathWorks, Inc.

https://ebooknice.com/product/matlab-mapping-toolboxtm-user-s-
guide-11236250

ebooknice.com

(Ebook) MATLAB Optimization Toolbox™ User's Guide by The

MathWorks, Inc.

https://ebooknice.com/product/matlab-optimization-toolboxtm-user-s-
guide-11236252

ebooknice.com
(Ebook) Deep Learning Toolbox Getting Started Guide -
MATLAB by The MathWorks, Inc.

https://ebooknice.com/product/deep-learning-toolbox-getting-started-
guide-matlab-43710832

ebooknice.com

(Ebook) MATLAB Computer Vision Toolbox™ User's Guide by

The MathWorks, Inc.

https://ebooknice.com/product/matlab-computer-vision-toolboxtm-user-s-
guide-11236240

ebooknice.com

(Ebook) MATLAB Curve Fitting Toolbox™ User's Guide by The

MathWorks, Inc.

https://ebooknice.com/product/matlab-curve-fitting-toolboxtm-user-s-
guide-11236242

ebooknice.com

(Ebook) MATLAB Fuzzy Logic Toolbox™ User's Guide by The

MathWorks, Inc.

https://ebooknice.com/product/matlab-fuzzy-logic-toolboxtm-user-s-
guide-11236244

ebooknice.com

(Ebook) MATLAB Global Optimization Toolbox™ User's Guide

by The MathWorks, Inc.

https://ebooknice.com/product/matlab-global-optimization-toolboxtm-
user-s-guide-11236246

ebooknice.com
Parallel Computing Toolbox™
User's Guide

R2020a
How to Contact MathWorks

Latest news: www.mathworks.com

Sales and services: www.mathworks.com/sales_and_services

User community: www.mathworks.com/matlabcentral

Technical support: www.mathworks.com/support/contact_us

Phone: 508-647-7000

The MathWorks, Inc.

1 Apple Hill Drive
Natick, MA 01760-2098
Parallel Computing Toolbox™ User's Guide
© COPYRIGHT 2004–2020 by The MathWorks, Inc.
The software described in this document is furnished under a license agreement. The software may be used or copied
only under the terms of the license agreement. No part of this manual may be photocopied or reproduced in any form
without prior written consent from The MathWorks, Inc.
FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through
the federal government of the United States. By accepting delivery of the Program or Documentation, the government
hereby agrees that this software or documentation qualifies as commercial computer software or commercial computer
software documentation as such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014.
Accordingly, the terms and conditions of this Agreement and only those rights specified in this Agreement, shall pertain
to and govern the use, modification, reproduction, release, performance, display, and disclosure of the Program and
Documentation by the federal government (or other entity acquiring for or through the federal government) and shall
supersede any conflicting contractual terms or conditions. If this License fails to meet the government's needs or is
inconsistent in any respect with federal procurement law, the government agrees to return the Program and
Documentation, unused, to The MathWorks, Inc.
Trademarks
MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See
www.mathworks.com/trademarks for a list of additional trademarks. Other product or brand names may be
trademarks or registered trademarks of their respective holders.
Patents
MathWorks products are protected by one or more U.S. patents. Please see www.mathworks.com/patents for
more information.
Revision History
November 2004 Online only New for Version 1.0 (Release 14SP1+)
March 2005 Online only Revised for Version 1.0.1 (Release 14SP2)
September 2005 Online only Revised for Version 1.0.2 (Release 14SP3)
November 2005 Online only Revised for Version 2.0 (Release 14SP3+)
March 2006 Online only Revised for Version 2.0.1 (Release 2006a)
September 2006 Online only Revised for Version 3.0 (Release 2006b)
March 2007 Online only Revised for Version 3.1 (Release 2007a)
September 2007 Online only Revised for Version 3.2 (Release 2007b)
March 2008 Online only Revised for Version 3.3 (Release 2008a)
October 2008 Online only Revised for Version 4.0 (Release 2008b)
March 2009 Online only Revised for Version 4.1 (Release 2009a)
September 2009 Online only Revised for Version 4.2 (Release 2009b)
March 2010 Online only Revised for Version 4.3 (Release 2010a)
September 2010 Online only Revised for Version 5.0 (Release 2010b)
April 2011 Online only Revised for Version 5.1 (Release 2011a)
September 2011 Online only Revised for Version 5.2 (Release 2011b)
March 2012 Online only Revised for Version 6.0 (Release 2012a)
September 2012 Online only Revised for Version 6.1 (Release 2012b)
March 2013 Online only Revised for Version 6.2 (Release 2013a)
September 2013 Online only Revised for Version 6.3 (Release 2013b)
March 2014 Online only Revised for Version 6.4 (Release 2014a)
October 2014 Online only Revised for Version 6.5 (Release 2014b)
March 2015 Online only Revised for Version 6.6 (Release 2015a)
September 2015 Online only Revised for Version 6.7 (Release 2015b)
March 2016 Online only Revised for Version 6.8 (Release 2016a)
September 2016 Online only Revised for Version 6.9 (Release 2016b)
March 2017 Online only Revised for Version 6.10 (Release 2017a)
September 2017 Online only Revised for Version 6.11 (Release 2017b)
March 2018 Online only Revised for Version 6.12 (Release 2018a)
September 2018 Online only Revised for Version 6.13 (Release 2018b)
March 2019 Online only Revised for Version 7.0 (Release 2019a)
September 2019 Online only Revised for Version 7.1 (Release 2019b)
March 2020 Online only Revised for Version 7.2 (Release 2020a)
 Contents

Getting Started
1
Parallel Computing Toolbox Product Description . . . . . . . . . . . . . . . . . . . . 1-2

Parallel Computing Support in MathWorks Products . . . . . . . . . . . . . . . . 1-3

Create and Use Distributed Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4

Creating Distributed Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4
Creating Codistributed Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5

Determine Product Installation and Versions . . . . . . . . . . . . . . . . . . . . . . . 1-6

Interactively Run a Loop in Parallel Using parfor . . . . . . . . . . . . . . . . . . . 1-7

Run Batch Parallel Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9

Run a Batch Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9
Run a Batch Job with a Parallel Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9
Run Script as Batch Job from the Current Folder Browser . . . . . . . . . . . . 1-11

Distribute Arrays and Run SPMD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12

Distributed Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12
Single Program Multiple Data (spmd) . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12
Composites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12

What Is Parallel Computing? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-14

Choose a Parallel Computing Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-16

Run MATLAB Functions with Automatic Parallel Support . . . . . . . . . . . . 1-20

Find Automatic Parallel Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-20

Run Non-Blocking Code in Parallel Using parfeval . . . . . . . . . . . . . . . . . 1-22

Evaluate Functions in the Background Using parfeval . . . . . . . . . . . . . . 1-23

Use Parallel Computing Toolbox with Cloud Center clusters in MATLAB

Online . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-24

v
 Parallel for-Loops (parfor)
2
Decide When to Use parfor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2
parfor-Loops in MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2
Deciding When to Use parfor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2
Example of parfor With Low Parallel Overhead . . . . . . . . . . . . . . . . . . . . . 2-3
Example of parfor With High Parallel Overhead . . . . . . . . . . . . . . . . . . . . 2-4

Convert for-Loops Into parfor-Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7

Ensure That parfor-Loop Iterations are Independent . . . . . . . . . . . . . . . 2-10

Nested parfor and for-Loops and Other parfor Requirements . . . . . . . . 2-13

Nested parfor-Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-13
Convert Nested for-Loops to parfor-Loops . . . . . . . . . . . . . . . . . . . . . . . 2-14
Nested for-Loops: Requirements and Limitations . . . . . . . . . . . . . . . . . . 2-16
parfor-Loop Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-17

Scale Up parfor-Loops to Cluster and Cloud . . . . . . . . . . . . . . . . . . . . . . . 2-21

Use parfor-Loops for Reduction Assignments . . . . . . . . . . . . . . . . . . . . . . 2-26

Use Objects and Handles in parfor-Loops . . . . . . . . . . . . . . . . . . . . . . . . . 2-27

Using Objects in parfor-Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-27
Handle Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-27
Sliced Variables Referencing Function Handles . . . . . . . . . . . . . . . . . . . 2-27

Troubleshoot Variables in parfor-Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-29

Ensure That parfor-Loop Variables Are Consecutive Increasing Integers
..................................................... 2-29
Avoid Overflows in parfor-Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-29
Solve Variable Classification Issues in parfor-Loops . . . . . . . . . . . . . . . . 2-30
Structure Arrays in parfor-Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-31
Converting the Body of a parfor-Loop into a Function . . . . . . . . . . . . . . . 2-32
Unambiguous Variable Names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-33
Transparent parfor-loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-33
Global and Persistent Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-33

Loop Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-35

Sliced Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-37

Characteristics of a Sliced Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-37
Sliced Input and Output Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-38
Nested for-Loops with Sliced Variables . . . . . . . . . . . . . . . . . . . . . . . . . . 2-39

Broadcast Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-41

Reduction Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-42

Notes About Required and Recommended Guidelines . . . . . . . . . . . . . . . 2-43
Basic Rules for Reduction Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-43
Requirements for Reduction Assignments . . . . . . . . . . . . . . . . . . . . . . . . 2-44
Using a Custom Reduction Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-45
Chaining Reduction Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-46

vi Contents
 Temporary Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-48
Uninitialized Temporaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-48
Temporary Variables Intended as Reduction Variables . . . . . . . . . . . . . . . 2-49
ans Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-49

Ensure Transparency in parfor-Loops or spmd Statements . . . . . . . . . . . 2-50

Parallel Simulink Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-51

Improve parfor Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-52

Where to Create Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-52
Profiling parfor-loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-53
Slicing Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-54
Optimizing on Local vs. Cluster Workers . . . . . . . . . . . . . . . . . . . . . . . . . 2-55

Run Code on Parallel Pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-56

What Is a Parallel Pool? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-56
Automatically Start and Stop a Parallel Pool . . . . . . . . . . . . . . . . . . . . . . 2-56
Alternative Ways to Start and Stop Pools . . . . . . . . . . . . . . . . . . . . . . . . . 2-57
Pool Size and Cluster Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-59

Choose Between Thread-Based and Process-Based Environments . . . . . 2-61

Select Parallel Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-61
Compare Process Workers and Thread Workers . . . . . . . . . . . . . . . . . . . 2-64
Solve Optimization Problem in Parallel on Thread-Based Pool . . . . . . . . . 2-65
What Are Thread-Based Environments? . . . . . . . . . . . . . . . . . . . . . . . . . 2-67
What are Process-Based Environments? . . . . . . . . . . . . . . . . . . . . . . . . . 2-67
Check Support for Thread-Based Environment . . . . . . . . . . . . . . . . . . . . 2-68

Repeat Random Numbers in parfor-Loops . . . . . . . . . . . . . . . . . . . . . . . . 2-70

Recommended System Limits for Macintosh and Linux . . . . . . . . . . . . . 2-71

Single Program Multiple Data (spmd)

3
Run Single Programs on Multiple Data Sets . . . . . . . . . . . . . . . . . . . . . . . . 3-2
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2
When to Use spmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2
Define an spmd Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2
Display Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4
MATLAB Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4
Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4
spmd Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4

Access Worker Variables with Composites . . . . . . . . . . . . . . . . . . . . . . . . . 3-7

Introduction to Composites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7
Create Composites in spmd Statements . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7
Variable Persistence and Sequences of spmd . . . . . . . . . . . . . . . . . . . . . . 3-8
Create Composites Outside spmd Statements . . . . . . . . . . . . . . . . . . . . . . 3-9

Distributing Arrays to Parallel Workers . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10

Using Distributed Arrays to Partition Data Across Workers . . . . . . . . . . . 3-10

vii
 Load Distributed Arrays in Parallel Using datastore . . . . . . . . . . . . . . . . 3-10
Alternative Methods for Creating Distributed and Codistributed Arrays . 3-12

Math with Codistributed Arrays

4
Nondistributed Versus Distributed Arrays . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Nondistributed Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Codistributed Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3

Working with Codistributed Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-4

How MATLAB Software Distributes Arrays . . . . . . . . . . . . . . . . . . . . . . . . 4-4
Creating a Codistributed Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5
Local Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-8
Obtaining information About the Array . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-9
Changing the Dimension of Distribution . . . . . . . . . . . . . . . . . . . . . . . . . 4-10
Restoring the Full Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-10
Indexing into a Codistributed Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-11
2-Dimensional Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-12

Looping Over a Distributed Range (for-drange) . . . . . . . . . . . . . . . . . . . . 4-16

Parallelizing a for-Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-16
Codistributed Arrays in a for-drange Loop . . . . . . . . . . . . . . . . . . . . . . . 4-17

Run MATLAB Functions with Distributed Arrays . . . . . . . . . . . . . . . . . . . 4-19

Check Distributed Array Support in Functions . . . . . . . . . . . . . . . . . . . . 4-19
Support for Sparse Distributed Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . 4-19

Programming Overview
5
How Parallel Computing Products Run a Job . . . . . . . . . . . . . . . . . . . . . . . 5-2
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Toolbox and Server Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3
Life Cycle of a Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-6

Program a Job on a Local Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-8

Specify Your Parallel Preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-9

Discover Clusters and Use Cluster Profiles . . . . . . . . . . . . . . . . . . . . . . . . 5-11

Create and Manage Cluster Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-11
Discover Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-12
Create Cloud Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-14
Add and Modify Cluster Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-14
Import and Export Cluster Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-18
Edit Number of Workers and Cluster Settings . . . . . . . . . . . . . . . . . . . . . 5-19
Use Your Cluster from MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-19

viii Contents
Apply Callbacks to MATLAB Job Scheduler Jobs and Tasks . . . . . . . . . . . 5-21

Job Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-24

Typical Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-24
Manage Jobs Using the Job Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-24
Identify Task Errors Using the Job Monitor . . . . . . . . . . . . . . . . . . . . . . . 5-25

Programming Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-26

Program Development Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-26
Current Working Directory of a MATLAB Worker . . . . . . . . . . . . . . . . . . 5-27
Writing to Files from Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-27
Saving or Sending Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-27
Using clear functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-28
Running Tasks That Call Simulink Software . . . . . . . . . . . . . . . . . . . . . . 5-28
Using the pause Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-28
Transmitting Large Amounts of Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-28
Interrupting a Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-28
Speeding Up a Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-28

Control Random Number Streams on Workers . . . . . . . . . . . . . . . . . . . . . 5-29

Client and Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-29
Different Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-30
Normally Distributed Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 5-31

Profiling Parallel Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-32

Profile Parallel Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-32
Analyze Parallel Profile Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-34

Troubleshooting and Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-42

Attached Files Size Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-42
File Access and Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-42
No Results or Failed Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-43
Connection Problems Between the Client and MATLAB Job Scheduler . . 5-44
SFTP Error: Received Message Too Long . . . . . . . . . . . . . . . . . . . . . . . . 5-44

Big Data Workflow Using Tall Arrays and Datastores . . . . . . . . . . . . . . . . 5-46

Running Tall Arrays in Parallel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-47
Use mapreducer to Control Where Your Code Runs . . . . . . . . . . . . . . . . . 5-47

Use Tall Arrays on a Parallel Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-48

Use Tall Arrays on a Spark Enabled Hadoop Cluster . . . . . . . . . . . . . . . . 5-51

Creating and Using Tall Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-51

Run mapreduce on a Parallel Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-54

Start Parallel Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-54
Compare Parallel mapreduce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-54

Run mapreduce on a Hadoop Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-57

Cluster Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-57
Output Format and Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-57
Calculate Mean Delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-57

Partition a Datastore in Parallel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-60

ix
 Program Independent Jobs
6
Program Independent Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2

Program Independent Jobs on a Local Cluster . . . . . . . . . . . . . . . . . . . . . . 6-3

Create and Run Jobs with a Local Cluster . . . . . . . . . . . . . . . . . . . . . . . . . 6-3
Local Cluster Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5

Program Independent Jobs for a Supported Scheduler . . . . . . . . . . . . . . . 6-7

Create and Run Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-7
Manage Objects in the Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-11

Share Code with the Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-13

Workers Access Files Directly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-13
Pass Data to and from Worker Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . 6-14
Pass MATLAB Code for Startup and Finish . . . . . . . . . . . . . . . . . . . . . . . 6-15

Plugin Scripts for Generic Schedulers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-17

Sample Plugin Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-17
Writing Custom Plugin Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-19
Adding User Customization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-24
Managing Jobs with Generic Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . 6-25
Submitting from a Remote Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-26
Submitting without a Shared File System . . . . . . . . . . . . . . . . . . . . . . . . 6-27

Program Communicating Jobs

7
Program Communicating Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-2

Program Communicating Jobs for a Supported Scheduler . . . . . . . . . . . . 7-3

Schedulers and Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-3
Code the Task Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-3
Code in the Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4

Further Notes on Communicating Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-6

Number of Tasks in a Communicating Job . . . . . . . . . . . . . . . . . . . . . . . . . 7-6
Avoid Deadlock and Other Dependency Errors . . . . . . . . . . . . . . . . . . . . . 7-6

GPU Computing
8
GPU Capabilities and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2
Capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2
Performance Benchmarking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2

x Contents
Establish Arrays on a GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-3
Create GPU Arrays from Existing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-3
Create GPU Arrays Directly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4
Examine gpuArray Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4
Save and Load gpuArrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-5

Random Number Streams on a GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-6

Client CPU and GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-6
Worker CPU and GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-7
Normally Distributed Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . 8-7

Run MATLAB Functions on a GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-9

MATLAB Functions with gpuArray Arguments . . . . . . . . . . . . . . . . . . . . . 8-9
Check or Select a GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-10
Use MATLAB Functions with a GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-10
Sharpen an Image Using the GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-12
Compute the Mandelbrot Set using GPU-Enabled Functions . . . . . . . . . . 8-13
Work with Sparse Arrays on a GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-15
Work with Complex Numbers on a GPU . . . . . . . . . . . . . . . . . . . . . . . . . 8-16
Special Conditions for gpuArray Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . 8-17
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-18

Identify and Select a GPU Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-19

Run CUDA or PTX Code on GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-20

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-20
Create a CUDAKernel Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-20
Run a CUDAKernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-24
Complete Kernel Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-26

Run MEX-Functions Containing CUDA Code . . . . . . . . . . . . . . . . . . . . . . . 8-28

Write a MEX-File Containing CUDA Code . . . . . . . . . . . . . . . . . . . . . . . . 8-28
Run the Resulting MEX-Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-28
Comparison to a CUDA Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-29
Access Complex Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-29
Compile a GPU MEX-File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-30

Measure and Improve GPU Performance . . . . . . . . . . . . . . . . . . . . . . . . . 8-31

Getting Started with GPU Benchmarking . . . . . . . . . . . . . . . . . . . . . . . . 8-31
Improve Performance Using Single Precision Calculations . . . . . . . . . . . 8-31
Basic Workflow for Improving Performance . . . . . . . . . . . . . . . . . . . . . . . 8-31
Advanced Tools for Improving Performance . . . . . . . . . . . . . . . . . . . . . . 8-32
Best Practices for Improving Performance . . . . . . . . . . . . . . . . . . . . . . . 8-33
Measure Performance on the GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-34
Vectorize for Improved GPU Performance . . . . . . . . . . . . . . . . . . . . . . . . 8-35
Troubleshooting GPUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-36

GPU Support by Release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-38

Supported GPUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-38
CUDA Toolkit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-39
Increase the CUDA Cache Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-39

xi
 Objects
9

Functions
10

xii Contents
 1

Getting Started

• “Parallel Computing Toolbox Product Description” on page 1-2

• “Parallel Computing Support in MathWorks Products” on page 1-3
• “Create and Use Distributed Arrays” on page 1-4
• “Determine Product Installation and Versions” on page 1-6
• “Interactively Run a Loop in Parallel Using parfor” on page 1-7
• “Run Batch Parallel Jobs” on page 1-9
• “Distribute Arrays and Run SPMD” on page 1-12
• “What Is Parallel Computing?” on page 1-14
• “Choose a Parallel Computing Solution” on page 1-16
• “Run MATLAB Functions with Automatic Parallel Support” on page 1-20
• “Run Non-Blocking Code in Parallel Using parfeval” on page 1-22
• “Evaluate Functions in the Background Using parfeval” on page 1-23
• “Use Parallel Computing Toolbox with Cloud Center clusters in MATLAB Online” on page 1-24
1 Getting Started

Parallel Computing Toolbox Product Description

Perform parallel computations on multicore computers, GPUs, and computer clusters

Parallel Computing Toolbox lets you solve computationally and data-intensive problems using
multicore processors, GPUs, and computer clusters. High-level constructs—parallel for-loops, special
array types, and parallelized numerical algorithms—enable you to parallelize MATLAB® applications
without CUDA or MPI programming. The toolbox lets you use parallel-enabled functions in MATLAB
and other toolboxes. You can use the toolbox with Simulink® to run multiple simulations of a model in
parallel. Programs and models can run in both interactive and batch modes.

The toolbox lets you use the full processing power of multicore desktops by executing applications on
workers (MATLAB computational engines) that run locally. Without changing the code, you can run
the same applications on clusters or clouds (using MATLAB Parallel Server™). You can also use the
toolbox with MATLAB Parallel Server to execute matrix calculations that are too large to fit into the
memory of a single machine.

1-2
 Parallel Computing Support in MathWorks Products

Parallel Computing Support in MathWorks Products

Parallel Computing Toolbox provides you with tools for a local cluster of workers on your client
machine. MATLAB Parallel Server software allows you to run as many MATLAB workers on a remote
cluster of computers as your licensing allows.

Most MathWorks products enable you to run applications in parallel. For example, Simulink models
can run simultaneously in parallel, as described in “Run Multiple Simulations” (Simulink). MATLAB
Compiler™ and MATLAB Compiler SDK™ software let you build and deploy parallel applications; for
example, see the “Parallel Computing” section of MATLAB Compiler “Standalone Applications”
(MATLAB Compiler).

Several MathWorks products now offer built-in support for the parallel computing products, without
requiring extra coding. For the current list of these products and their parallel functionality, see:
https://www.mathworks.com/products/parallel-computing/parallel-support.html

1-3
1 Getting Started

Create and Use Distributed Arrays

In this section...
“Creating Distributed Arrays” on page 1-4
“Creating Codistributed Arrays” on page 1-5

If your data is currently in the memory of your local machine, you can use the distributed function
to distribute an existing array from the client workspace to the workers of a parallel pool.
Distributed arrays use the combined memory of multiple workers in a parallel pool to store the
elements of an array. For alternative ways of partitioning data, see “Distributing Arrays to Parallel
Workers” on page 3-10.You can use distributed arrays to scale up your big data computation.
Consider distributed arrays when you have access to a cluster, as you can combine the memory of
multiple machines in your cluster.

A distributed array is a single variable, split over multiple workers in your parallel pool. You can
work with this variable as one single entity, without having to worry about its distributed nature.
Explore the functionalities available for distributed arrays in the Parallel Computing Toolbox:
“Run MATLAB Functions with Distributed Arrays” on page 4-19.

When you create a distributed array, you cannot control the details of the distribution. On the
other hand, codistributed arrays allow you to control all aspects of distribution, including
dimensions and partitions. In the following, you learn how to create both distributed and
codistributed arrays.

Creating Distributed Arrays

You can create a distributed array in different ways:

• Use the distributed function to distribute an existing array from the client workspace to the
workers of a parallel pool.
• You can directly construct a distributed array on the workers. You do not need to first create the
array in the client, so that client workspace memory requirements are reduced. The functions
available include eye(___,'distributed'), rand(___,'distributed'), etc. For a full list,
see the distributed object reference page.
• Create a codistributed array inside an spmd statement, see “Single Program Multiple Data
(spmd)” on page 1-12. Then access it as a distributed array outside the spmd statement. This
lets you use distribution schemes other than the default.

In this example, you create an array in the client workspace, then turn it into a distributed array:

parpool('local',4) % Create pool

A = magic(4); % Create magic 4-by-4 matrix
B = distributed(A); % Distribute to the workers
B % View results in client.
whos % B is a distributed array here.
delete(gcp) % Stop pool

You have createdB as a distributed array, split over the workers in your parallel pool. This is
shown in the figure.

1-4
 Create and Use Distributed Arrays

Creating Codistributed Arrays

Unlike distributed arrays, codistributed arrays allow you to control all aspects of distribution,
including dimensions and partitions. You can create a codistributed array in different ways:

• “Partitioning a Larger Array” on page 4-6 — Start with a large array that is replicated on all
workers, and partition it so that the pieces are distributed across the workers. This is most useful
when you have sufficient memory to store the initial replicated array.
• “Building from Smaller Arrays” on page 4-6 — Start with smaller replicated arrays stored on
each worker, and combine them so that each array becomes a segment of a larger codistributed
array. This method reduces memory requirements as it lets you build a codistributed array from
smaller pieces.
• “Using MATLAB Constructor Functions” on page 4-7 — Use any of the MATLAB constructor
functions like rand or zeros with a codistributor object argument. These functions offer a quick
means of constructing a codistributed array of any size in just one step.

In this example, you create a codistributed array inside an spmd statement, using a nondefault
distribution scheme. First, define 1-D distribution along the third dimension, with 4 parts on worker
1, and 12 parts on worker 2. Then create a 3-by-3-by-16 array of zeros.

parpool('local',2) % Create pool

spmd
codist = codistributor1d(3,[4,12]);
Z = zeros(3,3,16,codist);
Z = Z + labindex;
end
Z % View results in client.
whos % Z is a distributed array here.
delete(gcp) % Stop pool

For more details on codistributed arrays, see “Working with Codistributed Arrays” on page 4-4.

See Also

Related Examples
• “Distributing Arrays to Parallel Workers” on page 3-10
• “Big Data Workflow Using Tall Arrays and Datastores” on page 5-46
• “Single Program Multiple Data (spmd)” on page 1-12

1-5
1 Getting Started

Determine Product Installation and Versions

To determine if Parallel Computing Toolbox software is installed on your system, type this command
at the MATLAB prompt.

ver

When you enter this command, MATLAB displays information about the version of MATLAB you are
running, including a list of all toolboxes installed on your system and their version numbers.

If you want to run your applications on a cluster, see your system administrator to verify that the
version of Parallel Computing Toolbox you are using is the same as the version of MATLAB Parallel
Server installed on your cluster.

1-6
 Interactively Run a Loop in Parallel Using parfor

Interactively Run a Loop in Parallel Using parfor

In this example, you start with a slow for-loop, and you speed up the calculation using a parfor-
loop instead. parfor splits the execution of for-loop iterations over the workers in a parallel pool.

This example calculates the spectral radius of a matrix and converts a for-loop into a parfor-loop.
Find out how to measure the resulting speedup.
1 In the MATLAB Editor, enter the following for-loop. Add tic and toc to measure the time
elapsed.
tic
n = 200;
A = 500;
a = zeros(n);
for i = 1:n
a(i) = max(abs(eig(rand(A))));
end
toc
2 Run the script, and note the elapsed time.
Elapsed time is 31.935373 seconds.
3 In the script, replace the for-loop with a parfor-loop.
tic
n = 200;
A = 500;
a = zeros(n);
parfor i = 1:n
a(i) = max(abs(eig(rand(A))));
end
toc
4 Run the new script, and run it again. Note that the first run is slower than the second run,
because the parallel pool takes some time to start and make the code available to the workers.
Note the elapsed time for the second run.

By default, MATLAB automatically opens a parallel pool of workers on your local machine.
Starting parallel pool (parpool) using the 'local' profile ... connected to 4 workers.
...
Elapsed time is 10.760068 seconds.

1-7
1 Getting Started

The parfor run on four workers is about three times faster than the corresponding for-loop
run. The speed-up is smaller than the ideal speed-up of a factor of four on four workers. This is
due to parallel overhead, including the time required to transfer data from the client to the
workers and back. This example shows a good speed-up with relatively small parallel overhead,
and benefits from conversion into a parfor-loop. Not all for-loop iterations can be turned into
faster parfor-loops. To learn more, see “Decide When to Use parfor” on page 2-2.

One key requirement for using parfor-loops is that the individual iterations must be independent.
Independent problems suitable for parfor processing include Monte Carlo simulations and
parameter sweeps. For next steps, see “Convert for-Loops Into parfor-Loops” on page 2-7.

In this example, you managed to speed up the calculation by converting the for-loop into a parfor-
loop on four workers. You might reduce the elapsed time further by increasing the number of workers
in your parallel pool, see “Scale Up parfor-Loops to Cluster and Cloud” on page 2-21.

You can modify your cluster profiles to control how many workers run your loops, and whether the
workers are local or on a cluster. For more information on profiles, see “Discover Clusters and Use
Cluster Profiles” on page 5-11.

Modify your parallel preferences to control whether a parallel pool is created automatically, and how
long it remains available before timing out. For more information on preferences, see “Specify Your
Parallel Preferences” on page 5-9.

You can run Simulink models in parallel with the parsim command instead of using parfor-loops.
For more information and examples of using Simulink in parallel, see “Run Multiple Simulations”
(Simulink).

See Also
parfor | parpool | tic | toc

More About
• “Decide When to Use parfor” on page 2-2
• “Convert for-Loops Into parfor-Loops” on page 2-7
• “Scale Up parfor-Loops to Cluster and Cloud” on page 2-21

1-8
 Run Batch Parallel Jobs

Run Batch Parallel Jobs

Run a Batch Job
To offload work from your MATLAB session to run in the background in another session, you can use
the batch command inside a script.
1 To create the script, type:
edit mywave
2 In the MATLAB Editor, create a for-loop:
for i = 1:1024
A(i) = sin(i*2*pi/1024);
end
3 Save the file and close the Editor.
4 Use the batch command in the MATLAB Command Window to run your script on a separate
MATLAB worker:
job = batch('mywave')

5 batch does not block MATLAB and you can continue working while computations take place. If
you need to block MATLAB until the job finishes, use the wait function on the job object.
wait(job)
6 After the job finishes, you can retrieve and view its results. The load command transfers
variables created on the worker to the client workspace, where you can view the results:
load(job,'A')
plot(A)
7 When the job is complete, permanently delete its data and remove its reference from the
workspace:
delete(job)
clear job

batch runs your code on a local worker or a cluster worker, but does not require a parallel pool.

You can use batch to run either scripts or functions. For more details, see the batch reference page.

Run a Batch Job with a Parallel Pool

You can combine the abilities to offload a job and run a loop in a parallel pool. This example combines
the two to create a simple batch parfor-loop.
1 To create a script, type:
edit mywave

1-9
1 Getting Started

2 In the MATLAB Editor, create a parfor-loop:

parfor i = 1:1024
A(i) = sin(i*2*pi/1024);
end
3 Save the file and close the Editor.
4 Run the script in MATLAB with the batch command. Indicate that the script should use a
parallel pool for the loop:

job = batch('mywave','Pool',3)

This command specifies that three workers (in addition to the one running the batch script) are
to evaluate the loop iterations. Therefore, this example uses a total of four local workers,
including the one worker running the batch script. Altogether, there are five MATLAB sessions
involved, as shown in the following diagram.

5 To view the results:

wait(job)
load(job,'A')
plot(A)

The results look the same as before, however, there are two important differences in execution:

• The work of defining the parfor-loop and accumulating its results are offloaded to another
MATLAB session by batch.
• The loop iterations are distributed from one MATLAB worker to another set of workers
running simultaneously ('Pool' and parfor), so the loop might run faster than having only
one worker execute it.
6 When the job is complete, permanently delete its data and remove its reference from the
workspace:

delete(job)
clear job

1-10
 Run Batch Parallel Jobs

Run Script as Batch Job from the Current Folder Browser

From the Current Folder browser, you can run a MATLAB script as a batch job by browsing to the
file’s folder, right-clicking the file, and selecting Run Script as Batch Job. The batch job runs on the
cluster identified by the default cluster profile. The following figure shows the menu option to run the
script file script1.m:

Running a script as a batch from the browser uses only one worker from the cluster. So even if the
script contains a parfor loop or spmd block, it does not open an additional pool of workers on the
cluster. These code blocks execute on the single worker used for the batch job. If your batch script
requires opening an additional pool of workers, you can run it from the command line, as described in
“Run a Batch Job with a Parallel Pool” on page 1-9.

When you run a batch job from the browser, this also opens the Job Monitor. The Job Monitor is a tool
that lets you track your job in the scheduler queue. For more information about the Job Monitor and
its capabilities, see “Job Monitor” on page 5-24.

See Also
batch

Related Examples
• “Run Batch Job and Access Files from Workers”

1-11
1 Getting Started

Distribute Arrays and Run SPMD

Distributed Arrays
The workers in a parallel pool communicate with each other, so you can distribute an array among
the workers. Each worker contains part of the array, and all the workers are aware of which portion
of the array each worker has.

Use the distributed function to distribute an array among the workers:

M = magic(4) % a 4-by-4 magic square in the client workspace
MM = distributed(M)

Now MM is a distributed array, equivalent to M, and you can manipulate or access its elements in the
same way as any other array.
M2 = 2*MM; % M2 is also distributed, calculation performed on workers
x = M2(1,1) % x on the client is set to first element of M2

Single Program Multiple Data (spmd)

The single program multiple data (spmd) construct lets you define a block of code that runs in parallel
on all the workers in a parallel pool. The spmd block can run on some or all the workers in the pool.
spmd % By default creates pool and uses all workers
R = rand(4);
end

This code creates an individual 4-by-4 matrix, R, of random numbers on each worker in the pool.

Composites
Following an spmd statement, in the client context, the values from the block are accessible, even
though the data is actually stored on the workers. On the client, these variables are called Composite
objects. Each element of a composite is a symbol referencing the value (data) on a worker in the pool.
Note that because a variable might not be defined on every worker, a Composite might have
undefined elements.

Continuing with the example from above, on the client, the Composite R has one element for each
worker:
X = R{3}; % Set X to the value of R from worker 3.

The line above retrieves the data from worker 3 to assign the value of X. The following code sends
data to worker 3:
X = X + 2;
R{3} = X; % Send the value of X from the client to worker 3.

If the parallel pool remains open between spmd statements and the same workers are used, the data
on each worker persists from one spmd statement to another.
spmd
R = R + labindex % Use values of R from previous spmd.
end

1-12
 Distribute Arrays and Run SPMD

A typical use for spmd is to run the same code on a number of workers, each of which accesses a
different set of data. For example:

spmd
INP = load(['somedatafile' num2str(labindex) '.mat']);
RES = somefun(INP)
end

Then the values of RES on the workers are accessible from the client as RES{1} from worker 1,
RES{2} from worker 2, etc.

There are two forms of indexing a Composite, comparable to indexing a cell array:

• AA{n} returns the values of AA from worker n.

• AA(n) returns a cell array of the content of AA from worker n.

Although data persists on the workers from one spmd block to another as long as the parallel pool
remains open, data does not persist from one instance of a parallel pool to another. That is, if the pool
is deleted and a new one created, all data from the first pool is lost.

For more information about using distributed arrays, spmd, and Composites, see “Distributed
Arrays”.

1-13
1 Getting Started

What Is Parallel Computing?

Parallel computing allows you to carry out many calculations simultaneously. Large problems can
often be split into smaller ones, which are then solved at the same time.

The main reasons to consider parallel computing are to

• Save time by distributing tasks and executing these simultaneously

• Solve big data problems by distributing data
• Take advantage of your desktop computer resources and scale up to clusters and cloud computing

With Parallel Computing Toolbox, you can

• Accelerate your code using interactive parallel computing tools, such as parfor and parfeval
• Scale up your computation using interactive Big Data processing tools, such as distributed,
tall, datastore, and mapreduce
• Use gpuArray to speed up your calculation on the GPU of your computer
• Use batch to offload your calculation to computer clusters or cloud computing facilities

Here are some useful Parallel Computing concepts:

• Node: standalone computer, containing one or more CPUs / GPUs. Nodes are networked to form a
cluster or supercomputer
• Thread: smallest set of instructions that can be managed independently by a scheduler. On a GPU,
multiprocessor or multicore system, multiple threads can be executed simultaneously (multi-
threading)
• Batch: off-load execution of a functional script to run in the background
• Scalability: increase in parallel speedup with the addition of more resources

What tools do MATLAB and Parallel Computing Toolbox offer?

• MATLAB workers: MATLAB computational engines that run in the background without a graphical
desktop. You use functions in the Parallel Computing Toolbox to automatically divide tasks and
assign them to these workers to execute the computations in parallel. You can run local workers to
take advantage of all the cores in your multicore desktop computer. You can also scale up to run
your workers on a cluster of machines, using the MATLAB Parallel Server. The MATLAB session
you interact with is known as the MATLAB client. The client instructs the workers with parallel
language functions.
• Parallel pool: a parallel pool of MATLAB workers created using parpool or functions with
automatic parallel support. By default, parallel language functions automatically create a parallel
pool for you when necessary. To learn more, see “Run Code on Parallel Pools” on page 2-56.

For the default local profile, the default number of workers is one per physical CPU core using a
single computational thread. This is because even though each physical core can have several
virtual cores, the virtual cores share some resources, typically including a shared floating point
unit (FPU). Most MATLAB computations use this unit because they are double-precision floating
point. Restricting to one worker per physical core ensures that each worker has exclusive access
to a floating point unit, which generally optimizes performance of computational code. If your
code is not computationally intensive, for example, it is input/output (I/O) intensive, then consider

1-14
 What Is Parallel Computing?

using up to two workers per physical core. Running too many workers on too few resources may
impact performance and stability of your machine.
• Speed up: Accelerate your code by running on multiple MATLAB workers or GPUs, for example,
using parfor, parfeval, or gpuArray.
• Scale up your data: Partition your big data across multiple MATLAB workers, using tall arrays and
distributed arrays. To learn more, see “Big Data Processing”.
• Asynchronous processing: Use parfeval to execute a computing task in the background without
waiting for it to complete.
• Scale up to clusters and clouds: If your computing task is too big or too slow for your local
computer, you can offload your calculation to a cluster onsite or in the cloud using MATLAB
Parallel Server. For more information, see “Clusters and Clouds”.

See Also

Related Examples
• “Choose a Parallel Computing Solution” on page 1-16
• “Identify and Select a GPU Device” on page 8-19
• “Decide When to Use parfor” on page 2-2
• “Run Single Programs on Multiple Data Sets” on page 3-2
• “Evaluate Functions in the Background Using parfeval” on page 1-23
• “Distributing Arrays to Parallel Workers” on page 3-10
• “Run Batch Parallel Jobs” on page 1-9

1-15
1 Getting Started

Choose a Parallel Computing Solution

Process your data faster or scale up your big data computation using the capabilities of MATLAB,
Parallel Computing Toolbox and MATLAB Parallel Server.

Problem Solutions Required Products More Information

Do you want Profile your code. MATLAB “Profile Your Code to Improve
to process Performance” (MATLAB)
your data Vectorize your code. MATLAB “Vectorization” (MATLAB)
faster?
Use automatic parallel MATLAB “Run MATLAB Functions with
computing support in Automatic Parallel Support” on
MathWorks products. Parallel Computing page 1-20
Toolbox
If you have a GPU, try MATLAB “Run MATLAB Functions on a
gpuArray. GPU” on page 8-9
Parallel Computing
Toolbox
Use parfor. MATLAB “Interactively Run a Loop in
Parallel Using parfor” on page 1-
Parallel Computing 7
Toolbox
Are you Try parfeval. MATLAB “Evaluate Functions in the
looking for Background Using parfeval” on
other ways to Parallel Computing page 1-23
speed up your Toolbox
processing? Try spmd. MATLAB “Run Single Programs on
Multiple Data Sets” on page 3-
Parallel Computing 2
Toolbox
Do you want To work with out-of- MATLAB “Big Data Workflow Using Tall
to scale up memory data with any Arrays and Datastores” on page
your big data number of rows, use tall 5-46
calculation? arrays.

This workflow is well

suited to data analytics and
machine learning.
Use tall arrays in parallel MATLAB “Use Tall Arrays on a Parallel
on your local machine. Pool” on page 5-48
Parallel Computing
Toolbox
Use tall arrays in parallel MATLAB “Use Tall Arrays on a Spark
on your cluster. Enabled Hadoop Cluster” on
Parallel Computing page 5-51
Toolbox

MATLAB Parallel
Server

1-16
 Choose a Parallel Computing Solution

Problem Solutions Required Products More Information

If your data is large in MATLAB “Run MATLAB Functions with
multiple dimensions, use Distributed Arrays” on page 4-
distributed instead. Parallel Computing 19
Toolbox
This workflow is well
suited to linear algebra MATLAB Parallel
problems. Server
Do you want Use batch to run your MATLAB Parallel “Run Batch Parallel Jobs” on
to offload to a code on clusters and Server page 1-9
cluster? clouds.

1-17
Exploring the Variety of Random
Documents with Different Content
Israelites bitten by the fiery serpents by the sight of the brazen
image, and he cured Naaman’s leprosy by bathing in the waters of
Jordan. Whatever then be the means which He prescribes, our
highest wisdom is to make use of them. But as he has not prescribed
the means recommended by the Talmud, but forbidden them in his
general prohibition of magic, we must say that the man who uses
them has bid adieu to all true wisdom. No wonder, then, if his own
inventions are stamped with folly. But what will our readers think of
the cause of the canine madness here assigned? “Rav says, It
proceeds from the witches who are making their sport with him.
Samuel says, It is an evil spirit that rests upon him.” Rav believed,
then, that God, whose mercies are over all his works, allows wicked
women to torment his creatures, and to inflict upon them a dreadful
malady to make sport for themselves. Is this wise, is it according to
Scripture? This is the doctrine of the oral law; and if Jesus of
Nazareth had not protested against it, and taught a true doctrine by
asserting the truth of Scripture, this would be the universal doctrine
and practice of the Jews. Whoever believes the Talmud, must
believe in this and all the other follies which it contains. Whoever
rejects these things, confesses that the Talmud contains what is
false and foolish, and thereby shakes or rather overthrows its
authority. Some person will perhaps say that similar superstitions
and follies have been found amongst Christians. We grant that this
has been the case wherever Christians have departed from the
written Word of God, but can anything similar be found in the New
Testament? That book is our standard of Christianity. As you say that
the oral law is of divine authority, we say that the New Testament is
of divine authority. We point out to you these follies, not in individual
Jews, but in your book of authority. If you would make out a parallel
case, you must do the same. But you cannot. The New Testament
has nothing of the kind; and it is for you to explain how this happens
that the New Testament, which you believe to be false, is entirely
free from every thing of the kind.
Further, we ask every right-minded Israelite, whether he is not
shocked at that profanation of the reverend and holy names of God
which is here not only countenanced but prescribed. What can a
devout Jew think either of the man or the book that tells us to write
the names,
‫יה יה יהוה צבאות ׃‬
“Jah, Jah, the Lord of Hosts,” by the side of such nonsense as Kanti,
Kanti, Klurus? Would he say that this is consistent with true religion?
And yet this profane use of the name of God for magical purposes, is
not rare in the Talmud. The following is another instance:—
‫ האי גלא דמטבע לספינה מיתחזי כי צוציתא‬, ‫אמר רבה אשתעו לי נחותי ימא‬
‫דנורא חיוורתא ברישא ומחינן ליה באלוותא דחקיק עליה אהיה אשר אהיה יה ה׳‬
‫צבאות אמן אמן סלה ונייח אמר רבה אשתעו לי נחותי ימא בין גלא לגלא תלת‬
‫מאה פרסי זמנא חדא הוה אזלינן באורחא ודלינן גלא עד דחזינן בי מרבעתא‬
‫ ואי דלינן טפי מקלינן‬, ‫דכוכבא זוטא והויא בי מבזר ארבעין גריוי בזרא דחרדלא‬
‫ ורמי ליה גלא קלא לחברתה חברתי שבקת מידי בעלמא דלא שטפתיה‬, ‫מהבלי‬
‫דניתי אנא ונאבדיה א׳׳ל הזי גבורתא דמריך מלא חוטא חלא ולא עברי שנאמר‬
‫האותי לא תיראו נאום ה׳ אם מפני לא תחילו אשר שמתי חול גבול לים חק עולם‬
‫ולא יעברנהו ׃‬
“Rabbah says, They that go down to the sea have told me, that when
a wave is going to overwhelm a ship, sparks of white light are seen
on its head. But if we strike it with a staff on which are graved the
words, ‘I am that I am, Jah, Lord of Hosts, Amen, Amen, Selah,’ it
subsides. They that go down to the sea have told me, that the
distance between one wave and another, is three hundred miles. It
happened once that we were making a voyage, and we raised a
wave until we saw the resting-place of the least of all the stars. It
was large enough to sow forty bushels of mustard seed, and if we
had raised it more we should have been burned by the vapour of the
star. One wave raised its voice and called to its companion, O,
companion, hast thou left anything in the world that thou hast not
overflowed? Come, and let us destroy it. It replied, Come and see
the power of thy Lord. I could not overpass the sand even a hair’s-
breadth, for it is written, ‘Fear ye not me? saith the Lord: will ye not
tremble at my presence, which have placed the sand for the bound
of the sea, by a perpetual decree that it cannot pass it?’ (Jer. v. 22.)”
(Bava Bathra, fol. 73, col. 1.) Here is the same profanation of the
peculiar and holy names of God: it is to be engraved on a staff either
to lay or to raise the waves. But besides the profanity, just consider
the folly of this whole story. In the first place, it ascribes to men, no
matter whether they are good or wicked, absolute power over the
waves of the sea. Anybody can engrave those names of God upon a
staff, anybody can use the staff to strike the sea, and thus a wicked
man, without either faith, fear, or love of God, may make and use an
instrument which almost invests him with omnipotence. Is it possible
that any son of Israel can be so credulous as to believe such
manifest absurdity? But this story reminds us again of the utter
disregard of truth which characterises the Talmud. Here we are told
that, by power of this magic staff, a wave was raised so high as to
enable those travellers to see the resting-place of the smallest of all
the stars, and that so distinctly, too, as to be able to make a good
guess at its measure. The slightest knowledge of modern astronomy
is sufficient to show not only the improbability, but the utter
impossibility of anything of the kind. The least of the stars visible to
the naked eye is at an almost immeasurable distance from the earth,
so as to make it perfectly ludicrous to talk of a wave being raised to
such a height. All the water on the face of the globe would be far
from sufficient for the formation of one such wave. But the Talmud
intimates that they had the power of raising it still higher, and were
prevented only by the fear of being scorched. But the Talmud is not
satisfied with these wonders, it goes on to describe a conversation
between two waves. The commentator, who evidently believed every
word of the story, suggest that this conversation was carried on by
the angels presiding over the waves.
‫ ושמא מלאכים‬, ‫ורמי ליה גלא נתן קולו כלומר צעק כדוגמא תהום אל תהום קורא‬
‫הממונים עליהם הם ׃‬
“The wave lifted up his voice, that is, it cried, and so we find, ‘Deep
calleth unto deep.’ And perhaps this means the angels who were set
over them.” The commentator, it appears, had no doubt of the truth
of the story, and how should he have, if he believed in the Divine
authority of the Talmud? But we ask our readers do they believe this
story—and if they do not, why not? Because it is too absurd, and too
far beyond the bounds of possibility. Can, then, a book that swarms
with similar accounts be from God? By what means did all these
things about magic, astrology, amulets, magical cures, and staves,
get into the Talmud? No doubt they were put in by the authors.
Either, then, the authors believed in all these things, or they did not.
If they did not believe in them, then they were evidently bad men,
who deliberately wrote falsehood. But if they did believe these
things, then, though not guilty of wilful falsehood, they were
credulous, superstitious persons, who had no clear idea of the
religion of Moses and the prophets; and in either case they are most
unsafe guides in religion. It is for the Jews of the present day to
consider whether they will still adhere to a system that involves the
belief of so many incredibilities and sanctions the profanation of the
names of God for the purposes of magic. Eighteen centuries are
surely long enough to have remained in such thick darkness. Those
who have been brought up in such a system ought now, at least, to
arise and ask what have they and their forefathers been about all
this while? And how it is that the New Testament, which they have
rejected, is entirely free from such deformities? Something has been
decidedly wrong, or the chosen people of God could not have
remained so long in captivity, unheeded and unhelped by the Holy
One of Israel. An exhibition of the doctrines of the oral law explains
the cause. Israel has departed from the religion of Moses, and
pertinaciously adhered to a system compounded of human
inventions, and idolatrous heathenism. They call Moses their master,
and say that the oral law is derived from him, but if we may from the
work, form a conjecture about the author, it is much more probably a
tradition from the magicians of Egypt or the witch of Endor. And if it
had been handed down as such—if the Israelites had presented the
Talmud to the world and their posterity as part of the heavy yoke of
Egypt, we should not have been astonished at the universality of its
reception. But that Israel should ever have been so far imposed
upon, as to believe that Moses or the prophets ever had anything to
do with the oral law appears almost inexplicable. However unwilling
one may be to apply to fellow-sinners any prophecy that contains a
denunciation of God’s wrath, one cannot help asking, was it of this
that the prophet said, “The Lord hath poured out upon you the spirit
of deep sleep, and hath closed your eyes; the prophets and your
rulers the seers hath he covered. And the vision of all is become
unto you as the words of a book that is sealed, which men deliver to
one that is learned, saying, read this, I pray thee; and he saith, I
cannot, for it is sealed: and the book is delivered to him that is not
learned, saying, Read this, I pray thee, and he saith, I am not
learned.” (Isaiah xxix. 10-12.) This question is, however, far more
important to Israel than to us, and to them we leave the answer.
Some will still persist in the assertion that this heathenish compound
is the highest wisdom. The great majority of the nation is devoted to
the Talmud, which is still the cistern whence the synagogues
endeavour to draw the waters of life. The multitude does it in
ignorance, they are, therefore, not so culpable. But there are many
that know better, what then is the reason that they do not strain
every nerve to deliver their brethren? These few do not suffer the
oral law to interfere either with their business or their convenience.
They profane the Sabbath, eat Gentile food, carry on their business
on feasts and festivals. If they do all this on principle, why not protest
against error? Is it because they are indifferent to the welfare of their
brethren? If indifference be the only fruit of this intellectual progress,
instead of rising above, they have sunk below superstition itself.
 No. XXVI.
CHARMS CONTINUED.

If men would only employ in religion a little of that common sense

and earnestness, which they find so necessary for the affairs of this
life, they would by God’s blessing soon arrive at the truth. For
example, if the father of a family should find, that by following the
advice of a physician, sickness and death were constant guests, he
would soon look out for another; and he would be much quickened in
his measures, if this physician’s counsel had produced the same
results in the house of his father and his grandfather. He would not
think it any shame, under such circumstances, to change his father’s
physician for another; on the contrary, he would think, and most men
would agree with him, that it would be both a sin and a shame to
retain him. Now let Israel make the application to their spiritual
physicians, the Scribes, Pharisees, and Rabbies. For many centuries
they have punctually followed their advice, and the consequence has
been one misfortune after another, and centuries of exile from the
land which God gave to their fathers; the very contrary of that which
God has promised. God has said, if the Jews will obey the religion of
Moses, that they shall be restored to their land. “It shall come to pass
... if thou shalt return unto the Lord thy God, and shalt obey his voice
according to all that I command thee this day, thou and thy children,
with all thy heart and with all thy soul; that then the Lord thy God will
turn thy captivity, and have compassion upon thee, and will return
and gather thee from all nations,” &c. (Deut. xxx. 2, 3.) The Jews
have obeyed the commands of the rabbies, and have not been
gathered; what is the conclusion? Either that God’s promise has
failed, which is impossible, or that the religion of the rabbies is not
the religion of Moses. Such is the inevitable conclusion from the
words of Moses and the facts of the case; let it then lead the
sufferers to examine the religion which they have hitherto professed.
A very little examination will convince any reasonable man, that it is
a fearful corruption of divine truth, a compilation made by men who
professed to be astrologers and magicians. Let not the Jews think
that our opinion is the result of prejudice. It has been deliberately
formed on evidence furnished by the oral law itself. If we are wrong,
let the rabbies prove the contrary. Let them, for example, explain the
following law of modern Judaism.
‫מי שנשכו עקרב או נחש מותר ללחוש אל מקום הנשיכה ואפילו בשבת כדי לישב‬
‫דעתו ולחזק לבו אף על פי שאין הדבר מועיל כלום הואיל ומסוכן הוא התירו לו כדי‬
‫שלא תטרף דעתו עליו ׃‬
“If any person be bitten by a scorpion or a serpent, it is lawful to
charm the place of the bite, even on the Sabbath-day, in order to
quiet his mind, and to encourage his heart, although it is a thing
utterly profitless. Because the man is in danger, they have
pronounced this lawful for him that his mind may not be distracted.”
(Hilchoth Accum. c. xi. 11.) Here the rabbies have allowed what God
has absolutely forbidden. The men who profess such reverence for
the Sabbath allow it to be profaned by magic, which is one of the
works of the devil. Rambam, whose words we have just quoted, felt
that it was both wicked and foolish, and has therefore endeavoured
to furnish an excuse, saying that it is of no use, and is only allowed
to quiet the mind of the sufferer. But that does not alter the
unlawfulness. Besides, what sort of opinion could Rambam and the
rabbies have had of the Jews, when they say that magic is permitted
in order to quiet their minds? They evidently supposed that the Jews
were a weak and superstitious people, who believed so firmly in
charms, that the use of them would quiet the mind; and so ignorant
or careless about God’s commandments, that they could be
comforted by their transgressions. The excuse, therefore, only
makes the case worse. It takes for granted that the professors of the
oral law are ignorant and superstitious; and then to quiet their minds
allows the transgression of the law of Moses, and that on the
Sabbath-day. But this excuse is altogether Rambam’s invention. The
original passage in the Talmud says nothing about quieting the
man’s mind, it simply says—
‫ולוחשין לחישת נחשים ועקרבים בשבת ׃‬
“It is lawful to charm serpents and scorpions on the Sabbath-day.”
(Sanhedrin, fol. 101, col. 1.) And Rashi’s commentary on the
passage—
‫בשביל שלא יזיקו ׃‬
“That they may not do injury.” This man, then, who spent his life in
the study of the Talmud, knew nothing of Rambam’s apology. He
plainly believed that by charming serpents on the Sabbath, they
might be prevented from doing harm, and that on this account, and
not for the purpose of quieting the mind, they were permitted so to
do. This was also the opinion of that famous expounder of Jewish
law, the Baal Turim, for after quoting Rambam’s words, he adds:—
‫מי שרודפים אחריו נחש או עקרב מותר להבר ללחש כדי שלא יזיקוהו כתב‬
‫הרמב׳׳ם הלוחש על המכה והקורא פסוק מן התורה וכן הקורא על התינוק שלא‬
‫יבעת או מניח ס׳׳ת או תפילין על הקטן לא די להם שהם בכלל חברים ומנחשים‬
‫אלא שהם בכלל הכופרים בתורה שעושין דברי תורה רפואת הגוף ואינן אלא‬
‫רפואת הנפש ור׳׳י פירש דוקא בלוחש על המכה ומזכיר שם שמים ורוקק אותו‬
‫הוא שאין לו חלק לעולם הבא אבל אם אינו רוקק לא חמיר כולי האי ומיהו איסורא‬
‫ ואם יש‬, ‫איכא בלוחש פסוק על המכה אפילו בלא רקיקה ובלא הזכרת שם שמים‬
‫כו סכנת נפשות הכל מותר ומותר לקרוא פסוק פהגן כגון בלילה על מטתו ׃‬
“If any person be pursued by a serpent or a scorpion, it is lawful to
charm it to prevent it from doing injury. Rambam has written, He that
charms a wound, or reads a verse from the law (as a charm), and
also he that reads over an infant that it may not be afraid, or who
lays a roll of the law or phylacteries upon a child, are not only to be
accounted as one of the charmers and magicians, but as of the
deniers of the law, for they use the words of the law as medicine for
the body, whereas it is only a medicine for the soul. R. Isaac says
absolutely, that he who charms a wound, mentioning at the same
time the name of God and spitting, is the charmer of whom it is said
that he has no share in the world to come: but if he does not spit, the
matter is not so grave. It is, however, forbidden to use a verse as a
charm over a wound, even though there will be no spitting nor
mentioning the name of God. But if life be in danger, every thing is
lawful; and it is lawful to read a verse as a defence, for instance at
night in bed.” (Joreh Deah. § 179.) From this it is pretty plain that the
charming of serpents was allowed, not as Rambam says to quiet the
mind of him that had been bitten, but to prevent injury, for it is
allowed before the man is bitten at all, if he be only pursued by a
serpent or a scorpion. But what a picture does this whole passage
give us of the religious state of the Rabbinic Jews, both rabbies and
people. Here you have the people described, not by Christians, but
by the rabbies themselves, as sunk in the depths of superstition,
using a sepher torah, a roll of the law, or phylacteries as a sort of
charm for the benefit of children, and you have the rabbies forbidding
this at one time, but allowing what is equally forbidden by God, to
charm serpents: and, in case of danger, declaring that “Every thing is
lawful,” that is, allowing them to do what will make them, according
to Rambam’s opinion, charmers, magicians, and deniers of the law.
And this is the Jewish religion, and this is what the Jews have gained
by rejecting Christianity. We, poor Gentiles, who cannot trace our
pedigree to Abraham, Isaac, and Jacob, should be ashamed of such
follies. And if such wicked heathenish practices were to be found in
our religious books, we would not let an hour pass over until we had
lifted up our voice and protested against them, and should use every
lawful means to deliver our children from such ungodliness and error.
We have now given quotations from the two great digests of Jewish
law on the subject of using charms, but it is worth while to consider
the context of the original passage, upon which these laws are
based, as that will prove that the Talmud has not been
misrepresented by its compilers.
‫תנו רבנן סכין וממשמשין בבני מעיין בשבת ולוחשין לחישת נחשים ועקרבים‬
‫בשבת ומעבירין כלי על גב העין בשבת אמר רבן שמעון בן גמליאל במה דברים‬
‫אמורים בכלי הניטל אבל בכלי שאינו ניטל אסור ואין שואלין בדבר שדים בשבת ר׳‬
‫יוסי אומר אף בחול אסור אמר רב הונא אין הלכה כר׳ יוסי ואף ר׳ יוסי לא אמרה‬
‫אלא משום סכנה כי הא דרב יצחק בר יוסף דאיבלע בארזא ואתעביד ליה ניסא‬
‫פקע ארזא ופלטיה ׃‬
“Our rabbies have handed down the tradition that it is lawful to anoint
and rub the stomach (of a sick man) on the Sabbath, also to charm
serpents and scorpions on the Sabbath: also to pass an instrument
across the eye on the Sabbath. R. Simeon, the son of Gamaliel,
says, that this only applies to an instrument which may be moved,[26]
but with one that may not be moved, it is unlawful. But it is unlawful
on the Sabbath to make inquiry of demons. R. Jose says, this is also
unlawful on week-days. Rav Huna says, the decision is not
according to R. Jose: and R. Jose himself said this only on account
of danger, for that is what occurred in the case of R. Isaac, the son of
Joseph, who was swallowed up in a cedar tree, but a miracle was
wrought for him—the cedar opened and cast him out.” (Sanhedrin,
fol. 101, col. 1.) We have here, first, the charming of serpents; we
ask, then, could the Talmudic doctors really believe in such folly or
allow such wickedness on the Sabbath? Is there any
misunderstanding, or does the context show, that they were men of
that superstitious turn of mind to justify this idea? The context is all of
a piece, for after permitting the charming of serpents, it goes on to
discuss the lawfulness of asking counsel of demons, and here Rashi
shall explain what this means:—
‫בדבר שדים שכן עושין כשאובדין שום דבר שואלין במעשה שדים והם מּגידים‬
‫להם ואסור לעשות בשבת משום ממצוא חפציך ׃‬
“To make inquiry of demons, is what they do when any thing is lost.
They make inquiry by the work of demons, and they tell them, and
this is forbidden on the Sabbath, on account of the words, ‘Not
finding thine own pleasure.’ (Isaiah lviii. 13.)” This is plainly a magical
operation, but yet the rabbies do not say that it is unlawful because it
is magical, but because it would be attending to one’s own concerns.
In like manner, they say, it is unlawful on week-days, only on account
of the danger. And an instance is given in Rabbi Isaac of what might
happen; and here, again, we ask counsel of Rashi, in order to
understand what Rabbi Isaac was about. This commentator tells us:
—
‫היה שואל במעשה שדים ובקש השד להזיקו ונעשה לו נס ובלעו הארז ׃‬
“He was asking counsel, by means of a demoniacal operation, and
the demon sought to do him an injury, but a miracle was wrought for
him, and a cedar tree swallowed him.” Such, then, is the context,
those men who permit the charming of serpents, also teach the
doctrine of asking advice of demons, and give us a practical example
in one of their friends. There can, therefore, be no mistake; the one
feature of their religious system exactly agrees with the other: and
the authors of the oral law represent themselves as patrons and
practisers of charms and magic, and therefore to every lover of the
Mosaic law, as unwise and ungodly men. It is, however, curious to
see how they endeavoured to quiet their own conscience, and that of
the people, in a matter so evidently repugnant to the plain words of
Scripture. They pretended, that there was a holy sort of magic in the
practical Cabbala, which men might learn, and then perform the
greatest miracles.
‫אי בעו צדיקי ברו עלמא שנאמר כי עונותיכם היו מבדילים וגו׳ רבא ברא גברא‬
‫שדריה לקמיה דר׳ זירא הוה קא משתעי בהדיה ולא הוה קא מהדר ליה אמר ליה‬
‫מן חבריא את הדר לעפריך רב חנינא ורב אושעיא הוו יתבי כל מעלי שבתא ועסקו‬
‫בספר יצירה ומיברו להו עיגלא תילתא ואכלי ליה ׃‬
“If the righteous wished, they might create the world, for it is written,
‘But your sins separate, &c.’ Rabba created a man, and sent him to
Rabbi Zira. He spoke with him, but when the other did not answer
him, he said, Thou art from the magicians, return to thy dust. Rav
Chanina and Rav Oshaia used to sit every Sabbath eve and study
the book of Jetzirah, and then created for themselves a three-years-
old calf, and ate it.” (Sanhedrin, fol. 65, col. 2.) The second miracle is
here ascribed to the study of a certain book. In Rashi the first miracle
performed by Rabba is ascribed to the same source.
‫ברא גברא ע׳׳י ספר יצירה שמלמדו צרוף אותיות של שם ׃‬
“He created the man by means of the book of Jetzirah, for it taught
him the combination of the letters of the name of God.” According to
this account, these rabbies were much greater men than Moses or
any of the prophets, for in the whole Old Testament there is not one
such miracle recorded. Moses never created any thing, neither did
he perform any of his miracles without the help of God. Either the
Lord immediately commanded him, or he sought the Lord’s help. But
these rabbies acquired the power of omnipotence by studying a
particular book, and exercised it either for their amusement or their
profit. Rabba created a man, and sent him to Rabbi Zira, not as it
appears to do any good, or to glorify God, but simply to show his
power, or to act a little bit of waggery; and the other two created a fat
calf for themselves every Sabbath eve, that they might have a good
dinner. The difference between these miracles and those recorded in
Scripture is obvious. The Scripture miracles are either for the glory of
God, or the good of man. The rabbinical miracles are altogether for
the glory of man, and the gratification of self. Moses smote the rock,
and supplied all Israel with water. The rabbies create a calf, and eat
it themselves. No doubt there were many poor people in Israel at the
time of Rabbies Oshaia and Chanina, who would have been very
glad of a calf for their Sabbath dinner, why did they not create a calf
or two for them? This selfish falsehood betrays itself, and bears on
its front its own condemnation. The whole doctrine of the
combination of the letters in the name of God is a pure invention of
men, whose minds have been debased by superstition. There is not
a word about it in the whole Bible, and it is derogatory to the honour
of God, who is the only Creator.
The whole Talmudic doctrine of magic does, however, explain the
reason why the Scribes and Pharisees were so little moved by the
real miracles of Jesus of Nazareth and his disciples. Their minds
were fully possessed with faith in the power of cabalistic magic, they
therefore were insensible to the real displays of divine power. They
were in the same state of mind as Pharaoh and his magicians, who
looked on the miracles of Moses as a mere proof of magical skill,
and hardened their hearts. Even when they confessed “This is the
finger of God,” they were not converted. Pharaoh still persisted in his
resistance. And so it was with the Scribes and Pharisees. When the
Lord had raised Lazarus from the dead, “then gathered the chief
priests and the Pharisees in council, and said, What do we? for this
man doeth many miracles.” They acknowledged the fact of the
miracles, but did not receive their evidence, for they believed that the
study of the book of Jetzirah would enable them to do greater. No
miracle, therefore, could convince them. But besides this, their
hearts were corrupt, and they had apostatized from the law of
Moses; they therefore did not love the truth. They had turned aside
to charms and magic, and asking counsel of demons; and when men
do this, the understanding becomes darkened, so that it is rendered
impervious to the light. Their unbelief, therefore, becomes an
evidence to the truth of Christianity. If such transgressors of the law
of Moses, and such unblushing relaters of falsehood had believed, it
would have cast a shade of suspicion over the whole Gospel history.
If the men, who say that Rabba created a man, and the two other
worthies created a calf every week, had appeared as witnesses for
the truth of Christianity, the miracles of the Gospel would have
appeared in one category with these most absurd fictions. But when
such men appear as the enemies and persecutors of Jesus, it
testifies that He was not one of them, and that as they were bad
men, and loved a false system, his doctrine must necessarily have
had something good in it, or they would not have opposed it.
But this doctrine explains still more clearly the cause of God’s wrath
against Israel. The Jews boast that since the Babylonian captivity,
they have been free from idolatry, but this is not true. They have not
made images, that is, they have avoided the form, but they have
retained all the substance of idolatrous heathenism. The man who
charms a serpent is an idolater, and the religion which permits it is
idolatrous and heathenish. The man who asks counsel of demons is
an idolater of the worst class, for he does homage to unclean spirits.
He turns his back upon the allwise God, who ought to be the
counsellor of all his children, and by making demons his advisers,
makes them his gods, and yet this is also allowed in the religion of
the rabbies if it can be done without danger. Those Jews, therefore,
who believe in the oral law—that is, all Jews who make use of the
synagogue prayers, have departed from the law and the God of
Moses, and have chosen for themselves the doctrines and the gods
of the rabbies. How then can God have compassion upon them and
gather them? The thing is impossible, until they utterly renounce all
these delusions, confess their sin in having followed them so long,
and “return and seek the Lord their God and David their king.” A long
trial has been made of the rabbinical medicine, and it has altogether
failed. Wherever the religion of the oral law has been or is
predominant, its sway has been marked by the misery of the people.
And the first dawn of a happier day has appeared only since the time
that a part of the nation burst the fetters of rabbinic superstition.
Compare the state of the German Jews with that of their brethren in
Turkey or on the coast of Morocco. Some of the former have
abandoned the oral law, and the latter still cling to it with a bigoted
devotion; and yet the former have had a blessing in the improvement
of their temporal and intellectual condition, and the latter still remain
in mental and corporeal slavery. The mere renunciation of Rabbinism
has produced these beneficial effects, and if the Jews of Europe go
on from the renunciation of error to the attainment of truth, that is, if
they return to the religion of Moses and the prophets, the promises
of God will be fulfilled, and the nation will be restored to the land of
their fathers.
The Rabbinic Jews comfort themselves with the idea, that they
cannot have this world and the world to come too; but they confound
two things which are perfectly distinct, God’s mode of dealing with
individuals, and his mode of dealing with nations. Individuals have
not only an existence in time, but for eternity. Worldly misfortune to
an individual is, therefore, no proof of God’s displeasure, because
the world is only a part, and that the smallest part, of his existence.
But the case of nations is different. They exist only in time, and
therefore the rewards and punishments must be temporal, and so
God has uniformly promised to the Jewish people temporal
prosperity, in case of national obedience, and temporal calamity in
the former case. Whenever, therefore, we see Israel exiled from their
land and scattered among the nations, we must infer, if Moses has
spoken the truth, that it is because they have departed from the God
of their fathers.
 No. XXVII.
SABBATIC LAWS.

How little the oral law has hitherto done to promote the peace and
happiness of Israel, we considered in our last number. It may,
however, be replied, that it has not had a fair trial, and that the failure
is to be attributed rather to the people than to the law. This possible
reply naturally leads us to think, what then would be the state of
Israel and of the world at large, if the oral law were universally and
exactly observed, and its disciples had supreme dominion in the
world? Suppose that all the kingdoms of the world were melted into
one vast and universal monarchy, and the sceptre swayed by a
devout and learned rabbi, and all the magisterial offices filled by able
and zealous Talmudists, would the world be happy? This is a fair
question, and well deserves consideration, for there can be no doubt
that true religion was intended by its Divine Author to promote the
happiness of his creatures:—
‫דוכיה דרכי נועם וכל נתיבוביה שלום ׃‬
“Her ways are ways of pleasantness, and all her paths are peace.”
(Prov. iii. 17.) And that not of a few, but of all without exception.
‫ הלא אל אחד בראנו ׃‬, ‫הלא אב אחד לכלנו‬
“Have we not all one father? Hath not one God created us?” (Mal. ii.
10.) That religion, therefore, cannot be of God, which would make
the greatest portion of his creatures miserable, and confer happiness
on a very limited number. The religion that came from heaven,
wherever it exists, must contain the elements of happiness for all
nations, and include all the families of man. It must exclude none but
the wilfully and obstinately wicked, who carry the torments of hell in
their own bosom, and would be necessarily unhappy even in heaven
itself. A religion, whose principles, if triumphant, would effect so
desirable a consummation, must be true. The question is, whether
modern Judaism, if it had full and free scope for the realization of all
its principles, would bear such blessed fruit? Our late inquiries about
amulets and magic led us to consider some of the laws about the
Sabbath-day, and as when true religion prevails, this ought to be the
happiest day of the week, the laws respecting it shall furnish
materials for our answer. That a rabbinical Sabbath would be the
happiest day in the week we much doubt, for, in the first place, to
keep the rabbinical Sabbath aright, it is necessary to be perfectly
acquainted with all the laws relating to it, which are very many and
very intricate, occupying even in Rambam’s compendium, including
the notes, above one hundred and seventy folio pages.[27] That any
conscientious man can be happy with such a load of law about his
neck appears impossible. He must be in continual fear and trembling
lest he should through forgetfulness or inadvertence be guilty of
transgression, and the continued watchfulness and anxiety would be
more intolerable than the hardest labour. But if Rabbinism wielded
the supreme power, he would have to dread the most severe and
immediate punishment:—
‫ וכל העושה‬, ‫שביתה בשביעי ממלאכה מצות עשה שנאמר וביום השביעי תשבות‬
, ‫בו מלאכה ביטל מצות עשה ועבר על לא תעשה שנאמר לא תעשה כל מלאכה‬
‫ומהו חייב על עשיית מלאכה אם עשה ברצונו בזדון חייב כרת ואם היה שם עדים‬
‫והתראה נסקל ואם עשה בשגגה חייב קרבן הטאת קבועה ׃‬
“To rest on the seventh day from work is an affirmative precept, for it
is said, ‘On the seventh day thou shalt rest.’ Whosoever, therefore,
does any work, annuls an affirmative, and transgresses a negative
precept, for it is said, ‘Thou shalt do no manner of work.’ What is
meant by being guilty on account of doing work? If it be done
voluntarily and presumptuously, the meaning is, that he is liable to
excision, and if there were witnesses and a warning, he is to be
stoned. If he did it in error, he must bring a certain sin-offering.”
(Hilchoth Shabbath, c. i. 1.) This sounds something like the law of
Moses, but is in reality far more severe. The whole force depends
upon the meaning of the word “work,” and the rabbinical sense
would entirely destroy the peace of society. If, for instance, a poor
man could not afford to have his Sabbath lamp burn all day, and
should extinguish it to save the oil; or if a humane man should see
burning coals in some place likely to do injury to others, and should
extinguish them, they would both be guilty, and if some zealous
Talmudists happened to be present, and first remonstrated with them
on the unlawfulness of the act, they would both be tried, found guilty,
and stoned to death:—
‫ כיצד‬, ‫כל העושה מלאכה בשבת אע׳׳פ שאינו צריך לגופה של מלאכה חייב עליה‬
‫הרי שכבה את הנר מפני שהוא צריך לשמן או לפתילה כדי שלא יאבד או כדי שלא‬
‫ישרף או כדי שלא יבקע חרס של נר מפני שהכבוי מלאכה והרי נתכוון לכבות‬
‫ואע׳׳פ שאין צריך לגוף הכבוי ולא כבה אלא מפני השמן או מפני החרס או מפני‬
‫ וכן המעביר את הקוץ ד׳ אמות ברה׳׳ר או המכבה את‬, ‫הפתילה הרי זה חייב‬
‫הגחלת כדי שלא יזוקו בו רבים חייב ואע׳׳פ שאינו צריך לגוף הכבוי או לגוף‬
‫חהעברה אלא להרחיק ההיזק הרי זה חייב וכן כל כיוצא בזה ׃‬
“Whosoever does any work on the Sabbath, even though he does
not do it for the sake of the work itself, is nevertheless guilty. How
so? If, for instance, a man extinguishes a lamp, because he wants
the oil or the wick, and wishes that it should not waste, nor be
burned, or that the earthenware part of the lamp should not be
cracked; inasmuch as the extinguishing is work, and his intention
was to extinguish it: although the mere act of extinguishing it was not
the ultimate object, but on the contrary, the saving of the oil or the
wick, or the earthen lamp, he is, nevertheless, guilty. And in like
manner, whosoever, removes thorns a distance of four ells in a
public place, or whosoever extinguishes coals to prevent the public
from being injured, is guilty: although the ultimate object was not the
extinguishing nor the moving, but he simply intended to prevent the
injury, he is guilty, and so in all similar cases.” (Ibid.) If this were the
law of the land, and the executive were in the hands of Talmudistic
zealots, the peace of the world would be at an end. The poor man
could not be happy, when he saw his little property wasting; and the
humane man would either be made miserable at the thought of being
able to prevent much injury, and yet not doing it, or would have to
expose himself to the danger of a cruel and ignominious death. We
know enough of the general character of the Jewish nation to believe
that there are amongst them those who would brave the danger,
whose generous hearts would rise above personal considerations,
but how dreadful would be the consequences! A man of a tender
heart, the father of a family, would be induced, by the best of
feelings, to save his fellow-men from injury. He would return to his
family, and tell them how God had given him an opportunity of doing
good. The family worthy of such a father would rejoice to hear the
information, but the sequel of his story would turn their joy into
mourning. He would have to tell them that ignominious death would
be the consequence, and that because he dared to do an act of
charity, and to love his brother as himself, the morrow would see his
wife a widow and his children orphans. But suppose, that when he
performed the act, he had been attended by two of his sons, now
grown up, and zealots for the oral law—that they had warned him,
and then became his accusers, as they must, if firm believers in
Talmudic religion, he would have the additional pangs of seeing his
own flesh and blood as the foremost of his executioners. This one
law would clothe the world with mourning, and make the light of the
Sabbath sun the curse of mankind. Though men might be found at
first to brave the danger, the course of time and the inflexible severity
of the law would soon annihilate all generous feeling. Children would
be trained up with the idea that humanity is not a Sabbath virtue, and
the constant resistance of the tender feelings would harden the
heart, and mankind in time become totally insensible on week-days
as well as Sabbath-days; and thus the enforcement of this one law
would produce universal selfishness, and this would certainly not
promote the happiness of the world. But take another case of a man,
who leaves his home on the Friday morning to go a short distance
into the country, intending to return before the commencement of the
Sabbath; he meets with an accident, and breaks a limb; on the
Sabbath he is sufficiently restored to think of the anxiety of his family,
and writes a short note to inform them of his state, this act of
common love and kindness would cost him his life; nay, if he had
only begun the letter, and then overcome by fear or weakness, had
left it unfinished, a rabbinic tribunal would condemn him to be
stoned.
‫כל המתכוון לעשות מלאכה בשבת והתחיל בה ועשה כשיעור חייב אע׳׳פ שלא‬
‫ כיצד הרי שנתכוון לכתוב אגרת או שטר‬, ‫השלים כל המלאכה שנתכוון להשלימה‬
‫בשבת אין אומרים לא יתחייב זה עד שישלים חפצו ויכתות כל השטר או כל‬
‫האגרת אלא משיכתוב שתי אותיות חייב ׃‬
“Whosoever intends to do any work on the Sabbath, and begins it,
and does a certain measure, is guilty, although he does not finish all
that he intended. How so? Suppose he intended to write a letter, or a
contract on the Sabbath, it is not to be thought that he will not be
guilty until he finish his business, and write the whole contract or the
whole letter. On the contrary, as soon as he shall have written two
letters (of the alphabet) he is guilty.” (Ibid.) And consequently, if it can
be proved, must be stoned. Every one’s daily experience will tell
them of the many similar cases where a letter may be necessary for
the peace or well-being of an individual or a family, and where the
delay of a day would be a serious injury. If rabbinism held the reins
of power, the anxiety, the sorrow, the injury must all be endured; the
Sabbath-day must be made a burden and a curse, instead of a
blessing, or life itself must be exposed to danger. But this would not
be the only misery. These sanguinary laws would, as religious laws,
bind the consciences of the weak and superstitious. A man’s
domestics, or his children, or even his wife, would become spies
over all his Sabbath doings, and the denouncers of every
transgression; and thus domestic confidence, without which not even
the shadow of happiness can exist, would be destroyed, and a man’s
foes would be those of his own household. Much has lately been
thought and said about the sanguinary nature of the laws of England,
but the laws of Draco himself were merciful when compared with the
religious enactments of the rabbies. Draco only sentenced to death
men convicted of a crime. The oral law condemns to stoning the
man, woman, or child who will venture to write two letters of the
alphabet, or even who will extinguish fire to prevent a public injury.
Nay, in some cases, where it actually pronounces a man innocent, it
nevertheless commands him to be flogged.
‫נתכוון ללקוט תאנים שחורות וליקט לבנות או שנתכוון ללקוט תאנים ואחר כך‬
‫ענבים ונהפך הדבר וליקט הענבים בתחלה ואח׳׳כ תאנים פטור אע׳׳פ שליקט כל‬
‫מה שחשב הואיל ולא ליקט כסדר שחשב פטור שבלא כוונה עשה שלא אסרה‬
‫התורה אלא מלאכת מחשבת ׃‬
“If a man intended to gather black figs, but gathered white figs, or if
he intended to gather figs and afterwards grapes, but the matter has
been inverted, and he gathered the grapes first, and afterwards the
figs, he is not guilty. Although he have gathered all that he thought of
gathering, yet, because he did not gather them in the intended order,
he is not guilty, for he did what was unintentional, and the law forbids
only intentional work.” (Ibid.) We pass by the manifest absurdity of
this decision, which is, however, sufficient to prove that this law is not
of God, because it is more important to consider what is to be done
with a man not guilty. The law of England, or any other civilized
country, would say, of course, that he is to go free; but not so the oral
law, it commands that the man should be flogged.
‫ הרי זה פטור מן הכרת ומן הסקילה‬, ‫וכל מקום שנאמר שהעושה דבר זה פטור‬
‫ומן הקרבן אבל אסור לעשות אותו דבר בשבת ואיסורו מדברי סופרים והוא‬
‫הרחקה מן המלאכה והעושה אותו בזדון מכין אותו מכות מרדות ׃‬
“Wherever it is said, he that doeth anything is not guilty, the meaning
is, that he is not liable to excision, nor stoning, nor a sacrifice, but
that thing is unlawful to be done, and the prohibition is of the words
of the Scribes, and is intended as a removal from the possibility of
work: and he that does it presumptuously, is to be flogged with the
flogging of rebellion.” (Ibid.) Here, then, we have a whole class of
crimes which the oral law itself allows are no crimes according to the
law of Moses, but which it thinks fit to punish with that dreadful and
degrading infliction. Are the professors of this traditional religion
really acquainted with its ordinances? or can any man believe that a
religion which, if it had full scope and power, would become the
torment of the human race, can emanate from God?
If ever this religion attains supreme power, its adherents will be
reduced to a state of the most deplorable bondage, but what would
be its effect upon the other nations of the world? It would, in the first
place, deprive all other nations of a Sabbath; for we have already
quoted the law (No. 3, p. 22), which decides, “That a Gentile who
keeps a Sabbath, though it be on one of the week-days is guilty of
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebooknice.com