DeepSpeed
latest
Training Setup
DeepSpeed Configuration
Training API
Model Checkpointing
Activation Checkpointing
Transformer Kernels
Pipeline Parallelism
DeepSpeed
Docs
»
Index
Edit on GitHub
Index
A
|
B
|
C
|
D
|
E
|
F
|
G
|
I
|
L
|
M
|
N
|
O
|
P
|
R
|
S
|
T
A
add_config_arguments() (in module deepspeed)
allreduce_tied_weight_gradients() (deepspeed.pipe.PipelineModule method)
B
backward() (deepspeed.runtime.pipe.engine.PipelineEngine method)
(in module deepspeed.DeepSpeedEngine)
BackwardPass (class in deepspeed.runtime.pipe.schedule)
batch (deepspeed.config.TrainingConfig attribute)
BatchConfig (class in deepspeed.config)
BufferOpInstruction (class in deepspeed.runtime.pipe.schedule)
build() (deepspeed.pipe.LayerSpec method)
C
checkpoint() (in module deepspeed.checkpointing)
CheckpointFunction (class in deepspeed.checkpointing)
ckpt_layer_path() (deepspeed.pipe.PipelineModule method)
ckpt_prefix() (deepspeed.pipe.PipelineModule method)
clip (deepspeed.config.FP16Config attribute)
,
[1]
Config (class in deepspeed.config)
configure() (in module deepspeed.checkpointing)
CudaRNGStatesTracker (class in deepspeed.checkpointing)
D
DataParallelSchedule (class in deepspeed.runtime.pipe.schedule)
deepspeed.runtime.pipe.engine (module)
deepspeed.runtime.pipe.schedule (module)
DeepSpeedCPUAdam (class in deepspeed.ops.adam)
,
[1]
DeepSpeedTransformerConfig (class in deepspeed)
DeepSpeedTransformerLayer (class in deepspeed)
E
enabled (deepspeed.config.FP16Config attribute)
,
[1]
eval_batch() (deepspeed.runtime.pipe.engine.PipelineEngine method)
F
forward() (deepspeed.runtime.pipe.engine.PipelineEngine method)
(in module deepspeed.DeepSpeedEngine)
ForwardPass (class in deepspeed.runtime.pipe.schedule)
fp16 (deepspeed.config.TrainingConfig attribute)
FP16Config (class in deepspeed.config)
,
[1]
G
get_cuda_rng_tracker() (in module deepspeed.checkpointing)
gradient_accumulation_steps (deepspeed.config.BatchConfig attribute)
I
InferenceSchedule (class in deepspeed.runtime.pipe.schedule)
init_distributed() (in module deepspeed)
initialize() (in module deepspeed)
is_configured() (in module deepspeed.checkpointing)
is_first_stage (deepspeed.runtime.pipe.schedule.PipeSchedule attribute)
is_first_stage() (deepspeed.runtime.pipe.engine.PipelineEngine method)
is_gradient_accumulation_boundary() (deepspeed.runtime.pipe.engine.PipelineEngine method)
(in module deepspeed.DeepSpeedEngine)
is_last_stage (deepspeed.runtime.pipe.schedule.PipeSchedule attribute)
is_last_stage() (deepspeed.runtime.pipe.engine.PipelineEngine method)
is_valid() (deepspeed.config.BatchConfig method)
(deepspeed.config.Config method)
L
LayerSpec (class in deepspeed.pipe)
load_checkpoint() (in module deepspeed.DeepSpeedEngine)
load_module_state_dict() (deepspeed.runtime.pipe.engine.PipelineEngine method)
LoadMicroBatch (class in deepspeed.runtime.pipe.schedule)
M
model_parallel_cuda_manual_seed() (in module deepspeed.checkpointing)
module_state_dict() (deepspeed.runtime.pipe.engine.PipelineEngine method)
N
num_micro_batches (deepspeed.runtime.pipe.schedule.PipeSchedule attribute)
num_pipe_buffers() (deepspeed.runtime.pipe.schedule.DataParallelSchedule method)
(deepspeed.runtime.pipe.schedule.InferenceSchedule method)
(deepspeed.runtime.pipe.schedule.PipeSchedule method)
(deepspeed.runtime.pipe.schedule.TrainSchedule method)
num_stages (deepspeed.runtime.pipe.schedule.PipeSchedule attribute)
O
OptimizerStep (class in deepspeed.runtime.pipe.schedule)
P
PipeInstruction (class in deepspeed.runtime.pipe.schedule)
PipelineEngine (class in deepspeed.runtime.pipe.engine)
PipelineModule (class in deepspeed.pipe)
PipeSchedule (class in deepspeed.runtime.pipe.schedule)
R
RecvActivation (class in deepspeed.runtime.pipe.schedule)
RecvGrad (class in deepspeed.runtime.pipe.schedule)
ReduceGrads (class in deepspeed.runtime.pipe.schedule)
ReduceTiedGrads (class in deepspeed.runtime.pipe.schedule)
reset() (in module deepspeed.checkpointing)
resolve() (deepspeed.config.BatchConfig method)
(deepspeed.config.Config method)
S
save_checkpoint() (in module deepspeed.DeepSpeedEngine)
SendActivation (class in deepspeed.runtime.pipe.schedule)
SendGrad (class in deepspeed.runtime.pipe.schedule)
set_batch_fn() (deepspeed.runtime.pipe.engine.PipelineEngine method)
set_dataiterator() (deepspeed.runtime.pipe.engine.PipelineEngine method)
stage (deepspeed.runtime.pipe.schedule.PipeSchedule attribute)
step() (deepspeed.runtime.pipe.engine.PipelineEngine method)
(in module deepspeed.DeepSpeedEngine)
steps() (deepspeed.runtime.pipe.schedule.PipeSchedule method)
T
TiedLayerSpec (class in deepspeed.pipe)
topology() (deepspeed.pipe.PipelineModule method)
train_batch() (deepspeed.runtime.pipe.engine.PipelineEngine method)
train_batch_size (deepspeed.config.BatchConfig attribute)
train_micro_batch_size_per_gpu (deepspeed.config.BatchConfig attribute)
TrainingConfig (class in deepspeed.config)
TrainSchedule (class in deepspeed.runtime.pipe.schedule)
Read the Docs
v: latest
Versions
latest
Downloads
pdf
On Read the Docs
Project Home
Builds
Free document hosting provided by
Read the Docs
.