23 lines
669 B
ReStructuredText
23 lines
669 B
ReStructuredText
models.gpt package
|
|
==================
|
|
This is the implementation of the popular GPT model. It supports several features like model parallelization (Tensor Parallel, Pipeline Parallel, Data Parallel) , mixture of experts, FP8 , Distributed optimizer etc. We are constantly adding new features. So be on the lookout or raise an issue if you want to have something added.
|
|
|
|
Submodules
|
|
----------
|
|
|
|
models.gpt.gpt\_model module
|
|
----------------------------
|
|
|
|
.. automodule:: core.models.gpt.gpt_model
|
|
:members:
|
|
:undoc-members:
|
|
:show-inheritance:
|
|
|
|
Module contents
|
|
---------------
|
|
|
|
.. automodule:: core.models.gpt
|
|
:members:
|
|
:undoc-members:
|
|
:show-inheritance:
|