Search notes:

Python library: transformers

pip install transformers

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained(       "EleutherAI/gpt-j-6B")
model     = AutoModelForCausalLM.from_pretrained("EleutherAI/gpt-j-6B")

Standard classes

A model is required to use three classes derived from
Instances of these classes are created by calling from_pretrained(). This method downloads (if necessary), caches and loads the followin related data from a pretrained checkpoint provided on Hugging Face Hub/

«Base» classes

PreTrainedModel

PreTrainedModel is the base class for all models and implements the methods for loading and saving a model.
PreTrainedModel immediatly derives from: 
>>> import transformers
>>> for parent in transformers.PreTrainedModel.__bases__:
...     print(f'{parent.__name__:<20} {parent.__module__}')
...
Module               torch.nn.modules.module
ModuleUtilsMixin     transformers.modeling_utils
GenerationMixin      transformers.generation.utils
PushToHubMixin       transformers.utils.hub

TFPreTrainedModel

Same thing, but for TFPreTrainedModel: 
>>> import transformers
>>> for parent in transformers.TFPreTrainedModel.__bases__:
...     print(f'{parent.__name__:<20} {parent.__module__}')
...
Model                keras.engine.training
TFModelUtilsMixin    transformers.modeling_tf_utils
TFGenerationMixin    transformers.generation.tf_utils
PushToHubMixin       transformers.utils.hub

FlaxPreTrainedModel

Same thing, but for FlaxPreTrainedModel: 
>>> import transformers
>>> for parent in transformers.FlaxPreTrainedModel.__bases__:
...     print(f'{parent.__name__:<20} {parent.__module__}')
...
object               builtins

PreTrainedConfig

PreTrainedConfig is the base class that implements methods for loading and saving configurations.
Common attributes in derived classes include

GPT2LMHeadModel

GPT2LMHeadModel inherits from GPT2PreTrainedModel which inherits from PreTrainedModel: 
>>> import transformers
>>> for baseClass in transformers.GPT2LMHeadModel.__bases__:
...     print(f'{baseClass.__name__:<20} {baseClass.__module__}')
...
GPT2PreTrainedModel  transformers.models.gpt2.modeling_gpt

>>> for baseClass in transformers.GPT2PreTrainedModel.__bases__:
...     print(f'{baseClass.__name__:<20} {baseClass.__module__}')
...
PreTrainedModel      transformers.modeling_utils

from_pretrained()

…

Index