glasses.utils.weights.storage.hubs package

Submodules

glasses.utils.weights.storage.hubs.HFModelHub module

class glasses.utils.weights.storage.hubs.HFModelHub.HFModelHub[source]

Bases: object

static from_pretrained(pretrained_model_name_or_path: Optional[str], strict: bool = True, map_location: Optional[str] = 'cpu', force_download: bool = False, resume_download: bool = False, proxies: Optional[Dict] = None, use_auth_token: Optional[str] = None, cache_dir: Optional[str] = None, local_files_only: bool = False) → Dict[str, torch.Tensor][source]

Instantiate a pretrained pytorch model from a pre-trained model configuration from huggingface-hub. The model is set in evaluation mode by default using model.eval() (Dropout modules are deactivated). To train the model, you should first set it back in training mode with model.train().

Parameters

  • pretrained_model_name_or_path (str or os.PathLike, optional) –

    Can be either:

    • A string, the model id of a pretrained model hosted inside a model repo on huggingface.co. Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a user or organization name, like dbmdz/bert-base-german-cased.

    • You can add revision by appending @ at the end of model_id simply like this: dbmdz/bert-base-german-cased@main Revision is the specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so revision can be any identifier allowed by git.

    • A path to a directory containing model weights saved using save_pretrained(), e.g., ./my_model_directory/.

    • None if you are both providing the configuration and state dictionary (resp. with keyword arguments config and state_dict).

  • cache_dir (Union[str, os.PathLike], optional) – Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used.

  • force_download (bool, optional, defaults to False) – Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist.

  • resume_download (bool, optional, defaults to False) – Whether or not to delete incompletely received files. Will attempt to resume the download if such a file exists.

  • proxies (Dict[str, str], `optional) – A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.

  • local_files_only (bool, optional, defaults to False) – Whether or not to only look at local files (i.e., do not try to download the model).

  • use_auth_token (str or bool, optional) – The token to use as HTTP bearer authorization for remote files. If True, will use the token generated when running transformers-cli login (stored in huggingface).

  • model_kwargs (Dict, optional) – model_kwargs will be passed to the model during initialization

Note

Passing use_auth_token=True is required when you want to use a private model.

static push_to_hub(save_directory: Optional[str], model_id: Optional[str] = None, repo_url: Optional[str] = None, commit_message: Optional[str] = 'add model', organization: Optional[str] = None, private: Optional[bool] = None) → str[source]
Parameters

  • save_directory (Union[str, os.PathLike]) – Directory having model weights & config.

  • model_id (str, optional, defaults to save_directory) – Repo name in huggingface_hub. If not specified, repo name will be same as save_directory

  • repo_url (str, optional) – Specify this in case you want to push to existing repo in hub.

  • organization (str, optional) – Organization in which you want to push your model.

  • private (bool, optional) – private: Whether the model repo should be private (requires a paid huggingface.co account)

  • commit_message (str, optional, defaults to add model) – Message to commit while pushing

Returns

url to commit on remote repo.

static save_pretrained(model: torch.nn.modules.module.Module, save_directory: str, config: Optional[dict] = None, push_to_hub: bool = False, **kwargs)[source]

Saving weights in local directory.

Parameters

  • save_directory (str) – Specify directory in which you want to save weights.

  • config (dict, optional) – specify config (must be dict) incase you want to save it.

  • push_to_hub (bool, optional, defaults to False) – Set it to True in case you want to push your weights to huggingface_hub

  • model_id (str, optional, defaults to save_directory) – Repo name in huggingface_hub. If not specified, repo name will be same as save_directory

  • kwargs (Dict, optional) – kwargs will be passed to push_to_hub

Module contents