russell@unturf.com
81bef63b2e
transformers-0002: MOAD-0004 (CWE-312) regnet convert script logs HF_TOKEN verbatim transformers-0003: MOAD-0001 (CWE-407) convert_tokens_to_string O(T×S) list scan - marian, m2m_100, speech_to_text, siglip, gpt_sw3 all affected - all_special_tokens is list[str]; fix: cache set() before loop; 5x speedup vllm-0002: MOAD-0001 (CWE-407) Grok2Tokenizer O(N×V) dict.values() scan - decode() and convert_ids_to_tokens() use .values() view per token - sibling Mistral tokenizer already uses frozenset correctly - fix: add _special_token_ids frozenset at __init__; 10x speedup at N=2048, V=200 MOADs 0002/0003/0005 CLEAN for both repos
35 lines
1.8 KiB
Diff
35 lines
1.8 KiB
Diff
# transformers-0002: convert_regnet_seer_10b_to_pytorch.py logs HF_TOKEN verbatim
|
|
# CWE-312 — Cleartext Storage of Sensitive Information (credential in log)
|
|
# MOAD-0004 — The Logged Secret
|
|
#
|
|
# In convert_regnet_seer_10b_to_pytorch.py, when push_to_hub=True, the script
|
|
# logs the HuggingFace authentication token verbatim:
|
|
#
|
|
# logger.info(f"Token is {os.environ['HF_TOKEN']}")
|
|
#
|
|
# This exposes the secret token to any logging system, stdout capture, CI log
|
|
# archive, or anyone with access to the log output. The token grants full write
|
|
# access to the HuggingFace Hub account, including publishing new model revisions.
|
|
#
|
|
# Fix: remove the log line entirely. The token is not needed for diagnostics;
|
|
# the surrounding log messages already indicate the push_to_hub path.
|
|
# If a presence check is needed, log a redacted placeholder instead.
|
|
#
|
|
# Severity: MEDIUM — convert scripts are run by maintainers, but CI logs and
|
|
# shared environments may capture them. Token is a long-lived secret with
|
|
# push access to public/private model repos.
|
|
#
|
|
# File: src/transformers/models/regnet/convert_regnet_seer_10b_to_pytorch.py
|
|
# Function: convert_weights_and_push
|
|
# Line: 236
|
|
--- a/src/transformers/models/regnet/convert_regnet_seer_10b_to_pytorch.py
|
|
+++ b/src/transformers/models/regnet/convert_regnet_seer_10b_to_pytorch.py
|
|
@@ -233,7 +233,7 @@ def convert_weights_and_push(save_directory: Path, model_name: str | None = Non
|
|
else:
|
|
logger.info("The state_dict was already stored on disk.")
|
|
if push_to_hub:
|
|
- logger.info(f"Token is {os.environ['HF_TOKEN']}")
|
|
+ logger.info("Pushing model to the Hub (token loaded from HF_TOKEN env var).")
|
|
logger.info("Loading our model.")
|
|
# create our model
|
|
our_config = names_to_config[model_name]
|