Skip to content

Hugging Face

Hugging Face Inc.

一个 AI 社区平台，提供模型、数据集、应用和完整的工具链。

Official Site
01

概述

Hugging Face 是全球最大的 AI 社区平台，被誉为"AI 界的 GitHub"。它提供模型库（500k+ 模型）、数据集库（100k+ 数据集）、应用 Spaces 等。Transformers 库是最受欢迎的 NLP 库，支持 BERT、GPT、Llama、Mistral 等 100+ 模型。Hugging Face 还提供 Tokenizers、Diffusers、PEFT、Accelerate 等工具，覆盖整个 AI 开发工作流。无论是研究还是生产，Hugging Face 都是 AI 开发者的核心平台。

02

模型库

Models Hub 是 Hugging Face 的核心，托管 500k+ 模型。涵盖 NLP、计算机视觉、语音、多模态等领域。每个模型都有模型卡片，描述其用途、性能、限制等。支持按任务、语言、许可证等筛选。模型可一键下载使用。

bash
# Browse models
# https://huggingface.co/models

# Download a model
from huggingface_hub import snapshot_download
snapshot_download(repo_id="meta-llama/Llama-3.1-8B")

# Use the CLI
huggingface-cli download meta-llama/Llama-3.1-8B

# Upload a model
huggingface-cli upload my-model ./model_files
03

数据集

Datasets Hub 托管 100k+ 数据集，涵盖文本、图像、音频等。提供统一的数据加载接口，支持大数据集的流式加载。数据集有数据集卡片，描述其内容、格式、许可证等。支持数据集可视化预览。

bash
# Load a dataset
from datasets import load_dataset

# Load well-known datasets
dataset = load_dataset("squad")
dataset = load_dataset("imdb")

# Stream loading (large datasets)
dataset = load_dataset("oscar", streaming=True)
for example in dataset:
    print(example)
    break

# Upload a dataset
from huggingface_hub import HfApi
api = HfApi()
api.upload_folder(folder_path="./data", repo_id="my-dataset", repo_type="dataset")
04

Transformers 库

Transformers 是 Hugging Face 的核心库，提供 100+ 预训练模型。支持文本分类、生成、翻译、问答等任务。提供统一且易用的 API。支持 PyTorch、TensorFlow、JAX 后端。

bash
from transformers import pipeline

# Text classification
classifier = pipeline("sentiment-analysis")
result = classifier("I love AI!")

# Text generation
generator = pipeline("text-generation", model="gpt2")
text = generator("Once upon a time", max_length=50)

# Question answering
qa = pipeline("question-answering")
answer = qa(question="Who invented AI?", context="...")

# Use a specific model
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B")
05

Tokenizers

Tokenizers 库提供高性能的分词工具。支持 BPE、WordPiece、SentencePiece 等算法。用 Rust 实现，速度极快。支持训练自定义分词器、保存和加载。

bash
from tokenizers import Tokenizer
from tokenizers.models import BPE
from tokenizers.trainers import BpeTrainer
from tokenizers.pre_tokenizers import Whitespace

# Train a tokenizer
tokenizer = Tokenizer(BPE(unk_token="[UNK]"))
trainer = BpeTrainer(special_tokens=["[UNK]", "[CLS]", "[SEP]", "[PAD]", "[MASK]"])
tokenizer.pre_tokenizer = Whitespace()
tokenizer.train(files=["data.txt"], trainer=trainer)

# Use it
output = tokenizer.encode("Hello, world!")
print(output.tokens)

# Save
tokenizer.save("tokenizer.json")
06

Spaces

Spaces 是 Hugging Face 的应用托管平台。可以部署 ML 应用、演示和交互式可视化。支持 Gradio、Streamlit、Docker 等框架。提供免费 CPU/GPU 资源，适合展示和分享 ML 应用。

bash
# Gradio Space example
import gradio as gr
from transformers import pipeline

classifier = pipeline("sentiment-analysis")

def predict(text):
    return classifier(text)[0]

demo = gr.Interface(fn=predict, inputs="text", outputs="json")
demo.launch()

# Create a Space
# 1. Visit https://huggingface.co/new-space
# 2. Select an SDK (Gradio/Streamlit/Docker)
# 3. Upload code
# 4. Auto deploy
07

推理 API

Hugging Face 提供推理 API，允许你无需部署即可使用模型。免费版有速率限制，付费版提供更高配额。还提供 Inference Endpoints，允许你部署专用推理服务。支持多种任务类型。

bash
# Use the Inference API
import requests

API_URL = "https://api-inference.huggingface.co/models/distilbert-base-uncased-finetuned-sst-2-english"
headers = {"Authorization": "Bearer YOUR_API_TOKEN"}

response = requests.post(API_URL, headers=headers, json={
    "inputs": "I love AI!"
})
print(response.json())

# Use huggingface_hub
from huggingface_hub import InferenceClient
client = InferenceClient()
result = client.text_classification("I love AI!")
08

微调

Hugging Face 提供完整的微调工具链。PEFT 库支持 LoRA、QLoRA 等高效微调方法。TRL 库支持 RLHF、DPO 等对齐训练。Accelerate 库简化分布式训练。Transformers Trainer 提供统一的训练接口。

bash
# Fine-tune with PEFT
from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM, TrainingArguments, Trainer

model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")
peft_config = LoraConfig(
    r=8, lora_alpha=32, lora_dropout=0.1,
    target_modules=["q_proj", "v_proj"]
)
model = get_peft_model(model, peft_config)

# Train
training_args = TrainingArguments(output_dir="./results", num_train_epochs=3)
trainer = Trainer(model=model, args=training_args, train_dataset=dataset)
trainer.train()
09

部署

Hugging Face 提供多种部署方案。Inference Endpoints 提供托管推理服务，支持自动扩展。Transformers.js 支持在浏览器中运行模型。支持导出为 ONNX、TensorRT 等格式以部署到生产环境。

bash
# Create an Inference Endpoint
from huggingface_hub import HfApi
api = HfApi()
api.create_inference_endpoint(
    name="my-endpoint",
    repository="meta-llama/Llama-3.1-8B",
    framework="pytorch",
    accelerator="gpu",
    instance_type="nvidia-a10g",
    region="us-east-1"
)

# Export to ONNX
from transformers import AutoModelForSequenceClassification
from optimum.onnxruntime import ORTModelForSequenceClassification
model = ORTModelForSequenceClassification.from_pretrained("model", export=True)
10

社区

Hugging Face 拥有活跃的社区。通过 Discord、GitHub Discussions 和论坛交流。定期举办模型竞赛、黑客松、在线分享会等活动。社区贡献模型、数据集和应用，推动 AI 民主化。Hugging Face 还提供课程、教程、博客等学习资源。

11

Configuration

Hugging Face is configured through environment variables, the huggingface-cli, and library-level settings. HF_TOKEN authenticates gated-model downloads and the Inference API; HF_HOME sets the cache root (default ~/.cache/huggingface); and transformers.from_pretrained accepts local paths, repo ids, and revision pins. Spaces are configured through a README.md front-matter (SDK, hardware, secrets) and a requirements.txt. The Hub supports private repos, organizations, and access tokens with scoped permissions.

bash
# Environment variables
export HF_TOKEN=hf_xxx                 # authenticate gated models & API
export HF_HOME=/data/hf-cache          # move the model cache off the home disk
export HF_HUB_DOWNLOAD_TIMEOUT=60      # per-request timeout

# CLI
huggingface-cli login --token $HF_TOKEN
huggingface-cli whoami
huggingface-cli download meta-llama/Llama-3.1-8B --local-dir ./llama

# Python - pin a revision and pick a cache dir
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B",
    revision="refs/pr/42",
    cache_dir="/data/hf-cache",
    token=os.environ["HF_TOKEN"]
)

# Spaces config (README.md front-matter)
# ---
# title: My Space
# sdk: gradio
# hardware: a10g-small
# app_file: app.py
# ---

Use 'huggingface-cli scan-cache' to list cached models and free disk space with 'huggingface-cli delete-cache'.

12

FAQ

Common questions cover gated models, cache location, private repos, Spaces billing, and the Inference API. Gated models require accepting a license on the model page before download, then a HF_TOKEN. The cache lives under ~/.cache/huggingface by default and can grow large. Spaces have free CPU tiers and paid GPU tiers billed per hour. The Inference API returns JSON and is rate-limited per token.

bash
Q: How do I download a gated model?
A: Visit the model page, accept the license, then run
   'huggingface-cli login' and 'huggingface-cli download <repo>'.

Q: Where is the model cache?
A: Under ~/.cache/huggingface by default. Change it with HF_HOME or
   cache_dir= in from_pretrained. Scan it with 'huggingface-cli scan-cache'.

Q: Can I host private models?
A: Yes—create a private repo on the Hub or use a private Space; share it
   with org members or via read-only tokens.

Q: How much do Spaces cost?
A: Free CPU Spaces are available; GPU hardware (A10G, A100) is billed
   per hour. Stop the Space when you are not using it.

Q: What is the Inference API?
A: A hosted endpoint for popular models—POST to
   api-inference.huggingface.co/models/<repo> with your HF_TOKEN. It is
   rate-limited and not meant for production serving.

For production serving, self-host the model with vLLM or TGI instead of relying on the shared Inference API.

Ready to try Hugging Face?

Visit the official site for the latest version and full documentation.

Visit Hugging Face
All AI guides