[TOC] ## 开箱即用 * `feature-extraction`（获得文本的向量化表示） * `fill-mask`（填充被遮盖的词、片段） * `ner`（命名实体识别） * `question-answering`（自动问答） * `sentiment-analysis`（情感分析） * `summarization`（自动摘要） * `text-generation`（文本生成） * `translation`（机器翻译） * `zero-shot-classification`（零训练样本分类） 1. 可以直接在 pipline 中使用以上 模型, 会自动选择最合适的模型进行下载 2. 也就是说,上述 pipeline 并不是模型本身,只是分类 3. 可指定具体模型 `generator = pipeline("text-generation", model="distilgpt2") ` ## 示例 ### 情感分析 ``` from transformers import pipeline classifier = pipeline("sentiment-analysis") result = classifier("I've been waiting for a HuggingFace course my whole life.") print(result) results = classifier( ["I've been waiting for a HuggingFace course my whole life.", "I hate this so much!"] ) print(results) ``` 输出 ``` No model was supplied, defaulted to distilbert-base-uncased-finetuned-sst-2-english (https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english) [{'label': 'POSITIVE', 'score': 0.9598048329353333}] [{'label': 'POSITIVE', 'score': 0.9598048329353333}, {'label': 'NEGATIVE', 'score': 0.9994558691978455}] ``` 自动选择了 base-uncased-finetuned-sst-2-english 进行处理 ### 零训练样本分类 ``` from transformers import pipeline classifier = pipeline("zero-shot-classification") result = classifier( "This is a course about the Transformers library", candidate_labels=["education", "politics", "business"], ) print(result) ``` 输出 ``` No model was supplied, defaulted to facebook/bart-large-mnli (https://huggingface.co/facebook/bart-large-mnli) {'sequence': 'This is a course about the Transformers library', 'labels': ['education', 'business', 'politics'], 'scores': [0.8445973992347717, 0.11197526752948761, 0.043427325785160065]} ``` 自动选择了 facebook/bart-large-mnli 进行处理 ### 文本生成 #### 英文生成 ``` from transformers import pipeline generator = pipeline("text-generation") results = generator("In this course, we will teach you how to") print(results) results = generator( "In this course, we will teach you how to", num_return_sequences=2, max_length=50 ) print(results) ``` 输出 ``` No model was supplied, defaulted to gpt2 (https://huggingface.co/gpt2) [{'generated_text': "In this course, we will teach you how to use data and models that can be applied in any real-world, everyday situation. In most cases, the following will work better than other courses I've offered for an undergrad or student. In order"}] [{'generated_text': 'In this course, we will teach you how to make your own unique game called "Mono" from scratch by doing a game engine, a framework and the entire process starting with your initial project. We are planning to make some basic gameplay scenarios and'}, {'generated_text': 'In this course, we will teach you how to build a modular computer, how to run it on a modern Windows machine, how to install packages, and how to debug and debug systems. We will cover virtualization and virtualization without a programmer,'}] ``` ### 古诗词生成 ``` from transformers import pipeline generator = pipeline("text-generation", model="uer/gpt2-chinese-poem") results = generator( "[CLS] 万 叠 春 山 积 雨 晴 ，", max_length=40, num_return_sequences=2, ) print(results) ``` 输出 ``` [{'generated_text': '[CLS] 万 叠 春 山 积 雨 晴 ， 孤 舟 遥 送 子 陵 行 。 别 情 共 叹 孤 帆 远 ， 交 谊 深 怜 一 座 倾 。 白 日 风 波 身 外 幻'}, {'generated_text': '[CLS] 万 叠 春 山 积 雨 晴 ， 满 川 烟 草 踏 青 行 。 何 人 唤 起 伤 春 思 ， 江 畔 画 船 双 橹 声 。 桃 花 带 雨 弄 晴 光'}] ``` ### 命名实体识别 命名实体识别 (NER) pipeline 负责从文本中抽取出指定类型的实体，例如人物、地点、组织等等。 ``` from transformers import pipeline ner = pipeline("ner", grouped_entities=True) results = ner("My name is Sylvain and I work at Hugging Face in Brooklyn.") print(results) ``` ### 自动问答 * **抽取式 QA (extractive QA)：**假设答案就包含在文档中，因此直接从文档中抽取答案； * **多选 QA (multiple-choice QA)：**从多个给定的选项中选择答案，相当于做阅读理解题； * **无约束 QA (free-form QA)：**直接生成答案文本，并且对答案文本格式没有任何限制。 #### 抽取式 ``` from transformers import pipeline question_answerer = pipeline("question-answering") answer = question_answerer( question="Where do I work?", context="My name is Sylvain and I work at Hugging Face in Brooklyn", ) print(answer) ``` 输出 ``` No model was supplied, defaulted to distilbert-base-cased-distilled-squad (https://huggingface.co/distilbert-base-cased-distilled-squad) {'score': 0.6949771046638489, 'start': 33, 'end': 45, 'answer': 'Hugging Face'} ``` ### 自动摘要