Function calling 让人们能够使用大型语言模型(LLM),并利用模型响应来执行函数或与外部 API 交互。开发者通常定义一组函数及每个函数所需的输入 schema,我们称之为 tools。模型会智能地理解何时需要进行 tool call,并返回 JSON 输出,用户需要将其传递给另一个函数或 API。
本质上,function calling 允许你通过执行代码或发起额外 API 调用来使用 LLM 执行操作。
Workers AI 提供嵌入式 function calling,允许你在推理调用 alongside 执行函数代码。我们有一个名为 @cloudflare/ai-utils ↗ 的包来协助实现,已在 Github ↗ 开源。
对于行业标准 function calling,请参阅传统 Function Calling 文档。
为展示嵌入式 function calling 的价值,请看以下示例,对比传统 function calling 与嵌入式 function calling。嵌入式 function calling 将代码行数从 77 行减少到 31 行。
# The ai-utils package enables embedded function calling
npm i @cloudflare/ai-utilsimport {
createToolsFromOpenAPISpec,
runWithTools,
autoTrimTools,
} from "@cloudflare/ai-utils";
export default {
async fetch(request, env, ctx) {
const response = await runWithTools(
env.AI,
"@hf/nousresearch/hermes-2-pro-mistral-7b",
{
messages: [{ role: "user", content: "Who is Cloudflare on github?" }],
tools: [
// You can pass the OpenAPI spec link or contents directly
...(await createToolsFromOpenAPISpec(
"https://gist.githubusercontent.com/mchenco/fd8f20c8f06d50af40b94b0671273dc1/raw/f9d4b5cd5944cc32d6b34cad0406d96fd3acaca6/partial_api.github.com.json",
{
overrides: [
{
// for all requests on *.github.com, we'll need to add a User-Agent.
matcher: ({ url, method }) => {
return url.hostname === "api.github.com";
},
values: {
headers: {
"User-Agent":
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36",
},
},
},
],
},
)),
],
},
).then((response) => {
return response;
});
return new Response(JSON.stringify(response));
},
};export default {
async fetch(request, env, ctx) {
const response = await env.AI.run(
"@hf/nousresearch/hermes-2-pro-mistral-7b",
{
messages: [{ role: "user", content: "Who is Cloudflare on GitHub?" }],
tools: [
{
name: "getGithubUser",
description:
"Provides publicly available information about someone with a GitHub account.",
parameters: {
type: "object",
properties: {
username: {
type: "string",
description: "The handle for the GitHub user account.",
},
},
required: ["username"],
},
},
],
},
);
const selected_tool = response.tool_calls[0];
let res;
if (selected_tool.name == "getGithubUser") {
try {
const username = selected_tool.arguments.username;
const url = `https://api.github.com/users/${username}`;
res = await fetch(url, {
headers: {
// Github API requires a User-Agent header
"User-Agent":
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36",
},
}).then((res) => res.json());
} catch (error) {
return error;
}
}
const finalResponse = await env.AI.run(
"@hf/nousresearch/hermes-2-pro-mistral-7b",
{
messages: [
{
role: "user",
content: "Who is Cloudflare on GitHub?",
},
{
role: "assistant",
content: JSON.stringify(selected_tool),
},
{
role: "tool",
content: JSON.stringify(res),
},
],
tools: [
{
name: "getGithubUser",
description:
"Provides publicly available information about someone with a GitHub account.",
parameters: {
type: "object",
properties: {
username: {
type: "string",
description: "The handle for the GitHub user account.",
},
},
required: ["username"],
},
},
],
},
);
return new Response(JSON.stringify(finalResponse));
},
};有些开源模型经过微调以支持 function calling。浏览我们的模型目录时,查找带有 function calling 属性的模型。例如,@hf/nousresearch/hermes-2-pro-mistral-7b 是 Mistral 7B 的微调变体,可用于 function calling。