SQLite FTS5 扩展 --- SQLite FTS5 Extension

1. Overview of FTS5
1.FTS5概述

FTS5 is an SQLite virtual table module that provides full-text search functionality to database applications. In their most elementary form, full-text search engines allow the user to efficiently search a large collection of documents for the subset that contain one or more instances of a search term. The search functionality provided to world wide web users by Google is, among other things, a full-text search engine, as it allows users to search for all documents on the web that contain, for example, the term "fts5".
FTS5 是一个 SQLite虚拟表模块，为数据库应用程序提供全文搜索功能。在最基本的形式中，全文搜索引擎允许用户有效地搜索大量文档以查找包含一个或多个搜索项实例的子集。 Google向万维网用户提供的搜索功能是全文搜索引擎，因为它允许用户搜索网络上包含（例如）术语“fts5”的所有文档。

To use FTS5, the user creates an FTS5 virtual table with one or more columns. For example:
要使用 FTS5，用户需要创建一个包含一列或多列的 FTS5 虚拟表。例如：

CREATE VIRTUAL TABLE email USING fts5(sender, title, body);

It is an error to add types, constraints or PRIMARY KEY declarations to a CREATE VIRTUAL TABLE statement used to create an FTS5 table. Once created, an FTS5 table may be populated using INSERT, UPDATE or DELETE statements like any other table. Like any other table with no PRIMARY KEY declaration, an FTS5 table has an implicit INTEGER PRIMARY KEY field named rowid.
将类型、约束或PRIMARY KEY声明添加到用于创建 FTS5 表的 CREATE VIRTUAL TABLE 语句是错误的。创建后，FTS5 表可以像任何其他表一样使用INSERT 、 UPDATE或DELETE语句进行填充。与任何其他没有 PRIMARY KEY 声明的表一样，FTS5 表有一个名为 rowid 的隐式 INTEGER PRIMARY KEY 字段。

Not shown in the example above is that there are also various options that may be provided to FTS5 as part of the CREATE VIRTUAL TABLE statement to configure various aspects of the new table. These may be used to modify the way in which the FTS5 table extracts terms from documents and queries, to create extra indexes on disk to speed up prefix queries, or to create an FTS5 table that acts as an index on content stored elsewhere.
上面的示例中未显示的是，还可以作为 CREATE VIRTUAL TABLE 语句的一部分向 FTS5 提供各种选项，以配置新表的各个方面。这些可用于修改 FTS5 表从文档和查询中提取术语的方式，在磁盘上创建额外的索引以加速前缀查询，或创建充当其他地方存储的内容的索引的 FTS5 表。

Once populated, there are three ways to execute a full-text query against the contents of an FTS5 table:
填充后，可以通过三种方式对 FTS5 表的内容执行全文查询：

• Using a MATCH operator in the WHERE clause of a SELECT statement, or
  在 SELECT 语句的 WHERE 子句中使用 MATCH 运算符，或者
• Using an equals ("=") operator in the WHERE clause of a SELECT statement, or
  在 SELECT 语句的 WHERE 子句中使用等于 (“=”) 运算符，或者
• using the table-valued function syntax.
  使用表值函数语法。

If using the MATCH or = operators, the expression to the left of the MATCH operator is usually the name of the FTS5 table (the exception is when specifying a column-filter). The expression on the right must be a text value specifying the term to search for. For the table-valued function syntax, the term to search for is specified as the first table argument. For example:
如果使用 MATCH 或 = 运算符，则 MATCH 运算符左侧的表达式通常是 FTS5 表的名称（指定列过滤器时例外）。右侧的表达式必须是指定要搜索的术语的文本值。对于表值函数语法，要搜索的术语被指定为第一个表参数。例如：

-- Query for all rows that contain at least once instance of the term
-- "fts5" (in any column). The following three queries are equivalent.
SELECT * FROM email WHERE email MATCH 'fts5';
SELECT * FROM email WHERE email = 'fts5';
SELECT * FROM email('fts5');

By default, FTS5 full-text searches are case-independent. Like any other SQL query that does not contain an ORDER BY clause, the example above returns results in an arbitrary order. To sort results by relevance (most to least relevant), an ORDER BY may be added to a full-text query as follows:
默认情况下，FTS5 全文搜索不区分大小写。与任何其他不包含 ORDER BY 子句的 SQL 查询一样，上面的示例以任意顺序返回结果。要按相关性（从最相关到​​最不相关）对结果进行排序，可以将 ORDER BY 添加到全文查询中，如下所示：

-- Query for all rows that contain at least once instance of the term
-- "fts5" (in any column). Return results in order from best to worst
-- match.  
SELECT * FROM email WHERE email MATCH 'fts5' ORDER BY rank;

As well as the column values and rowid of a matching row, an application may use FTS5 auxiliary functions to retrieve extra information regarding the matched row. For example, an auxiliary function may be used to retrieve a copy of a column value for a matched row with all instances of the matched term surrounded by html <b></b> tags. Auxiliary functions are invoked in the same way as SQLite scalar functions, except that the name of the FTS5 table is specified as the first argument. For example:
除了匹配行的列值和 rowid 之外，应用程序还可以使用FTS5 辅助函数来检索有关匹配行的额外信息。例如，辅助函数可用于检索匹配行的列值的副本，其中匹配术语的所有实例都被 html 包围标签。辅助函数的调用方式与 SQLite标量函数相同，只是将 FTS5 表的名称指定为第一个参数。例如：

-- Query for rows that match "fts5". Return a copy of the "body" column
-- of each row with the matches surrounded by <b></b> tags.
SELECT highlight(email, 2, '<b>', '</b>') FROM email('fts5');

A description of the available auxiliary functions, and more details regarding configuration of the special "rank" column, are available below. Custom auxiliary functions may also be implemented in C and registered with FTS5, just as custom SQL functions may be registered with the SQLite core.

As well as searching for all rows that contain a term, FTS5 allows the user to search for rows that contain:

• any terms that begin with a specified prefix,
• "phrases" - sequences of terms or prefix terms that must feature in a document for it to match the query,
• sets of terms, prefix terms or phrases that appear within a specified proximity of each other (these are called "NEAR queries"), or
• boolean combinations of any of the above.

Such advanced searches are requested by providing a more complicated FTS5 query string as the text to the right of the MATCH operator (or = operator, or as the first argument to a table-valued function syntax). The full query syntax is described here.

2. Compiling and Using FTS5

2.1. Building FTS5 as part of SQLite

As of version 3.9.0 (2015-10-14), FTS5 is included as part of the SQLite amalgamation. If using one of the two autoconf build system, FTS5 is enabled by specifying the "--enable-fts5" option when running the configure script. (FTS5 is currently disabled by default for the source-tree configure script and enabled by default for the amalgamation configure script, but these defaults might change in the future.)

Or, if sqlite3.c is compiled using some other build system, by arranging for the SQLITE_ENABLE_FTS5 pre-processor symbol to be defined.

2.2. Building a Loadable Extension

Alternatively, FTS5 may be built as a loadable extension.

The canonical FTS5 source code consists of a series of *.c and other files in the "ext/fts5" directory of the SQLite source tree. A build process reduces this to just two files - "fts5.c" and "fts5.h" - which may be used to build an SQLite loadable extension.

1. Obtain the latest SQLite code from fossil.
2. Create a Makefile as described in How To Compile SQLite.
3. Build the "fts5.c" target. Which also creates fts5.h.

$ wget -c https://www.sqlite.org/src/tarball/SQLite-trunk.tgz?uuid=trunk -O SQLite-trunk.tgz
.... output ...
$ tar -xzf SQLite-trunk.tgz
$ cd SQLite-trunk
$ ./configure && make fts5.c
... lots of output ...
$ ls fts5.[ch]
fts5.c        fts5.h

The code in "fts5.c" may then be compiled into a loadable extension or statically linked into an application as described in Compiling Loadable Extensions. There are two entry points defined, both of which do the same thing:

• sqlite3_fts_init
• sqlite3_fts5_init

The other file, "fts5.h", is not required to compile the FTS5 extension. It is used by applications that implement custom FTS5 tokenizers or auxiliary functions.
编译 FTS5 扩展不需要另一个文件“fts5.h”。它由实现自定义 FTS5 标记器或辅助功能的应用程序使用。

3. Full-text Query Syntax
3.全文查询语法

The following block contains a summary of the FTS query syntax in BNF form. A detailed explanation follows.
以下块包含 BNF 形式的 FTS 查询语法的摘要。详细解释如下。

<phrase>    := string [*]
<phrase>    := <phrase> + <phrase>
<neargroup> := NEAR ( <phrase> <phrase> ... [, N] )
<query>     := [ [-] <colspec> :] [^] <phrase>
<query>     := [ [-] <colspec> :] <neargroup>
<query>     := [ [-] <colspec> :] ( <query> )
<query>     := <query> AND <query>
<query>     := <query> OR <query>
<query>     := <query> NOT <query>
<colspec>   := colname
<colspec>   := { colname1 colname2 ... }

3.1. FTS5 Strings
3.1. FTS5 字符串

Within an FTS expression a string may be specified in one of two ways:
在 FTS 表达式中，可以通过以下两种方式之一指定字符串：

• By enclosing it in double quotes ("). Within a string, any embedded double quote characters may be escaped SQL-style - by adding a second double-quote character.
  通过将其括在双引号 (") 中。在字符串中，任何嵌入的双引号字符都可以通过添加第二个双引号字符来转义 SQL 样式。

• As an FTS5 bareword that is not "AND", "OR" or "NOT" (case sensitive). An FTS5 bareword is a string of one or more consecutive characters that are all either:
  作为非“AND”、“OR”或“NOT”（区分大小写）的 FTS5 裸字。 FTS5 裸字是由一个或多个连续字符组成的字符串，这些字符均为：

  • Non-ASCII range characters (i.e. unicode codepoints greater than 127), or
    非 ASCII 范围字符（即大于 127 的 unicode 代码点），或
  • One of the 52 upper and lower case ASCII characters, or
    52 个大写和小写 ASCII 字符之一，或
  • One of the 10 decimal digit ASCII characters, or
    10 个十进制数字 ASCII 字符之一，或
  • The underscore character (unicode codepoint 96).
    下划线字符（unicode 代码点 96）。
  • The substitute character (unicode codepoint 26).
    替代字符（unicode 代码点 26）。
  Strings that include any other characters must be quoted. Characters that are not currently allowed in barewords, are not quote characters and do not currently serve any special purpose in FTS5 query expressions may at some point in the future be allowed in barewords or used to implement new query functionality. This means that queries that are currently syntax errors because they include such a character outside of a quoted string may be interpreted differently by some future version of FTS5.
  包含任何其他字符的字符串必须用引号引起来。人物 目前在裸词中不允许使用的字符不是引号字符并且 目前在 FTS5 查询表达式中不具有任何特殊用途 在将来的某个时候允许使用裸字或用于实现 新的查询功能。这意味着当前的查询 语法错误，因为它们在引号之外包含这样的字符 FTS5 的某些未来版本可能会对字符串进行不同的解释。

3.2. FTS5 Phrases
3.2. FTS5 短语

Each string in an fts5 query is parsed ("tokenized") by the tokenizer and a list of zero or more tokens, or terms, extracted. For example, the default tokenizer tokenizes the string "alpha beta gamma" to three separate tokens - "alpha", "beta" and "gamma" - in that order.
fts5 查询中的每个字符串都由分词器进行解析（“分词化”），并提取零个或多个分词或术语的列表。例如，默认标记生成器将字符串“alpha beta gamma”按顺序标记为三个单独的标记 - “alpha”、“beta”和“gamma”。

FTS queries are made up of phrases. A phrase is an ordered list of one or more tokens. The tokens from each string in the query each make up a single phrase. Two phrases can be concatenated into a single large phrase using the "+" operator. For example, assuming the tokenizer module being used tokenizes the input "one.two.three" to three separate tokens, the following four queries all specify the same phrase:
FTS 查询由短语组成。短语是一个或多个标记的有序列表。查询中每个字符串的标记各自组成一个短语。可以使用“+”运算符将两个短语连接成一个大短语。例如，假设使用的分词器模块将输入“one.two. Three”分词为三个单独的分词，则以下四个查询都指定相同的短语：

... MATCH '"one two three"'
... MATCH 'one + two + three'
... MATCH '"one two" + three'
... MATCH 'one.two.three'

A phrase matches a document if the document contains at least one sub-sequence of tokens that matches the sequence of tokens that make up the phrase.
如果文档包含至少一个与组成短语的标记序列相匹配的标记子序列，则该短语与文档匹配。

3.3. FTS5 Prefix Queries
3.3. FTS5 前缀查询

If a "*" character follows a string within an FTS expression, then the final token extracted from the string is marked as a prefix token. As you might expect, a prefix token matches any document token of which it is a prefix. For example, the first two queries in the following block will match any document that contains the token "one" immediately followed by the token "two" and then any token that begins with "thr".
如果 FTS 表达式中的字符串后面有“*”字符，则从字符串中提取的最终标记将被标记为前缀标记。正如您所期望的，前缀标记与它作为前缀的任何文档标记相匹配。例如，以下块中的前两个查询将匹配包含标记“one”、紧随其后的标记“two”以及以“thr”开头的任何标记的任何文档。

... MATCH '"one two thr" * '
... MATCH 'one + two + thr*'
... MATCH '"one two thr*"'      -- May not work as expected!

The final query in the block above may not work as expected. Because the "*" character is inside the double-quotes, it will be passed to the tokenizer, which will likely discard it (or perhaps, depending on the specific tokenizer in use, include it as part of the final token) instead of recognizing it as a special FTS character.
上面块中的最终查询可能无法按预期工作。因为“*”字符位于双引号内，所以它将被传递给分词器，分词器可能会丢弃它（或者可能，根据所使用的特定分词器，将其作为最终标记的一部分包含在内）而不是识别它作为一个特殊的 FTS 字符。

3.4. FTS5 Initial Token Queries
3.4. FTS5 初始令牌查询

If a "^" character appears immediately before a phrase that is not part of a NEAR query, then that phrase only matches a document only if it starts at the first token in a column. The "^" syntax may be combined with a column filter, but may not be inserted into the middle of a phrase.
如果“^”字符紧接出现在不属于 NEAR 查询的短语之前，则该短语仅与从列中的第一个标记开始的文档匹配。 “^”语法可以与列过滤器组合，但不能插入短语的中间。

... MATCH '^one'              -- first token in any column must be "one"
... MATCH '^ one + two'       -- phrase "one two" must appear at start of a column
... MATCH '^ "one two"'       -- same as previous 
... MATCH 'a : ^two'          -- first token of column "a" must be "two"
... MATCH 'NEAR(^one, two)'   -- syntax error! 
... MATCH 'one + ^two'        -- syntax error! 
... MATCH '"^one two"'        -- May not work as expected!

3.5. FTS5 NEAR Queries
3.5. FTS5 NEAR 查询

Two or more phrases may be grouped into a NEAR group. A NEAR group is specified by the token "NEAR" (case sensitive) followed by an open parenthesis character, followed by two or more whitespace separated phrases, optionally followed by a comma and the numeric parameter N, followed by a close parenthesis. For example:
两个或多个短语可以分为一个NEAR 组。 NEAR 组由标记“NEAR”（区分大小写）指定，后跟左括号字符，后跟两个或多个空格分隔的短语，可选后跟逗号和数字参数N ，后跟右括号。例如：

... MATCH 'NEAR("one two" "three four", 10)'
... MATCH 'NEAR("one two" thr* + four)'

If no N parameter is supplied, it defaults to 10. A NEAR group matches a document if the document contains at least one clump of tokens that:
如果未提供N参数，则默认为 10。如果文档包含至少一组符合以下条件的标记，则 NEAR 组与文档匹配：

1. contains at least one instance of each phrase, and
   每个短语至少包含一个实例，并且
2. for which the number of tokens between the end of the first phrase and the beginning of the last phrase in the clump is less than or equal to N.
   其中丛中第一个短语的末尾和最后一个短语的开头之间的标记数小于或等于N 。

For example:  例如：

CREATE VIRTUAL TABLE ft USING fts5(x);
INSERT INTO ft(rowid, x) VALUES(1, 'A B C D x x x E F x');

... MATCH 'NEAR(e d, 4)';                      -- Matches!
... MATCH 'NEAR(e d, 3)';                      -- Matches!
... MATCH 'NEAR(e d, 2)';                      -- Does not match!

... MATCH 'NEAR("c d" "e f", 3)';              -- Matches!
... MATCH 'NEAR("c"   "e f", 3)';              -- Does not match!

... MATCH 'NEAR(a d e, 6)';                    -- Matches!
... MATCH 'NEAR(a d e, 5)';                    -- Does not match!

... MATCH 'NEAR("a b c d" "b c" "e f", 4)';    -- Matches!
... MATCH 'NEAR("a b c d" "b c" "e f", 3)';    -- Does not match!

3.6. FTS5 Column Filters
3.6. FTS5 柱过滤器

A single phrase or NEAR group may be restricted to matching text within a specified column of the FTS table by prefixing it with the column name followed by a colon character. Or to a set of columns by prefixing it with a whitespace separated list of column names enclosed in parenthesis ("curly brackets") followed by a colon character. Column names may be specified using either of the two forms described for strings above. Unlike strings that are part of phrases, column names are not passed to the tokenizer module. Column names are case-insensitive in the usual way for SQLite column names - upper/lower case equivalence is understood for ASCII-range characters only.
单个短语或 NEAR 组可以被限制为匹配 FTS 表的指定列中的文本，方法是在其前面加上列名，后跟冒号字符。或者通过在一组列前添加一个空格分隔的列名列表作为前缀，该列表括在括号（“大括号”）中，后跟冒号字符。可以使用上述字符串的两种形式之一来指定列名称。与作为短语一部分的字符串不同，列名称不会传递到分词器模块。按照 SQLite 列名称的通常方式，列名称不区分大小写 - 仅对于 ASCII 范围字符理解大写/小写等效性。

... MATCH 'colname : NEAR("one two" "three four", 10)'
... MATCH '"colname" : one + two + three'

... MATCH '{col1 col2} : NEAR("one two" "three four", 10)'
... MATCH '{col2 col1 col3} : one + two + three'

If a column filter specification is preceded by a "-" character, then it is interpreted as a list of column not to match against. For example:
如果列过滤器规范前面有“-”字符，则它将被解释为不匹配的列的列表。例如：

-- Search for matches in all columns except "colname"
... MATCH '- colname : NEAR("one two" "three four", 10)'

-- Search for matches in all columns except "col1", "col2" and "col3"
... MATCH '- {col2 col1 col3} : one + two + three'

Column filter specifications may also be applied to arbitrary expressions enclosed in parenthesis. In this case the column filter applies to all phrases within the expression. Nested column filter operations may only further restrict the subset of columns matched, they can not be used to re-enable filtered columns. For example:
列过滤器规范也可以应用于括号内的任意表达式。在这种情况下，列过滤器适用于表达式中的所有短语。嵌套列过滤操作可能只会进一步限制匹配的列的子集，它们不能用于重新启用过滤列。例如：

-- The following are equivalent:
... MATCH '{a b} : ( {b c} : "hello" AND "world" )'
... MATCH '(b : "hello") AND ({a b} : "world")'

Finally, a column filter for a single column may be specified by using the column name as the LHS of a MATCH operator (instead of the usual table name). For example:
最后，可以通过使用列名作为 MATCH 运算符的 LHS（而不是通常的表名）来指定单个列的列过滤器。例如：

-- Given the following table
CREATE VIRTUAL TABLE ft USING fts5(a, b, c);

-- The following are equivalent
SELECT * FROM ft WHERE b MATCH 'uvw AND xyz';
SELECT * FROM ft WHERE ft MATCH 'b : (uvw AND xyz)';

-- This query cannot match any rows (since all columns are filtered out): 
SELECT * FROM ft WHERE b MATCH 'a : xyz';

3.7. FTS5 Boolean Operators
3.7. FTS5 布尔运算符

Phrases and NEAR groups may be arranged into expressions using boolean operators. In order of precedence, from highest (tightest grouping) to lowest (loosest grouping), the operators are:
短语和 NEAR 组可以使用布尔运算符排列成表达式。按照优先级顺序，从最高（最紧密分组）到最低（最松散分组），运算符是：

Parenthesis may be used to group expressions in order to modify operator precedence in the usual ways. For example:
括号可用于对表达式进行分组，以便以通常的方式修改运算符优先级。例如：

-- Because NOT groups more tightly than OR, either of the following may
-- be used to match all documents that contain the token "two" but not
-- "three", or contain the token "one".  
... MATCH 'one OR two NOT three'
... MATCH 'one OR (two NOT three)'

-- Matches documents that contain at least one instance of either "one"
-- or "two", but do not contain any instances of token "three".
... MATCH '(one OR two) NOT three'

Phrases and NEAR groups may also be connected by implicit AND operators. For simplicity, these are not shown in the BNF grammar above. Essentially, any sequence of phrases or NEAR groups (including those restricted to matching specified columns) separated only by whitespace are handled as if there were an implicit AND operator between each pair of phrases or NEAR groups. Implicit AND operators are never inserted after or before an expression enclosed in parenthesis. Implicit AND operators group more tightly than all other operators, including NOT. For example:
短语和 NEAR 组也可以通过隐式 AND 运算符连接。为了简单起见，上面的 BNF 语法中没有显示这些。本质上，仅由空格分隔的任何短语或 NEAR 组序列（包括限制为匹配指定列的序列）都将被处理，就好像每对短语或 NEAR 组之间存在隐式 AND 运算符一样。隐式 AND 运算符绝不会插入括号内的表达式之后或之前。隐式 AND 运算符比所有其他运算符（包括 NOT）分组更紧密。例如：

... MATCH 'one two three'         -- 'one AND two AND three'
... MATCH 'three "one two"'       -- 'three AND "one two"'
... MATCH 'NEAR(one two) three'   -- 'NEAR(one two) AND three'
... MATCH 'one OR two three'      -- 'one OR two AND three'
... MATCH 'one NOT two three'     -- 'one NOT (two AND three)'

... MATCH '(one OR two) three'    -- Syntax error!
... MATCH 'func(one two)'         -- Syntax error!

4. FTS5 Table Creation and Initialization
4.FTS5表创建和初始化

Each argument specified as part of a "CREATE VIRTUAL TABLE ... USING fts5 ..." statement is either a column declaration or a configuration option. A column declaration consists of one or more whitespace separated FTS5 barewords or string literals quoted in any manner acceptable to SQLite.
指定为“CREATE VIRTUAL TABLE ... USING fts5 ...”语句一部分的每个参数都是列声明或配置选项。列声明由一个或多个空格分隔的 FTS5 裸字或以 SQLite 可接受的任何方式引用的字符串文字组成。

The first string or bareword in a column declaration is the column name. It is an error to attempt to name an fts5 table column "rowid" or "rank", or to assign the same name to a column as is used by the table itself. This is not supported.
列声明中的第一个字符串或裸字是列名称。尝试将 fts5 表列命名为“rowid”或“rank”，或者为表本身使用的列分配相同的名称是错误的。不支持此操作。

Each subsequent string or bareword in a column declaration is a column option that modifies the behaviour of that column. Column options are case-independent. Unlike the SQLite core, FTS5 considers unrecognized column options to be errors. Currently, the only option recognized is "UNINDEXED" (see below).
列声明中的每个后续字符串或裸字都是修改该列行为的列选项。列选项与大小写无关。与 SQLite 核心不同，FTS5 将无法识别的列选项视为错误。目前，唯一识别的选项是“UNINDEXED”（见下文） 。

A configuration option consists of an FTS5 bareword - the option name - followed by an "=" character, followed by the option value. The option value is specified using either a single FTS5 bareword or a string literal, again quoted in any manner acceptable to the SQLite core. For example:
配置选项由 FTS5 裸字（选项名称）组成，后跟“=”字符，然后是选项值。选项值使用单个 FTS5 裸字或字符串文字指定，再次以 SQLite 核心可接受的任何方式引用。例如：

CREATE VIRTUAL TABLE mail USING fts5(sender, title, body, tokenize = 'porter ascii');

There are currently the following configuration options:
目前有以下配置选项：

• The "tokenize" option, used to configure a custom tokenizer.
  “tokenize”选项，用于配置自定义分词器。
• The "prefix" option, used to add prefix indexes to an FTS5 table.
  “前缀”选项，用于向 FTS5 表添加前缀索引。
• The "content" option, used to make the FTS5 table an external content or contentless table.
  “内容”选项，用于使 FTS5 表成为外部内容表或无内容表。
• The "content_rowid" option, used to set the rowid field of an external content table.
  “content_rowid”选项，用于设置外部内容表的rowid字段。
• The "columnsize" option, used to configure whether or not the size in tokens of each value in the FTS5 table is stored separately within the database.
  “columnsize”选项，用于配置 FTS5 表中每个值的标记大小是否单独存储在数据库中。
• The "detail" option. This option may be used to reduce the size of the FTS index on disk by omitting some information from it.
  “详细信息”选项。此选项可用于通过省略其中的一些信息来减小磁盘上 FTS 索引的大小。

4.1. The UNINDEXED column option
4.1. UNINDEXED 列选项

The contents of columns qualified with the UNINDEXED column option are not added to the FTS index. This means that for the purposes of MATCH queries and FTS5 auxiliary functions, the column contains no matchable tokens.
使用 UNINDEXED 列选项限定的列的内容不会添加到 FTS 索引中。这意味着，出于 MATCH 查询和FTS5 辅助函数的目的，该列不包含可匹配的标记。

For example, to avoid adding the contents of the "uuid" field to the FTS index:
例如，为了避免将“uuid”字段的内容添加到FTS索引中：

CREATE VIRTUAL TABLE customers USING fts5(name, addr, uuid UNINDEXED);

4.2. Prefix Indexes
4.2.前缀索引

By default, FTS5 maintains a single index recording the location of each token instance within the document set. This means that querying for complete tokens is fast, as it requires a single lookup, but querying for a prefix token can be slow, as it requires a range scan. For example, to query for the prefix token "abc*" requires a range scan of all tokens greater than or equal to "abc" and less than "abd".
默认情况下，FTS5 维护一个记录文档集中每个标记实例位置的索引。这意味着查询完整令牌很快，因为它需要单次查找，但查询前缀令牌可能很慢，因为它需要范围扫描。例如，要查询前缀标记“abc*”，需要对所有大于或等于“abc”且小于“abd”的标记进行范围扫描。

A prefix index is a separate index that records the location of all instances of prefix tokens of a certain length in characters used to speed up queries for prefix tokens. For example, optimizing a query for prefix token "abc*" requires a prefix index of three-character prefixes.
前缀索引是一个单独的索引，它记录一定长度字符的前缀标记的所有实例的位置，用于加速前缀标记的查询。例如，优化前缀标记“abc*”的查询需要三字符前缀的前缀索引。

To add prefix indexes to an FTS5 table, the "prefix" option is set to either a single positive integer or a text value containing a white-space separated list of one or more positive integer values. A prefix index is created for each integer specified. If more than one "prefix" option is specified as part of a single CREATE VIRTUAL TABLE statement, all apply.
要将前缀索引添加到 FTS5 表，“前缀”选项设置为单个正整数或包含一个或多个正整数值的空格分隔列表的文本值。为每个指定的整数创建一个前缀索引。如果在单个 CREATE VIRTUAL TABLE 语句中指定了多个“前缀”选项，则所有选项均适用。

-- Two ways to create an FTS5 table that maintains prefix indexes for
-- two and three character prefix tokens.
CREATE VIRTUAL TABLE ft USING fts5(a, b, prefix='2 3');
CREATE VIRTUAL TABLE ft USING fts5(a, b, prefix=2, prefix=3);

4.3. Tokenizers
4.3.分词器

The CREATE VIRTUAL TABLE "tokenize" option is used to configure the specific tokenizer used by the FTS5 table. The option argument must be either an FTS5 bareword, or an SQL text literal. The text of the argument is itself treated as a white-space series of one or more FTS5 barewords or SQL text literals. The first of these is the name of the tokenizer to use. The second and subsequent list elements, if they exist, are arguments passed to the tokenizer implementation.
CREATE VIRTUAL TABLE“tokenize”选项用于配置 FTS5 表使用的特定标记生成器。选项参数必须是 FTS5 裸字或 SQL 文本文字。参数文本本身被视为一个或多个 FTS5 裸字或 SQL 文本文字的空白系列。第一个是要使用的分词器的名称。第二个和后续列表元素（如果存在）是传递给标记生成器实现的参数。

Unlike option values and column names, SQL text literals intended as tokenizers must be quoted using single quote characters. For example:
与选项值和列名不同，用作标记器的 SQL 文本文字必须使用单引号字符引起来。例如：

-- The following are all equivalent
CREATE VIRTUAL TABLE ft USING fts5(x, tokenize = 'porter ascii');
CREATE VIRTUAL TABLE ft USING fts5(x, tokenize = "porter ascii");
CREATE VIRTUAL TABLE ft USING fts5(x, tokenize = "'porter' 'ascii'");
CREATE VIRTUAL TABLE ft USING fts5(x, tokenize = '''porter'' ''ascii''');

-- But this will fail:
CREATE VIRTUAL TABLE ft USING fts5(x, tokenize = '"porter" "ascii"');

-- This will fail too:
CREATE VIRTUAL TABLE ft USING fts5(x, tokenize = 'porter' 'ascii');

FTS5 features four built-in tokenizer modules, described in subsequent sections:
FTS5 具有四个内置分词器模块，如后续部分所述：

• The unicode61 tokenizer, based on the Unicode 6.1 standard. This is the default.
  unicode61分词器，基于 Unicode 6.1 标准。这是默认设置。
• The ascii tokenizer, which assumes all characters outside of the ASCII codepoint range (0-127) are to be treated as token characters.
  ascii标记生成器，假定 ASCII 代码点范围 (0-127) 之外的所有字符都被视为标记字符。
• The porter tokenizer, which implements the porter stemming algorithm.
  porter tokenizer，它实现了porter 词干算法。
• The trigram tokenizer, which treats each contiguous sequence of three characters as a token, allowing FTS5 to support more general substring matching.
  trigram tokenizer，它将三个字符的每个连续序列视为一个标记，允许 FTS5 支持更通用的子字符串匹配。

It is also possible to create custom tokenizers for FTS5. The API for doing so is described here.
还可以为 FTS5 创建自定义标记器。此处描述了执行此操作的 API。

4.3.1. Unicode61 Tokenizer
4.3.1. Unicode61 分词器

The unicode tokenizer classifies all unicode characters as either "separator" or "token" characters. By default all space and punctuation characters, as defined by Unicode 6.1, are considered separators, and all other characters as token characters. More specifically, all unicode characters assigned to a general category beginning with "L" or "N" (letters and numbers, specifically) or to category "Co" ("other, private use") are considered tokens. All other characters are separators.
unicode 标记生成器将所有 unicode 字符分类为“分隔符”或“标记”字符。默认情况下，Unicode 6.1 定义的所有空格和标点字符都被视为分隔符，所有其他字符被视为标记字符。更具体地说，分配给以“L”或“N”（特别是字母和数字）开头的一般类别或分配给类别“Co”（“其他私人用途”）的所有 unicode 字符都被视为令牌。所有其他字符都是分隔符。

Each contiguous run of one or more token characters is considered to be a token. The tokenizer is case-insensitive according to the rules defined by Unicode 6.1.
每个连续的一个或多个标记字符都被视为一个标记。根据 Unicode 6.1 定义的规则，分词器不区分大小写。

By default, diacritics are removed from all Latin script characters. This means, for example, that "A", "a", "À", "à", "Â" and "â" are all considered to be equivalent.
默认情况下，将从所有拉丁脚本字符中删除变音符号。例如，这意味着“A”、“a”、“À”、“à”、“”和“â”都被认为是等效的。

Any arguments following "unicode61" in the token specification are treated as a list of alternating option names and values. Unicode61 supports the following options:
令牌规范中“unicode61”后面的任何参数都被视为交替选项名称和值的列表。 Unicode61 支持以下选项：

For example:  例如：

-- Create an FTS5 table that does not remove diacritics from Latin
-- script characters, and that considers hyphens and underscore characters
-- to be part of tokens. 
CREATE VIRTUAL TABLE ft USING fts5(a, b,
    tokenize = "unicode61 remove_diacritics 0 tokenchars '-_'"
);

or:  或者：

-- Create an FTS5 table that, as well as the default token character classes,
-- considers characters in class "Mn" to be token characters.
CREATE VIRTUAL TABLE ft USING fts5(a, b,
    tokenize = "unicode61 categories 'L* N* Co Mn'"
);

The fts5 unicode61 tokenizer is byte-for-byte compatible with the fts3/4 unicode61 tokenizer.
fts5 unicode61 分词器与 fts3/4 unicode61 分词器逐字节兼容。

4.3.2. Ascii Tokenizer
4.3.2. Ascii 分词器

The Ascii tokenizer is similar to the Unicode61 tokenizer, except that:
Ascii 分词器与 Unicode61 分词器类似，不同之处在于：

• All non-ASCII characters (those with codepoints greater than 127) are always considered token characters. If any non-ASCII characters are specified as part of the separators option, they are ignored.
  所有非 ASCII 字符（代码点大于 127 的字符）始终被视为标记字符。如果任何非 ASCII 字符被指定为分隔符选项的一部分，它们将被忽略。
• Case-folding is only performed for ASCII characters. So while "A" and "a" are considered to be equivalent, "Ã" and "ã" are distinct.
  仅对 ASCII 字符执行大小写折叠。因此，虽然“A”和“a”被认为是等效的，但“à”和“ã”是不同的。
• The remove_diacritics option is not supported.
  不支持remove_diacritics 选项。

For example:  例如：

-- Create an FTS5 table that uses the ascii tokenizer, but does not
-- consider numeric characters to be part of tokens.
CREATE VIRTUAL TABLE ft USING fts5(a, b,
    tokenize = "ascii separators '0123456789'"
);

4.3.3. Porter Tokenizer
4.3.3.波特分词器

The porter tokenizer is a wrapper tokenizer. It takes the output of some other tokenizer and applies the porter stemming algorithm to each token before it returns it to FTS5. This allows search terms like "correction" to match similar words such as "corrected" or "correcting". The porter stemmer algorithm is designed for use with English language terms only - using it with other languages may or may not improve search utility.
波特分词器是一个包装分词器。它获取其他一些分词器的输出，并对每个分词应用波特词干算法，然后将其返回到 FTS5。这允许像“更正”这样的搜索词匹配类似的词，例如“已更正”或“正在纠正”。波特词干分析器算法仅设计用于英语语言术语 - 将其与其他语言一起使用可能会也可能不会提高搜索实用性。

By default, the porter tokenizer operates as a wrapper around the default tokenizer (unicode61). Or, if one or more extra arguments are added to the "tokenize" option following "porter", they are treated as a specification for the underlying tokenizer that the porter stemmer uses. For example:
默认情况下，porter 分词器作为默认分词器 (unicode61) 的包装器运行。或者，如果将一个或多个额外参数添加到“porter”后面的“tokenize”选项中，则它们将被视为 porter 词干分析器使用的底层分词器的规范。例如：

-- Two ways to create an FTS5 table that uses the porter tokenizer to
-- stem the output of the default tokenizer (unicode61). 
CREATE VIRTUAL TABLE ft USING fts5(x, tokenize = porter);
CREATE VIRTUAL TABLE ft USING fts5(x, tokenize = 'porter unicode61');

-- A porter tokenizer used to stem the output of the unicode61 tokenizer,
-- with diacritics removed before stemming.
CREATE VIRTUAL TABLE ft USING fts5(x, tokenize = 'porter unicode61 remove_diacritics 1');

4.3.4. The Trigram Tokenizer
4.3.4. Trigram 分词器

The trigram tokenizer extends FTS5 to support substring matching in general, instead of the usual token matching. When using the trigram tokenizer, a query or phrase token may match any sequence of characters within a row, not just a complete token. For example:
trigram tokenizer 扩展了 FTS5 以支持一般的子字符串匹配，而不是通常的标记匹配。使用 trigram tokenizer 时，查询或短语标记可以匹配行中的任何字符序列，而不仅仅是完整的标记。例如：

CREATE VIRTUAL TABLE tri USING fts5(a, tokenize="trigram");
INSERT INTO tri VALUES('abcdefghij KLMNOPQRST uvwxyz');

-- The following queries all match the single row in the table
SELECT * FROM tri('cdefg');
SELECT * FROM tri('cdefg AND pqr');
SELECT * FROM tri('"hij klm" NOT stuv');

The trigram tokenizer supports the following options:
trigram tokenizer 支持以下选项：

-- A case-sensitive trigram index
CREATE VIRTUAL TABLE tri USING fts5(a, tokenize="trigram case_sensitive 1");

Unless the remove_diacritics option is set, FTS5 tables that use the trigram tokenizer also support indexed GLOB and LIKE pattern matching. For example:
除非设置了remove_diacritics选项，否则使用trigram tokenizer的FTS5表还支持索引GLOB和LIKE模式匹配。例如：

SELECT * FROM tri WHERE a LIKE '%cdefg%';
SELECT * FROM tri WHERE a GLOB '*ij klm*xyz';

If an FTS5 trigram tokenizer is created with the case_sensitive option set to 1, it may only index GLOB queries, not LIKE.
如果创建 FTS5 trigram tokenizer 时将 case_sensitive 选项设置为 1，则它只能索引 GLOB 查询，而不是 LIKE。

Notes:  笔记：

• Substrings consisting of fewer than 3 unicode characters do not match any rows when used with a full-text query. If a LIKE or GLOB pattern does not contain at least one sequence of non-wildcard unicode characters, FTS5 falls back to a linear scan of the entire table.
  与全文查询一起使用时，由少于 3 个 unicode 字符组成的子字符串不匹配任何行。如果 LIKE 或 GLOB 模式不包含至少一个非通配符 unicode 字符序列，FTS5 将回退到整个表的线性扫描。
• If the FTS5 table is created with the detail=none or detail=column option specified, full-text queries may not contain any tokens longer than 3 unicode characters. LIKE and GLOB pattern matching may be slightly slower, but still works. If the index is to be used only for LIKE and/or GLOB pattern matching, these options are worth experimenting with to reduce the index size.
  如果创建 FTS5 表时指定了Detail=none 或detail=column 选项，则全文查询不得包含任何长度超过3 个unicode 字符的标记。 LIKE 和 GLOB 模式匹配可能会稍微慢一些，但仍然有效。如果索引仅用于 LIKE 和/或 GLOB 模式匹配，则值得尝试使用这些选项来减小索引大小。
• The index cannot be used to optimize LIKE patterns if the LIKE operator has an ESCAPE clause.
  如果 LIKE 运算符具有 ESCAPE 子句，则索引不能用于优化 LIKE 模式。

4.4. External Content and Contentless Tables
4.4.外部内容和无内容表

Normally, when a row is inserted into an FTS5 table, in addition to building the index, FTS5 makes a copy of the original row content. When column values are requested from the FTS5 table by the user or by an auxiliary function implementation, those values are read from that private copy of the content. The "content" option may be used to create an FTS5 table that stores only FTS full-text index entries. Because the column values themselves are usually much larger than the associated full-text index entries, this can save significant database space.
通常，当向 FTS5 表中插入一行时，除了构建索引之外，FTS5 还会复制原始行内容。当用户或辅助功能实现从 FTS5 表请求列值时，将从内容的私有副本中读取这些值。 “内容”选项可用于创建仅存储 FTS 全文索引条目的 FTS5 表。由于列值本身通常比关联的全文索引条目大得多，因此这可以节省大量数据库空间。

There are two ways to use the "content" option:
有两种方法可以使用“内容”选项：

• By setting it to an empty string to create a contentless FTS5 table. In this case FTS5 assumes that the original column values are unavailable to it when processing queries. Full-text queries and some auxiliary functions can still be used, but no column values apart from the rowid may be read from the table.
  通过将其设置为空字符串来创建无内容的 FTS5 表。在这种情况下，FTS5 假定在处理查询时原始列值对其不可用。全文查询和一些辅助函数仍然可以使用，但不能从表中读取除 rowid 之外的任何列值。
• By setting it to the name of a database object (table, virtual table or view) that may be queried by FTS5 at any time to retrieve the column values. This is known as an "external content" table. In this case all FTS5 functionality may be used, but it is the responsibility of the user to ensure that the contents of the full-text index are consistent with the named database object. If they are not, query results may be unpredictable.
  通过将其设置为 FTS5 可以随时查询以检索列值的数据库对象（表、虚拟表或视图）的名称。这称为“外部内容”表。在这种情况下，可以使用所有 FTS5 功能，但用户有责任确保全文索引的内容与指定的数据库对象一致。如果不是，查询结果可能是不可预测的。

4.4.1. Contentless Tables
4.4.1.无内容表

A contentless FTS5 table is created by setting the "content" option to an empty string. For example:
通过将“content”选项设置为空字符串来创建无内容的 FTS5 表。例如：

CREATE VIRTUAL TABLE ft USING fts5(a, b, c, content='');

Contentless FTS5 tables do not support UPDATE or DELETE statements, or INSERT statements that do not supply a non-NULL value for the rowid field. Contentless tables do not support REPLACE conflict handling. REPLACE and INSERT OR REPLACE statements are treated as regular INSERT statements. Rows may be deleted from a contentless table using an FTS5 delete command.
无内容 FTS5 表不支持 UPDATE 或 DELETE 语句，也不支持不为 rowid 字段提供非 NULL 值的 INSERT 语句。无内容表不支持 REPLACE 冲突处理。 REPLACE 和 INSERT OR REPLACE 语句被视为常规 INSERT 语句。可以使用FTS5 删除命令从无内容表中删除行。

Attempting to read any column value except the rowid from a contentless FTS5 table returns an SQL NULL value.
尝试从无内容的 FTS5 表中读取除 rowid 之外的任何列值都会返回 SQL NULL 值。

4.4.2. Contentless-Delete Tables
4.4.2.无内容删除表

As of version 3.43.0, also available are contentless-delete tables. A contentless-delete table is created by setting the content option to an empty string and also setting the contentless_delete option to 1. For example:
从版本 3.43.0 开始，还可以使用无内容删除表。通过将 content 选项设置为空字符串并将 contentless_delete 选项设置为 1 来创建无内容删除表。例如：

CREATE VIRTUAL TABLE ft USING fts5(a, b, c, content='', contentless_delete=1);

A contentless-delete table differs from a contentless table in that:
无内容删除表与无内容表的不同之处在于：

• Contentless-delete tables support both DELETE and "INSERT OR REPLACE INTO" statements.
  无内容删除表支持 DELETE 和“INSERT OR REPLACE INTO”语句。
• Contentless-delete tables support UPDATE statements, but only if new values are supplied for all user-defined columns of the fts5 table.
  无内容删除表支持 UPDATE 语句，但前提是为 fts5 表的所有用户定义列提供了新值。
• Contentless-delete tables do not support the FTS5 delete command.
  无内容删除表不支持FTS5 删除命令。

-- Supported UPDATE statement:
UPDATE ft SET a=?, b=?, c=? WHERE rowid=?;

-- This UPDATE is not supported, as it does not supply a new value
-- for column "c".
UPDATE ft SET a=?, b=? WHERE rowid=?;

Unless backwards compatibility is required, new code should prefer contentless-delete tables to contentless tables.
除非需要向后兼容，否则新代码应该更喜欢无内容删除表而不是无内容表。

4.4.3. External Content Tables
4.4.3.外部内容表

An external content FTS5 table is created by setting the content option to the name of a table, virtual table or view (hereafter the "content table") within the same database. Whenever column values are required by FTS5, it queries the content table as follows, with the rowid of the row for which values are required bound to the SQL variable:
通过将 content 选项设置为同一数据库内的表、虚拟表或视图（以下简称“内容表”）的名称来创建外部内容 FTS5 表。每当 FTS5 需要列值时，它都会按如下方式查询内容表，并将需要值的行的 rowid 绑定到 SQL 变量：

SELECT <content_rowid>, <cols> FROM <content> WHERE <content_rowid> = ?;

In the above, <content> is replaced by the name of the content table. By default, <content_rowid> is replaced by the literal text "rowid". Or, if the "content_rowid" option is set within the CREATE VIRTUAL TABLE statement, by the value of that option. <cols> is replaced by a comma-separated list of the FTS5 table column names. For example:
在上面，<content> 被替换为内容表的名称。默认情况下，<content_rowid> 被替换为文字文本“rowid”。或者，如果在 CREATE VIRTUAL TABLE 语句中设置了“content_rowid”选项，则按该选项的值设置。 <cols> 替换为以逗号分隔的 FTS5 表列名称列表。例如：

-- If the database schema is: 
CREATE TABLE t1 (a, b, c, d INTEGER PRIMARY KEY);
CREATE VIRTUAL TABLE ft USING fts5(a, c, content=t1, content_rowid=d);

-- Fts5 may issue queries such as:
SELECT d, a, c FROM t1 WHERE d = ?;

The content table may also be queried as follows:
还可以查询内容表，如下：

SELECT <content_rowid>, <cols> FROM <content> ORDER BY <content_rowid> ASC;
SELECT <content_rowid>, <cols> FROM <content> ORDER BY <content_rowid> DESC;

It is still the responsibility of the user to ensure that the contents of an external content FTS5 table are kept up to date with the content table. One way to do this is with triggers. For example:
用户仍然有责任确保外部内容 FTS5 表的内容与内容表保持最新。一种方法是使用触发器。例如：

-- Create a table. And an external content fts5 table to index it.
CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
CREATE VIRTUAL TABLE fts_idx USING fts5(b, c, content='t1', content_rowid='a');

-- Triggers to keep the FTS index up to date.
CREATE TRIGGER t1_ai AFTER INSERT ON t1 BEGIN
  INSERT INTO fts_idx(rowid, b, c) VALUES (new.a, new.b, new.c);
END;
CREATE TRIGGER t1_ad AFTER DELETE ON t1 BEGIN
  INSERT INTO fts_idx(fts_idx, rowid, b, c) VALUES('delete', old.a, old.b, old.c);
END;
CREATE TRIGGER t1_au AFTER UPDATE ON t1 BEGIN
  INSERT INTO fts_idx(fts_idx, rowid, b, c) VALUES('delete', old.a, old.b, old.c);
  INSERT INTO fts_idx(rowid, b, c) VALUES (new.a, new.b, new.c);
END;

Like contentless tables, external content tables do not support REPLACE conflict handling. Any operations that specify REPLACE conflict handling are handled using ABORT.
与无内容表一样，外部内容表不支持 REPLACE 冲突处理。任何指定 REPLACE 冲突处理的操作都使用 ABORT 进行处理。

4.4.4. External Content Table Pitfalls
4.4.4.外部内容表陷阱

It is the responsibility of the user to ensure that an FTS5 external content table (one with a non-empty content= option) is kept consistent with the content table itself (the table named by the content= option). If these are allowed to become inconsistent, then the results of queries against the FTS5 table may become unintuitive and appear inconsistent.
用户有责任确保 FTS5 外部内容表（带有非空 content= 选项的表）与内容表本身（由 content= 选项命名的表）保持一致。如果允许这些变得不一致，那么针对 FTS5 表的查询结果可能会变得不直观并且显得不一致。

In these situations, the apparently inconsistent results produced by queries against the FTS5 external content table may be understood as follows:
在这些情况下，针对 FTS5 外部内容表的查询产生的明显不一致的结果可以理解如下：

• If the query does not use the full-text index - does not contain a MATCH operator or equivalent table-valued function syntax - then the query is effectively passed through to the external content table. In this case the contents of the FTS index have no effect on the results of the query.
  如果查询不使用全文索引 - 不包含 MATCH 运算符或等效的表值函数语法 - 则查询将有效地传递到外部内容表。在这种情况下，FTS 索引的内容对查询结果没有影响。

• If the query does use the full text index, then the FTS5 module queries it for the set of rowid values corresponding to documents that match the query. For each such rowid, it then runs a query similar to the following to retrieve any required column values, where '?' is replaced by the rowid value, and <content> and <content_rowid> by the values specified for the content= and content_rowid= options:
  如果查询确实使用全文索引，则 FTS5 模块会向其查询与匹配查询的文档对应的 rowid 值集。对于每个这样的 rowid，它会运行类似于以下内容的查询来检索任何所需的列值，其中“？”由 rowid 值替换，<content> 和 <content_rowid> 由为 content= 和 content_rowid= 选项指定的值替换：

SELECT <content_rowid>, <cols> FROM <content> WHERE <content_rowid> = ?;

For example, if a database is created using the following script:
例如，如果使用以下脚本创建数据库：

-- Create and populate a table. 
CREATE TABLE t1(a INTEGER PRIMARY KEY, t TEXT);
INSERT INTO t1 VALUES(1, 'all that glitters');
INSERT INTO t1 VALUES(2, 'is not gold');

-- Create an external content FTS5 table 
CREATE VIRTUAL TABLE ft USING fts5(t, content='t1', content_rowid='a');

then the content table contains two rows, but the FTS index contains no entries corresponding to them. In this case the following queries will return inconsistent results as follows:
那么内容表包含两行，但 FTS 索引不包含与它们对应的条目。在这种情况下，以下查询将返回不一致的结果，如下所示：

-- Returns 2 rows.  Because the query does not use the FTS index, it is
-- effectively executed against table 't1' directly, and so returns
-- both rows.
SELECT * FROM ft;

-- Returns 0 rows.  This query does use the FTS index, which currently
-- contains no entries. So it returns 0 rows.
SELECT rowid, t FROM ft('gold')

Alternatively, if the database were created and populated as follows:
或者，如果数据库是按如下方式创建和填充的：

-- Create and populate a table. 
CREATE TABLE t1(a INTEGER PRIMARY KEY, t TEXT);

-- Create an external content FTS5 table 
CREATE VIRTUAL TABLE ft USING fts5(t, content='t1', content_rowid='a');
INSERT INTO ft(rowid, t) VALUES(1, 'all that glitters');
INSERT INTO ft(rowid, t) VALUES(2, 'is not gold');

then the content table is empty, but the FTS index contains entries for 6 different tokens. In this case the following queries will return inconsistent results as follows:
则内容表为空，但 FTS 索引包含 6 个不同标记的条目。在这种情况下，以下查询将返回不一致的结果，如下所示：

-- Returns 0 rows.  Since it does not use the FTS index, the query is
-- passed directly through to table 't1', which contains no data.
SELECT * FROM ft;

-- Returns 1 row. The "rowid" field of the returned row is 2, and
-- the "t" field set to NULL. "t" is set to NULL because when the external
-- content table "t1" was queried for the data associated with the row
-- with a=2 ("a" is the content_rowid column), none could be found.
SELECT rowid, t FROM ft('gold')

As described in the previous section, triggers on the content table are a good way to ensure that an FTS5 external content table is kept consistent. However, triggers are only fired when rows are inserted, updated or deleted in the content table. This means that if, for example, a database is created as follows:
如上一节所述，内容表上的触发器是确保 FTS5 外部内容表保持一致的好方法。但是，仅当在内容表中插入、更新或删除行时才会触发触发器。这意味着，例如，如果按如下方式创建数据库：

-- Create and populate a table. 
CREATE TABLE t1(a INTEGER PRIMARY KEY, t TEXT);
INSERT INTO t1 VALUES(1, 'all that glitters');
INSERT INTO t1 VALUES(2, 'is not gold');

-- Create an external content FTS5 table 
CREATE VIRTUAL TABLE ft USING fts5(t, content='t1', content_rowid='a');

-- Create triggers to keep the FTS5 table up to date
CREATE TRIGGER t1_ai AFTER INSERT ON t1 BEGIN
  INSERT INTO ft(rowid, t) VALUES (new.a, new.t);
END;
<similar triggers for update + delete>

then the content table and external content FTS5 table are inconsistent, as creating the triggers does not copy existing rows from the content table into the FTS index. The triggers are only able to ensure that updates made to the content table after they are created are reflected in the FTS index.
那么内容表和外部内容 FTS5 表不一致，因为创建触发器不会将现有行从内容表复制到 FTS 索引中。触发器只能确保内容表创建后所做的更新反映在 FTS 索引中。

In this, and any other situation where the FTS index and its content table have become inconsistent, the 'rebuild' command may be used to completely discard the contents of the FTS index and rebuild it based on the current contents of the content table.
在这种情况下，以及FTS索引及其内容表变得不一致的任何其他情况下， “重建”命令可用于完全丢弃FTS索引的内容并基于内容表的当前内容重建它。

4.5. The Columnsize Option
4.5.列大小选项

Normally, FTS5 maintains a special backing table within the database that stores the size of each column value in tokens inserted into the main FTS5 table in a separate table. This backing table is used by the xColumnSize API function, which is in turn used by the built-in bm25 ranking function (and is likely to be useful to other ranking functions as well).
通常，FTS5 在数据库中维护一个特殊的后备表，该表将插入到主 FTS5 表中的标记中的每个列值的大小存储在单独的表中。此支持表由xColumnSize API 函数使用，该函数又由内置bm25 排名函数使用（并且可能对其他排名函数也很有用）。

In order to save space, this backing table may be omitted by setting the columnsize option to zero. For example:
为了节省空间，可以通过将columnsize选项设置为零来省略该后备表。例如：

-- A table without the xColumnSize() values stored on disk:
CREATE VIRTUAL TABLE ft USING fts5(a, b, c, columnsize=0);

-- Three equivalent ways of creating a table that does store the
-- xColumnSize() values on disk:
CREATE VIRTUAL TABLE ft USING fts5(a, b, c);
CREATE VIRTUAL TABLE ft USING fts5(a, b, c, columnsize=1);
CREATE VIRTUAL TABLE ft USING fts5(a, b, columnsize='1', c);

It is an error to set the columnsize option to any value other than 0 or 1.
将 columnsize 选项设置为 0 或 1 以外的任何值都是错误的。

If an FTS5 table is configured with columnsize=0 but is not a contentless table, the xColumnSize API function still works, but runs much more slowly. In this case, instead of reading the value to return directly from the database, it reads the text value itself and count the tokens within it on demand.
如果 FTS5 表配置为 columnsize=0 但不是无内容表，则 xColumnSize API 函数仍然有效，但运行速度要慢得多。在这种情况下，它不是读取直接从数据库返回的值，而是读取文本值本身并根据需要计算其中的标记。

Or, if the table is also a contentless table, then the following apply:
或者，如果该表也是无内容表，则以下情况适用：

• The xColumnSize API always returns -1. There is no way to determine the number of tokens in a value stored within a contentless FTS5 table configured with columnsize=0.
  xColumnSize API 始终返回 -1。无法确定配置为 columnsize=0 的无内容 FTS5 表中存储的值中的标记数量。

• Each inserted row must be accompanied by an explicitly specified rowid value. If a contentless table is configured with columnsize=0, attempting to insert a NULL value into the rowid is an SQLITE_MISMATCH error.
  每个插入的行必须附有显式指定的 rowid 值。如果无内容表配置为columnsize = 0，则尝试将NULL值插入rowid会出现SQLITE_MISMATCH错误。

• All queries on the table must be full-text queries. In other words, they must use the MATCH or = operator with the table-name column as the left-hand operand, or else use the table-valued function syntax. Any query that is not a full-text query results in an error.
  表上的所有查询都必须是全文查询。换句话说，它们必须使用 MATCH 或 = 运算符并将表名列作为左侧操作数，否则使用表值函数语法。任何非全文查询的查询都会导致错误。

The name of the table in which the xColumnSize values are stored (unless columnsize=0 is specified) is "<name>_docsize", where <name> is the name of the FTS5 table itself. The sqlite3_analyzer tool may be used on an existing database in order to determine how much space might be saved by recreating an FTS5 table using columnsize=0.
存储 xColumnSize 值的表的名称（除非指定了 columnsize=0）是“<name>_docsize”，其中 <name> 是 FTS5 表本身的名称。 sqlite3_analyzer工具可用于现有数据库，以确定通过使用 columnsize=0 重新创建 FTS5 表可以节省多少空间。

4.6. The Detail Option
4.6.详细选项

For each term in a document, the FTS index maintained by FTS5 stores the rowid of the document, the column number of the column that contains the term and the offset of the term within the column value. The "detail" option may be used to omit some of this information. This reduces the space that the index consumes within the database file, but also reduces the capability and efficiency of the system.
对于文档中的每个术语，FTS5 维护的 FTS 索引存储文档的 rowid、包含该术语的列的列号以及该术语在列值中的偏移量。 “详细信息”选项可用于省略一些此类信息。这减少了索引在数据库文件中消耗的空间，但也降低了系统的能力和效率。

The detail option may be set to "full" (the default value), "column" or "none". For example:
详细信息选项可以设置为“full”（默认值）、“column”或“none”。例如：

-- The following two lines are equivalent (because the default value
-- of "detail" is "full". 
CREATE VIRTUAL TABLE ft USING fts5(a, b, c);
CREATE VIRTUAL TABLE ft USING fts5(a, b, c, detail=full);

CREATE VIRTUAL TABLE ft USING fts5(a, b, c, detail=column);
CREATE VIRTUAL TABLE ft USING fts5(a, b, c, detail=none);

If the detail option is set to column, then for each term the FTS index records the rowid and column number only, omitting the term offset information. This results in the following restrictions:
如果详细信息选项设置为column ，则对于每个术语，FTS索引仅记录rowid和列号，省略术语偏移量信息。这会导致以下限制：

• NEAR queries are not available.
  NEAR 查询不可用。
• Phrase queries are not available.
  短语查询不可用。
• Assuming the table is not also a contentless table, the xInstCount, xInst, xPhraseFirst and xPhraseNext are slower than usual. This is because instead of reading the required data directly from the FTS index they have to load and tokenize the document text on demand.
  假设该表不是无内容表，则xInstCount 、 xInst 、 xPhraseFirst和xPhraseNext比平常慢。这是因为，他们必须按需加载和标记文档文本，而不是直接从 FTS 索引读取所需的数据。
• If the table is also a contentless table, the xInstCount, xInst, xPhraseFirst and xPhraseNext APIs behave as if the current row contains no phrase matches at all (i.e. xInstCount() returns 0).
  如果该表也是无内容表，则 xInstCount、xInst、xPhraseFirst 和 xPhraseNext API 的行为就好像当前行根本不包含短语匹配（即 xInstCount() 返回 0）。

If the detail option is set to none, then for each term the FTS index records just the rowid is stored. Both column and offset information are omitted. As well as the restrictions itemized above for detail=column mode, this imposes the following extra limitations:
如果详细信息选项设置为none ，则对于每个术语，FTS 索引记录仅存储 rowid。列和偏移信息都被省略。除了上面列出的详细信息=列模式的限制之外，这还施加了以下额外限制：

• Column filter queries are not available.
  列过滤器查询不可用。
• Assuming the table is not also a contentless table, the xPhraseFirstColumn and xPhraseNextColumn are slower than usual.
  假设该表不是无内容表，则xPhraseFirstColumn和xPhraseNextColumn比平常慢。
• If the table is also a contentless table, the xPhraseFirstColumn and xPhraseNextColumn APIs behave as if the current row contains no phrase matches at all (i.e. xPhraseFirstColumn() sets the iterator to EOF).
  如果表也是无内容表，则 xPhraseFirstColumn 和 xPhraseNextColumn API 的行为就好像当前行根本不包含短语匹配（即 xPhraseFirstColumn() 将迭代器设置为 EOF）。

In one test that indexed a large set of emails (1636 MiB on disk), the FTS index was 743 MiB on disk with detail=full, 340 MiB with detail=column and 134 MiB with detail=none.
在对大量电子邮件（磁盘上 1636 MiB）建立索引的一项测试中，磁盘上的 FTS 索引为 743 MiB（detail=full）、340 MiB（detail=column）和 134 MiB（detail=none）。

4.7. The Tokendata Option
4.7.令牌数据选项

This option is only useful to applications that implement custom tokenizers. Usually, tokenizers may return tokens that consist of any sequence of bytes, including 0x00 bytes. However, if the table specifies the tokendata=1 option, then fts5 ignores the first 0x00 byte and any trailing data in the token for the purposes of matching. It still stores the entire token as returned by the tokenizer, but it is ignored by the fts5 core.
此选项仅对实现自定义分词器的应用程序有用。通常，标记生成器可能返回由任何字节序列组成的标记，包括 0x00 字节。但是，如果表指定 tokendata=1 选项，则 fts5 出于匹配目的将忽略第一个 0x00 字节和令牌中的任何尾随数据。它仍然存储标记生成器返回的整个标记，但 fts5 核心会忽略它。

The full version of the token, including any 0x00 byte and trailing data, is available to custom auxiliary functions via the xQueryToken and xInstToken APIs.
令牌的完整版本（包括任何 0x00 字节和尾随数据）可通过xQueryToken和xInstToken API 供自定义辅助函数使用。

This may be useful for ranking functions. A custom tokenizer may add extra data to some document tokens allowing a ranking function to give more weight to hits of some tokens (e.g. those in document headings).
这对于排序函数可能很有用。自定义分词器可以向某些文档标记添加额外的数据，从而允许排名功能对某些标记（例如文档标题中的标记）的命中给予更多权重。

The combination of a custom tokenizer and a custom auxiliary function may be used to implement asymmetric search. The tokenizer could (say) for each document token return the case-normalized and unmarked version of the token, followed by an 0x00 byte, followed by the full text of the token from the document. When queried, fts5 would provide results as if all characters in the query were case-normalized and unmarked. The custom auxiliary function could then be used in the WHERE clause of the query to filter out any rows that do not match based on secondary or tertiary markings in the document or query terms.
自定义分词器和自定义辅助函数的组合可用于实现非对称搜索。标记生成器可以（比如说）为每个文档标记返回标记的大小写标准化且未标记的版本，后跟 0x00 字节，然后是文档中标记的全文。查询时，fts5 将提供结果，就好像查询中的所有字符都已大小写标准化且未标记。然后，可以在查询的 WHERE 子句中使用自定义辅助函数，以根据文档或查询术语中的二级或三级标记过滤掉任何不匹配的行。

4.8. The Locale Option
4.8.区域设置选项

This option is only useful to applications that implement custom tokenizers. If an fts5 table is created with the "locale=1" option specified, then the fts5_locale() SQL function may be used to associate a locale value (e.g. "th_TH" or "en_US") with strings passed to FTS5. FTS5 itself does not use locale values, but makes them available to the tokenizer implementation whenever the string is tokenized. The tokenizer may then adjust its behaviour based on the locale.
此选项仅对实现自定义分词器的应用程序有用。如果创建 fts5 表时指定了“locale=1”选项，则 fts5_locale() SQL 函数可用于将区域设置值（例如“th_TH”或“en_US”）与传递给 FTS5 的字符串相关联。 FTS5 本身不使用区域设置值，但每当字符串被标记化时，它们都可供标记生成器实现使用。然后，分词器可以根据区域设置调整其行为。

-- The following statement creates an fts5 table with locale support.
-- The "tokenizer=..." option below must be replaced with a real tokenizer
-- specification for a tokenizer that supports locales.
CREATE VIRTUAL TABLE ft USING fts5(a, b, locale=1, tokenizer=...);

-- This statement inserts a row into the table. The value inserted into
-- column "a" uses locale "th_TH", the value written to column "b" uses the
-- tokenizer's default locale
INSERT INTO ft(a, b) VALUES(
     fts5_locale('th_TH', 'Tokenize this in Thai locale'),
     'Tokenize this in the default locale.'
);

-- The "en_US" locale is used to tokenize the query terms in the 
--following query.
SELECT * FROM ft( fts5_locale('en_US', 'query terms') );

Attempting to pass an fts5_locale() string to an fts5 table that was not created with the locale=1 option is an error.
尝试将 fts5_locale() 字符串传递到不是使用 locale=1 选项创建的 fts5 表是错误的。

When an fts5_locale() string is stored in a normal content table (i.e. not a contentless or external content table), the attached locale is stored along with it. If the string is tokenized by FTS5 again, for example because its row is being deleted or as part of an auxiliary function evaluation, the attached locale is again passed to the tokenizer implementation.
当 fts5_locale() 字符串存储在普通内容表（即不是无内容或外部内容表）中时，附加的语言环境将与其一起存储。如果字符串再次被 FTS5 标记化，例如因为其行被删除或作为辅助函数评估的一部分，则附加的语言环境将再次传递给标记化器实现。

In order to support locales, an FTS5 external-content table may use an SQL view that returns fts5_locale() values as the content table. For example:
为了支持区域设置，FTS5 外部内容表可以使用返回 fts5_locale() 值作为内容表的 SQL 视图。例如：

-- Each row of this table contains a string and its locale.
CREATE TABLE t1(val, locale);
INSERT INTO t1 VALUES('a text value', 'en_US');

-- A view to combine the string and locale from table t1.
CREATE VIEW v1 AS SELECT rowid, fts5_locale(val, locale) AS val FROM t1;

-- An FTS5 table to read locale-enabled strings from view v1.
CREATE VIRTUAL TABLE ft USING fts5(val, locale=1, content=v1, tokenize=...);

If an fts5_locale() value is written to an UNINDEXED column of an fts5 table, the locale value is discarded and the string stored by itself.
如果将 fts5_locale() 值写入 fts5 表的 UNINDEXED 列，则区域设置值将被丢弃，字符串将自行存储。

The fts5_get_locale() function may be used to retreive the locale of a value stored in an FTS5 table.
fts5_get_locale()函数可用于检索存储在 FTS5 表中的值的区域设置。

4.9. The Contentless-Unindexed Option
4.9.无内容无索引选项

Usually, UNINDEXED columns belonging to contentless tables are not very useful. Values written to them are not indexed or stored, and reading from such an UNINDEXED column always returns NULL. However, if the "contentless_unindexed=1" option is specified on a contentless table, then the values of UNINDEXED columns are stored persistently, even though values written to other columns are not.
通常，属于无内容表的UNINDEXED列不是很有用。写入其中的值不会被索引或存储，并且从此类 UNINDEXED 列中读取始终返回 NULL。但是，如果在无内容表上指定“contentless_unindexed=1”选项，则 UNINDEXED 列的值将被持久存储，即使写入其他列的值不会被持久存储。

-- Create a contentless table with the contentless_unindexed=1 option.
-- Of the row written to it, the value 'one' will be indexed and then
-- discarded, and the value "1" will be stored but not indexed.
CREATE VIRTUAL TABLE ft USING fts5(a, b UNINDEXED, content='', contentless_unindexed=1);
INSERT INTO ft(a, b) VALUES('one', 1);

-- This query returns 1 row with 2 columns - (NULL, 1). Reading from 
-- column "a" is always NULL, as the table is contentless. But reading from
-- "b" returns the value, as the table uses contentless_unindexed=1.
SELECT a, b FROM ft('one');

It is an error to specify contentless_unindexed=1 for an fts5 table that is not a contentless or contentless-delete table.
对于不是无内容或无内容删除表的 fts5 表指定 contentless_unindexed=1 是错误的。

5. Auxiliary Functions
5.辅助功能

Auxiliary functions are similar to SQL scalar functions, except that they may only be used within full-text queries (those that use the MATCH operator, or LIKE/GLOB with the trigram tokenizer) on an FTS5 table. Their results are calculated based not only on the arguments passed to them, but also on the current match and matched row. For example, an auxiliary function may return a numeric value indicating the accuracy of the match (see the bm25() function), or a fragment of text from the matched row that contains one or more instances of the search terms (see the snippet() function).
辅助函数与SQL 标量函数类似，不同之处在于它们只能在 FTS5 表的全文查询（使用 MATCH 运算符或带有三元标记生成器的 LIKE/GLOB 的查询）中使用。它们的结果不仅根据传递给它们的参数计算，还根据当前匹配和匹配的行计算。例如，辅助函数可能会返回一个指示匹配准确性的数值（请参阅bm25()函数），或者来自匹配行的包含一个或多个搜索词实例的文本片段（请参阅代码片段（ ）功能）。

To invoke an auxiliary function, the name of the FTS5 table should be specified as the first argument. Other arguments may follow the first, depending on the specific auxiliary function being invoked. For example, to invoke the "highlight" function:
要调用辅助函数，应将 FTS5 表的名称指定为第一个参数。其他参数可能跟随第一个参数，具体取决于所调用的特定辅助函数。例如，要调用“突出显示”功能：

-- Assuming fts5 table:
CREATE VIRTUAL TABLE ft USING fts5(a, b, c);

-- Invoke the highlight() function:
SELECT highlight(ft, 2, '<b>', '</b>') FROM ft WHERE ft MATCH 'fts5'

The built-in auxiliary functions provided as part of FTS5 are described in the following section. Applications may also implement custom auxiliary functions in C.
作为 FTS5 一部分提供的内置辅助功能将在以下部分中描述。应用程序还可以用 C 实现自定义辅助函数。

5.1. Built-in Auxiliary Functions
5.1.内置辅助功能

FTS5 provides three built-in auxiliary functions:
FTS5提供了三个内置辅助功能：

• The bm25() auxiliary function returns a real value reflecting the accuracy of the current match. Better matches are assigned numerically lower values.
  bm25() 辅助函数返回反映当前匹配精度的实际值。更好的匹配被分配较低的数值。
• The highlight() auxiliary function returns a copy of the text from one of the columns of the current match with each instance of a queried term within the result surrounded by specified markup (for example "<b>" and "</b>").
  highlight() 辅助函数返回当前匹配的一列中的文本副本，其中包含结果中被指定标记包围的查询术语的每个实例（例如“<b>”和“</b”） >”）。
• The snippet() auxiliary function selects a short fragment of text from one of the columns of the matched row and returns it with each instance of a queried term surrounded by markup in the same manner as the highlight() function. The fragment of text is selected so as to maximize the number of distinct queried terms it contains. Higher weight is given to snippets that occur at the start of a column value, or that immediately follow "." or ":" characters in the text.
  snippet() 辅助函数从匹配行的一列中选择一小段文本，并将其与由标记包围的查询术语的每个实例一起返回，其方式与highlight() 函数相同。选择文本片段以使其包含的不同查询术语的数量最大化。出现在列值开头或紧跟在“.”之后的片段的权重较高。或文本中的“：”字符。
• The fts5_get_locale() auxiliary function is used to retrieve the locale, if any, associated with a value stored in an FTS5 table.
  fts5_get_locale() 辅助函数用于检索与存储在 FTS5 表中的值关联的区域设置（如果有）。

5.1.1. The bm25() function
5.1.1. bm25() 函数

The built-in auxiliary function bm25() returns a real value indicating how well the current row matches the full-text query. The better the match, the numerically smaller the value returned. A query such as the following may be used to return matches in order from best to worst match:
内置辅助函数 bm25() 返回一个实数，指示当前行与全文查询的匹配程度。匹配越好，返回的值在数值上越小。如下所示的查询可用于按从最佳匹配到最差匹配的顺序返回匹配项：

SELECT * FROM ft WHERE ft MATCH ? ORDER BY bm25(ft)

In order to calculate a documents score, the full-text query is separated into its component phrases. The bm25 score for document D and query Q is then calculated as follows:
为了计算文档分数，全文查询被分成其组成短语。然后，文档D和查询Q的 bm25 分数计算如下：

In the above, nPhrase is the number of phrases in the query. |D| is the number of tokens in the current document, and avgdl is the average number of tokens in all documents within the FTS5 table. k₁ and b are both constants, hard-coded at 1.2 and 0.75 respectively.
在上面， nPhrase是查询中的短语数。 |D|是当前文档中的标记数量， avgdl是 FTS5 表中所有文档的平均标记数量。 k ₁和b都是常数，分别硬编码为 1.2 和 0.75。

The "-1" term at the start of the formula is not found in most implementations of the BM25 algorithm. Without it, a better match is assigned a numerically higher BM25 score. Since the default sorting order is "ascending", this means that appending "ORDER BY bm25(ft)" to a query would cause results to be returned in order from worst to best. The "DESC" keyword would be required in order to return the best matches first. In order to avoid this pitfall, the FTS5 implementation of BM25 multiplies the result by -1 before returning it, ensuring that better matches are assigned numerically lower scores.
在 BM25 算法的大多数实现中都找不到公式开头的“-1”项。如果没有它，更好的匹配会分配更高的 BM25 分数。由于默认排序顺序是“升序”，这意味着将“ORDER BY bm25(ft)”附加到查询将导致结果按从最差到最佳的顺序返回。为了首先返回最佳匹配，需要使用“DESC”关键字。为了避免这个陷阱，BM25 的 FTS5 实现在返回之前将结果乘以 -1，确保更好的匹配被分配较低的分数。

IDF(q_i) is the inverse-document-frequency of query phrase i. It is calculated as follows, where N is the total number of rows in the FTS5 table and n(q_i) is the total number of rows that contain at least one instance of phrase i:
IDF(q _i )是查询短语i的逆文档频率。其计算方式如下，其中N是 FTS5 表中的总行数， n(q _i )是包含至少一个短语i实例的总行数：

Finally, f(q_i,D) is the phrase frequency of phrase i. By default, this is simply the number of occurrences of the phrase within the current row. However, by passing extra real value arguments to the bm25() SQL function, each column of the table may be assigned a different weight and the phrase frequency calculated as follows:
最后， f(q _i ,D)是短语i的短语频率。默认情况下，这只是该短语在当前行中出现的次数。然而，通过将额外的实数值参数传递给 bm25() SQL 函数，表的每一列可以被分配不同的权重，并且短语频率计算如下：

where w_c is the weight assigned to column c and n(q_i,c) is the number of occurrences of phrase i in column c of the current row. The first argument passed to bm25() following the table name is the weight assigned to the leftmost column of the FTS5 table. The second is the weight assigned to the second leftmost column, and so on. If there are not enough arguments for all table columns, remaining columns are assigned a weight of 1.0. If there are too many trailing arguments, the extras are ignored. For example:
其中w _c是分配给c列的权重， n(q _i ,c)是当前行c列中短语i出现的次数。表名后面传递给 bm25() 的第一个参数是分配给 FTS5 表最左边列的权重。第二个是分配给最左边第二列的权重，依此类推。如果所有表列都没有足够的参数，则为剩余列分配权重 1.0。如果尾随参数太多，则多余的参数将被忽略。例如：

-- Assuming the following schema:
CREATE VIRTUAL TABLE email USING fts5(sender, title, body);

-- Return results in bm25 order, with each phrase hit in the "sender"
-- column considered the equal of 10 hits in the "body" column, and
-- each hit in the "title" column considered as valuable as 5 hits in
-- the "body" column.
SELECT * FROM email WHERE email MATCH ? ORDER BY bm25(email, 10.0, 5.0);

Refer to wikipedia for more information regarding BM25 and its variants.
有关 BM25 及其变体的更多信息，请参阅维基百科。

5.1.2. The highlight() function
5.1.2.高亮显示()函数

The highlight() function returns a copy of the text from a specified column of the current row with extra markup text inserted to mark the start and end of phrase matches.
highlight() 函数返回当前行指定列中的文本副本，并插入额外的标记文本来标记短语匹配的开始和结束。

The highlight() must be invoked with exactly three arguments following the table name. To be interpreted as follows:
必须使用紧跟在表名后面的三个参数来调用highlight()。解释如下：

1. An integer indicating the index of the FTS table column to read the text from. Columns are numbered from left to right starting at zero.
   一个整数，指示要从中读取文本的 FTS 表列的索引。列从零开始从左到右编号。
2. The text to insert before each phrase match.
   要在每个短语匹配之前插入的文本。
3. The text to insert after each phrase match.
   要在每个短语匹配后插入的文本。

For example:  例如：

-- Return a copy of the text from the leftmost column of the current
-- row, with phrase matches marked using html "b" tags.
SELECT highlight(ft, 0, '<b>', '</b>') FROM ft WHERE ft MATCH ?

In cases where two or more phrase instances overlap (share one or more tokens in common), a single open and close marker is inserted for each set of overlapping phrases. For example:
在两个或多个短语实例重叠（共享一个或多个共同标记）的情况下，将为每组重叠短语插入一个开始和结束标记。例如：

-- Assuming this:
CREATE VIRTUAL TABLE ft USING fts5(a);
INSERT INTO ft VALUES('a b c x c d e');
INSERT INTO ft VALUES('a b c c d e');
INSERT INTO ft VALUES('a b c d e');

-- The following SELECT statement returns these three rows:
--   '[a b c] x [c d e]'
--   '[a b c] [c d e]'
--   '[a b c d e]'
SELECT highlight(ft, 0, '[', ']') FROM ft WHERE ft MATCH 'a+b+c AND c+d+e';

5.1.3. The snippet() function
5.1.3. snippet() 函数

The snippet() function is similar to highlight(), except that instead of returning entire column values, it automatically selects and extracts a short fragment of document text to process and return. The snippet() function must be passed five parameters following the table name argument:
snippet() 函数与highlight() 类似，不同之处在于它不是返回整个列值，而是自动选择并提取一小段文档文本来处理和返回。 snippet() 函数必须在表名参数之后传递五个参数：

1. An integer indicating the index of the FTS table column to select the returned text from. Columns are numbered from left to right starting at zero. A negative value indicates that the column should be automatically selected.
   一个整数，指示要从中选择返回文本的 FTS 表列的索引。列从零开始从左到右编号。负值表示应自动选择该列。
2. The text to insert before each phrase match within the returned text.
   要在返回的文本中匹配的每个短语之前插入的文本。
3. The text to insert after each phrase match within the returned text.
   要在返回文本中的每个短语匹配之后插入的文本。
4. The text to add to the start or end of the selected text to indicate that the returned text does not occur at the start or end of its column, respectively.
   添加到所选文本的开头或结尾的文本，以指示返回的文本分别不会出现在其列的开头或结尾。
5. The maximum number of tokens in the returned text. This must be greater than zero and equal to or less than 64.
   返回文本中的最大标记数。该值必须大于零且等于或小于 64。

5.1.4. The fts5_get_locale() function
5.1.4. fts5_get_locale() 函数

The fts5_get_locale() function is used to retrieve the locale, if any, associated with a value stored in an FTS5 table. It accepts a single argument following the table name, the index of the column of the current row to query. Columns are numbered in the order they appeared in the CREATE VIRTUAL TABLE statement starting from 0.
fts5_get_locale() 函数用于检索与 FTS5 表中存储的值关联的区域设置（如果有）。它接受表名后面的单个参数，即要查询的当前行的列的索引。列按照它们在 CREATE VIRTUAL TABLE 语句中出现的顺序从 0 开始编号。

If the FTS5 table does not support locales (i.e. was not created with the locale=1 option), or if there is no locale associated with the nominated value, then this function returns NULL. Otherwise it returns a text value, the name of the locale that the value in question is associated with.
如果 FTS5 表不支持区域设置（即不是使用locale=1选项创建的），或者如果没有与指定值关联的区域设置，则此函数返回 NULL。否则，它返回一个文本值，即与该值关联的区域设置的名称。

CREATE VIRTUAL TABLE ft USING fts5(a, b, c, locale=1);
INSERT INTO ft VALUES(
    'no locale', 
    fts5_locale('th_TH', 'Thai locale'), 
    fts5_locale('en_US', 'US locale')
);

-- The following statement returns a single row containing three values:
-- NULL, text value 'th_TH', and text value 'en_US'.
SELECT 
    fts5_get_locale(ft, 0), 
    fts5_get_locale(ft, 1), 
    fts5_get_locale(ft, 2)
FROM ft;

5.2. Sorting by Auxiliary Function Results
5.2.按辅助功能结果排序

All FTS5 tables feature a special hidden column named "rank". If the current query is not a full-text query (i.e. if it does not include a MATCH operator), the value of the "rank" column is always NULL. Otherwise, in a full-text query, column rank contains by default the same value as would be returned by executing the bm25() auxiliary function with no trailing arguments.
所有 FTS5 表都有一个名为“rank”的特殊隐藏列。如果当前查询不是全文查询（即，如果它不包含 MATCH 运算符），则“rank”列的值始终为 NULL。否则，在全文查询中，列排名默认包含与执行不带尾随参数的 bm25() 辅助函数返回的值相同的值。

The difference between reading from the rank column and using the bm25() function directly within the query is only significant when sorting by the returned value. In this case, using "rank" is faster than using bm25().
仅当按返回值排序时，从排名列读取与直接在查询中使用 bm25() 函数之间的差异才显着。在这种情况下，使用“rank”比使用 bm25() 更快。

-- The following queries are logically equivalent. But the second may
-- be faster, particularly if the caller abandons the query before
-- all rows have been returned (or if the queries were modified to 
-- include LIMIT clauses).
SELECT * FROM ft WHERE ft MATCH ? ORDER BY bm25(ft);
SELECT * FROM ft WHERE ft MATCH ? ORDER BY rank;

Instead of using bm25() with no trailing arguments, the specific auxiliary function mapped to the rank column may be configured either on a per-query basis, or by setting a different persistent default for the FTS table.
映射到排名列的特定辅助函数可以基于每个查询进行配置，也可以通过为 FTS 表设置不同的持久默认值来配置，而不是使用不带尾随参数的 bm25()。

In order to change the mapping of the rank column for a single query, a term similar to either of the following is added to the WHERE clause of a query:
为了更改单个查询的排名列的映射，将类似于以下任一的术语添加到查询的 WHERE 子句中：

rank MATCH 'auxiliary-function-name(arg1, arg2, ...)'
rank = 'auxiliary-function-name(arg1, arg2, ...)'

The right-hand-side of the MATCH or = operator must be a constant expression that evaluates to a string consisting of the auxiliary function to invoke, followed by zero or more comma separated arguments within parenthesis. Arguments must be SQL literals. For example:
MATCH 或 = 运算符的右侧必须是常量表达式，其计算结果为由要调用的辅助函数组成的字符串，后跟括号内的零个或多个逗号分隔的参数。参数必须是 SQL 文字。例如：

-- The following queries are logically equivalent. But the second may
-- be faster. See above. 
SELECT * FROM ft WHERE ft MATCH ? ORDER BY bm25(ft, 10.0, 5.0);
SELECT * FROM ft WHERE ft MATCH ? AND rank MATCH 'bm25(10.0, 5.0)' ORDER BY rank;

The table-valued function syntax may also be used to specify an alternative ranking function. In this case the text describing the ranking function should be specified as the second table-valued function argument. The following three queries are equivalent:
表值函数语法也可用于指定替代的排名函数。在这种情况下，描述排名函数的文本应指定为第二个表值函数参数。以下三个查询是等效的：

SELECT * FROM ft WHERE ft MATCH ? AND rank MATCH 'bm25(10.0, 5.0)' ORDER BY rank;
SELECT * FROM ft WHERE ft = ? AND rank = 'bm25(10.0, 5.0)' ORDER BY rank;
SELECT * FROM ft WHERE ft(?, 'bm25(10.0, 5.0)') ORDER BY rank;

The default mapping of the rank column for a table may be modified using the FTS5 rank configuration option.

6. Special INSERT Commands

6.1. The 'automerge' Configuration Option

Instead of using a single data structure on disk to store the full-text index, FTS5 uses a series of b-trees. Each time a new transaction is committed, a new b-tree containing the contents of the committed transaction is written into the database file. When the full-text index is queried, each b-tree must be queried individually and the results merged before being returned to the user.

In order to prevent the number of b-trees in the database from becoming too large (slowing down queries), smaller b-trees are periodically merged into single larger b-trees containing the same data. By default, this happens automatically within INSERT, UPDATE or DELETE statements that modify the full-text index. The 'automerge' parameter determines how many smaller b-trees are merged together at a time. Setting it to a small value can speed up queries (as they have to query and merge the results from fewer b-trees), but can also slow down writing to the database (as each INSERT, UPDATE or DELETE statement has to do more work as part of the automatic merging process).

Each of the b-trees that make up the full-text index is assigned to a "level" based on its size. Level-0 b-trees are the smallest, as they contain the contents of a single transaction. Higher level b-trees are the result of merging two or more level-0 b-trees together and so they are larger. FTS5 begins to merge b-trees together once there exist M or more b-trees with the same level, where M is the value of the 'automerge' parameter.

The maximum allowed value for the 'automerge' parameter is 16. The default value is 4. Setting the 'automerge' parameter to 0 disables the automatic incremental merging of b-trees altogether.

INSERT INTO ft(ft, rank) VALUES('automerge', 8);

6.2. The 'crisismerge' Configuration Option

The 'crisismerge' option is similar to 'automerge', in that it determines how and how often the component b-trees that make up the full-text index are merged together. Once there exist C or more b-trees on a single level within the full-text index, where C is the value of the 'crisismerge' option, all b-trees on the level are immediately merged into a single b-tree.

The difference between this option and the 'automerge' option is that when the 'automerge' limit is reached FTS5 only begins to merge the b-trees together. Most of the work is performed as part of subsequent INSERT, UPDATE or DELETE operations. Whereas when the 'crisismerge' limit is reached, the offending b-trees are all merged immediately. This means that an INSERT, UPDATE or DELETE that triggers a crisis-merge may take a long time to complete.

The default 'crisismerge' value is 16. There is no maximum limit. Attempting to set the 'crisismerge' parameter to a value of 0 or 1 is equivalent to setting it to the default value (16). It is an error to attempt to set the 'crisismerge' option to a negative value.

INSERT INTO ft(ft, rank) VALUES('crisismerge', 16);

6.3. The 'delete' Command

This command is only available with external content and contentless tables. It is used to delete the index entries associated with a single row from the full-text index. This command and the delete-all command are the only ways to remove entries from the full-text index of a contentless table.

In order to use this command to delete a row, the text value 'delete' must be inserted into the special column with the same name as the table. The rowid of the row to delete is inserted into the rowid column. The values inserted into the other columns must match the values currently stored in the table. For example:

-- Insert a row with rowid=14 into the fts5 table.
INSERT INTO ft(rowid, a, b, c) VALUES(14, $a, $b, $c);

-- Remove the same row from the fts5 table.
INSERT INTO ft(ft, rowid, a, b, c) VALUES('delete', 14, $a, $b, $c);

If the values "inserted" into the text columns as part of a 'delete' command are not the same as those currently stored within the table, the results may be unpredictable.

The reason for this is easy to understand: When a document is inserted into the FTS5 table, an entry is added to the full-text index to record the position of each token within the new document. When a document is removed, the original data is required in order to determine the set of entries that need to be removed from the full-text index. So if the data supplied to FTS5 when a row is deleted using this command is different from that used to determine the set of token instances when it was inserted, some full-text index entries may not be correctly deleted, or FTS5 may try to remove index entries that do not exist. This can leave the full-text index in an unpredictable state, making future query results unreliable.

6.4. The 'delete-all' Command

This command is only available with external content and contentless tables (including contentless-delete tables). It deletes all entries from the full-text index.

INSERT INTO ft(ft) VALUES('delete-all');

6.5. The 'deletemerge' Configuration Option

The 'deletemerge' option is only used by contentless-delete tables.

When a row is deleted from a contentless-delete table, the entries associated with its tokens are not immediately removed from the FTS index. Instead, a "tombstone" marker containing the rowid of the deleted row is attached to the b-tree that contains the row's FTS index entries. When the b-tree is queried, any query result rows for which there exist tombstone markers are omitted from the results. When the b-tree is merged with other b-trees, both the deleted rows and their tombstone markers are discarded.
当从无内容删除表中删除一行时，与其标记关联的条目不会立即从 FTS 索引中删除。相反，包含已删除行的 rowid 的“逻辑删除”标记将附加到包含该行的 FTS 索引条目的 b 树。查询 b 树时，结果中会省略任何存在逻辑删除标记的查询结果行。当 B 树与其他 B 树合并时，删除的行及其逻辑删除标记都将被丢弃。

This option specifies a minimum percentage of rows in a b-tree that must have tombstone markers before the b-tree is made eligible for merging - either by automatic merges or explicit user 'merge' commands - even if it does not meet the usual criteria as determined by the 'automerge' and 'usermerge' options.
此选项指定在使 B 树符合合并条件之前，B 树中必须具有逻辑删除标记的行的最小百分比 - 通过自动合并或显式用户“合并”命令 - 即使它不符合通常的标准由“automerge”和“usermerge”选项确定。

For example, to specify that FTS5 should consider merging a component b-tree after 15% of its rows have associated tombstone markers:
例如，要指定 FTS5 应考虑在 15% 的行具有关联的逻辑删除标记后合并组件 B 树：

INSERT INTO ft(ft, rank) VALUES('deletemerge', 15);

The default value of this option is 10. Attempting to set it to less than zero restores the default value. Setting this option to 0 or to greater than 100 ensures that b-trees are never made eligible for merging due to tombstone markers.
该选项的默认值为 10。尝试将其设置为小于零会恢复默认值。将此选项设置为 0 或大于 100 可确保 B 树永远不会因逻辑删除标记而符合合并条件。

6.6. The 'integrity-check' Command
6.6。 “完整性检查”命令

This command is used to verify that the full-text index is internally consistent, and, optionally, that it is consistent with any external content table.
此命令用于验证全文索引是否内部一致，并且（可选）它是否与任何外部内容表一致。

The integrity-check command is invoked by inserting the text value 'integrity-check' into the special column with the same name as the FTS5 table. If a value is supplied for the "rank" column, it must be either 0 or 1. For example:
通过将文本值“integrity-check”插入与 FTS5 表同名的特殊列来调用完整性检查命令。如果为“rank”列提供了值，则该值必须是 0 或 1。例如：

INSERT INTO ft(ft) VALUES('integrity-check');
INSERT INTO ft(ft, rank) VALUES('integrity-check', 0);
INSERT INTO ft(ft, rank) VALUES('integrity-check', 1);

The three forms above are equivalent for all FTS tables that are not external content tables. They check that the index data structures are not corrupt, and, if the FTS table is not contentless, that the contents of the index match the contents of the table itself.
上述三种形式对于所有非外部内容表的 FTS 表是等效的。它们检查索引数据结构是否未损坏，并且如果 FTS 表不是无内容的，则索引的内容是否与表本身的内容匹配。

For an external content table, the contents of the index are only compared to the contents of the external content table if the value specified for the rank column is 1.
对于外部内容表，如果为排名列指定的值为 1，则仅将索引的内容与外部内容表的内容进行比较。

In all cases, if any discrepancies are found, the command fails with an SQLITE_CORRUPT_VTAB error.
在所有情况下，如果发现任何差异，命令都会失败并出现SQLITE_CORRUPT_VTAB错误。

6.7. The 'merge' Command
6.7. “合并”命令

INSERT INTO ft(ft, rank) VALUES('merge', 500);

This command merges b-tree structures together until roughly N pages of merged data have been written to the database, where N is the absolute value of the parameter specified as part of the 'merge' command. The size of each page is as configured by the FTS5 pgsz option.
此命令将 b 树结构合并在一起，直到大约 N 页的合并数据写入数据库，其中 N 是“merge”命令中指定的参数的绝对值。每个页面的大小由FTS5 pgsz 选项配置。

If the parameter is a positive value, B-tree structures are only eligible for merging if one of the following is true:
如果参数为正值，则仅当满足以下条件之一时，B 树结构才符合合并条件：

• There are U or more such b-trees on a single level (see the documentation for the FTS5 automerge option for an explanation of b-tree levels), where U is the value assigned to the FTS5 usermerge option option.
  单个级别上有 U 个或更多此类 B 树（有关 B 树级别的说明，请参阅FTS5 自动合并选项的文档），其中 U 是分配给FTS5 用户合并选项的值。
• A merge has already been started (perhaps by a 'merge' command that specified a negative parameter).
  合并已经开始（可能是通过指定负参数的“合并”命令）。

It is possible to tell whether or not the 'merge' command found any b-trees to merge together by checking the value returned by the sqlite3_total_changes() API before and after the command is executed. If the difference between the two values is 2 or greater, then work was performed. If the difference is less than 2, then the 'merge' command was a no-op. In this case there is no reason to execute the same 'merge' command again, at least until after the FTS table is next updated.
通过在执行命令之前和之后检查sqlite3_total_changes() API 返回的值，可以判断“merge”命令是否找到任何要合并在一起的 b 树。如果两个值之间的差值是 2 或更大，则完成工作。如果差值小于 2，则“合并”命令无效。在这种情况下，没有理由再次执行相同的“合并”命令，至少在 FTS 表下次更新之前是这样。

If the parameter is negative, and there are B-tree structures on more than one level within the FTS index, all B-tree structures are assigned to the same level before the merge operation is commenced. Additionally, if the parameter is negative, the value of the usermerge configuration option is not respected - as few as two b-trees from the same level may be merged together.
如果该参数为负，并且 FTS 索引内的多个 B 树结构存在于一级，则在开始合并操作之前，所有 B 树结构都将分配到同一级别。此外，如果参数为负数，则不考虑 usermerge 配置选项的值 - 同一级别的两棵 B 树可能会合并在一起。

The above means that executing the 'merge' command with a negative parameter until the before and after difference in the return value of sqlite3_total_changes() is less than two optimizes the FTS index in the same way as the FTS5 optimize command. However, if a new b-tree is added to the FTS index while this process is ongoing, FTS5 will move the new b-tree to the same level as the existing b-trees and restart the merge. To avoid this, only the first call to 'merge' should specify a negative parameter. Each subsequent call to 'merge' should specify a positive value so that the merge started by the first call is run to completion even if new b-trees are added to the FTS index.
上面的意思是，执行带有负参数的 'merge' 命令，直到sqlite3_total_changes()返回值的前后差值小于 2 为止，以与FTS5 优化命令相同的方式优化 FTS 索引。但是，如果在此过程正在进行时将新的 B 树添加到 FTS 索引中，FTS5 会将新的 B 树移动到与现有 B 树相同的级别并重新启动合并。为了避免这种情况，只有第一次调用“merge”时才应该指定负参数。对“merge”的每个后续调用都应指定一个正值，以便即使将新的 b 树添加到 FTS 索引，第一次调用启动的合并也会运行完成。

6.8. The 'optimize' Command
6.8。 “优化”命令

This command merges all individual b-trees that currently make up the full-text index into a single large b-tree structure. This ensures that the full-text index consumes the minimum space within the database and is in the fastest form to query.
此命令将当前构成全文索引的所有单独的 B 树合并为一个大型 B 树结构。这确保全文索引占用数据库内最小的空间并且以最快的形式进行查询。

Refer to the documentation for the FTS5 automerge option for more details regarding the relationship between the full-text index and its component b-trees.
有关全文索引及其组件 B 树之间关系的更多详细信息，请参阅FTS5 自动合并选项的文档。

INSERT INTO ft(ft) VALUES('optimize');

Because it reorganizes the entire FTS index, the optimize command can take a long time to run. The FTS5 merge command can be used to divide the work of optimizing the FTS index into multiple steps. To do this:
由于它会重新组织整个 FTS 索引，因此优化命令可能需要很长时间才能运行。 FTS5 merge命令可用于将优化FTS索引的工作分为多个步骤。为此：

• Invoke the 'merge' command once with the parameter set to -N, then
  调用“merge”命令一次，并将参数设置为-N，然后
• Invoke the 'merge' command zero or more times with the parameter set to N.
  调用“merge”命令零次或多次，并将参数设置为 N。

where N is the number of pages of data to merge within each invocation of the merge command. The application should stop invoking merge when the difference in the value returned by the sqlite3_total_changes() function before and after the merge command drops to below two. The merge commands may be issued as part of the same or separate transactions, and by the same or different database clients. Refer to the documentation for the merge command for further details.
其中 N 是每次调用合并命令时要合并的数据页数。当合并命令前后 sqlite3_total_changes() 函数返回的值的差值降至 2 以下时，应用程序应停止调用合并。合并命令可以作为相同或单独事务的一部分并且由相同或不同的数据库客户端发出。有关更多详细信息，请参阅合并命令的文档。

6.9. The 'pgsz' Configuration Option
6.9。 “pgsz”配置选项

This command is used to set the persistent "pgsz" option.
该命令用于设置持久的“pgsz”选项。

The full-text index maintained by FTS5 is stored as a series of fixed-size blobs in a database table. It is not strictly necessary for all blobs that make up a full-text index to be the same size. The pgsz option determines the size of all blobs created by subsequent index writers. The default value is 4050.
FTS5 维护的全文索引作为一系列固定大小的 blob 存储在数据库表中。构成全文索引的所有 blob 并不严格需要具有相同的大小。 pgsz 选项确定后续索引写入器创建的所有 blob 的大小。默认值为 4050。

INSERT INTO ft(ft, rank) VALUES('pgsz', 4072);

6.10. The 'rank' Configuration Option
6.10。 “等级”配置选项

This command is used to set the persistent "rank" option.
该命令用于设置持久的“rank”选项。

The rank option is used to change the default auxiliary function mapping for the rank column. The option should be set to a text value in the same format as described for "rank MATCH ?" terms above. For example:
排名选项用于更改排名列的默认辅助函数映射。该选项应设置为与“rank MATCH ?”所述格式相同的文本值。上述条款。例如：

INSERT INTO ft(ft, rank) VALUES('rank', 'bm25(10.0, 5.0)');

6.11. The 'rebuild' Command
6.11. “重建”命令

This command first deletes the entire full-text index, then rebuilds it based on the contents of the table or content table. It is not available with contentless tables.
该命令首先删除整个全文索引，然后根据表或内容表的内容重建它。它不适用于无内容表。

INSERT INTO ft(ft) VALUES('rebuild');

6.12. The 'secure-delete' Configuration Option
6.12. “安全删除”配置选项

This command is used to set the persistent boolean "secure-delete" option. For example:
该命令用于设置持久布尔“secure-delete”选项。例如：

INSERT INTO ft(ft, rank) VALUES('secure-delete', 1);

Normally, when an entry in an fts5 table is updated or deleted, instead of removing entries from the full-text index, delete-keys are added to the new b-tree created by the transaction. This is efficient, but it means that the old full-text index entries remain in the database file until they are eventually removed by merge operations on the full-text index. Anyone with access to the database can use these entries to trivially reconstruct the contents of deleted FTS5 table rows. However, if the 'secure-delete' option is set to 1, then full-text entries are actually removed from the database when existing FTS5 table rows are updated or deleted. This is slower, but it prevents old full-text entries from being used to reconstruct deleted table rows.
通常，当更新或删除 fts5 表中的条目时，不是从全文索引中删除条目，而是将删除键添加到事务创建的新 B 树中。这是有效的，但这意味着旧的全文索引条目保留在数据库文件中，直到它们最终被全文索引上的合并操作删除。任何有权访问数据库的人都可以使用这些条目轻松重建已删除的 FTS5 表行的内容。但是，如果“secure-delete”选项设置为 1，则当更新或删除现有 FTS5 表行时，实际上会从数据库中删除全文条目。这速度较慢，但​​可以防止使用旧的全文条目来重建已删除的表行。

This option ensures that old full-text entries are not available to attackers with SQL access to the database. To also ensure that they may not be recovered by attackers with access to the SQLite database file itself, the application must also enable the SQLite core secure-delete option with a command like "PRAGMA secure_delete = 1".
此选项可确保对数据库具有 SQL 访问权限的攻击者无法使用旧的全文条目。为了确保它们不会被有权访问 SQLite 数据库文件本身的攻击者恢复，应用程序还必须使用“PRAGMA secure_delete = 1”等命令启用 SQLite 核心安全删除选项。

Warning: Once one or more table rows have been updated or deleted with this option set, the FTS5 table may no longer be read or written by any version of FTS5 earlier than 3.42.0 (the first version in which this option was available). Attempting to do so results in an error, with an error message like "invalid fts5 file format (found 5, expected 4) - run 'rebuild'". The FTS5 file format may be reverted, so that it may be read by earlier versions of FTS5, by running the 'rebuild' command on the table using version 3.42.0 or later.
警告：使用此选项集更新或删除一个或多个表行后，任何早于 3.42.0 的 FTS5 版本（此选项可用的第一个版本）可能无法再读取或写入 FTS5 表。尝试这样做会导致错误，并显示一条错误消息，例如“无效的 fts5 文件格式（找到 5 个，预期 4 个） - 运行‘重建’”。通过在使用 3.42.0 或更高版本的表上运行“rebuild”命令，可以恢复 FTS5 文件格式，以便早期版本的 FTS5 可以读取它。

The default value of the secure-delete option is 0.
安全删除选项的默认值为 0。

6.13. The 'usermerge' Configuration Option
6.13。 “usermerge”配置选项

This command is used to set the persistent "usermerge" option.
该命令用于设置持久的“usermerge”选项。

The usermerge option is similar to the automerge and crisismerge options. It is the minimum number of b-tree segments that will be merged together by a 'merge' command with a positive parameter. For example:
usermerge 选项类似于 automerge 和 Crisismerge 选项。它是通过带有正参数的“合并”命令合并在一起的 B 树段的最小数量。例如：

INSERT INTO ft(ft, rank) VALUES('usermerge', 4);

The default value of the usermerge option is 4. The minimum allowed value is 2, and the maximum 16.
usermerge 选项的默认值为 4。允许的最小值为 2，最大值为 16。

7. Extending FTS5
7.扩展 FTS5

FTS5 features APIs allowing it to be extended by:
FTS5 具有 API，允许通过以下方式进行扩展：

• Adding new auxiliary functions implemented in C, and
  添加用 C 实现的新辅助函数，以及
• Adding new tokenizers, also implemented in C.
  添加新的分词器，也是用 C 实现的。

The built-in tokenizers and auxiliary functions described in this document are all implemented using the publicly available API described below.
本文档中描述的内置分词器和辅助函数都是使用下面描述的公开可用的 API 实现的。

Before a new auxiliary function or tokenizer implementation may be registered with FTS5, an application must obtain a pointer to the "fts5_api" structure. There is one fts5_api structure for each database connection with which the FTS5 extension is registered. To obtain the pointer, the application invokes the SQL user-defined function fts5() with a single argument. That argument must be set to a pointer to a pointer to an fts5_api object using the sqlite3_bind_pointer() interface. The following example code demonstrates the technique:
在向 FTS5 注册新的辅助函数或分词器实现之前，应用程序必须获取指向“fts5_api”结构的指针。注册 FTS5 扩展的每个数据库连接都有一个 fts5_api 结构。为了获取指针，应用程序使用单个参数调用 SQL 用户定义函数 fts5()。该参数必须使用sqlite3_bind_pointer()接口设置为指向 fts5_api 对象的指针。以下示例代码演示了该技术：

/*
** Return a pointer to the fts5_api pointer for database connection db.
** If an error occurs, return NULL and leave an error in the database
** handle (accessible using sqlite3_errcode()/errmsg()).
*/
fts5_api *fts5_api_from_db(sqlite3 *db){
  fts5_api *pRet = 0;
  sqlite3_stmt *pStmt = 0;

  if( SQLITE_OK==sqlite3_prepare(db, "SELECT fts5(?1)", -1, &pStmt, 0) ){
    sqlite3_bind_pointer(pStmt, 1, (void*)&pRet, "fts5_api_ptr", NULL);
    sqlite3_step(pStmt);
  }
  sqlite3_finalize(pStmt);
  return pRet;
}

Backwards Compatibility Warning: Prior to SQLite version 3.20.0 (2017-08-01), the fts5() worked slightly differently. Older applications that extend FTS5 must be revised to use the new technique shown above.
向后兼容性警告：在 SQLite 版本 3.20.0 (2017-08-01) 之前，fts5() 的工作方式略有不同。必须修改扩展 FTS5 的旧应用程序才能使用上面所示的新技术。

The fts5_api structure is defined as follows. It exposes five methods:
fts5_api结构体定义如下。它公开了五种方法：

• xCreateTokenizer() and xCreateTokenizer_v2(), for registering new custom tokenizer implementations.
  xCreateTokenizer() 和 xCreateTokenizer_v2()，用于注册新的自定义标记生成器实现。
• xFindTokenizer() and xFindTokenizer_v2(), for retrieving existing tokenizer implementations. This can be useful for implementing "tokenizer wrappers", similar to the built-in porter tokenizer.
  xFindTokenizer() 和 xFindTokenizer_v2()，用于检索现有标记生成器实现。这对于实现“标记器包装器”非常有用，类似于内置的 porter 标记器。
• xCreateFunction(), for registering new auxiliary function implementations.
  xCreateFunction()，用于注册新的辅助函数实现。

The two "v2" methods above are only available if the fts5_api.iVersion field is set to 3 or greater. Attempting to access the "v2" APIs via an fts5_api object with a lower value for iVersion results in undefined behaviour.
仅当 fts5_api.iVersion 字段设置为 3 或更大时，上述两个“v2”方法才可用。尝试通过 iVersion 值较低的 fts5_api 对象访问“v2”API 会导致未定义的行为。

typedef struct fts5_api fts5_api;
struct fts5_api {
  int iVersion;                   /* Currently always set to 3 */

  /* Create a new tokenizer */
  int (*xCreateTokenizer)(
    fts5_api *pApi,
    const char *zName,
    void *pUserData,
    fts5_tokenizer *pTokenizer,
    void (*xDestroy)(void*)
  );

  /* Find an existing tokenizer */
  int (*xFindTokenizer)(
    fts5_api *pApi,
    const char *zName,
    void **ppUserData,
    fts5_tokenizer *pTokenizer
  );

  /* Create a new auxiliary function */
  int (*xCreateFunction)(
    fts5_api *pApi,
    const char *zName,
    void *pUserData,
    fts5_extension_function xFunction,
    void (*xDestroy)(void*)
  );

  /* APIs below this point are only available if iVersion>=3 */

  /* Create a new tokenizer */
  int (*xCreateTokenizer_v2)(
    fts5_api *pApi,
    const char *zName,
    void *pUserData,
    fts5_tokenizer_v2 *pTokenizer,
    void (*xDestroy)(void*)
  );

  /* Find an existing tokenizer */
  int (*xFindTokenizer_v2)(
    fts5_api *pApi,
    const char *zName,
    void **ppUserData,
    fts5_tokenizer_v2 **ppTokenizer
  );
};

To invoke a method of the fts5_api object, the fts5_api pointer itself should be passed as the methods first argument followed by the other, method specific, arguments. For example:
要调用 fts5_api 对象的方法，fts5_api 指针本身应作为方法的第一个参数传递，后跟其他特定于方法的参数。例如：

rc = pFts5Api->xCreateTokenizer(pFts5Api, ... other args ...);

The fts5_api structure methods are described individually in the following sections.
fts5_api 结构方法将在以下部分中单独描述。

7.1. Custom Tokenizers
7.1.自定义分词器

To create a custom tokenizer, an application must implement three functions: a tokenizer constructor (xCreate), a destructor (xDelete) and a function to do the actual tokenization (xTokenize). The type of each function is as for the member variables of the fts5_tokenizer_v2 struct:
要创建自定义标记生成器，应用程序必须实现三个函数：标记生成器构造函数 (xCreate)、析构函数 (xDelete) 和执行实际标记化的函数 (xTokenize)。每个函数的类型与 fts5_tokenizer_v2 结构体的成员变量相同：

typedef struct Fts5Tokenizer Fts5Tokenizer;
typedef struct fts5_tokenizer_v2 fts5_tokenizer_v2;
struct fts5_tokenizer_v2 {
  int iVersion;             /* Currently always 2 */

  int (*xCreate)(void*, const char **azArg, int nArg, Fts5Tokenizer **ppOut);
  void (*xDelete)(Fts5Tokenizer*);
  int (*xTokenize)(Fts5Tokenizer*, 
      void *pCtx,
      int flags,            /* Mask of FTS5_TOKENIZE_* flags */
      const char *pText, int nText, 
      const char *pLocale, int nLocale,
      int (*xToken)(
        void *pCtx,         /* Copy of 2nd argument to xTokenize() */
        int tflags,         /* Mask of FTS5_TOKEN_* flags */
        const char *pToken, /* Pointer to buffer containing token */
        int nToken,         /* Size of token in bytes */
        int iStart,         /* Byte offset of token within input text */
        int iEnd            /* Byte offset of end of token within input text */
      )
  );
};

/* Flags that may be passed as the third argument to xTokenize() */
#define FTS5_TOKENIZE_QUERY     0x0001
#define FTS5_TOKENIZE_PREFIX    0x0002
#define FTS5_TOKENIZE_DOCUMENT  0x0004
#define FTS5_TOKENIZE_AUX       0x0008

/* Flags that may be passed by the tokenizer implementation back to FTS5
** as the third argument to the supplied xToken callback. */
#define FTS5_TOKEN_COLOCATED    0x0001      /* Same position as prev. token */

The implementation is registered with the FTS5 module by populating an instance of the fts5_tokenizer_v2 struct and passing a pointer to it to the xCreateTokenizer_v2() method of the fts5_api object. If there is already a tokenizer with the same name, it is replaced. If a non-NULL xDestroy parameter is passed to xCreateTokenizer(), it is invoked with a copy of the pUserData pointer passed as the only argument when the database handle is closed or when the tokenizer is replaced.
通过填充 fts5_tokenizer_v2 结构的实例并将指向该实例的指针传递给 fts5_api 对象的 xCreateTokenizer_v2() 方法，该实现在 FTS5 模块中注册。如果已经存在同名的分词器，则将其替换。如果将非 NULL xDestroy 参数传递给 xCreateTokenizer()，则在关闭数据库句柄或替换标记生成器时，将使用作为唯一参数传递的 pUserData 指针的副本来调用该参数。

If successful, xCreateTokenizer() returns SQLITE_OK. Otherwise, it returns an SQLite error code. In this case the xDestroy function is not invoked.
如果成功，xCreateTokenizer() 返回 SQLITE_OK。否则，它返回 SQLite 错误代码。在这种情况下，不会调用 xDestroy 函数。

When an FTS5 table uses the custom tokenizer, the FTS5 core calls xCreate() once to create a tokenizer, then xTokenize() zero or more times to tokenize strings, then xDelete() to free any resources allocated by xCreate(). More specifically:
当 FTS5 表使用自定义分词器时，FTS5 核心调用 xCreate() 一次来创建分词器，然后调用 xTokenize() 零次或多次来分词字符串，然后调用 xDelete() 来释放 xCreate() 分配的任何资源。更具体地说：

xCreate: x创建：

This function is used to allocate and initialize a tokenizer instance. A tokenizer instance is required to actually tokenize text.
该函数用于分配和初始化分词器实例。需要一个分词器实例来实际分词文本。

The first argument passed to this function is a copy of the (void*) pointer provided by the application when the fts5_tokenizer_v2 object was registered with FTS5 (the third argument to xCreateTokenizer()). The second and third arguments are an array of nul-terminated strings containing the tokenizer arguments, if any, specified following the tokenizer name as part of the CREATE VIRTUAL TABLE statement used to create the FTS5 table.
传递给此函数的第一个参数是在 fts5_tokenizer_v2 对象向 FTS5 注册时由应用程序提供的 (void*) 指针的副本（xCreateTokenizer() 的第三个参数）。第二个和第三个参数是一个以 nul 结尾的字符串数组，其中包含分词器参数（如果有），在分词器名称后面指定，作为用于创建 FTS5 表的 CREATE VIRTUAL TABLE 语句的一部分。

The final argument is an output variable. If successful, (*ppOut) should be set to point to the new tokenizer handle and SQLITE_OK returned. If an error occurs, some value other than SQLITE_OK should be returned. In this case, fts5 assumes that the final value of *ppOut is undefined.
最后一个参数是输出变量。如果成功， (*ppOut) 应设置为指向新的分词器句柄并返回 SQLITE_OK。如果发生错误，则应返回 SQLITE_OK 以外的其他值。在这种情况下，fts5 假定 *ppOut 的最终值未定义。

xDelete: x删除：

This function is invoked to delete a tokenizer handle previously allocated using xCreate(). Fts5 guarantees that this function will be invoked exactly once for each successful call to xCreate().
调用此函数来删除先前使用 xCreate() 分配的分词器句柄。 Fts5 保证每次成功调用 xCreate() 时都会调用该函数一次。

xTokenize: x标记化：

This function is expected to tokenize the nText byte string indicated by argument pText. pText may or may not be nul-terminated. The first argument passed to this function is a pointer to an Fts5Tokenizer object returned by an earlier call to xCreate().
该函数预计会对参数 pText 指示的 nText 字节字符串进行标记。 pText 可能会也可能不会以 null 结尾。传递给此函数的第一个参数是指向先前调用 xCreate() 返回的 Fts5Tokenizer 对象的指针。

The third argument indicates the reason that FTS5 is requesting tokenization of the supplied text. This is always one of the following four values:
第三个参数指示 FTS5 请求对所提供文本进行标记化的原因。这始终是以下四个值之一：

• FTS5_TOKENIZE_DOCUMENT - A document is being inserted into or removed from the FTS table. The tokenizer is being invoked to determine the set of tokens to add to (or delete from) the FTS index.
  FTS5_TOKENIZE_DOCUMENT - 正在向 FTS 表插入或删除文档。调用标记生成器来确定要添加到 FTS 索引（或从中删除）的标记集。
• FTS5_TOKENIZE_QUERY - A MATCH query is being executed against the FTS index. The tokenizer is being called to tokenize a bareword or quoted string specified as part of the query.
  FTS5_TOKENIZE_QUERY - 正在针对 FTS 索引执行 MATCH 查询。调用分词器来对指定为查询一部分的裸字或带引号的字符串进行分词。
• (FTS5_TOKENIZE_QUERY | FTS5_TOKENIZE_PREFIX) - Same as FTS5_TOKENIZE_QUERY, except that the bareword or quoted string is followed by a "*" character, indicating that the last token returned by the tokenizer will be treated as a token prefix.
  (FTS5_TOKENIZE_QUERY | FTS5_TOKENIZE_PREFIX) - 与 FTS5_TOKENIZE_QUERY 相同，只是裸字或带引号的字符串后跟一个“*”字符，表示分词器返回的最后一个标记将被视为标记前缀。
• FTS5_TOKENIZE_AUX - The tokenizer is being invoked to satisfy an fts5_api.xTokenize() request made by an auxiliary function. Or an fts5_api.xColumnSize() request made by the same on a columnsize=0 database.
  FTS5_TOKENIZE_AUX - 正在调用标记生成器以满足辅助函数发出的 fts5_api.xTokenize() 请求。或者由同一用户在 columnsize=0 数据库上发出 fts5_api.xColumnSize() 请求。

The sixth and seventh arguments passed to xTokenize() - pLocale and nLocale - are a pointer to a buffer containing the locale to use for tokenization (e.g. "en_US") and its size in bytes, respectively. The pLocale buffer is not nul-terminated. pLocale may be passed NULL (in which case nLocale is always 0) to indicate that the tokenizer should use its default locale.
传递给 xTokenize() 的第六个和第七个参数 - pLocale 和 nLocale - 是指向缓冲区的指针，其中包含用于标记化的区域设置（例如“en_US”）及其大小（以字节为单位）。 pLocale 缓冲区不是以 null 结尾的。 pLocale 可以传递 NULL（在这种情况下 nLocale 始终为 0）以指示标记生成器应使用其默认区域设置。

For each token in the input string, the supplied callback xToken() must be invoked. The first argument to it should be a copy of the pointer passed as the second argument to xTokenize(). The third and fourth arguments are a pointer to a buffer containing the token text, and the size of the token in bytes. The 4th and 5th arguments are the byte offsets of the first byte of and first byte immediately following the text from which the token is derived within the input.
对于输入字符串中的每个标记，必须调用提供的回调 xToken()。它的第一个参数应该是作为第二个参数传递给 xTokenize() 的指针的副本。第三个和第四个参数是指向包含令牌文本的缓冲区的指针，以及令牌的大小（以字节为单位）。第四个和第五个参数是输入中派生令牌的文本的第一个字节和紧随其后的第一个字节的字节偏移量。

The second argument passed to the xToken() callback ("tflags") should normally be set to 0. The exception is if the tokenizer supports synonyms. In this case see the discussion below for details.
传递给 xToken() 回调的第二个参数（“tflags”）通常应设置为 0。例外情况是标记生成器支持同义词。在这种情况下，请参阅下面的讨论以了解详细信息。

FTS5 assumes the xToken() callback is invoked for each token in the order that they occur within the input text.
FTS5 假定按照每个标记在输入文本中出现的顺序调用 xToken() 回调。

If an xToken() callback returns any value other than SQLITE_OK, then the tokenization should be abandoned and the xTokenize() method should immediately return a copy of the xToken() return value. Or, if the input buffer is exhausted, xTokenize() should return SQLITE_OK. Finally, if an error occurs with the xTokenize() implementation itself, it may abandon the tokenization and return any error code other than SQLITE_OK or SQLITE_DONE.
如果 xToken() 回调返回 SQLITE_OK 以外的任何值，则应放弃标记化，并且 xTokenize() 方法应立即返回 xToken() 返回值的副本。或者，如果输入缓冲区已耗尽，xTokenize() 应返回 SQLITE_OK。最后，如果 xTokenize() 实现本身发生错误，它可能会放弃标记化并返回 SQLITE_OK 或 SQLITE_DONE 之外的任何错误代码。

If the tokenizer is registered using an fts5_tokenizer_v2 object, then the xTokenize() method has two additional arguments - pLocale and nLocale. These specify the locale that the tokenizer should use for the current request. If pLocale and nLocale are both 0, then the tokenizer should use its default locale. Otherwise, pLocale points to an nLocale byte buffer containing the name of the locale to use as utf-8 text. pLocale is not nul-terminated.
如果使用 fts5_tokenizer_v2 对象注册标记生成器，则 xTokenize() 方法有两个附加参数 - pLocale 和 nLocale。这些指定标记生成器应用于当前请求的区域设置。如果 pLocale 和 nLocale 均为 0，则标记生成器应使用其默认区域设置。否则，pLocale 指向一个 nLocale 字节缓冲区，其中包含用作 utf-8 文本的语言环境名称。 pLocale 不是以 null 结尾的。

There is also an fts5_tokenizer object. This is an older, deprecated, version of fts5_tokenizer_v2. It is similar except that:
还有一个 fts5_tokenizer 对象。这是 fts5_tokenizer_v2 的旧版本，已弃用。其相似之处在于：

• There is no "iVersion" field, and
  没有“iVersion”字段，并且
• The xTokenize() method does not take a locale argument.
  xTokenize() 方法不采用语言环境参数。

Legacy fts5_tokenizer tokenizers must be registered using the legacy xCreateTokenizer() function, instead of xCreateTokenizer_v2().
旧版 fts5_tokenizer 分词器必须使用旧版 xCreateTokenizer() 函数（而不是 xCreateTokenizer_v2()）进行注册。

Tokenizer implementations registered using either API may be retrieved using both xFindTokenizer() and xFindTokenizer_v2().
使用任一 API 注册的 Tokenizer 实现都可以使用 xFindTokenizer() 和 xFindTokenizer_v2() 进行检索。

7.1.1. Synonym Support
7.1.1.同义词支持

Custom tokenizers may also support synonyms. Consider a case in which a user wishes to query for a phrase such as "first place". Using the built-in tokenizers, the FTS5 query 'first + place' will match instances of "first place" within the document set, but not alternative forms such as "1st place". In some applications, it would be better to match all instances of "first place" or "1st place" regardless of which form the user specified in the MATCH query text.
自定义分词器也可能支持同义词。考虑用户希望查询诸如“第一名”之类的短语的情况。使用内置标记器，FTS5 查询“first + place”将匹配文档集中“first place”的实例，但不匹配“1st place”等替代形式。在某些应用程序中，最好匹配“first place”或“1st place”的所有实例，无论用户在 MATCH 查询文本中指定哪种形式。

There are several ways to approach this in FTS5:
FTS5 中有多种方法可以解决此问题：

1. By mapping all synonyms to a single token. In this case, using the above example, this means that the tokenizer returns the same token for inputs "first" and "1st". Say that token is in fact "first", so that when the user inserts the document "I won 1st place" entries are added to the index for tokens "i", "won", "first" and "place". If the user then queries for '1st + place', the tokenizer substitutes "first" for "1st" and the query works as expected.
   通过将所有同义词映射到单个标记。在这种情况下，使用上面的示例，这意味着标记生成器为输入“first”和“1st”返回相同的标记。假设该标记实际上是“first”，这样当用户插入文档“I won 1st place”时，条目就会添加到标记“i”、“won”、“first”和“place”的索引中。如果用户随后查询“1st + place”，分词器会将“first”替换为“1st”，并且查询将按预期工作。
2. By querying the index for all synonyms of each query term separately. In this case, when tokenizing query text, the tokenizer may provide multiple synonyms for a single term within the document. FTS5 then queries the index for each synonym individually. For example, faced with the query:
   通过在索引中查询每个查询词的所有同义词 分别地。在这种情况下，当对查询文本进行标记时， 分词器可以为单个术语提供多个同义词 在文档内。 FTS5 然后查询每个的索引 同义词单独。例如，面对这样的查询：

   ... MATCH 'first place'
   

   the tokenizer offers both "1st" and "first" as synonyms for the first token in the MATCH query and FTS5 effectively runs a query similar to:
   标记生成器提供“1st”和“first”作为 MATCH 查询中第一个标记的同义词，FTS5 有效地运行类似于以下内容的查询：

   ... MATCH '(first OR 1st) place'
   

   except that, for the purposes of auxiliary functions, the query still appears to contain just two phrases - "(first OR 1st)" being treated as a single phrase.
   除此之外，出于辅助功能的目的，查询似乎仍然只包含两个短语 - “(first OR 1st)”被视为单个短语。

3. By adding multiple synonyms for a single term to the FTS index. Using this method, when tokenizing document text, the tokenizer provides multiple synonyms for each token. So that when a document such as "I won first place" is tokenized, entries are added to the FTS index for "i", "won", "first", "1st" and "place".
   通过将单个术语的多个同义词添加到 FTS 索引。 使用这种方法，在对文档文本进行分词时，分词器 为每个标记提供多个同义词。这样当一个 例如“我赢得了第一名”的文档被标记化，条目是 添加到 FTS 索引中的“i”、“won”、“first”、“1st”和 “地方”。

   This way, even if the tokenizer does not provide synonyms when tokenizing query text (it should not - to do so would be inefficient), it doesn't matter if the user queries for 'first + place' or '1st + place', as there are entries in the FTS index corresponding to both forms of the first token.
   这样，即使标记生成器在标记查询文本时不提供同义词（它不应该 - 这样做效率低下），用户是否查询“first + place”或“1st + place”也没关系，因为 FTS 索引中存在与第一个令牌的两种形式相对应的条目。

Whether it is parsing document or query text, any call to xToken that specifies a tflags argument with the FTS5_TOKEN_COLOCATED bit is considered to supply a synonym for the previous token. For example, when parsing the document "I won first place", a tokenizer that supports synonyms would call xToken() 5 times, as follows:
无论是解析文档还是查询文本，对使用 FTS5_TOKEN_COLOCATED 位指定tflags参数的 xToken 的任何调用都被视为提供前一个标记的同义词。例如，在解析文档“我赢得了第一名”时，支持同义词的分词器将调用 xToken() 5 次，如下所示：

xToken(pCtx, 0, "i",                      1,  0,  1);
xToken(pCtx, 0, "won",                    3,  2,  5);
xToken(pCtx, 0, "first",                  5,  6, 11);
xToken(pCtx, FTS5_TOKEN_COLOCATED, "1st", 3,  6, 11);
xToken(pCtx, 0, "place",                  5, 12, 17);

It is an error to specify the FTS5_TOKEN_COLOCATED flag the first time xToken() is called. Multiple synonyms may be specified for a single token by making multiple calls to xToken(FTS5_TOKEN_COLOCATED) in sequence. There is no limit to the number of synonyms that may be provided for a single token.
第一次调用 xToken() 时指定 FTS5_TOKEN_COLOCATED 标志是错误的。通过按顺序多次调用 xToken(FTS5_TOKEN_COLOCATED) 可以为单个令牌指定多个同义词。可以为单个标记提供的同义词数量没有限制。

In many cases, method (1) above is the best approach. It does not add extra data to the FTS index or require FTS5 to query for multiple terms, so it is efficient in terms of disk space and query speed. However, it does not support prefix queries very well. If, as suggested above, the token "first" is substituted for "1st" by the tokenizer, then the query:
在许多情况下，上述方法（1）是最好的方法。它不会向 FTS 索引添加额外的数据，也不需要 FTS5 查询多个术语，因此在磁盘空间和查询速度方面都很高效。但是，它不能很好地支持前缀查询。如果如上所述，标记器将标记“first”替换为“1st”，则查询：

... MATCH '1s*'

will not match documents that contain the token "1st" (as the tokenizer will probably not map "1s" to any prefix of "first").
将不匹配包含标记“1st”的文档（因为标记生成器可能不会将“1”映射到“first”的任何前缀）。

For full prefix support, method (3) may be preferred. In this case, because the index contains entries for both "first" and "1st", prefix queries such as 'fi*' or '1s*' will match correctly. However, because extra entries are added to the FTS index, this method uses more space within the database.
对于完整的前缀支持，方法 (3) 可能是首选。在这种情况下，由于索引包含“first”和“1st”的条目，因此“fi*”或“1s*”等前缀查询将正确匹配。但是，由于将额外的条目添加到 FTS 索引中，因此此方法会使用数据库中的更多空间。

Method (2) offers a midpoint between (1) and (3). Using this method, a query such as '1s*' will match documents that contain the literal token "1st", but not "first" (assuming the tokenizer is not able to provide synonyms for prefixes). However, a non-prefix query like '1st' will match against "1st" and "first". This method does not require extra disk space, as no extra entries are added to the FTS index. On the other hand, it may require more CPU cycles to run MATCH queries, as separate queries of the FTS index are required for each synonym.
方法（2）提供了（1）和（3）之间的中点。使用此方法，诸如“1s*”之类的查询将匹配包含文字标记“1st”但不包含“first”的文档（假设标记生成器无法提供前缀的同义词）。但是，像“1st”这样的非前缀查询将与“1st”和“first”匹配。此方法不需要额外的磁盘空间，因为不会向 FTS 索引添加额外的条目。另一方面，它可能需要更多的 CPU 周期来运行 MATCH 查询，因为每个同义词都需要单独的 FTS 索引查询。

When using methods (2) or (3), it is important that the tokenizer only provide synonyms when tokenizing document text (method (3)) or query text (method (2)), not both. Doing so will not cause any errors, but is inefficient.
使用方法 (2) 或 (3) 时，重要的是分词器仅在分词文档文本（方法 (3)）或查询文本（方法 (2)）时提供同义词，而不是两者都提供。这样做不会导致任何错误，但效率低下。

7.2. Custom Auxiliary Functions
7.2.自定义辅助功能

Implementing a custom auxiliary function is similar to implementing a scalar SQL function. The implementation should be a C function of type fts5_extension_function, defined as follows:
实现自定义辅助函数与实现标量 SQL 函数类似。实现应该是 fts5_extension_function 类型的 C 函数，定义如下：

typedef struct Fts5ExtensionApi Fts5ExtensionApi;
typedef struct Fts5Context Fts5Context;
typedef struct Fts5PhraseIter Fts5PhraseIter;

typedef void (*fts5_extension_function)(
  const Fts5ExtensionApi *pApi,   /* API offered by current FTS version */
  Fts5Context *pFts,              /* First arg to pass to pApi functions */
  sqlite3_context *pCtx,          /* Context for returning result/error */
  int nVal,                       /* Number of values in apVal[] array */
  sqlite3_value **apVal           /* Array of trailing arguments */
);

The implementation is registered with the FTS5 module by calling the xCreateFunction() method of the fts5_api object. If there is already an auxiliary function with the same name, it is replaced by the new function. If a non-NULL xDestroy parameter is passed to xCreateFunction(), it is invoked with a copy of the pUserData pointer passed as the only argument when the database handle is closed or when the registered auxiliary function is replaced.
通过调用 fts5_api 对象的 xCreateFunction() 方法将实现注册到 FTS5 模块。如果已经存在同名的辅助函数，则将其替换为新函数。如果将非 NULL xDestroy 参数传递给 xCreateFunction()，则在关闭数据库句柄或替换已注册的辅助函数时，将使用作为唯一参数传递的 pUserData 指针的副本来调用该参数。

If successful, xCreateFunction() returns SQLITE_OK. Otherwise, it returns an SQLite error code. In this case the xDestroy function is not invoked.
如果成功，xCreateFunction() 返回 SQLITE_OK。否则，它返回 SQLite 错误代码。在这种情况下，不会调用 xDestroy 函数。

The final three arguments passed to the auxiliary function callback (pCtx, nVal and apVal above) are similar to the three arguments passed to the implementation of a scalar SQL function. The apVal[] array contains all SQL arguments except the first passed to the auxiliary function. The implementation should return a result or error via the content handle pCtx.
传递给辅助函数回调的最后三个参数（上面的 pCtx、nVal 和 apVal）与传递给标量 SQL 函数的实现的三个参数类似。 apVal[] 数组包含除第一个传递给辅助函数的参数之外的所有 SQL 参数。实现应通过内容句柄 pCtx 返回结果或错误。

The first argument passed to an auxiliary function callback is a pointer to a structure (pApi above) containing methods that may be invoked in order to obtain information regarding the current query or row. The second argument is an opaque handle (pFts above) that should be passed as the first argument to any such method invocation. For example, the following auxiliary function returns the total number of tokens in all columns of the current row:
传递给辅助函数回调的第一个参数是一个指向结构体（上面的 pApi）的指针，该结构体包含可以调用的方法，以便获取有关当前查询或行的信息。第二个参数是一个不透明句柄（上面的 pFts），应将其作为第一个参数传递给任何此类方法调用。例如，以下辅助函数返回当前行所有列中的标记总数：

/*
** Implementation of an auxiliary function that returns the number
** of tokens in the current row (including all columns).
*/
static void column_size_imp(
  const Fts5ExtensionApi *pApi,
  Fts5Context *pFts,
  sqlite3_context *pCtx,
  int nVal,
  sqlite3_value **apVal
){
  int rc;
  int nToken;
  rc = pApi->xColumnSize(pFts, -1, &nToken);
  if( rc==SQLITE_OK ){
    sqlite3_result_int(pCtx, nToken);
  }else{
    sqlite3_result_error_code(pCtx, rc);
  }
}

The following section describes the API offered to auxiliary function implementations in detail. Further examples may be found in the "fts5_aux.c" file of the source code.
以下部分详细描述了为辅助功能实现提供的 API。更多示例可以在源代码的“fts5_aux.c”文件中找到。

7.2.1. Custom Auxiliary Functions API Overview
7.2.1.自定义辅助函数API概述

This section provides an overview of the capabilities of the auxiliary function API. It does not describe every function. Refer to the reference text below for a complete description.
本节概述了辅助功能 API 的功能。它没有描述每个功能。请参阅下面的参考文本以获取完整的说明。

When invoked, an auxiliary function implementation has access to APIs that allow it to query FTS5 for various information. Some of these APIs return information relating to the current row of the FTS5 table being visited, some relating to the entire set of rows that will be visited by the FTS5 query, and some relating to the FTS5 table. Given an FTS5 table populated as follows:
调用时，辅助函数实现可以访问 API，从而可以查询 FTS5 以获取各种信息。其中一些 API 返回与正在访问的 FTS5 表的当前行相关的信息，一些返回与 FTS5 查询将访问的整个行集相关的信息，还有一些与 FTS5 表相关。给定一个 FTS5 表，填充如下：

CREATE VIRTUAL TABLE ft USING fts5(a, b);
INSERT INTO ft(rowid, a, b) VALUES
        (1, 'ab cd', 'cd de one'),
        (2, 'de fg', 'fg gh'),
        (3, 'gh ij', 'ij ab three four');

and the query:  和查询：

SELECT my_aux_function(ft) FROM ft('ab')

then the custom auxiliary function will be invoked for rows 1 and 3 (all rows that contain the token "ab" and therefore match the query).
那么将为第 1 行和第 3 行（包含标记“ab”并因此与查询匹配的所有行）调用自定义辅助函数。

Number of rows/columns in table: xRowCount, xColumnCount
表中的行/列数：xRowCount、xColumnCount

The system may be queried for the total number of rows in the FTS5 table using the xRowCount API. This provides the total number of rows in the table, not the number that match the current query.
可以使用xRowCount API 查询系统 FTS5 表中的总行数。这提供了表中的总行数，而不是与当前查询匹配的行数。

Table columns are numbered from left to right starting from 0. The "rowid" column does not count - only user declared columns - so in the example above column "a" is column 0 and column "b" is column 1. From within an auxiliary function implementation, the xColumnCount API may be used to determine how many columns the table being queried has. If the xColumnCount() API is invoked from within the implementation of the auxiliary function my_aux_function in the example above, it returns 2.
表列从左到右从 0 开始编号。“rowid”列不计数 - 仅用户声明的列 - 因此在上面的示例中，列“a”是列 0，列“b”是列 1。辅助函数实现中， xColumnCount API 可用于确定正在查询的表有多少列。如果在上例中从辅助函数 my_aux_function 的实现中调用 xColumnCount() API，则它将返回 2。

Data From Current Row: xColumnText, xRowid
当前行的数据：xColumnText、xRowid

The xRowid API may be used to find the rowid value for the current row. The xColumnText may be used to obtain the text stored in a specified column of the current row.
xRowid API 可用于查找当前行的 rowid 值。 xColumnText可用于获取存储在当前行的指定列中的文本。

Token Counts: xColumnSize, xColumnTotalSize
令牌计数：xColumnSize、xColumnTotalSize

FTS5 divides documents inserted into an fts5 table into tokens. These are usually just words, perhaps folded to either upper or lower case and with any punctuation removed. For example, the default unicode61 tokenizer tokenizes the text "The tokenizer is case-insensitive" to a list of 5 tokens - "the", "tokenizer", is", "case" and "insensitive". Exactly how tokens are extracted from text is determined by the tokenizer.
FTS5 将插入到 fts5 表中的文档划分为标记。这些通常只是单词，可能折叠为大写或小写，并删除了所有标点符号。例如，默认的unicode61 tokenizer将文本“The tokenizer is case-insensitive”标记为 5 个标记的列表 - “the”、“tokenizer”、“is”、“case”和“insensitive”。确切地说，标记是如何从文本由分词器确定。

The auxiliary functions API provides functions to query for both the number of tokens in a specified column of the current row (the xColumnSize API), or for the number of tokens in a specified column of all rows of the table (the xColumnTotalSize API). For the example at the top of this section, when visiting row 1, xColumnSize returns 2 for column 0 and 3 for column 1. xColumnTotalSize returns 6 for column 0 and 9 for column 1 regardless of the current row.
辅助函数 API 提供了查询当前行的指定列中的标记数（ xColumnSize API）或表的所有行的指定列中的标记数（ xColumnTotalSize API）的函数。对于本节顶部的示例，访问第 1 行时，xColumnSize 对第 0 列返回 2，对第 1 列返回 3。无论当前行如何，xColumnTotalSize 对第 0 列返回 6，对第 1 列返回 9。

The Current Full-Text Query: xPhraseCount, xPhraseSize, xQueryToken
当前全文查询：xPhraseCount、xPhraseSize、xQueryToken

An FTS5 query contains one or more phrases. The xPhraseCount, xPhraseSize and xQueryToken APIs allow an auxiliary function implementation to query the system for details of the current query. The xPhraseCount API returns the number of phrases in the current query. For example, if an FTS5 table is queried as follows:
FTS5 查询包含一个或多个短语。 xPhraseCount 、 xPhraseSize和xQueryToken API 允许辅助函数实现向系统查询当前查询的详细信息。 xPhraseCount API 返回当前查询中的短语数。例如，如果查询FTS5表如下：

SELECT my_aux_function(ft) FROM ft('ab AND "cd ef gh" OR ij + kl')

and the xPhraseCount() API invoked from within the implementation of the auxiliary function, it returns 3 (the three phrases being "ab", "ce ef gh" and "ij kl").
从辅助函数的实现中调用 xPhraseCount() API，它返回 3（这三个短语是“ab”、“ce ef gh”和“ij kl”）。

Phrases are numbered in order of appearance within a query starting from 0. The xPhraseSize() API may be used to query for the number of tokens in a specified phrase of the query. In the example above, phrase 0 contains 1 token, phrase 1 contains 3 tokens, and phrase 2 contains 2.
短语按照查询中出现的顺序从 0 开始编号。 xPhraseSize() API 可用于查询查询的指定短语中的标记数量。在上面的示例中，短语 0 包含 1 个标记，短语 1 包含 3 个标记，短语 2 包含 2 个标记。

The xQueryToken API may be used to access the text of a specified token within a specified phrase of the query. Tokens are numbered within their phrases from left to right starting from 0. For example, if the xQueryToken API is used to request token 1 of phrase 2 in the example above, it returns the text "kl". Token 0 of phrase 0 is "ab".
xQueryToken API 可用于访问查询的指定短语内指定令牌的文本。令牌在其短语中从左到右从 0 开始编号。例如，如果使用 xQueryToken API 请求上例中短语 2 的令牌 1，它将返回文本“kl”。短语 0 的标记 0 是“ab”。

Phrase Hits in the Current Row: xPhraseFirst, xPhraseNext
当前行中的短语命中：xPhraseFirst、xPhraseNext

These two API functions may be used to iterate through the matches for a specified phrase of the query within the current row. Phrase matches are identified by the column and token offset within the current row. For example, say the following example table:
这两个 API 函数可用于迭代当前行中查询的指定短语的匹配项。短语匹配由当前行中的列和标记偏移量来标识。例如，下面的示例表：

CREATE VIRTUAL TABLE ft2 USING fts5(x, y);
INSERT INTO ft2(rowid, x, y) VALUES
        (1, 'xxx one two xxx five xxx six', 'seven four'),
        (2, 'five four four xxx six', 'three four five six four five six');

is queried with:  被查询为：

SELECT my_aux_function(ft2) FROM ft2(
    '("one two" OR "three") AND y:four NEAR(five six, 2)'
);

The query above contains 5 phrases - "one two", "three", "four", "five" and "six". It matches all rows of the table, so the auxiliary function is invoked for each row.
上面的查询包含 5 个短语 - “一二”、“三”、“四”、“五”和“六”。它匹配表的所有行，因此为每一行调用辅助函数。

In row 1, for phrase 0, "one two", there is exactly one match to iterate through - at column 0 token offset 1. The column number is 0 because the match appears in the left most column. The token offset is 1 because there is exactly one token ("xxx") before the phrase match in the column value. For phrase 1, "three", there are no matches. Phrase 2, "four", has one match, at column 1, token offset 0. Phrase 3, "five", has one match at column 0, token offset 4, and phrase 4, "six", has one match at column 0 token offset 6.
在第 1 行中，对于短语 0“一二”，只有一个匹配项需要迭代 - 在第 0 列标记偏移量 1 处。列号为 0，因为匹配项出现在最左边的列中。标记偏移量为 1，因为列值中的短语匹配之前恰好有一个标记 (“xxx”)。对于短语 1“三”，没有匹配项。短语 2“四”在第 1 列、标记偏移量 0 处有 1 个匹配项。短语 3“5”在第 0 列、标记偏移量 4 处有 1 个匹配项，短语 4“6”在第 1 列有 1 个匹配项0 令牌偏移量 6。

The set of matches for each phrase in each row of the example is presented in the table below. Each match is notated as (column-number, token-offset):
下表列出了示例中每行中每个短语的匹配集。每个匹配都标记为（列号，标记偏移量）：

The second row is slightly more complicated. There were no occurrences of phrase 0. Phrase 1 ("three") appears once, at column 1 token offset 0. Although there are instances of phrase 2 ("four") in column 0, none of them are reported by the API, as phrase 4 has a column filter - "y:". Matches that are filtered out by column filters do not count. Similarly, although phrases 3 and 4 do occur in column "x" of row 2, they are filtered out by the NEAR filter. Matches that are filtered out by NEAR filters do not count either.
第二行稍微复杂一些。没有出现短语 0。短语 1（“三”）在第 1 列标记偏移 0 处出现一次。尽管第 0 列中有短语 2（“四”）的实例，但 API 均未报告，因为短语 4 有一个列过滤器- “y:”。被列过滤器过滤掉的匹配项不计入在内。类似地，虽然短语 3 和 4 确实出现在第 2 行的“x”列中，但它们被NEAR 过滤器过滤掉。由 NEAR 过滤器过滤掉的匹配项也不计入在内。

Phrase Hits in the Current Row (2): xInstCount, xInst
当前行中的短语命中 (2)：xInstCount、xInst

The xInstCount and xInst APIs provide access to the same information as the xPhraseFirst and xPhraseNext described above. The difference is that instead of iterating through the matches for a single, specified phrase, the xInstCount/xInst APIs collate all matches into a single flat array, sorted in order of occurrence within the current row. Elements of this array may then be accessed randomly.
xInstCount和xInst API 提供对与上述 xPhraseFirst 和 xPhraseNext 相同信息的访问。不同之处在于，xInstCount/xInst API 不是迭代单个指定短语的匹配项，而是将所有匹配项整理到单个平面数组中，并按当前行中出现的顺序排序。然后可以随机访问该数组的元素。

Each array element consists of three values:
每个数组元素由三个值组成：

• A phrase number,  一个短语编号，
• A column number, and  列号，以及
• A token offset  令牌偏移量

Using the same example data and query as for xPhraseFirst/xPhraseNext above, the array accessible via xInstCount/xInst consists of the following entries for each row:
使用与上面的 xPhraseFirst/xPhraseNext 相同的示例数据和查询，可通过 xInstCount/xInst 访问的数组由每行的以下条目组成：

Each entry of the array is called a phrase match. Phrase matches are numbered in order, starting from 0. So, in the example above, in row 2, phrase match 3 is (4, 1, 3) - phrase 4 of the query matches at column 1, token offset 3.
数组的每个条目称为短语匹配。短语匹配按顺序编号，从 0 开始。因此，在上面的示例中，在第 2 行中，短语匹配 3 是 (4, 1, 3) - 查询的短语 4 在第 1 列、标记偏移量 3 处匹配。

7.2.2. Custom Auxiliary Functions API Reference
7.2.2.自定义辅助函数API参考

struct Fts5ExtensionApi {
  int iVersion;                   /* Currently always set to 4 */

  void *(*xUserData)(Fts5Context*);

  int (*xColumnCount)(Fts5Context*);
  int (*xRowCount)(Fts5Context*, sqlite3_int64 *pnRow);
  int (*xColumnTotalSize)(Fts5Context*, int iCol, sqlite3_int64 *pnToken);

  int (*xTokenize)(Fts5Context*, 
    const char *pText, int nText, /* Text to tokenize */
    void *pCtx,                   /* Context passed to xToken() */
    int (*xToken)(void*, int, const char*, int, int, int)       /* Callback */
  );

  int (*xPhraseCount)(Fts5Context*);
  int (*xPhraseSize)(Fts5Context*, int iPhrase);

  int (*xInstCount)(Fts5Context*, int *pnInst);
  int (*xInst)(Fts5Context*, int iIdx, int *piPhrase, int *piCol, int *piOff);

  sqlite3_int64 (*xRowid)(Fts5Context*);
  int (*xColumnText)(Fts5Context*, int iCol, const char **pz, int *pn);
  int (*xColumnSize)(Fts5Context*, int iCol, int *pnToken);

  int (*xQueryPhrase)(Fts5Context*, int iPhrase, void *pUserData,
    int(*)(const Fts5ExtensionApi*,Fts5Context*,void*)
  );
  int (*xSetAuxdata)(Fts5Context*, void *pAux, void(*xDelete)(void*));
  void *(*xGetAuxdata)(Fts5Context*, int bClear);

  int (*xPhraseFirst)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*, int*);
  void (*xPhraseNext)(Fts5Context*, Fts5PhraseIter*, int *piCol, int *piOff);

  int (*xPhraseFirstColumn)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*);
  void (*xPhraseNextColumn)(Fts5Context*, Fts5PhraseIter*, int *piCol);

  /* Below this point are iVersion>=3 only */
  int (*xQueryToken)(Fts5Context*, 
      int iPhrase, int iToken, 
      const char **ppToken, int *pnToken
  );
  int (*xInstToken)(Fts5Context*, int iIdx, int iToken, const char**, int*);

  /* Below this point are iVersion>=4 only */
  int (*xColumnLocale)(Fts5Context*, int iCol, const char **pz, int *pn);
  int (*xTokenize_v2)(Fts5Context*,
    const char *pText, int nText,      /* Text to tokenize */
    const char *pLocale, int nLocale,  /* Locale to pass to tokenizer */
    void *pCtx,                        /* Context passed to xToken() */
    int (*xToken)(void*, int, const char*, int, int, int)       /* Callback */
  );
};

void *(*xUserData)(Fts5Context*)
无效*（*xUserData）（Fts5Context*）

Return a copy of the pUserData pointer passed to the xCreateFunction() API when the extension function was registered.
返回注册扩展函数时传递给 xCreateFunction() API 的 pUserData 指针的副本。

int (*xColumnTotalSize)(Fts5Context*, int iCol, sqlite3_int64 *pnToken)

If parameter iCol is less than zero, set output variable *pnToken to the total number of tokens in the FTS5 table. Or, if iCol is non-negative but less than the number of columns in the table, return the total number of tokens in column iCol, considering all rows in the FTS5 table.
如果参数 iCol 小于零，则将输出变量 *pnToken 设置为 FTS5 表中的令牌总数。或者，如果 iCol 为非负数但小于表中的列数，则考虑 FTS5 表中的所有行，返回 iCol 列中的标记总数。

If parameter iCol is greater than or equal to the number of columns in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g. an OOM condition or IO error), an appropriate SQLite error code is returned.
如果参数 iCol 大于或等于表中的列数，则返回 SQLITE_RANGE。或者，如果发生错误（例如 OOM 条件或 IO 错误），则会返回适当的 SQLite 错误代码。

int (*xColumnCount)(Fts5Context*)

Return the number of columns in the table.
返回表中的列数。

int (*xColumnSize)(Fts5Context*, int iCol, int *pnToken)

If parameter iCol is less than zero, set output variable *pnToken to the total number of tokens in the current row. Or, if iCol is non-negative but less than the number of columns in the table, set *pnToken to the number of tokens in column iCol of the current row.
如果参数 iCol 小于零，则将输出变量 *pnToken 设置为当前行中的标记总数。或者，如果 iCol 为非负数但小于表中的列数，则将 *pnToken 设置为当前行的列 iCol 中的标记数。

If parameter iCol is greater than or equal to the number of columns in the table, SQLITE_RANGE is returned. Or, if an error occurs (e.g. an OOM condition or IO error), an appropriate SQLite error code is returned.
如果参数 iCol 大于或等于表中的列数，则返回 SQLITE_RANGE。或者，如果发生错误（例如 OOM 条件或 IO 错误），则会返回适当的 SQLite 错误代码。

This function may be quite inefficient if used with an FTS5 table created with the "columnsize=0" option.
如果与使用“columnsize=0”选项创建的 FTS5 表一起使用，此函数的效率可能会非常低。

int (*xColumnText)(Fts5Context*, int iCol, const char **pz, int *pn)

If parameter iCol is less than zero, or greater than or equal to the number of columns in the table, SQLITE_RANGE is returned.
如果参数 iCol 小于零，或者大于或等于表中的列数，则返回 SQLITE_RANGE。

Otherwise, this function attempts to retrieve the text of column iCol of the current document. If successful, (*pz) is set to point to a buffer containing the text in utf-8 encoding, (*pn) is set to the size in bytes (not characters) of the buffer and SQLITE_OK is returned. Otherwise, if an error occurs, an SQLite error code is returned and the final values of (*pz) and (*pn) are undefined.
否则，此函数尝试检索当前文档的 iCol 列的文本。如果成功，(*pz) 设置为指向包含 utf-8 编码文本的缓冲区，(*pn) 设置为缓冲区的大小（以字节为单位）（而不是字符），并返回 SQLITE_OK。否则，如果发生错误，则会返回 SQLite 错误代码，并且 (*pz) 和 (*pn) 的最终值未定义。

int (*xPhraseCount)(Fts5Context*)

Returns the number of phrases in the current query expression.
返回当前查询表达式中的短语数。

int (*xPhraseSize)(Fts5Context*, int iPhrase)

If parameter iCol is less than zero, or greater than or equal to the number of phrases in the current query, as returned by xPhraseCount, 0 is returned. Otherwise, this function returns the number of tokens in phrase iPhrase of the query. Phrases are numbered starting from zero.
如果参数 iCol 小于零，或者大于或等于 xPhraseCount 返回的当前查询中的短语数，则返回 0。否则，此函数返回查询的短语 iPhrase 中的标记数。短语从零开始编号。

int (*xInstCount)(Fts5Context*, int *pnInst)

Set *pnInst to the total number of occurrences of all phrases within the query within the current row. Return SQLITE_OK if successful, or an error code (i.e. SQLITE_NOMEM) if an error occurs.
将 *pnInst 设置为当前行中查询中所有短语出现的总次数。如果成功则返回 SQLITE_OK，如果发生错误则返回错误代码（即 SQLITE_NOMEM）。

This API can be quite slow if used with an FTS5 table created with the "detail=none" or "detail=column" option. If the FTS5 table is created with either "detail=none" or "detail=column" and "content=" option (i.e. if it is a contentless table), then this API always returns 0.
如果与使用“detail=none”或“detail=column”选项创建的 FTS5 表一起使用，此 API 可能会非常慢。如果 FTS5 表是使用“detail=none”或“detail=column”和“content=”选项创建的（即，如果它是无内容表），则此 API 始终返回 0。

int (*xInst)(Fts5Context*, int iIdx, int *piPhrase, int *piCol, int *piOff)
int (*xInst)(Fts5Context*、int iIdx、int *piPhrase、int *piCol、int *piOff)

Query for the details of phrase match iIdx within the current row. Phrase matches are numbered starting from zero, so the iIdx argument should be greater than or equal to zero and smaller than the value output by xInstCount(). If iIdx is less than zero or greater than or equal to the value returned by xInstCount(), SQLITE_RANGE is returned.
查询当前行内短语匹配 iIdx 的详细信息。短语匹配从零开始编号，因此 iIdx 参数应大于或等于零且小于 xInstCount() 输出的值。如果 iIdx 小于零或大于或等于 xInstCount() 返回的值，则返回 SQLITE_RANGE。

Otherwise, output parameter *piPhrase is set to the phrase number, *piCol to the column in which it occurs and *piOff the token offset of the first token of the phrase. SQLITE_OK is returned if successful, or an error code (i.e. SQLITE_NOMEM) if an error occurs.
否则，输出参数 *piPhrase 设置为短语编号，*piCol 设置为它出现的列，*piOff 设置为短语的第一个标记的标记偏移量。如果成功则返回SQLITE_OK，如果发生错误则返回错误代码（即SQLITE_NOMEM）。

This API can be quite slow if used with an FTS5 table created with the "detail=none" or "detail=column" option.
如果与使用“detail=none”或“detail=column”选项创建的 FTS5 表一起使用，此 API 可能会非常慢。

sqlite3_int64 (*xRowid)(Fts5Context*)

Returns the rowid of the current row.
返回当前行的 rowid。

int (*xTokenize)(Fts5Context*, const char *pText, int nText, void *pCtx, int (*xToken)(void*, int, const char*, int, int, int) )
int (*xTokenize)(Fts5Context*, const char *pText, int nText, 无效*pCtx， int (*xToken)(void*, int, const char*, int, int, int) ）

Tokenize text using the tokenizer belonging to the FTS5 table.
使用属于 FTS5 表的分词器对文本进行分词。

int (*xQueryPhrase)(Fts5Context*, int iPhrase, void *pUserData, int(*)(const Fts5ExtensionApi*,Fts5Context*,void*) )
int (*xQueryPhrase)(Fts5Context*, int iPhrase, void *pUserData, int(*)(const Fts5ExtensionApi*,Fts5Context*,void*) ）

This API function is used to query the FTS table for phrase iPhrase of the current query. Specifically, a query equivalent to:
该API函数用于在FTS表中查询当前查询的短语iPhrase。具体来说，查询相当于：

... FROM ftstable WHERE ftstable MATCH $p ORDER BY rowid

with $p set to a phrase equivalent to the phrase iPhrase of the current query is executed. Any column filter that applies to phrase iPhrase of the current query is included in $p. For each row visited, the callback function passed as the fourth argument is invoked. The context and API objects passed to the callback function may be used to access the properties of each matched row. Invoking Api.xUserData() returns a copy of the pointer passed as the third argument to pUserData.
将 $p 设置为与当前查询的短语 iPhrase 等效的短语。任何适用于当前查询的短语 iPhrase 的列过滤器都包含在 $p 中。对于访问的每一行，都会调用作为第四个参数传递的回调函数。传递给回调函数的上下文和 API 对象可用于访问每个匹配行的属性。调用 Api.xUserData() 返回作为第三个参数传递给 pUserData 的指针的副本。

If parameter iPhrase is less than zero, or greater than or equal to the number of phrases in the query, as returned by xPhraseCount(), this function returns SQLITE_RANGE.
如果参数 iPhrase 小于零，或者大于或等于 xPhraseCount() 返回的查询中的短语数，则此函数返回 SQLITE_RANGE。

If the callback function returns any value other than SQLITE_OK, the query is abandoned and the xQueryPhrase function returns immediately. If the returned value is SQLITE_DONE, xQueryPhrase returns SQLITE_OK. Otherwise, the error code is propagated upwards.
如果回调函数返回 SQLITE_OK 以外的任何值，则查询将被放弃，并且 xQueryPhrase 函数立即返回。如果返回值为 SQLITE_DONE，则 xQueryPhrase 返回 SQLITE_OK。否则，错误代码将向上传播。

If the query runs to completion without incident, SQLITE_OK is returned. Or, if some error occurs before the query completes or is aborted by the callback, an SQLite error code is returned.
如果查询顺利完成，则返回 SQLITE_OK。或者，如果在查询完成之前发生某些错误或被回调中止，则会返回 SQLite 错误代码。

int (*xSetAuxdata)(Fts5Context*, void *pAux, void(*xDelete)(void*))

Save the pointer passed as the second argument as the extension function's "auxiliary data". The pointer may then be retrieved by the current or any future invocation of the same fts5 extension function made as part of the same MATCH query using the xGetAuxdata() API.
将作为第二个参数传递的指针保存为扩展函数的“辅助数据”。然后，可以通过使用 xGetAuxdata() API 作为同一 MATCH 查询的一部分的同一 fts5 扩展函数的当前或任何未来调用来检索该指针。

Each extension function is allocated a single auxiliary data slot for each FTS query (MATCH expression). If the extension function is invoked more than once for a single FTS query, then all invocations share a single auxiliary data context.
每个扩展函数都为每个 FTS 查询（MATCH 表达式）分配一个辅助数据槽。如果针对单个 FTS 查询多次调用扩展函数，则所有调用共享单个辅助数据上下文。

If there is already an auxiliary data pointer when this function is invoked, then it is replaced by the new pointer. If an xDelete callback was specified along with the original pointer, it is invoked at this point.
如果调用此函数时已经存在辅助数据指针，则它将被新指针替换。如果 xDelete 回调与原始指针一起指定，则此时将调用它。

The xDelete callback, if one is specified, is also invoked on the auxiliary data pointer after the FTS5 query has finished.
xDelete 回调（如果指定了）也会在 FTS5 查询完成后在辅助数据指针上调用。

If an error (e.g. an OOM condition) occurs within this function, the auxiliary data is set to NULL and an error code returned. If the xDelete parameter was not NULL, it is invoked on the auxiliary data pointer before returning.
如果此函数内发生错误（例如 OOM 条件），则辅助数据将设置为 NULL 并返回错误代码。如果 xDelete 参数不为 NULL，则在返回之前在辅助数据指针上调用它。

void *(*xGetAuxdata)(Fts5Context*, int bClear)

Returns the current auxiliary data pointer for the fts5 extension function. See the xSetAuxdata() method for details.
返回 fts5 扩展函数的当前辅助数据指针。有关详细信息，请参阅 xSetAuxdata() 方法。

If the bClear argument is non-zero, then the auxiliary data is cleared (set to NULL) before this function returns. In this case the xDelete, if any, is not invoked.
如果 bClear 参数非零，则在此函数返回之前辅助数据将被清除（设置为 NULL）。在这种情况下，不会调用 xDelete（如果有）。

int (*xRowCount)(Fts5Context*, sqlite3_int64 *pnRow)

This function is used to retrieve the total number of rows in the table. In other words, the same value that would be returned by:

SELECT count(*) FROM ftstable;

int (*xPhraseFirst)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*, int*)

This function is used, along with type Fts5PhraseIter and the xPhraseNext method, to iterate through all instances of a single query phrase within the current row. This is the same information as is accessible via the xInstCount/xInst APIs. While the xInstCount/xInst APIs are more convenient to use, this API may be faster under some circumstances. To iterate through instances of phrase iPhrase, use the following code:

Fts5PhraseIter iter;
int iCol, iOff;
for(pApi->xPhraseFirst(pFts, iPhrase, &iter, &iCol, &iOff);
    iCol>=0;
    pApi->xPhraseNext(pFts, &iter, &iCol, &iOff)
){
  // An instance of phrase iPhrase at offset iOff of column iCol
}

The Fts5PhraseIter structure is defined above. Applications should not modify this structure directly - it should only be used as shown above with the xPhraseFirst() and xPhraseNext() API methods (and by xPhraseFirstColumn() and xPhraseNextColumn() as illustrated below).

This API can be quite slow if used with an FTS5 table created with the "detail=none" or "detail=column" option. If the FTS5 table is created with either "detail=none" or "detail=column" and "content=" option (i.e. if it is a contentless table), then this API always iterates through an empty set (all calls to xPhraseFirst() set iCol to -1).

In all cases, matches are visited in (column ASC, offset ASC) order. i.e. all those in column 0, sorted by offset, followed by those in column 1, etc.

void (*xPhraseNext)(Fts5Context*, Fts5PhraseIter*, int *piCol, int *piOff)

See xPhraseFirst above.

int (*xPhraseFirstColumn)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*)

This function and xPhraseNextColumn() are similar to the xPhraseFirst() and xPhraseNext() APIs described above. The difference is that instead of iterating through all instances of a phrase in the current row, these APIs are used to iterate through the set of columns in the current row that contain one or more instances of a specified phrase. For example:

Fts5PhraseIter iter;
int iCol;
for(pApi->xPhraseFirstColumn(pFts, iPhrase, &iter, &iCol);
    iCol>=0;
    pApi->xPhraseNextColumn(pFts, &iter, &iCol)
){
  // Column iCol contains at least one instance of phrase iPhrase
}

This API can be quite slow if used with an FTS5 table created with the "detail=none" option. If the FTS5 table is created with either "detail=none" "content=" option (i.e. if it is a contentless table), then this API always iterates through an empty set (all calls to xPhraseFirstColumn() set iCol to -1).

The information accessed using this API and its companion xPhraseFirstColumn() may also be obtained using xPhraseFirst/xPhraseNext (or xInst/xInstCount). The chief advantage of this API is that it is significantly more efficient than those alternatives when used with "detail=column" tables.

void (*xPhraseNextColumn)(Fts5Context*, Fts5PhraseIter*, int *piCol)

See xPhraseFirstColumn above.

int (*xQueryToken)(Fts5Context*, int iPhrase, int iToken, const char **ppToken, int *pnToken )

This is used to access token iToken of phrase iPhrase of the current query. Before returning, output parameter *ppToken is set to point to a buffer containing the requested token, and *pnToken to the size of this buffer in bytes.

If iPhrase or iToken are less than zero, or if iPhrase is greater than or equal to the number of phrases in the query as reported by xPhraseCount(), or if iToken is equal to or greater than the number of tokens in the phrase, SQLITE_RANGE is returned and *ppToken and *pnToken are both zeroed.

The output text is not a copy of the query text that specified the token. It is the output of the tokenizer module. For tokendata=1 tables, this includes any embedded 0x00 and trailing data.

int (*xInstToken)(Fts5Context*, int iIdx, int iToken, const char**, int*)

This is used to access token iToken of phrase hit iIdx within the current row. If iIdx is less than zero or greater than or equal to the value returned by xInstCount(), SQLITE_RANGE is returned. Otherwise, output variable (*ppToken) is set to point to a buffer containing the matching document token, and (*pnToken) to the size of that buffer in bytes. This API is not available if the specified token matches a prefix query term. In that case both output variables are always set to 0.

The output text is not a copy of the document text that was tokenized. It is the output of the tokenizer module. For tokendata=1 tables, this includes any embedded 0x00 and trailing data.

This API can be quite slow if used with an FTS5 table created with the "detail=none" or "detail=column" option.

int (*xColumnLocale)(Fts5Context*, int iCol, const char **pz, int *pn)

If parameter iCol is less than zero, or greater than or equal to the number of columns in the table, SQLITE_RANGE is returned.

Otherwise, this function attempts to retrieve the locale associated with column iCol of the current row. Usually, there is no associated locale, and output parameters (*pzLocale) and (*pnLocale) are set to NULL and 0, respectively. However, if the fts5_locale() function was used to associate a locale with the value when it was inserted into the fts5 table, then (*pzLocale) is set to point to a nul-terminated buffer containing the name of the locale in utf-8 encoding. (*pnLocale) is set to the size in bytes of the buffer, not including the nul-terminator.

If successful, SQLITE_OK is returned. Or, if an error occurs, an SQLite error code is returned. The final value of the output parameters is undefined in this case.

int (*xTokenize_v2)(Fts5Context*, const char *pText, int nText, const char *pLocale, int nLocale, void *pCtx, int (*xToken)(void*, int, const char*, int, int, int) )

Tokenize text using the tokenizer belonging to the FTS5 table. This API is the same as the xTokenize() API, except that it allows a tokenizer locale to be specified.

8. The fts5vocab Virtual Table Module

The fts5vocab virtual table module allows users to extract information from an FTS5 full-text index directly. The fts5vocab module is a part of FTS5 - it is available whenever FTS5 is.

Each fts5vocab table is associated with a single FTS5 table. An fts5vocab table is usually created by specifying two arguments in place of column names in the CREATE VIRTUAL TABLE statement - the name of the associated FTS5 table and the type of fts5vocab table. Currently there are three types of fts5vocab table; "row", "col" and "instance". Unless the fts5vocab table is created within the "temp" database, it must be part of the same database as the associated FTS5 table.

-- Create an fts5vocab "row" table to query the full-text index belonging
-- to FTS5 table "ft1".
CREATE VIRTUAL TABLE ft1_v USING fts5vocab('ft1', 'row');

-- Create an fts5vocab "col" table to query the full-text index belonging
-- to FTS5 table "ft2".
CREATE VIRTUAL TABLE ft2_v USING fts5vocab(ft2, col);

-- Create an fts5vocab "instance" table to query the full-text index
-- belonging to FTS5 table "ft3".
CREATE VIRTUAL TABLE ft3_v USING fts5vocab(ft3, instance);

If an fts5vocab table is created in the temp database, it may be associated with an FTS5 table in any attached database. In order to attach the fts5vocab table to an FTS5 table located in a database other than "temp", the name of the database is inserted before the FTS5 table name in the CREATE VIRTUAL TABLE arguments. For example:

-- Create an fts5vocab "row" table to query the full-text index belonging
-- to FTS5 table "ft1" in database "main".
CREATE VIRTUAL TABLE temp.ft1_v USING fts5vocab(main, 'ft1', 'row');

-- Create an fts5vocab "col" table to query the full-text index belonging
-- to FTS5 table "ft2" in attached database "aux".
CREATE VIRTUAL TABLE temp.ft2_v USING fts5vocab('aux', ft2, col);

-- Create an fts5vocab "instance" table to query the full-text index
-- belonging to FTS5 table "ft3" in attached database "other".
CREATE VIRTUAL TABLE temp.ft2_v USING fts5vocab('aux', ft3, 'instance');

Specifying three arguments when creating an fts5vocab table in any database other than "temp" results in an error.

An fts5vocab table of type "row" contains one row for each distinct term in the associated FTS5 table. The table columns are as follows:

An fts5vocab table of type "col" contains one row for each distinct term/column combination in the associated FTS5 table. Table columns are as follows:

An fts5vocab table of type "instance" contains one row for each term instance stored in the associated FTS index. Assuming the FTS5 table is created with the 'detail' option set to 'full', table columns are as follows:

If the FTS5 table is created with the 'detail' option set to 'col', then the offset column of an instance virtual table always contains NULL. In this case there is one row in the table for each unique term/doc/col combination. Or, if the FTS5 table is created with 'detail' set to 'none', then both offset and col always contain NULL values. For detail=none FTS5 tables, there is one row in the fts5vocab table for each unique term/doc combination.

Example:

-- Assuming a database created using:
CREATE VIRTUAL TABLE ft USING fts5(c1, c2);
INSERT INTO ft VALUES('apple banana cherry', 'banana banana cherry');
INSERT INTO ft VALUES('cherry cherry cherry', 'date date date');

-- Then querying the following fts5vocab table (type "col") returns:
--
--    apple  | c1 | 1 | 1
--    banana | c1 | 1 | 1
--    banana | c2 | 1 | 2
--    cherry | c1 | 2 | 4
--    cherry | c2 | 1 | 1
--    date   | c3 | 1 | 3
--
CREATE VIRTUAL TABLE ft_v_col USING fts5vocab(ft, col);

-- Querying an fts5vocab table of type "row" returns:
--
--    apple  | 1 | 1
--    banana | 1 | 3
--    cherry | 2 | 5
--    date   | 1 | 3
--
CREATE VIRTUAL TABLE ft_v_row USING fts5vocab(ft, row);

-- And, for type "instance"
INSERT INTO ft VALUES('apple banana cherry', 'banana banana cherry');
INSERT INTO ft VALUES('cherry cherry cherry', 'date date date');
--
--    apple  | 1 | c1 | 0
--    banana | 1 | c1 | 1
--    banana | 1 | c2 | 0
--    banana | 1 | c2 | 1
--    cherry | 1 | c1 | 2
--    cherry | 1 | c2 | 2
--    cherry | 2 | c1 | 0
--    cherry | 2 | c1 | 1
--    cherry | 2 | c1 | 2
--    date   | 2 | c2 | 0
--    date   | 2 | c2 | 1
--    date   | 2 | c2 | 2
--
CREATE VIRTUAL TABLE ft_v_instance USING fts5vocab(ft, instance);

9. FTS5 Data Structures

This section describes at a high-level the way the FTS module stores its index and content in the database. It is not necessary to read or understand the material in this section in order to use FTS in an application. However, it may be useful to application developers attempting to analyze and understand FTS performance characteristics, or to developers contemplating enhancements to the existing FTS feature set.

When an FTS5 virtual table is created in a database, between 3 and 5 real tables are created in the database. These are known as "shadow tables", and are used by the virtual table module to store persistent data. They should not be accessed directly by the user. Many other virtual table modules, including FTS3 and rtree, also create and use shadow tables.

FTS5 creates the following shadow tables. In each case the actual table name is based on the name of the FTS5 virtual table (in the following, replace % with the name of the virtual table to find the actual shadow table name).

-- This table contains most of the full-text index data. 
CREATE TABLE %_data(id INTEGER PRIMARY KEY, block BLOB);

-- This table contains the remainder of the full-text index data. 
-- It is almost always much smaller than the %_data table. 
CREATE TABLE %_idx(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;

-- Contains the values of persistent configuration parameters.
CREATE TABLE %_config(k PRIMARY KEY, v) WITHOUT ROWID;

-- Contains the size of each column of each row in the virtual table
-- in tokens. This shadow table is not present if the "columnsize"
-- option is set to 0.
CREATE TABLE %_docsize(id INTEGER PRIMARY KEY, sz BLOB);

-- Contains the actual data inserted into the FTS5 table. There
-- is one "cN" column for each indexed column in the FTS5 table.
-- This shadow table is not present for contentless or external 
-- content FTS5 tables. 
CREATE TABLE %_content(id INTEGER PRIMARY KEY, c0, c1...);

The following sections describe in more detail how these five tables are used to store FTS5 data.

9.1. Varint Format

The sections below refer to 64-bit signed integers stored in "varint" form. FTS5 uses the same varint format as used in various places by the SQLite core.

A varint is between 1 and 9 bytes in length. The varint consists of either zero or more bytes which have the high-order bit set followed by a single byte with the high-order bit clear, or nine bytes, whichever is shorter. The lower seven bits of each of the first eight bytes and all 8 bits of the ninth byte are used to reconstruct the 64-bit twos-complement integer. Varints are big-endian: bits taken from the earlier byte of the varint are more significant than bits taken from the later bytes.

9.2. The FTS Index (%_idx and %_data tables)

The FTS index is an ordered key-value store where the keys are document terms or term prefixes and the associated values are "doclists". A doclist is a packed array of varints that encodes the position of each instance of the term within the FTS5 table. The position of a single term instance is defined as the combination of:

• The rowid of the FTS5 table row it appears in,
• The index of the column the term instance appears in (columns are numbered from left to right starting from zero), and
• The offset of the term within the column value (i.e. the number of tokens that appear within the column value before this one).

The FTS index contains up to (nPrefix+1) entries for each token in the data set, where nPrefix is the number of defined prefix indexes.

Keys associated with the main FTS index (the one that is not a prefix index) are prefixed with the character "0". Keys for the first prefix index are prefixed with "1". Keys for the second prefix index are prefixed with "2", and so on. For example, if the token "document" is inserted into an FTS5 table with prefix indexes specified by prefix="2 4", then the keys added to the FTS index would be "0document", "1do" and "2docu".

The FTS index entries are not stored in a single tree or hash table structure. Instead, they are stored in a series of immutable b-tree like structures referred to as "segment b-trees". Each time a write to the FTS5 table is committed, one or more (but usually just one) new segment b-trees are added containing both the new entries and tombstones for any deleted entries. When the FTS index is queried, the reader queries each segment b-tree in turn and merges the results, giving priority to newer data.

Each segment b-tree is assigned a numerical level. When a new segment b-tree is written to the database as part of committing a transaction, it is assigned to level 0. Segment b-trees belonging to a single level are periodically merged together to create a single, larger segment b-tree that is assigned to the next level (i.e. level 0 segment b-trees are merged to become a single level 1 segment b-tree). Thus the numerically larger levels contain older data in (usually) larger segment b-trees. Refer to the 'automerge', 'crisismerge' and 'usermerge' options, along with the 'merge' and 'optimize' commands for details on how to control the merging.

In cases where the doclist associated with a term or term prefix is very large, there may be an associated doclist index. A doclist index is similar to the set of internal nodes of a b-tree. It allows a large doclist to be efficiently queried for rowids or ranges of rowids. For example, when processing a query like:

SELECT ... FROM ft('term') WHERE rowid BETWEEN ? AND ?

FTS5 uses the segment b-tree index to locate the doclist for term "term", then uses its doclist index (assuming it is present) to efficiently identify the subset of matches with rowids in the required range.

9.2.1. The %_data Table Rowid Space

CREATE TABLE %_data(
  id INTEGER PRIMARY KEY,
  block BLOB
);

The %_data table is used to store three types of records:

• The special structure record, stored with id=10.
• The special averages record, stored with id=1.
• A record to store each segment b-tree leaf and doclist index leaf and internal node. See below for how id values are calculated for these records.

Each segment b-tree in the system is assigned a unique 16-bit segment id. Segment ids may only be reused after the original owner segment b-tree is completely merged into a higher level segment b-tree. Within a segment b-tree, each leaf page is assigned a unique page number - 1 for the first leaf page, 2 for the second, and so on.

Each doclist index leaf page is also assigned a page number. The first (leftmost) leaf page in a doclist index is assigned the same page number as the segment b-tree leaf page on which its term appears (because doclist indexes are only created for terms with very long doclists, at most one term per segment b-tree leaf has an associated doclist index). Call this page number P. If the doclist is so large that it requires a second leaf, the second leaf is assigned page number P+1. The third leaf P+2. Each tier of a doclist index b-tree (leaves, parents of leaves, grandparents etc.) is assigned page numbers in this fashion, starting with page number P.

The "id" value used in the %_data table to store any given segment b-tree leaf or doclist index leaf or node is composed as follows:

9.2.2. Structure Record Format

The structure record identifies the set of segment b-trees that make up the current FTS index, along with details of any ongoing incremental merge operations. It is stored in the %_data table with id=10. A structure record begins with a single 32-bit unsigned value - the cookie value. This value is incremented each time the structure is modified. Following the cookie value are three varint values, as follows:

• The number of levels in the index (i.e. the maximum level associated with any segment b-tree plus one).
• The total number of segment b-trees in the index.
• The total number of segment b-tree leaves written to level 0 trees since the FTS5 table was created.

Then, for each level from 0 to nLevel:

• The number of input segments from the previous level being used as inputs for the current incremental merge, or zero if there is no ongoing incremental merge to create a new segment b-tree for this level.
• The total number of segment b-trees on the level.
• Then, for each segment b-tree, from oldest to newest:
  • The segment id.
  • Page number of first leaf (often 1, always >0).
  • Page number of last leaf (always >0).

9.2.3. Averages Record Format

The averages record, which is always stored with id=1 in the %_data table, does not store the average of anything. Instead, it contains a vector of (nCol+1) packed varint values, where nCol is the number of columns in the FTS5 table, including unindexed columns. The first varint contains the total number of rows in the FTS5 table. The second contains the total number of tokens in all values stored in the leftmost FTS5 table column. The third the number of tokens in all values for the next leftmost, and so on. The value for unindexed columns is always zero.

9.2.4. Segment B-Tree Format

9.2.4.1. The Key/Doclist Format

The key/doclist format is a format used to store a series of keys (document terms or term prefixes prefixed by a single character to indentify the specific index to which they belong) in sorted order, each with their associated doclist. The format consists of alternating keys and doclists packed together.

The first key is stored as:

• A varint indicating the number of bytes in the key (N), followed by
• The key data itself (N bytes).

Each subsequent key is stored as:

• A varint indicating the size of the prefix that the key has in common with the previous key in bytes,
• A varint indicating the number of bytes in the key following the common prefix (N), followed by
• The key suffix data itself (N bytes).

For example, if the first two keys in an FTS5 key/doclist record are "0challenger" and "0chandelier", then the first key is stored as varint 11 followed by the 11 bytes "0challenger", and the second key is stored as varints 4 and 7, followed by the 7 bytes "ndelier".

Each doclist identifies the rows (by their rowid values) that contain at least one instance of the term or term prefix and an associated position list, or "poslist" enumerating the position of each term instance within the row. In this sense a "position" is defined as a column number and term offset within the column value.

Within a doclist, documents are always stored in order sorted by rowid. The first rowid in a doclist is stored as is, as a varint. It is immediately followed by its associated position list. Following this, the difference between the first rowid and the second, as a varint, followed by the doclist associated with the second rowid in the doclist. And so on.

There is no way to determine the size of a doclist by parsing it. This must be stored externally. See the section below for details of how this is accomplished in FTS5.

A position list - often shortened to "poslist" - identifies the column and token offset within the row of each instance of the token in question. The format of a poslist is:

• Varint set to twice the size of the poslist, not including this field, plus one if the "delete" flag is set on the entry.
• A (possibly empty) list of offsets for column 0 (the leftmost column) of the row. Each offset is stored as a varint. The first varint contains the value of the first offset, plus 2. The second variant contains the difference between the second and first offsets, plus 2. etc. For example, if the offset list is to contain offsets 0, 10, 15 and 16, it is encoded by packing the following values, encoded as varints, end to end:

             2, 12, 7, 3
  

• For each column other than column 0 that contains one of more instances of the token:
  • Byte value 0x01.
  • The column number, as a varint.
  • An offset list, in the same format as the offset list for column 0.

If it is small enough (by default this means smaller than 4000 bytes), the entire contents of a segment b-tree may be stored in the key/doclist format described in the previous section as a single blob within the %_data table. Otherwise, the key/doclist is split into pages (by default, of approximately 4000 bytes each) and stored in a contiguous set of entries in the %_data table (see above for details).

When a key/doclist is divided into pages, the following modifications are made to the format:

• A single varint or key data field never spans two pages.
• The first key on each page is not prefix-compressed. It is stored in the format described above for the first key of a doclist - its size as a varint followed by the key data.
• If there are one or more rowids on a page before the first key, then the first of them is not delta compressed. It is stored as is, just as if it were the first rowid of its doclist (which it may or may not be).

Each page also has fixed-size 4-byte header and a variably-sized footer. The header is divided into 2 16-bit big-endian integer fields. They contain:

• The byte offset of the first rowid value on the page, if it occurs before the first key, or 0 otherwise.
• The byte offset of the page footer.

The page footer consists of a series of varints containing the byte offset of each key that appears on the page. The page footer is zero bytes in size if there are no keys on the page.

9.2.4.3. Segment Index Format

The result of formatting the contents of the segment b-tree in the key/doclist format and then splitting it into pages is something very similar to the leaves of a b+tree. Instead of creating a format for the internal nodes of this b+tree and storing them in the %_data table alongside the leaves, the keys that would have been stored on such nodes are added to the %_idx table, defined as:

CREATE TABLE %_idx(
  segid INTEGER,              -- segment id
  term TEXT,                  -- prefix of first key on page
  pgno INTEGER,               -- (2*pgno + bDoclistIndex)
  PRIMARY KEY(segid, term)
);

For each "leaf" page that contains at least one key, an entry is added to the %_idx table. Fields are set as follows:

       (pgno*2 + bDoclistIndexFlag)

Then, to find the leaf for segment i that may contain term t, instead of searching through internal nodes, FTS5 runs the query:

SELECT pgno FROM %_idx WHERE segid=$i AND term>=$t ORDER BY term LIMIT 1

9.2.4.4. Doclist Index Format

The segment index described in the previous section allows a segment b-tree to be efficiently queried by term or, assuming there is a prefix index of the required size, a term prefix. The data structure described in this section, doclist indexes, allows FTS5 to efficiently search for a rowid or range or rowids within the doclist associated with a single term or term prefix.

Not all keys have associated doclists indexes. By default, a doclist index is only added for a key if its doclist spans more than 4 segment b-tree leaf pages. Doclist indexes are themselves b-trees, with both leaves and internal nodes stored as entries in the %_data table, but in practice most doclists are small enough to fit on a single leaf. FTS5 uses the same rough size for doclist index node and leaves as it does for segment b-tree leaves (by default 4000 bytes).

Doclist index leaves and internal nodes use the same page format. The first byte is a "flags" byte. This is set to 0x00 for the root page of the doclist index b-tree, and 0x01 for all other pages. The remainder of the page is a series of tightly packed varints, as follows:

• page number of leftmost child page, followed by
• the smallest rowid value on the left most child page, followed by
• one varint for each subsequent child page, containing the value:
  • 0x00 if there are no rowids on the child page (this can only happen when the "child" page is actually a segment b-tree leaf), or
  • the difference between the smallest rowid on the child page and the previous rowid value stored on the doclist index page.

For the leftmost doclist index leaf in a doclist index, the leftmost child page is the first segment b-tree leaf after the one that contains the key itself.

9.3. Document Sizes Table (%_docsize table)

CREATE TABLE %_docsize(
    id INTEGER PRIMARY KEY,   -- id of FTS5 row this record pertains to
    sz BLOB                   -- blob containing nCol packed varints
);

Many common search result ranking functions require as an input the size in tokens of the result document (as a search term hit in a short document is considered more significant than one in a long document). To provide fast access to this information, for each row in the FTS5 table there exists a corresponding record (with the same rowid) in the %_docsize shadow table that contains the size of each column value in the row, in tokens.

The column value sizes are stored in a blob containing one packed varint for each column of the FTS5 table, from left to right. The varint contains, of course, the total number of tokens in the corresponding column value. Unindexed columns are included in this vector of varints; for them the value is always set to zero.

This table is used by the xColumnSize API. It can be omitted altogether by specifying the columnsize=0 option. In that case the xColumnSize API is still available to auxiliary functions, but runs much more slowly.

9.4. The Table Contents (%_content table)

-- locale=0 (the default) table 
CREATE TABLE %_content(id INTEGER PRIMARY KEY, c0, c1...);

-- locale=1 table 
CREATE TABLE %_content(id INTEGER PRIMARY KEY, c0, c1..., l0, l1...);

The actual table content - the values inserted into the FTS5 table, is stored in the %_content table. This table is created with one "c*" column for each column of the FTS5 table, including any unindexed columns. The values for the leftmost FTS5 table column are stored in column "c0" of the %_content table, the values from the next FTS5 table column in column "c1", and so on.

For an FTS5 table with the locale option set to 1, the %_content table also contains one "l*" column for each indexed (i.e. not UNINDEXED) column of the table. For values that were written to the fts5 table using the default locale, this column contains a NULL. Or, for values that were written with an associated locale (fts5_locale() values), this column contains the name of the locale, as text.

Each "l*" column name has the same integer component as its associated "c*" column. This means that if the fts5 table has one or more UNINDEXED columns, the set of "l*" column names may not contain a contiguous set of integer components. For example:

-- This fts5 table: 
CREATE VIRTUAL TABLE ft USING fts5(a, b UNINDEXED, c, locale=1);

-- uses a %_content table with no "l1" column:
CREATE TABLE ft_content(id INTEGER PRIMARY KEY, c0, c1, c2, l0, l2);

Unless the contentless_unindexed=1 option is specified, this table is omitted completely for external content or contentless FTS5 tables. For contentless tables that do specify the contentless_unindexed=1 option, the %_content table is created, but contains only those "c*" columns that correspond to UNINDEXED columns of the fts5 table. For example:

-- This fts5 table: 
CREATE VIRTUAL TABLE ft USING fts5(a, b UNINDEXED, c, contentless_unindexed=1);

-- uses a %_content table with only the "c1" (b) column
CREATE TABLE ft_content(id INTEGER PRIMARY KEY, c1);

9.5. Configuration Options (%_config table)

CREATE TABLE %_config(k PRIMARY KEY, v) WITHOUT ROWID;

This table stores the values of any persistent configuration options. Column "k" stores the name of the option (text) and column "v" the value. Example contents:

sqlite> SELECT * FROM ft_config;
┌─────────────┬──────┐
│      k      │  v   │
├─────────────┼──────┤
│ crisismerge │ 8    │
│ pgsz        │ 8000 │
│ usermerge   │ 4    │
│ version     │ 4    │
└─────────────┴──────┘

Appendix A: Comparison with FTS3/4

Also available is the similar but more mature FTS3/4 module. FTS5 is a new version of FTS4 that includes various fixes and solutions for problems that could not be fixed in FTS4 without sacrificing backwards compatibility. Some of these problems are described below.

Application Porting Guide

In order to use FTS5 instead of FTS3 or FTS4, applications usually require minimal modifications. Most of these fall into three categories - changes required to the CREATE VIRTUAL TABLE statement used to create the FTS table, changes required to SELECT queries used to execute queries against the table, and changes required to applications that use FTS auxiliary functions.

Changes to CREATE VIRTUAL TABLE statements

1. The module name must be changed from "fts3" or "fts4" to "fts5".

2. All type information or constraint specifications must be removed from column definitions. FTS3/4 ignores everything following the column name in a column definition, FTS5 attempts to parse it (and will report an error if it fails to).

3. The "matchinfo=fts3" option is not available. The "columnsize=0" option is equivalent.

4. The notindexed= option is not available. Adding UNINDEXED to the column definition is equivalent.

5. The ICU tokenizer is not available.

6. The compress=, uncompress= and languageid= options are not available. There is as of yet no equivalent for their functionality.

 -- FTS3/4 statement 
CREATE VIRTUAL TABLE ft USING fts4(
  linkid INTEGER,
  header CHAR(20),
  text VARCHAR,
  notindexed=linkid,
  matchinfo=fts3,
  tokenizer=unicode61
);

 -- FTS5 equivalent (note - the "tokenizer=unicode61" option is not
 -- required as this is the default for FTS5 anyway)
CREATE VIRTUAL TABLE ft USING fts5(
  linkid UNINDEXED,
  header,
  text,
  columnsize=0
);

Changes to SELECT statements

1. The "docid" alias does not exist. Applications must use "rowid" instead.

2. The behaviour of queries when a column-filter is specified both as part of the FTS query and by using a column as the LHS of a MATCH operator is slightly different. For a table with columns "a" and "b" and a query similar to:

   ... a MATCH 'b: string'
   

   FTS3/4 searches for matches in column "b". However, FTS5 always returns zero rows, as results are first filtered for column "b", then for column "a", leaving no results. In other words, in FTS3/4 the inner filter overrides the outer, in FTS5 both filters are applied.

3. The FTS query syntax (right hand side of the MATCH operator) has changed in some ways. The FTS5 syntax is quite close to the FTS4 "enhanced syntax". The main difference is that FTS5 is fussier about unrecognized punctuation characters and similar within query strings. Most queries that work with FTS3/4 should also work with FTS5, and those that do not should return parse errors.

Auxiliary Function Changes

FTS5 has no matchinfo() or offsets() function, and the snippet() function is not as fully-featured as in FTS3/4. However, since FTS5 does provide an API allowing applications to create custom auxiliary functions, any required functionality may be implemented within the application code.

The set of built-in auxiliary functions provided by FTS5 may be improved upon in the future.

Other Issues

1. The functionality provided by the fts4aux module is now provided by fts5vocab. The schema of these two tables is slightly different.

2. The FTS3/4 "merge=X,Y" command has been replaced by the FTS5 merge command.

3. The FTS3/4 "automerge=X" command has been replaced by the FTS5 automerge option.
   FTS3/4“automerge=X”命令已被FTS5 automerge 选项取代。

Summary of Technical Differences
技术差异总结

FTS5 is similar to FTS3/4 in that the primary task of each is to maintain an index mapping from each unique token to a list of instances of that token within a set of documents, where each instance is identified by the document in which it appears and its position within that document. For example:
FTS5 与 FTS3/4 类似，两者的主要任务是维护从每个唯一标记到一组文档中该标记的实例列表的索引映射，其中每个实例由它出现的文档来标识及其在该文件中的位置。例如：

-- Given the following SQL:
CREATE VIRTUAL TABLE ft USING fts5(a, b);
INSERT INTO ft(rowid, a, b) VALUES(1, 'X Y', 'Y Z');
INSERT INTO ft(rowid, a, b) VALUES(2, 'A Z', 'Y Y');

-- The FTS5 module creates the following mapping on disk:
A --> (2, 0, 0)
X --> (1, 0, 0)
Y --> (1, 0, 1) (1, 1, 0) (2, 1, 0) (2, 1, 1)
Z --> (1, 1, 1) (2, 0, 1)

In the example above, each triple identifies the location of a token instance by rowid, column number (columns are numbered sequentially starting at 0 from left to right) and position within the column value (the first token in a column value is 0, the second is 1, and so on). Using this index, FTS5 is able to provide timely answers to queries such as "the set of all documents that contain the token 'A'", or "the set of all documents that contain the sequence 'Y Z'". The list of instances associated with a single token is called an "instance-list".
在上面的示例中，每个三元组通过 rowid、列号（从左到右从 0 开始按顺序编号）和列值中的位置（列值中的第一个标记为 0，列值中的第一个标记为 0）来标识标记实例的位置。第二个是 1，依此类推）。使用该索引，FTS5 能够及时回答诸如“包含标记‘A’的所有文档的集合”或“包含序列‘Y Z’的所有文档的集合”等查询。与单个令牌关联的实例列表称为“实例列表”。

The principle difference between FTS3/4 and FTS5 is that in FTS3/4, each instance-list is stored as a single large database record, whereas in FTS5 large instance-lists are divided between multiple database records. This has the following implications for dealing with large databases that contain large lists:
FTS3/4 和 FTS5 之间的主要区别在于，在 FTS3/4 中，每个实例列表存储为单个大型数据库记录，而在 FTS5 中，大型实例列表分为多个数据库记录。这对于处理包含大型列表的大型数据库有以下影响：

• FTS5 is able to load instance-lists into memory incrementally in order to reduce memory usage and peak allocation size. FTS3/4 very often loads entire instance-lists into memory.
  FTS5 能够将实例列表增量加载到内存中，以减少内存使用量和峰值分配大小。 FTS3/4 经常将整个实例列表加载到内存中。

• When processing queries that feature more than one token, FTS5 is sometimes able to determine that the query can be answered by inspecting a subset of a large instance-list. FTS3/4 almost always has to traverse entire instance-lists.
  当处理具有多个令牌的查询时，FTS5 有时能够通过检查大型实例列表的子集来确定可以回答该查询。 FTS3/4 几乎总是必须遍历整个实例列表。

• If an instance-list grows so large that it exceeds the SQLITE_MAX_LENGTH limit, FTS3/4 is unable to handle it. FTS5 does not have this problem.
  如果实例列表变得太大以至于超过了SQLITE_MAX_LENGTH限制，FTS3/4 将无法处理它。 FTS5不存在这个问题。

For these reasons, many complex queries may use less memory and run faster using FTS5.
由于这些原因，许多复杂的查询使用 FTS5 可能会使用更少的内存并运行得更快。

Some other ways in which FTS5 differs from FTS3/4 are:
FTS5 与 FTS3/4 的其他一些不同之处包括：

• FTS5 supports "ORDER BY rank" for returning results in order of decreasing relevancy.
  FTS5 支持“ORDER BY 排名”，用于按相关性递减的顺序返回结果。

• FTS5 features an API allowing users to create custom auxiliary functions for advanced ranking and text processing applications. The special "rank" column may be mapped to a custom auxiliary function so that adding "ORDER BY rank" to a query works as expected.
  FTS5 具有 API，允许用户为高级排名和文本处理应用程序创建自定义辅助功能。特殊的“排名”列可以映射到自定义辅助函数，以便将“ORDER BY 排名”添加到查询按预期工作。

• FTS5 recognizes unicode separator characters and case equivalence by default. This is also possible using FTS3/4, but must be explicitly enabled.
  默认情况下，FTS5 识别 unicode 分隔符和大小写等效。使用 FTS3/4 也可以实现此目的，但必须明确启用。

• The query syntax has been revised where necessary to remove ambiguities and to make it possible to escape special characters in query terms.
  查询语法已在必要时进行了修订，以消除歧义并使查询术语中的特殊字符转义成为可能。

• By default, FTS3/4 occasionally merges together two or more of the b-trees that make up its full-text index within an INSERT, UPDATE or DELETE statement executed by the user. This means that any operation on an FTS3/4 table may turn out to be surprisingly slow, as FTS3/4 may unpredictably choose to merge together two or more large b-trees within it. FTS5 uses incremental merging by default, which limits the amount of processing that may take place within any given INSERT, UPDATE or DELETE operation.
  默认情况下，FTS3/4 有时会在用户执行的 INSERT、UPDATE 或 DELETE 语句中将组成其全文索引的两个或多个 B 树合并在一起。这意味着 FTS3/4 表上的任何操作可能会出奇地慢，因为 FTS3/4 可能会不可预测地选择将其中的两个或多个大型 B 树合并在一起。 FTS5 默认情况下使用增量合并，这限制了任何给定 INSERT、UPDATE 或 DELETE 操作中可能发生的处理量。

This page last modified on 2024-10-22 17:07:33 UTC
本页面最后修改时间： 2024-10-22 17:07:33 UTC